2024-09-17T05:07:56.071 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T05:07:56.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T05:07:56.187 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} email: pdonnell@redhat.com first_in_suite: false job_id: '7908606' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVFijXA5oXIMS5FWWjKJ9ohT3FkjrQ9Vc8KgrQHB6dt15v45lXgasZEpSRcX4qRXbu9IBK2dNxx/qfXG9Eke58= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM7aXWy+vrWX7qJlh0bvpeZLqz9g1SKXWHriE99HsvuoBb9GXfoViAHONezhGTA6L7u3H9/AAM81o1t6PIyfiw0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T05:07:56.187 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T05:07:56.188 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T05:07:56.188 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T05:07:56.190 INFO:teuthology.task.internal:Checking packages... 2024-09-17T05:07:56.211 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T05:07:56.211 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T05:07:56.211 INFO:teuthology.packaging:ref: None 2024-09-17T05:07:56.211 INFO:teuthology.packaging:tag: None 2024-09-17T05:07:56.212 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T05:07:56.212 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T05:07:56.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T05:07:56.357 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:07:56.358 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T05:07:56.366 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T05:07:56.366 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T05:07:56.426 INFO:teuthology.task.internal:Saving configuration 2024-09-17T05:07:56.443 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T05:07:56.464 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T05:07:56.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 05:02:25.839871', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEiF4UT9Z72s2jkvfbU/+C7PF+Bf/fV/yibUw244FEL1'} 2024-09-17T05:07:56.517 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 05:02:25.838379', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGuhqWPaqVG7Qc91I02RMSSc0GY58f1W1+eJPR237J0X'} 2024-09-17T05:07:56.517 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T05:07:56.526 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T05:07:56.526 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T05:07:56.526 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T05:07:56.614 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f50c328bac0>, signals=[15]) 2024-09-17T05:07:56.614 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T05:07:56.625 INFO:teuthology.task.internal:Opening connections... 2024-09-17T05:07:56.625 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:07:56.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:07:56.709 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:07:56.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:07:56.788 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T05:07:56.800 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-17T05:07:56.805 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-17T05:07:56.805 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="Ubuntu" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="22.04" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_CODENAME=jammy 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:ID=ubuntu 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE=debian 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T05:07:56.857 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T05:07:56.858 INFO:teuthology.orchestra.run.smithi043.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T05:07:56.858 INFO:teuthology.orchestra.run.smithi043.stdout:UBUNTU_CODENAME=jammy 2024-09-17T05:07:56.858 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-17T05:07:56.883 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-09-17T05:07:56.888 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-09-17T05:07:56.889 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-09-17T05:07:56.937 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T05:07:56.937 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Ubuntu" 2024-09-17T05:07:56.937 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="22.04" 2024-09-17T05:07:56.937 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_CODENAME=jammy 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:ID=ubuntu 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE=debian 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T05:07:56.938 INFO:teuthology.orchestra.run.smithi062.stdout:UBUNTU_CODENAME=jammy 2024-09-17T05:07:56.938 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-09-17T05:07:56.958 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T05:07:56.973 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T05:07:57.031 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T05:07:57.031 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-17T05:07:57.034 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-09-17T05:07:57.039 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T05:07:57.053 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T05:07:57.053 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-17T05:07:57.082 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-09-17T05:07:57.090 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T05:07:57.145 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T05:07:57.163 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-17T05:07:57.164 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-17T05:07:57.164 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-17T05:07:57.164 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T05:07:57.165 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T05:07:57.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:07:57.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T05:07:57.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:07:57.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T05:07:57.350 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:07:57.351 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T05:07:57.351 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-17T05:07:57.352 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:07:57.352 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T05:07:57.352 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-09-17T05:07:57.357 INFO:teuthology.orchestra.run.smithi043.stdout:5.15.0-56-generic 2024-09-17T05:07:57.358 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:07:57.358 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T05:07:57.358 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-17T05:07:57.358 DEBUG:teuthology.orchestra.run.smithi043:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T05:07:57.359 INFO:teuthology.orchestra.run.smithi062.stdout:5.15.0-56-generic 2024-09-17T05:07:57.360 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:07:57.360 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T05:07:57.360 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-17T05:07:57.360 DEBUG:teuthology.orchestra.run.smithi062:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T05:07:57.410 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T05:07:57.418 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: git config --global init.defaultBranch 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T05:07:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:hint: 2024-09-17T05:07:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:hint: git branch -m 2024-09-17T05:07:57.547 INFO:teuthology.orchestra.run.smithi043.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T05:07:57.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T05:07:57.552 INFO:teuthology.orchestra.run.smithi062.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: git config --global init.defaultBranch 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: 2024-09-17T05:07:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:hint: git branch -m 2024-09-17T05:07:57.554 INFO:teuthology.orchestra.run.smithi062.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T05:07:57.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T05:07:57.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T05:07:57.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T05:07:57.675 DEBUG:teuthology.orchestra.run.smithi043:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T05:07:57.679 DEBUG:teuthology.orchestra.run.smithi062:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T05:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T05:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr: * [new branch] main -> origin/main 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T05:08:27.421 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T05:08:27.422 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T05:08:27.422 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T05:08:27.422 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T05:08:27.424 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T05:08:27.425 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T05:08:27.428 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T05:08:27.429 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T05:08:27.430 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T05:08:27.430 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T05:08:27.430 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T05:08:27.430 INFO:teuthology.orchestra.run.smithi043.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T05:08:29.359 INFO:teuthology.orchestra.run.smithi062.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T05:08:29.359 INFO:teuthology.orchestra.run.smithi062.stderr: * [new branch] main -> origin/main 2024-09-17T05:08:29.399 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T05:08:29.399 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T05:08:29.399 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T05:08:29.399 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T05:08:29.401 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T05:08:29.402 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T05:08:29.402 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T05:08:29.402 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T05:08:29.402 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T05:08:29.404 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T05:08:29.405 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T05:08:29.407 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T05:08:29.407 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T05:08:29.407 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T05:08:29.407 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T05:08:29.408 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T05:08:29.409 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T05:08:29.411 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T05:08:29.412 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T05:08:29.412 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T05:08:29.412 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T05:08:29.412 INFO:teuthology.orchestra.run.smithi062.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T05:08:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:Updating files: 28% (1004/3520) Updating files: 29% (1021/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2054/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2426/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2884/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T05:08:32.297 INFO:teuthology.orchestra.run.smithi043.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T05:08:32.322 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T05:08:32.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:08:32.469 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:08:32.469 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T05:08:32.473 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi043.front.sepia.ceph.com to complete... 2024-09-17T05:08:33.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17 05:08:33 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T05:08:33.463 INFO:teuthology.orchestra.run.smithi043.stderr:FINISHED --2024-09-17 05:08:33-- 2024-09-17T05:08:33.464 INFO:teuthology.orchestra.run.smithi043.stderr:Total wall clock time: 1.0s 2024-09-17T05:08:33.464 INFO:teuthology.orchestra.run.smithi043.stderr:Downloaded: 1 files, 43M in 0.9s (50.6 MB/s) 2024-09-17T05:08:33.468 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T05:08:33.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T05:08:33.735 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T05:08:34.288 INFO:teuthology.orchestra.run.smithi062.stderr:Updating files: 28% (998/3520) Updating files: 29% (1021/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2053/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2426/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2884/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T05:08:34.294 INFO:teuthology.orchestra.run.smithi062.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T05:08:34.317 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T05:08:34.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:08:34.461 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:08:34.461 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T05:08:34.463 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2024-09-17T05:08:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17 05:08:35 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T05:08:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2024-09-17 05:08:35-- 2024-09-17T05:08:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 1.0s 2024-09-17T05:08:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 43M in 0.8s (52.1 MB/s) 2024-09-17T05:08:35.441 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T05:08:35.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T05:08:35.704 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T05:08:35.753 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T05:08:35.757 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T05:08:35.864 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:08:37.730 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T05:08:37.735 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T05:08:37.918 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:08:39.868 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:08:39.972 INFO:teuthology.orchestra.run.smithi043.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:08:42.223 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:08:42.322 INFO:teuthology.orchestra.run.smithi062.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:02.862 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:09:02.874 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:09:02.897 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-17T05:09:03.238 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:03.574 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:09:03.580 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:09:03.645 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:09:03.651 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:09:03.806 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:09:03.806 INFO:teuthology.orchestra.run.smithi043.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:09:03.806 INFO:teuthology.orchestra.run.smithi043.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:09:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-17T05:09:04.019 DEBUG:teuthology.orchestra.run.smithi043:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T05:09:04.026 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T05:09:04.026 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T05:09:04.026 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.045 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.046 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.070 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T05:09:04.071 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T05:09:04.072 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T05:09:04.073 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T05:09:04.074 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T05:09:04.074 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T05:09:04.074 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T05:09:04.074 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T05:09:04.075 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T05:09:04.089 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T05:09:04.089 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T05:09:04.089 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T05:09:04.090 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T05:09:04.091 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T05:09:04.092 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T05:09:04.093 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T05:09:04.094 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T05:09:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T05:09:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T05:09:04.097 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T05:09:04.098 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T05:09:04.099 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T05:09:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T05:09:04.101 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T05:09:04.146 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T05:09:04.147 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T05:09:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T05:09:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T05:09:04.150 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T05:09:04.151 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T05:09:04.152 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T05:09:04.153 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T05:09:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T05:09:04.155 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T05:09:04.156 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T05:09:04.157 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T05:09:04.158 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T05:09:04.159 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T05:09:04.160 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T05:09:04.161 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T05:09:04.162 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T05:09:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T05:09:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T05:09:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T05:09:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T05:09:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T05:09:04.172 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T05:09:04.177 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T05:09:04.178 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T05:09:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T05:09:04.181 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T05:09:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T05:09:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T05:09:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T05:09:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T05:09:04.186 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T05:09:04.187 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T05:09:04.188 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T05:09:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T05:09:04.190 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T05:09:04.191 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T05:09:04.192 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T05:09:04.193 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T05:09:04.239 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T05:09:04.240 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T05:09:04.241 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T05:09:04.242 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T05:09:04.243 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T05:09:04.244 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T05:09:04.245 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T05:09:04.246 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T05:09:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T05:09:04.248 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T05:09:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T05:09:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T05:09:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T05:09:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T05:09:04.253 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T05:09:04.254 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T05:09:04.255 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T05:09:04.256 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T05:09:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T05:09:04.258 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T05:09:04.259 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T05:09:04.260 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T05:09:04.261 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T05:09:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T05:09:04.263 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T05:09:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T05:09:04.265 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T05:09:04.266 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T05:09:04.267 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T05:09:04.268 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T05:09:04.269 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T05:09:04.270 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T05:09:04.271 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T05:09:04.272 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T05:09:04.273 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T05:09:04.274 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T05:09:04.275 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T05:09:04.276 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T05:09:04.277 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T05:09:04.278 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T05:09:04.279 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T05:09:04.280 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T05:09:04.281 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T05:09:04.282 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T05:09:04.282 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T05:09:04.284 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T05:09:04.285 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T05:09:04.286 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T05:09:04.287 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T05:09:04.288 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T05:09:04.289 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T05:09:04.290 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T05:09:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T05:09:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T05:09:04.336 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T05:09:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T05:09:04.338 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T05:09:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T05:09:04.340 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T05:09:04.341 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T05:09:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T05:09:04.343 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T05:09:04.344 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T05:09:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T05:09:04.346 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T05:09:04.347 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T05:09:04.348 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T05:09:04.349 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T05:09:04.350 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T05:09:04.351 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T05:09:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T05:09:04.353 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T05:09:04.354 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T05:09:04.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T05:09:04.357 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T05:09:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T05:09:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T05:09:04.361 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T05:09:04.362 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T05:09:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T05:09:04.366 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T05:09:04.367 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T05:09:04.368 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T05:09:04.369 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T05:09:04.370 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T05:09:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T05:09:04.372 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T05:09:04.373 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T05:09:04.374 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T05:09:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T05:09:04.376 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T05:09:04.377 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T05:09:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T05:09:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T05:09:04.380 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T05:09:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T05:09:04.382 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T05:09:04.383 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T05:09:04.384 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T05:09:04.385 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T05:09:04.385 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T05:09:04.420 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T05:09:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T05:09:04.422 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T05:09:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T05:09:04.424 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T05:09:04.425 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T05:09:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T05:09:04.427 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T05:09:04.428 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T05:09:04.429 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T05:09:04.430 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T05:09:04.431 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T05:09:04.432 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T05:09:04.433 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T05:09:04.434 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T05:09:04.435 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T05:09:04.436 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T05:09:04.437 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T05:09:04.438 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T05:09:04.439 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T05:09:04.440 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T05:09:04.441 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T05:09:04.442 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T05:09:04.443 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T05:09:04.444 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T05:09:04.445 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T05:09:04.446 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T05:09:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T05:09:04.448 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T05:09:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T05:09:04.450 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T05:09:04.451 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T05:09:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T05:09:04.453 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T05:09:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T05:09:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T05:09:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T05:09:04.457 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T05:09:04.458 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T05:09:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T05:09:04.460 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T05:09:04.461 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T05:09:04.462 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T05:09:04.468 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.468 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.469 DEBUG:teuthology.orchestra.run.smithi043:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T05:09:04.475 INFO:teuthology.orchestra.run.smithi043.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:04.476 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T05:09:04.476 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:04.477 DEBUG:teuthology.orchestra.run.smithi043:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T05:09:04.518 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi043.front.sepia.ceph.com to complete... 2024-09-17T05:09:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:09:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:09:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-17T05:09:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:09:05.368 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:09:05.388 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-09-17T05:09:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:09:05.558 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:09:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:09:05.606 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:09:05.690 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:05.716 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:05.819 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:09:05.819 INFO:teuthology.orchestra.run.smithi043.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:09:05.819 INFO:teuthology.orchestra.run.smithi043.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:09:05.830 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-17T05:09:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:09:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:09:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:09:06.296 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:09:06.297 INFO:teuthology.orchestra.run.smithi062.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:09:06.297 INFO:teuthology.orchestra.run.smithi062.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:09:06.335 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-09-17T05:09:06.567 DEBUG:teuthology.orchestra.run.smithi062:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T05:09:06.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T05:09:06.575 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T05:09:06.575 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:06.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:06.594 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T05:09:06.619 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T05:09:06.620 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T05:09:06.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T05:09:06.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T05:09:06.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T05:09:06.621 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T05:09:06.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T05:09:06.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T05:09:06.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T05:09:06.622 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T05:09:06.623 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T05:09:06.624 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T05:09:06.642 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T05:09:06.643 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T05:09:06.644 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T05:09:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T05:09:06.646 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T05:09:06.647 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T05:09:06.648 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T05:09:06.649 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T05:09:06.650 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T05:09:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T05:09:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T05:09:06.686 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T05:09:06.686 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T05:09:06.686 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T05:09:06.686 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T05:09:06.687 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T05:09:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T05:09:06.689 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T05:09:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T05:09:06.691 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T05:09:06.692 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T05:09:06.693 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T05:09:06.694 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T05:09:06.695 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T05:09:06.696 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T05:09:06.697 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T05:09:06.698 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T05:09:06.699 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T05:09:06.700 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T05:09:06.701 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T05:09:06.702 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T05:09:06.703 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T05:09:06.704 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T05:09:06.705 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T05:09:06.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T05:09:06.707 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T05:09:06.708 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T05:09:06.709 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T05:09:06.710 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T05:09:06.711 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T05:09:06.712 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T05:09:06.713 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T05:09:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T05:09:06.715 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T05:09:06.716 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T05:09:06.717 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T05:09:06.747 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T05:09:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T05:09:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T05:09:06.750 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T05:09:06.751 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T05:09:06.752 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T05:09:06.753 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T05:09:06.754 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T05:09:06.755 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T05:09:06.756 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T05:09:06.757 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T05:09:06.758 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T05:09:06.759 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T05:09:06.760 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T05:09:06.761 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T05:09:06.762 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T05:09:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T05:09:06.764 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T05:09:06.765 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T05:09:06.766 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T05:09:06.767 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T05:09:06.768 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T05:09:06.769 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T05:09:06.770 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T05:09:06.771 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T05:09:06.772 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T05:09:06.773 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T05:09:06.774 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T05:09:06.775 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T05:09:06.776 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T05:09:06.776 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T05:09:06.776 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T05:09:06.776 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T05:09:06.776 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T05:09:06.812 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T05:09:06.812 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T05:09:06.813 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T05:09:06.814 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T05:09:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T05:09:06.816 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T05:09:06.817 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T05:09:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T05:09:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T05:09:06.820 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T05:09:06.821 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T05:09:06.822 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T05:09:06.823 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T05:09:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T05:09:06.825 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T05:09:06.826 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T05:09:06.827 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T05:09:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T05:09:06.829 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T05:09:06.830 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T05:09:06.831 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T05:09:06.832 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T05:09:06.833 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T05:09:06.834 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T05:09:06.835 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T05:09:06.836 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T05:09:06.837 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T05:09:06.838 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T05:09:06.839 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T05:09:06.840 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T05:09:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T05:09:06.842 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T05:09:06.843 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T05:09:06.844 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T05:09:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T05:09:06.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T05:09:06.846 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T05:09:06.890 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T05:09:06.890 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T05:09:06.890 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T05:09:06.891 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T05:09:06.892 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T05:09:06.893 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T05:09:06.894 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T05:09:06.895 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T05:09:06.896 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T05:09:06.897 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T05:09:06.898 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T05:09:06.899 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T05:09:06.900 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T05:09:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T05:09:06.902 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T05:09:06.903 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T05:09:06.904 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T05:09:06.905 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T05:09:06.906 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T05:09:06.907 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T05:09:06.908 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T05:09:06.909 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T05:09:06.910 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T05:09:06.911 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T05:09:06.912 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T05:09:06.913 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T05:09:06.914 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T05:09:06.915 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T05:09:06.916 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T05:09:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T05:09:06.918 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T05:09:06.919 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T05:09:06.920 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T05:09:06.921 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T05:09:06.922 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T05:09:06.922 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T05:09:06.922 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T05:09:06.922 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T05:09:06.922 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T05:09:06.923 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T05:09:06.924 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T05:09:06.925 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T05:09:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T05:09:06.927 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T05:09:06.928 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T05:09:06.929 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T05:09:06.930 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T05:09:06.931 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T05:09:06.932 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T05:09:07.006 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T05:09:07.006 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T05:09:07.007 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T05:09:07.008 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T05:09:07.009 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T05:09:07.010 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T05:09:07.016 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T05:09:07.020 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T05:09:07.023 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T05:09:07.024 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T05:09:07.025 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T05:09:07.027 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T05:09:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T05:09:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T05:09:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T05:09:07.031 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T05:09:07.032 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T05:09:07.033 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T05:09:07.034 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T05:09:07.035 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T05:09:07.036 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T05:09:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T05:09:07.038 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T05:09:07.039 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T05:09:07.040 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T05:09:07.041 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T05:09:07.042 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T05:09:07.043 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T05:09:07.044 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T05:09:07.045 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T05:09:07.046 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T05:09:07.047 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T05:09:07.048 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T05:09:07.049 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T05:09:07.050 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T05:09:07.051 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T05:09:07.052 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T05:09:07.053 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T05:09:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T05:09:07.055 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T05:09:07.056 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T05:09:07.057 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T05:09:07.058 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T05:09:07.059 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T05:09:07.060 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T05:09:07.061 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T05:09:07.062 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T05:09:07.063 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T05:09:07.068 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:07.068 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:07.068 DEBUG:teuthology.orchestra.run.smithi062:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T05:09:07.075 INFO:teuthology.orchestra.run.smithi062.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:07.075 INFO:teuthology.orchestra.run.smithi062.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:07.075 INFO:teuthology.orchestra.run.smithi062.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:09:07.076 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T05:09:07.076 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:07.076 DEBUG:teuthology.orchestra.run.smithi062:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T05:09:07.118 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi062.front.sepia.ceph.com to complete... 2024-09-17T05:09:07.552 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:09:07.554 INFO:teuthology.orchestra.run.smithi062.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:09:07.579 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-09-17T05:09:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:09:08.036 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:09:08.041 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:09:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:09:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:09:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:09:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:09:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:09:08.334 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-09-17T05:09:36.971 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T05:09:36.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-17T05:09:36.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:09:39.454 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-17T05:09:39.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:09:39.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:10:37.029 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T05:10:39.459 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T05:10:46.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-17T05:10:46.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:10:46.440 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-17T05:10:47.300 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-17T05:10:47.300 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T05:10:47.301 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-17T05:10:47.348 INFO:teuthology.orchestra.run.smithi043.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:10:47.349 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:10:47.349 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T05:10:47.349 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T05:10:48.352 DEBUG:teuthology.parallel:result is None 2024-09-17T05:10:48.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:10:48.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:10:48.464 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-09-17T05:11:00.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:11:00.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:11:00.675 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-09-17T05:11:01.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:11:01.355 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T05:11:01.356 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-09-17T05:11:01.410 INFO:teuthology.orchestra.run.smithi062.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.410 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.411 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T05:11:01.411 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T05:11:02.412 DEBUG:teuthology.parallel:result is None 2024-09-17T05:11:02.413 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T05:11:02.421 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T05:11:02.422 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T05:11:02.426 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T05:11:02.435 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T05:11:02.459 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T05:11:02.493 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T05:11:02.493 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T05:11:02.497 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T05:11:02.536 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T05:11:02.543 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T05:11:02.543 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T05:11:02.576 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T05:11:02.660 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:11:02.682 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:11:02.687 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:11:02.723 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:11:02.726 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T05:11:02.733 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T05:11:02.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T05:11:02.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T05:11:02.758 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T05:11:02.768 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T05:11:02.768 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T05:11:02.804 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T05:11:02.813 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T05:11:02.861 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T05:11:02.913 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:11:02.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T05:11:02.986 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T05:11:02.995 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T05:11:03.049 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:11:03.049 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T05:11:03.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-17T05:11:03.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-09-17T05:11:03.240 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T05:11:03.250 INFO:teuthology.task.internal:Starting timer... 2024-09-17T05:11:03.251 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T05:11:03.281 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T05:11:03.321 DEBUG:teuthology.task.selinux:Excluding smithi043: OS 'ubuntu' does not support SELinux 2024-09-17T05:11:03.321 DEBUG:teuthology.task.selinux:Excluding smithi062: OS 'ubuntu' does not support SELinux 2024-09-17T05:11:03.322 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T05:11:03.322 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T05:11:03.322 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T05:11:03.322 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T05:11:03.355 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T05:11:03.357 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T05:11:03.468 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T05:11:03.479 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T05:11:03.480 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T05:19:26.425 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2024-09-17T05:19:26.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-17T05:19:26.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:19:26.512 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-17T05:19:26.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-17T05:19:26.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:19:26.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:19:26.721 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-09-17T05:19:26.859 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-09-17T05:19:26.859 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T05:19:26.871 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T05:19:26.871 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T05:19:26.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:19:26.875 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T05:19:26.875 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:19:26.912 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Command line: ntpd -gq 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: ---------------------------------------------------- 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: corporation. Support and training for ntp-4 are 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: available at https://www.nwtime.org/support 2024-09-17T05:19:26.913 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: ---------------------------------------------------- 2024-09-17T05:19:26.914 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: proto: precision = 0.043 usec (-24) 2024-09-17T05:19:26.914 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: basedate set to 2022-02-04 2024-09-17T05:19:26.914 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: gps base set to 2022-02-06 (week 2196) 2024-09-17T05:19:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:19:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen normally on 3 ens1f0 172.21.15.43:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen normally on 4 lo [::1]:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1358%3]:123 2024-09-17T05:19:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:26 ntpd[18367]: Listening on routing socket on fd #22 for interface updates 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Command line: ntpd -gq 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: ---------------------------------------------------- 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: corporation. Support and training for ntp-4 are 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: available at https://www.nwtime.org/support 2024-09-17T05:19:26.938 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: ---------------------------------------------------- 2024-09-17T05:19:26.939 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: proto: precision = 0.043 usec (-24) 2024-09-17T05:19:26.939 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: basedate set to 2022-02-04 2024-09-17T05:19:26.939 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: gps base set to 2022-02-06 (week 2196) 2024-09-17T05:19:26.940 INFO:teuthology.orchestra.run.smithi062.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:19:26.940 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:19:26.940 INFO:teuthology.orchestra.run.smithi062.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:19:26.941 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:19:26.941 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T05:19:26.941 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T05:19:26.942 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T05:19:26.942 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen normally on 3 enp3s0f1 172.21.15.62:123 2024-09-17T05:19:26.942 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen normally on 4 lo [::1]:123 2024-09-17T05:19:26.942 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:febb:d1c3%5]:123 2024-09-17T05:19:26.942 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:26 ntpd[18480]: Listening on routing socket on fd #22 for interface updates 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:27 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:27.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:27.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:27 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:28 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:28.938 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:28 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:28.938 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:28 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:29 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:29.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:29.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:29.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:29 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:30 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:31 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:31.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:31.941 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:31 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:32.913 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:32.913 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:32.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:32 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:33.915 INFO:teuthology.orchestra.run.smithi043.stdout:17 Sep 05:19:33 ntpd[18367]: ntpd: time slew +0.001001 s 2024-09-17T05:19:33.915 INFO:teuthology.orchestra.run.smithi043.stdout:ntpd: time slew +0.001001s 2024-09-17T05:19:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T05:19:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:17 Sep 05:19:33 ntpd[18367]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.939 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stdout:17 Sep 05:19:33 ntpd[18480]: ntpd: time slew -0.001764 s 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stdout:ntpd: time slew -0.001764s 2024-09-17T05:19:33.940 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T05:19:33.941 INFO:teuthology.orchestra.run.smithi062.stderr:17 Sep 05:19:33 ntpd[18480]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T05:19:34.024 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:19:34.024 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-09-17T05:19:34.024 INFO:teuthology.orchestra.run.smithi043.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.024 INFO:teuthology.orchestra.run.smithi043.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.025 INFO:teuthology.orchestra.run.smithi043.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.025 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================== 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.046 INFO:teuthology.orchestra.run.smithi062.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:19:34.047 INFO:teuthology.run_tasks:Running task install... 2024-09-17T05:19:34.055 DEBUG:teuthology.task.install:project ceph 2024-09-17T05:19:34.055 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T05:19:34.056 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T05:19:34.056 INFO:teuthology.task.install:Using flavor: default 2024-09-17T05:19:34.061 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T05:19:34.061 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T05:19:34.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-key list | grep Ceph 2024-09-17T05:19:34.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-key list | grep Ceph 2024-09-17T05:19:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T05:19:34.195 INFO:teuthology.orchestra.run.smithi062.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T05:19:34.228 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T05:19:34.228 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T05:19:34.229 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T05:19:34.229 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T05:19:34.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T05:19:34.247 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T05:19:34.247 INFO:teuthology.orchestra.run.smithi062.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T05:19:34.248 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T05:19:34.248 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T05:19:34.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T05:19:34.377 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T05:19:34.377 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:19:34.387 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T05:19:34.387 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:19:34.490 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:19:34.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T05:19:34.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-09-17T05:19:34.520 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:19:34.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T05:19:34.539 DEBUG:teuthology.orchestra.run.smithi062:> sudo apt-get update 2024-09-17T05:19:34.698 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T05:19:34.698 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T05:19:34.704 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T05:19:34.721 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T05:19:34.748 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T05:19:34.770 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T05:19:34.780 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T05:19:34.792 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T05:19:34.801 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T05:19:34.813 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T05:19:34.824 INFO:teuthology.orchestra.run.smithi062.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T05:19:34.845 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T05:19:34.846 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T05:19:34.867 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T05:19:34.867 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T05:19:34.888 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T05:19:34.929 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T05:19:35.010 INFO:teuthology.orchestra.run.smithi062.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T05:19:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 29.2 kB in 1s (51.2 kB/s) 2024-09-17T05:19:35.484 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 29.2 kB in 1s (41.3 kB/s) 2024-09-17T05:19:36.408 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-17T05:19:36.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:19:36.503 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-17T05:19:36.565 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-09-17T05:19:36.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:19:36.660 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-09-17T05:19:36.691 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-17T05:19:36.692 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-17T05:19:36.847 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-09-17T05:19:36.847 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-09-17T05:19:36.886 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:19:36.886 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:19:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:19:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:19:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:19:36.887 INFO:teuthology.orchestra.run.smithi043.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:19:36.887 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:19:36.888 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-09-17T05:19:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T05:19:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T05:19:36.888 INFO:teuthology.orchestra.run.smithi043.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:19:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T05:19:36.889 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T05:19:36.890 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-09-17T05:19:36.891 INFO:teuthology.orchestra.run.smithi043.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout:Recommended packages: 2024-09-17T05:19:36.892 INFO:teuthology.orchestra.run.smithi043.stdout: btrfs-tools 2024-09-17T05:19:37.047 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:19:37.048 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:19:37.048 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:19:37.048 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T05:19:37.049 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T05:19:37.050 INFO:teuthology.orchestra.run.smithi062.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:19:37.050 INFO:teuthology.orchestra.run.smithi062.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T05:19:37.050 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T05:19:37.051 INFO:teuthology.orchestra.run.smithi062.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T05:19:37.051 INFO:teuthology.orchestra.run.smithi062.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T05:19:37.052 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T05:19:37.053 INFO:teuthology.orchestra.run.smithi062.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T05:19:37.054 INFO:teuthology.orchestra.run.smithi062.stdout:Recommended packages: 2024-09-17T05:19:37.054 INFO:teuthology.orchestra.run.smithi062.stdout: btrfs-tools 2024-09-17T05:19:37.055 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:19:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T05:19:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T05:19:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T05:19:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T05:19:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T05:19:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T05:19:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: socat unzip xmlstarlet zip 2024-09-17T05:19:37.124 INFO:teuthology.orchestra.run.smithi043.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T05:19:37.124 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 239 MB of archives. 2024-09-17T05:19:37.125 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T05:19:37.125 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T05:19:37.143 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T05:19:37.153 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T05:19:37.195 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T05:19:37.206 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T05:19:37.212 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2024-09-17T05:19:37.212 INFO:teuthology.orchestra.run.smithi062.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T05:19:37.212 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T05:19:37.212 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T05:19:37.212 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:19:37.214 INFO:teuthology.orchestra.run.smithi062.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T05:19:37.215 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi062.stdout: socat unzip xmlstarlet zip 2024-09-17T05:19:37.216 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T05:19:37.219 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T05:19:37.231 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T05:19:37.271 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T05:19:37.273 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T05:19:37.280 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T05:19:37.283 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T05:19:37.284 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T05:19:37.284 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T05:19:37.291 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T05:19:37.293 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T05:19:37.294 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T05:19:37.296 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T05:19:37.309 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T05:19:37.310 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T05:19:37.312 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T05:19:37.315 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T05:19:37.316 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T05:19:37.327 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T05:19:37.329 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T05:19:37.329 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T05:19:37.330 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T05:19:37.330 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T05:19:37.345 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T05:19:37.345 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T05:19:37.345 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T05:19:37.347 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T05:19:37.348 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T05:19:37.362 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T05:19:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T05:19:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T05:19:37.364 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T05:19:37.364 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T05:19:37.369 INFO:teuthology.orchestra.run.smithi062.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T05:19:37.369 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 239 MB of archives. 2024-09-17T05:19:37.369 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T05:19:37.369 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T05:19:37.380 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T05:19:37.380 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T05:19:37.381 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T05:19:37.381 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T05:19:37.386 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T05:19:37.386 INFO:teuthology.orchestra.run.smithi043.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T05:19:37.398 INFO:teuthology.orchestra.run.smithi043.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T05:19:37.398 INFO:teuthology.orchestra.run.smithi043.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T05:19:37.399 INFO:teuthology.orchestra.run.smithi043.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T05:19:37.401 INFO:teuthology.orchestra.run.smithi043.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T05:19:37.402 INFO:teuthology.orchestra.run.smithi043.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T05:19:37.412 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T05:19:37.416 INFO:teuthology.orchestra.run.smithi043.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T05:19:37.417 INFO:teuthology.orchestra.run.smithi043.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T05:19:37.419 INFO:teuthology.orchestra.run.smithi043.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T05:19:37.419 INFO:teuthology.orchestra.run.smithi043.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T05:19:37.420 INFO:teuthology.orchestra.run.smithi043.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T05:19:37.430 INFO:teuthology.orchestra.run.smithi043.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T05:19:37.435 INFO:teuthology.orchestra.run.smithi043.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T05:19:37.443 INFO:teuthology.orchestra.run.smithi043.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T05:19:37.447 INFO:teuthology.orchestra.run.smithi043.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T05:19:37.447 INFO:teuthology.orchestra.run.smithi043.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T05:19:37.454 INFO:teuthology.orchestra.run.smithi043.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T05:19:37.455 INFO:teuthology.orchestra.run.smithi043.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T05:19:37.461 INFO:teuthology.orchestra.run.smithi043.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T05:19:37.481 INFO:teuthology.orchestra.run.smithi043.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T05:19:37.482 INFO:teuthology.orchestra.run.smithi043.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T05:19:37.483 INFO:teuthology.orchestra.run.smithi043.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T05:19:37.484 INFO:teuthology.orchestra.run.smithi043.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T05:19:37.485 INFO:teuthology.orchestra.run.smithi043.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T05:19:37.498 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T05:19:37.504 INFO:teuthology.orchestra.run.smithi043.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T05:19:37.507 INFO:teuthology.orchestra.run.smithi043.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T05:19:37.508 INFO:teuthology.orchestra.run.smithi043.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T05:19:37.525 INFO:teuthology.orchestra.run.smithi043.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T05:19:37.525 INFO:teuthology.orchestra.run.smithi043.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T05:19:37.526 INFO:teuthology.orchestra.run.smithi043.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T05:19:37.527 INFO:teuthology.orchestra.run.smithi043.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T05:19:37.527 INFO:teuthology.orchestra.run.smithi043.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T05:19:37.528 INFO:teuthology.orchestra.run.smithi043.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T05:19:37.541 INFO:teuthology.orchestra.run.smithi043.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T05:19:37.542 INFO:teuthology.orchestra.run.smithi043.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T05:19:37.543 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T05:19:37.545 INFO:teuthology.orchestra.run.smithi043.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T05:19:37.559 INFO:teuthology.orchestra.run.smithi043.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T05:19:37.560 INFO:teuthology.orchestra.run.smithi043.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T05:19:37.561 INFO:teuthology.orchestra.run.smithi043.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T05:19:37.565 INFO:teuthology.orchestra.run.smithi043.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T05:19:37.567 INFO:teuthology.orchestra.run.smithi043.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T05:19:37.568 INFO:teuthology.orchestra.run.smithi043.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T05:19:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T05:19:37.577 INFO:teuthology.orchestra.run.smithi043.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T05:19:37.581 INFO:teuthology.orchestra.run.smithi043.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T05:19:37.582 INFO:teuthology.orchestra.run.smithi043.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T05:19:37.594 INFO:teuthology.orchestra.run.smithi043.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T05:19:37.595 INFO:teuthology.orchestra.run.smithi043.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T05:19:37.595 INFO:teuthology.orchestra.run.smithi043.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T05:19:37.596 INFO:teuthology.orchestra.run.smithi043.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T05:19:37.598 INFO:teuthology.orchestra.run.smithi043.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T05:19:37.598 INFO:teuthology.orchestra.run.smithi043.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T05:19:37.605 INFO:teuthology.orchestra.run.smithi043.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T05:19:37.612 INFO:teuthology.orchestra.run.smithi043.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T05:19:37.612 INFO:teuthology.orchestra.run.smithi043.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T05:19:37.614 INFO:teuthology.orchestra.run.smithi043.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T05:19:37.624 INFO:teuthology.orchestra.run.smithi043.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T05:19:37.625 INFO:teuthology.orchestra.run.smithi043.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T05:19:37.629 INFO:teuthology.orchestra.run.smithi043.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T05:19:37.632 INFO:teuthology.orchestra.run.smithi043.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T05:19:37.742 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T05:19:37.834 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T05:19:37.847 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T05:19:37.897 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T05:19:38.034 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T05:19:38.070 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T05:19:38.080 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T05:19:38.106 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T05:19:38.115 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T05:19:38.117 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T05:19:38.118 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T05:19:38.136 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T05:19:38.141 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T05:19:38.144 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T05:19:38.152 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T05:19:38.170 INFO:teuthology.orchestra.run.smithi043.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T05:19:38.229 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T05:19:38.232 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T05:19:38.235 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T05:19:38.236 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T05:19:38.237 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T05:19:38.237 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T05:19:38.238 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T05:19:38.239 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T05:19:38.239 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T05:19:38.259 INFO:teuthology.orchestra.run.smithi043.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T05:19:38.262 INFO:teuthology.orchestra.run.smithi043.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T05:19:38.264 INFO:teuthology.orchestra.run.smithi043.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T05:19:38.290 INFO:teuthology.orchestra.run.smithi043.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T05:19:38.319 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T05:19:38.320 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T05:19:38.401 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T05:19:38.401 INFO:teuthology.orchestra.run.smithi062.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T05:19:38.403 INFO:teuthology.orchestra.run.smithi062.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T05:19:38.404 INFO:teuthology.orchestra.run.smithi062.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T05:19:38.404 INFO:teuthology.orchestra.run.smithi062.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T05:19:38.405 INFO:teuthology.orchestra.run.smithi062.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T05:19:38.406 INFO:teuthology.orchestra.run.smithi062.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T05:19:38.407 INFO:teuthology.orchestra.run.smithi062.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T05:19:38.418 INFO:teuthology.orchestra.run.smithi043.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T05:19:38.462 INFO:teuthology.orchestra.run.smithi062.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T05:19:38.483 INFO:teuthology.orchestra.run.smithi062.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T05:19:38.483 INFO:teuthology.orchestra.run.smithi062.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T05:19:38.520 INFO:teuthology.orchestra.run.smithi062.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T05:19:38.564 INFO:teuthology.orchestra.run.smithi062.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T05:19:38.564 INFO:teuthology.orchestra.run.smithi062.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T05:19:38.570 INFO:teuthology.orchestra.run.smithi062.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T05:19:38.570 INFO:teuthology.orchestra.run.smithi062.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T05:19:38.570 INFO:teuthology.orchestra.run.smithi062.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T05:19:38.571 INFO:teuthology.orchestra.run.smithi062.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T05:19:38.571 INFO:teuthology.orchestra.run.smithi062.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T05:19:38.572 INFO:teuthology.orchestra.run.smithi062.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T05:19:38.572 INFO:teuthology.orchestra.run.smithi062.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T05:19:38.574 INFO:teuthology.orchestra.run.smithi062.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T05:19:38.575 INFO:teuthology.orchestra.run.smithi062.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T05:19:38.576 INFO:teuthology.orchestra.run.smithi062.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T05:19:38.627 INFO:teuthology.orchestra.run.smithi062.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T05:19:38.646 INFO:teuthology.orchestra.run.smithi062.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T05:19:38.647 INFO:teuthology.orchestra.run.smithi062.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T05:19:38.659 INFO:teuthology.orchestra.run.smithi062.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T05:19:38.662 INFO:teuthology.orchestra.run.smithi062.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T05:19:38.662 INFO:teuthology.orchestra.run.smithi062.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T05:19:38.727 INFO:teuthology.orchestra.run.smithi062.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T05:19:38.729 INFO:teuthology.orchestra.run.smithi062.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T05:19:38.838 INFO:teuthology.orchestra.run.smithi062.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T05:19:38.839 INFO:teuthology.orchestra.run.smithi062.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T05:19:38.841 INFO:teuthology.orchestra.run.smithi062.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T05:19:38.841 INFO:teuthology.orchestra.run.smithi062.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T05:19:38.842 INFO:teuthology.orchestra.run.smithi062.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T05:19:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T05:19:38.878 INFO:teuthology.orchestra.run.smithi043.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T05:19:38.885 INFO:teuthology.orchestra.run.smithi043.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T05:19:38.915 INFO:teuthology.orchestra.run.smithi062.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T05:19:38.917 INFO:teuthology.orchestra.run.smithi062.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T05:19:38.918 INFO:teuthology.orchestra.run.smithi062.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T05:19:38.937 INFO:teuthology.orchestra.run.smithi043.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T05:19:38.950 INFO:teuthology.orchestra.run.smithi043.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T05:19:38.952 INFO:teuthology.orchestra.run.smithi043.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T05:19:38.990 INFO:teuthology.orchestra.run.smithi062.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T05:19:38.990 INFO:teuthology.orchestra.run.smithi062.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T05:19:38.990 INFO:teuthology.orchestra.run.smithi062.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T05:19:38.991 INFO:teuthology.orchestra.run.smithi062.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T05:19:38.992 INFO:teuthology.orchestra.run.smithi062.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T05:19:38.992 INFO:teuthology.orchestra.run.smithi062.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T05:19:39.056 INFO:teuthology.orchestra.run.smithi062.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T05:19:39.056 INFO:teuthology.orchestra.run.smithi062.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T05:19:39.059 INFO:teuthology.orchestra.run.smithi062.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T05:19:39.138 INFO:teuthology.orchestra.run.smithi062.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T05:19:39.139 INFO:teuthology.orchestra.run.smithi062.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T05:19:39.140 INFO:teuthology.orchestra.run.smithi062.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T05:19:39.143 INFO:teuthology.orchestra.run.smithi062.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T05:19:39.146 INFO:teuthology.orchestra.run.smithi062.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T05:19:39.146 INFO:teuthology.orchestra.run.smithi062.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T05:19:39.147 INFO:teuthology.orchestra.run.smithi062.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T05:19:39.219 INFO:teuthology.orchestra.run.smithi062.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T05:19:39.224 INFO:teuthology.orchestra.run.smithi062.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T05:19:39.225 INFO:teuthology.orchestra.run.smithi062.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T05:19:39.288 INFO:teuthology.orchestra.run.smithi062.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T05:19:39.289 INFO:teuthology.orchestra.run.smithi062.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T05:19:39.301 INFO:teuthology.orchestra.run.smithi062.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T05:19:39.301 INFO:teuthology.orchestra.run.smithi062.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T05:19:39.302 INFO:teuthology.orchestra.run.smithi062.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T05:19:39.302 INFO:teuthology.orchestra.run.smithi062.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T05:19:39.304 INFO:teuthology.orchestra.run.smithi062.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T05:19:39.305 INFO:teuthology.orchestra.run.smithi062.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T05:19:39.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T05:19:39.340 INFO:teuthology.orchestra.run.smithi062.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T05:19:39.383 INFO:teuthology.orchestra.run.smithi062.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T05:19:39.384 INFO:teuthology.orchestra.run.smithi062.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T05:19:39.386 INFO:teuthology.orchestra.run.smithi062.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T05:19:39.465 INFO:teuthology.orchestra.run.smithi062.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T05:19:40.521 INFO:teuthology.orchestra.run.smithi043.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T05:19:40.559 INFO:teuthology.orchestra.run.smithi062.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T05:19:40.710 INFO:teuthology.orchestra.run.smithi062.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T05:19:40.723 INFO:teuthology.orchestra.run.smithi062.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T05:19:40.726 INFO:teuthology.orchestra.run.smithi062.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T05:19:40.776 INFO:teuthology.orchestra.run.smithi062.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T05:19:40.806 INFO:teuthology.orchestra.run.smithi043.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T05:19:40.806 INFO:teuthology.orchestra.run.smithi043.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T05:19:40.959 INFO:teuthology.orchestra.run.smithi062.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T05:19:41.685 INFO:teuthology.orchestra.run.smithi062.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T05:19:41.685 INFO:teuthology.orchestra.run.smithi062.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T05:19:41.714 INFO:teuthology.orchestra.run.smithi062.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T05:19:41.782 INFO:teuthology.orchestra.run.smithi062.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T05:19:41.824 INFO:teuthology.orchestra.run.smithi062.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T05:19:41.826 INFO:teuthology.orchestra.run.smithi062.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T05:19:42.191 INFO:teuthology.orchestra.run.smithi043.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T05:19:42.192 INFO:teuthology.orchestra.run.smithi043.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T05:19:42.194 INFO:teuthology.orchestra.run.smithi043.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T05:19:42.203 INFO:teuthology.orchestra.run.smithi043.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T05:19:42.204 INFO:teuthology.orchestra.run.smithi043.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T05:19:42.205 INFO:teuthology.orchestra.run.smithi043.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T05:19:42.783 INFO:teuthology.orchestra.run.smithi043.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T05:19:43.327 INFO:teuthology.orchestra.run.smithi062.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T05:19:43.451 INFO:teuthology.orchestra.run.smithi062.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T05:19:43.451 INFO:teuthology.orchestra.run.smithi062.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T05:19:43.464 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 239 MB in 6s (41.9 MB/s) 2024-09-17T05:19:43.592 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T05:19:43.659 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T05:19:43.663 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T05:19:43.724 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T05:19:44.026 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T05:19:44.040 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T05:19:44.085 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:19:44.134 INFO:teuthology.orchestra.run.smithi062.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T05:19:44.134 INFO:teuthology.orchestra.run.smithi062.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T05:19:44.137 INFO:teuthology.orchestra.run.smithi062.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T05:19:44.137 INFO:teuthology.orchestra.run.smithi062.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T05:19:44.137 INFO:teuthology.orchestra.run.smithi062.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T05:19:44.138 INFO:teuthology.orchestra.run.smithi062.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T05:19:44.423 INFO:teuthology.orchestra.run.smithi062.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T05:19:44.429 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T05:19:44.444 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T05:19:44.487 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T05:19:44.824 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T05:19:44.840 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T05:19:44.924 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:19:45.104 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 239 MB in 7s (33.3 MB/s) 2024-09-17T05:19:45.252 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T05:19:45.277 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T05:19:45.293 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T05:19:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T05:19:45.323 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T05:19:45.393 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T05:19:45.503 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:19:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T05:19:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T05:19:45.754 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:19:45.856 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T05:19:45.871 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T05:19:45.914 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:19:46.082 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T05:19:46.096 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T05:19:46.140 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T05:19:46.384 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T05:19:46.400 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:46.451 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:46.509 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T05:19:46.524 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T05:19:46.593 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:19:46.820 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T05:19:46.836 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:46.870 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:46.913 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T05:19:46.927 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T05:19:46.971 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:19:47.232 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T05:19:47.248 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:47.282 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T05:19:47.290 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:47.298 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T05:19:47.332 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:19:47.652 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T05:19:47.667 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T05:19:47.710 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:19:47.744 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T05:19:47.760 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:47.819 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:48.013 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librados2. 2024-09-17T05:19:48.029 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:48.062 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:48.431 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T05:19:48.446 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:48.480 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:48.566 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libnbd0. 2024-09-17T05:19:48.581 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T05:19:48.624 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T05:19:48.792 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T05:19:48.807 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:19:48.841 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:19:48.968 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librbd1. 2024-09-17T05:19:48.984 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:49.018 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:49.178 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T05:19:49.193 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T05:19:49.236 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:19:49.413 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T05:19:49.428 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:49.463 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:49.555 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librados2. 2024-09-17T05:19:49.571 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:49.614 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:49.773 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rados. 2024-09-17T05:19:49.790 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:49.832 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:50.083 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libnbd0. 2024-09-17T05:19:50.098 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T05:19:50.141 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T05:19:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T05:19:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:19:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:50.378 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librbd1. 2024-09-17T05:19:50.393 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:50.436 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:50.463 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T05:19:50.476 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:50.520 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:50.823 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T05:19:50.837 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:19:50.881 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:50.888 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T05:19:50.904 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:50.947 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:51.224 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rados. 2024-09-17T05:19:51.235 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T05:19:51.239 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:51.248 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T05:19:51.282 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:51.293 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T05:19:51.587 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T05:19:51.601 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:51.601 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T05:19:51.617 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:19:51.645 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:51.659 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:51.895 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T05:19:51.911 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:51.970 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:51.982 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T05:19:51.998 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T05:19:52.040 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:19:52.273 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T05:19:52.289 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:19:52.332 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:52.360 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T05:19:52.374 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T05:19:52.417 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:19:52.651 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T05:19:52.666 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T05:19:52.709 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T05:19:52.720 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T05:19:52.735 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T05:19:52.779 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:19:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T05:19:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:53.053 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:53.124 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua5.1. 2024-09-17T05:19:53.138 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T05:19:53.174 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:19:53.331 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T05:19:53.347 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T05:19:53.381 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:19:53.477 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-any. 2024-09-17T05:19:53.491 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T05:19:53.535 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T05:19:53.650 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T05:19:53.666 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T05:19:53.708 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:19:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package zip. 2024-09-17T05:19:53.819 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T05:19:53.854 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T05:19:54.003 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T05:19:54.018 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T05:19:54.069 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:19:54.157 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package unzip. 2024-09-17T05:19:54.172 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T05:19:54.241 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:19:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package lua5.1. 2024-09-17T05:19:54.438 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T05:19:54.472 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:19:54.619 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package luarocks. 2024-09-17T05:19:54.637 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T05:19:54.678 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:19:54.750 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package lua-any. 2024-09-17T05:19:54.766 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T05:19:54.808 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T05:19:55.061 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package zip. 2024-09-17T05:19:55.077 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T05:19:55.128 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T05:19:55.352 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librgw2. 2024-09-17T05:19:55.368 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:55.402 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:55.498 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package unzip. 2024-09-17T05:19:55.513 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T05:19:55.564 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:19:55.959 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package luarocks. 2024-09-17T05:19:55.975 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T05:19:56.017 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:19:56.063 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T05:19:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:56.121 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:56.467 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package librgw2. 2024-09-17T05:19:56.468 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T05:19:56.481 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:56.483 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T05:19:56.526 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:19:56.533 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:56.854 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T05:19:56.870 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:56.912 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:57.177 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T05:19:57.193 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:57.244 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:57.307 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-common. 2024-09-17T05:19:57.323 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:57.357 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:57.705 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T05:19:57.721 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T05:19:57.771 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:19:58.083 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T05:19:58.098 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:58.141 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:58.746 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-base. 2024-09-17T05:19:58.762 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:58.769 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-common. 2024-09-17T05:19:58.781 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:19:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:58.819 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:19:59.442 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T05:19:59.457 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T05:19:59.500 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:19:59.778 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T05:19:59.794 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T05:19:59.836 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:20:00.183 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-base. 2024-09-17T05:20:00.199 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:00.250 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:00.381 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T05:20:00.391 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T05:20:00.431 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:20:00.767 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T05:20:00.777 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T05:20:00.817 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:20:00.828 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T05:20:00.842 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T05:20:00.878 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:20:01.070 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T05:20:01.080 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T05:20:01.111 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:20:01.139 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T05:20:01.153 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T05:20:01.189 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:20:01.423 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T05:20:01.433 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T05:20:01.459 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T05:20:01.473 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T05:20:01.473 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T05:20:01.509 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:20:01.776 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-portend. 2024-09-17T05:20:01.785 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T05:20:01.836 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T05:20:01.842 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T05:20:01.853 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T05:20:01.903 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:20:02.129 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T05:20:02.138 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T05:20:02.178 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T05:20:02.206 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T05:20:02.221 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T05:20:02.264 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:20:02.398 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T05:20:02.411 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T05:20:02.448 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:20:02.542 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T05:20:02.557 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T05:20:02.592 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T05:20:02.810 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T05:20:02.821 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T05:20:02.859 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T05:20:02.861 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-portend. 2024-09-17T05:20:02.876 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T05:20:02.936 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T05:20:03.171 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T05:20:03.182 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T05:20:03.221 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T05:20:03.239 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T05:20:03.254 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T05:20:03.297 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T05:20:03.507 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T05:20:03.517 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T05:20:03.557 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:20:03.583 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T05:20:03.598 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T05:20:03.641 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:20:03.868 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-mako. 2024-09-17T05:20:03.879 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T05:20:03.918 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:20:04.028 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T05:20:04.043 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T05:20:04.095 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T05:20:04.247 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T05:20:04.257 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T05:20:04.296 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:20:04.406 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T05:20:04.424 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T05:20:04.464 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T05:20:04.624 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T05:20:04.638 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T05:20:04.682 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:20:04.742 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T05:20:04.757 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T05:20:04.792 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:20:04.952 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webob. 2024-09-17T05:20:04.964 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T05:20:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:20:05.103 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-mako. 2024-09-17T05:20:05.118 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T05:20:05.161 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:20:05.372 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T05:20:05.383 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T05:20:05.424 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:20:05.490 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T05:20:05.505 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T05:20:05.548 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:20:05.733 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T05:20:05.745 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T05:20:05.816 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:20:05.884 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T05:20:05.899 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T05:20:05.950 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:20:06.119 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-paste. 2024-09-17T05:20:06.131 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T05:20:06.169 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:20:06.328 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webob. 2024-09-17T05:20:06.343 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T05:20:06.395 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:20:06.514 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T05:20:06.526 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T05:20:06.564 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:20:06.748 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T05:20:06.763 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T05:20:06.809 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:20:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T05:20:06.878 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T05:20:06.916 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:20:07.109 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T05:20:07.124 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T05:20:07.159 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:20:07.228 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T05:20:07.243 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T05:20:07.295 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T05:20:07.420 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-paste. 2024-09-17T05:20:07.435 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T05:20:07.478 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:20:07.556 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T05:20:07.569 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T05:20:07.606 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:20:07.781 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T05:20:07.795 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T05:20:07.839 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:20:08.019 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T05:20:08.032 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T05:20:08.069 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T05:20:08.117 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T05:20:08.131 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T05:20:08.167 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:20:08.405 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-idna. 2024-09-17T05:20:08.418 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T05:20:08.470 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T05:20:08.480 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:20:08.484 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T05:20:08.528 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T05:20:08.741 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T05:20:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T05:20:08.783 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:20:08.823 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T05:20:08.838 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T05:20:08.881 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:20:09.053 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-requests. 2024-09-17T05:20:09.066 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T05:20:09.128 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:20:09.252 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T05:20:09.263 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T05:20:09.310 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T05:20:09.439 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T05:20:09.452 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T05:20:09.489 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:20:09.613 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-idna. 2024-09-17T05:20:09.626 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T05:20:09.655 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:20:09.809 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T05:20:09.821 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:09.858 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:09.949 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T05:20:09.960 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T05:20:10.674 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:20:10.986 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-requests. 2024-09-17T05:20:10.996 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T05:20:11.035 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:20:11.039 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T05:20:11.051 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:11.088 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:11.346 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T05:20:11.357 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T05:20:11.397 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:20:11.433 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T05:20:11.445 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T05:20:11.483 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:20:11.683 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T05:20:11.694 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:11.733 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:11.807 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T05:20:11.818 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T05:20:11.861 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T05:20:12.112 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T05:20:12.122 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:12.162 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:12.172 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T05:20:12.184 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:12.213 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:12.398 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T05:20:12.409 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T05:20:12.439 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:20:12.609 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T05:20:12.621 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:12.659 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:12.767 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T05:20:12.778 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T05:20:12.817 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T05:20:13.128 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T05:20:13.139 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:13.178 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:13.271 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T05:20:13.283 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:13.321 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:13.556 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T05:20:13.567 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:13.606 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:14.091 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph. 2024-09-17T05:20:14.102 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:14.134 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T05:20:14.140 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:14.148 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:14.192 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:14.485 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T05:20:14.496 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:14.534 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:14.896 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T05:20:14.908 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:14.945 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:14.962 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph. 2024-09-17T05:20:14.973 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:15.012 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:15.283 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T05:20:15.295 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:15.324 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephadm. 2024-09-17T05:20:15.336 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:15.341 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:15.365 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:15.678 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T05:20:15.690 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:15.693 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T05:20:15.705 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T05:20:15.727 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:15.743 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T05:20:16.121 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T05:20:16.130 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephadm. 2024-09-17T05:20:16.133 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:20:16.142 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:16.170 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:16.180 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:16.475 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T05:20:16.487 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T05:20:16.524 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T05:20:16.819 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T05:20:16.830 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:20:16.869 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-babel. 2024-09-17T05:20:17.194 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:20:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:17.562 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T05:20:17.573 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T05:20:17.611 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:20:17.865 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-babel. 2024-09-17T05:20:17.876 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:20:17.915 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:18.007 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T05:20:18.018 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:18.056 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:18.267 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T05:20:18.279 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T05:20:18.317 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:20:18.443 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T05:20:18.455 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T05:20:18.493 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T05:20:18.670 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T05:20:18.682 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:18.720 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:18.796 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-routes. 2024-09-17T05:20:18.808 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T05:20:18.846 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:20:19.233 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T05:20:19.245 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:19.282 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:19.324 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T05:20:19.336 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T05:20:19.373 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T05:20:19.676 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-routes. 2024-09-17T05:20:19.688 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T05:20:19.726 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:20:20.062 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T05:20:20.077 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:20.120 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:21.988 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T05:20:22.000 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T05:20:22.038 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:20:22.392 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T05:20:22.405 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T05:20:22.450 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:20:22.508 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T05:20:22.519 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T05:20:22.651 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:20:22.920 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T05:20:22.932 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T05:20:23.022 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T05:20:23.033 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T05:20:23.046 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:20:23.072 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:20:23.392 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T05:20:23.400 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T05:20:23.404 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T05:20:23.412 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T05:20:23.442 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:20:23.450 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:20:23.728 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T05:20:23.740 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T05:20:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:20:24.175 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T05:20:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:24.234 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:24.403 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T05:20:24.415 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:24.444 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:25.254 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T05:20:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T05:20:25.312 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T05:20:25.422 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T05:20:25.436 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T05:20:25.480 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T05:20:25.582 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T05:20:25.595 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T05:20:25.640 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:20:25.742 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T05:20:25.756 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T05:20:25.792 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:20:26.001 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T05:20:26.015 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T05:20:26.059 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:20:26.087 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T05:20:26.100 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T05:20:26.145 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:20:26.363 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T05:20:26.376 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T05:20:26.446 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T05:20:26.490 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T05:20:26.503 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T05:20:26.548 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T05:20:26.766 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T05:20:26.780 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T05:20:26.825 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T05:20:26.860 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T05:20:26.873 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T05:20:26.918 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T05:20:27.169 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T05:20:27.186 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T05:20:27.213 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T05:20:27.226 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T05:20:27.227 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:20:27.262 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:20:27.531 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T05:20:27.545 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T05:20:27.589 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T05:20:27.590 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T05:20:27.604 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T05:20:27.649 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T05:20:27.926 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T05:20:27.940 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T05:20:27.969 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T05:20:27.982 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T05:20:27.984 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:20:28.027 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:20:28.690 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T05:20:28.703 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:28.706 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T05:20:28.719 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:28.740 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:28.764 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:29.075 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T05:20:29.076 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T05:20:29.088 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T05:20:29.089 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T05:20:29.126 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:20:29.133 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:20:29.421 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T05:20:29.433 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:20:29.470 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T05:20:29.484 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:20:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:20:29.528 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:20:29.723 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package jq. 2024-09-17T05:20:29.736 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:20:29.773 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T05:20:29.839 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package jq. 2024-09-17T05:20:29.854 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:20:29.897 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T05:20:30.018 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package socat. 2024-09-17T05:20:30.031 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T05:20:30.084 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:20:30.151 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package socat. 2024-09-17T05:20:30.165 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T05:20:30.208 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:20:30.437 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T05:20:30.451 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T05:20:30.503 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:20:30.511 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T05:20:30.526 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T05:20:30.570 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:20:30.882 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-test. 2024-09-17T05:20:30.895 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:30.932 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:30.941 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-test. 2024-09-17T05:20:30.954 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:30.999 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:33.012 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T05:20:33.026 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:33.078 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:33.337 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T05:20:33.352 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:33.412 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:33.466 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T05:20:33.480 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:33.524 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:33.766 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T05:20:33.781 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:33.816 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:33.819 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T05:20:33.832 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T05:20:33.877 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T05:20:34.154 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T05:20:34.168 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:34.177 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T05:20:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T05:20:34.212 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:34.235 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T05:20:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T05:20:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:34.490 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:34.505 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T05:20:34.519 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:34.563 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:34.841 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T05:20:34.855 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T05:20:34.899 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:35.001 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T05:20:35.015 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:35.059 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:35.135 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T05:20:35.149 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:35.184 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:35.378 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T05:20:35.392 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T05:20:35.436 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:20:35.523 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T05:20:35.537 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T05:20:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:20:35.765 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T05:20:35.779 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T05:20:35.823 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:20:35.918 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T05:20:35.933 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T05:20:35.968 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T05:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T05:20:36.269 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:20:36.339 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T05:20:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T05:20:36.397 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:20:36.682 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package pkg-config. 2024-09-17T05:20:36.695 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T05:20:36.740 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:20:36.844 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package pkg-config. 2024-09-17T05:20:36.858 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T05:20:36.902 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:20:37.002 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T05:20:37.016 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T05:20:37.060 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:20:37.163 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T05:20:37.178 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T05:20:37.221 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:20:37.529 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T05:20:37.543 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T05:20:37.588 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T05:20:37.725 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T05:20:37.739 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T05:20:37.783 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T05:20:37.882 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T05:20:37.896 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T05:20:37.932 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T05:20:38.102 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T05:20:38.117 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T05:20:38.160 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T05:20:38.210 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T05:20:38.224 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T05:20:38.268 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:20:38.505 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T05:20:38.520 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T05:20:38.563 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:20:38.621 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T05:20:38.635 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T05:20:38.680 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T05:20:38.900 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T05:20:38.914 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T05:20:38.958 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T05:20:39.016 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-py. 2024-09-17T05:20:39.029 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T05:20:39.073 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T05:20:39.303 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-py. 2024-09-17T05:20:39.317 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T05:20:39.394 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T05:20:39.402 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T05:20:39.416 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T05:20:39.485 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:20:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T05:20:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T05:20:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:20:39.931 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T05:20:39.944 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T05:20:39.981 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:20:40.267 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-toml. 2024-09-17T05:20:40.281 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T05:20:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T05:20:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T05:20:40.325 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T05:20:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:20:40.621 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-toml. 2024-09-17T05:20:40.635 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T05:20:40.644 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T05:20:40.658 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T05:20:40.670 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T05:20:40.703 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:20:40.982 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T05:20:40.997 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T05:20:40.997 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T05:20:41.011 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T05:20:41.039 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:20:41.040 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:20:41.283 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T05:20:41.296 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T05:20:41.350 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:20:41.402 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T05:20:41.416 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T05:20:41.451 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:20:41.704 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T05:20:41.719 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T05:20:41.762 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:20:42.095 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package radosgw. 2024-09-17T05:20:42.108 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:42.137 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:42.491 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package radosgw. 2024-09-17T05:20:42.505 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:42.549 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:43.007 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T05:20:43.020 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:43.064 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:43.375 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T05:20:43.428 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T05:20:43.442 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T05:20:43.477 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:43.653 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:20:43.779 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:20:43.781 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T05:20:44.066 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T05:20:44.082 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:20:44.209 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:20:44.471 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T05:20:44.483 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T05:20:44.901 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T05:20:45.112 INFO:teuthology.orchestra.run.smithi062.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T05:20:45.155 INFO:teuthology.orchestra.run.smithi062.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T05:20:45.198 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:45.557 INFO:teuthology.orchestra.run.smithi043.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T05:20:45.607 INFO:teuthology.orchestra.run.smithi043.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T05:20:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:45.733 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user cephadm....done 2024-09-17T05:20:45.878 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:20:46.205 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:20:46.456 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:20:46.574 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:20:46.606 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user cephadm....done 2024-09-17T05:20:46.767 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:20:46.816 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T05:20:47.060 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:20:47.067 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:20:47.227 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T05:20:47.311 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:20:47.421 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:20:47.554 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:20:47.680 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T05:20:47.914 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T05:20:48.190 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T05:20:48.248 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:20:48.374 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T05:20:48.492 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T05:20:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:20:48.776 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:20:49.027 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:20:49.144 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T05:20:49.296 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:20:49.420 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T05:20:49.579 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:20:49.680 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T05:20:49.847 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:50.014 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:20:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:20:50.398 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:20:50.525 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:20:50.549 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T05:20:50.684 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:20:50.801 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:20:50.809 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T05:20:51.052 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:51.085 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:20:51.211 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:20:51.337 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:20:51.462 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T05:20:51.519 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:20:51.605 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:20:51.723 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T05:20:51.870 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T05:20:51.901 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T05:20:51.982 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:20:51.999 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:20:52.108 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T05:20:52.125 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:20:52.392 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:20:52.493 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:52.643 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T05:20:52.720 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T05:20:52.773 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T05:20:52.837 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:20:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:20:52.963 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T05:20:53.196 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:20:53.357 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:20:53.482 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T05:20:53.513 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T05:20:53.622 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:20:53.767 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:20:54.043 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:20:54.352 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T05:20:54.358 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:20:54.453 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:20:54.617 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:20:54.743 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:20:54.894 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:20:55.220 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T05:20:55.231 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:20:55.506 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:20:55.629 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T05:20:55.632 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:20:55.758 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:20:55.897 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:20:56.093 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T05:20:56.290 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:20:56.502 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T05:20:56.575 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T05:20:56.786 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:20:56.901 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:20:57.221 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:20:57.294 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:20:57.539 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T05:20:57.620 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:20:57.721 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:57.882 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:20:58.064 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:20:58.242 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:20:58.423 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:20:58.534 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:20:58.661 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:20:58.978 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:20:59.329 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:20:59.704 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:20:59.947 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:00.081 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T05:21:00.341 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:21:00.469 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:21:00.586 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:21:00.602 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:21:00.853 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:00.871 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:21:00.962 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T05:21:01.132 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T05:21:01.231 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:21:01.357 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:21:01.399 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:01.482 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:21:01.499 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:21:01.801 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:21:01.801 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:21:02.068 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T05:21:02.077 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T05:21:02.311 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:02.353 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T05:21:02.445 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:21:02.633 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:21:02.771 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:21:02.880 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T05:21:03.031 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T05:21:03.165 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:21:03.291 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T05:21:03.434 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T05:21:03.575 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:21:03.700 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:21:03.701 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:21:03.993 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T05:21:04.011 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:21:04.252 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:21:04.320 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:21:04.352 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T05:21:04.605 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:21:04.662 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:21:04.714 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:21:04.780 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:21:04.840 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:21:04.929 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T05:21:05.089 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:21:05.175 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:21:05.309 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:21:05.399 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:21:05.666 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:21:05.728 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:21:05.817 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:21:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:21:05.996 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T05:21:06.032 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T05:21:06.097 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T05:21:06.278 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:21:06.356 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:06.421 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:21:06.457 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:21:06.783 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:21:06.831 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:21:07.101 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:21:07.115 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T05:21:07.241 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T05:21:07.479 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T05:21:07.509 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:07.580 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T05:21:07.610 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:21:07.865 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T05:21:07.961 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:21:08.133 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:21:08.278 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:21:08.409 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:21:08.690 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T05:21:08.816 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T05:21:08.828 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T05:21:09.101 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T05:21:09.154 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:21:09.369 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:21:09.515 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:21:09.616 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:09.628 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:21:09.734 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:09.860 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:21:10.055 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T05:21:10.221 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:10.348 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:10.373 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:21:10.457 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T05:21:10.734 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:21:10.742 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:10.835 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:10.868 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:10.936 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:11.002 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:11.087 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:21:11.257 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:21:11.257 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:21:11.431 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:11.557 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:11.708 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T05:21:11.937 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.002 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.063 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.127 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.214 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.253 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.332 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:12.433 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:21:12.518 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:21:12.518 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:21:13.180 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:13.315 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:13.440 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:13.566 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:13.734 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:21:13.753 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:13.879 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:14.005 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:14.131 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:14.232 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:14.333 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:14.912 INFO:teuthology.orchestra.run.smithi062.stdout:Adding group ceph....done 2024-09-17T05:21:15.062 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.188 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.271 INFO:teuthology.orchestra.run.smithi062.stdout:Adding system user ceph....done 2024-09-17T05:21:15.314 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.407 INFO:teuthology.orchestra.run.smithi062.stdout:Setting system user ceph properties....done 2024-09-17T05:21:15.425 INFO:teuthology.orchestra.run.smithi062.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T05:21:15.441 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.566 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.581 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T05:21:15.701 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:15.996 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T05:21:16.254 INFO:teuthology.orchestra.run.smithi043.stdout:Adding group ceph....done 2024-09-17T05:21:16.630 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user ceph....done 2024-09-17T05:21:16.768 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:16.799 INFO:teuthology.orchestra.run.smithi043.stdout:Setting system user ceph properties....done 2024-09-17T05:21:16.820 INFO:teuthology.orchestra.run.smithi043.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T05:21:16.975 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T05:21:16.985 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:21:16.985 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:21:17.390 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T05:21:17.661 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:17.787 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:18.060 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:18.275 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:21:18.275 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:21:18.326 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:21:18.326 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:21:18.953 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:18.997 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:19.079 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:19.356 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T05:21:19.641 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:21:19.641 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:21:20.108 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:20.289 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:21:20.322 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:21:20.660 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T05:21:21.160 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:21.332 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:21.365 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:21:21.365 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:21:21.555 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:21:21.555 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:21:22.187 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:22.234 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:22.447 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:21:22.447 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:21:22.493 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:21:22.494 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:21:23.085 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:23.163 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:23.289 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:23.401 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:21:23.401 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:21:23.432 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:23.650 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:21:23.650 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:21:24.070 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.255 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.316 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.406 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.459 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.569 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:24.623 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:21:24.623 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:21:24.686 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:25.048 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:21:25.148 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:21:25.257 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:25.334 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:21:25.400 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:25.542 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:25.660 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T05:21:26.029 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:21:26.155 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:21:26.307 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:21:26.520 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:21:27.222 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:21:27.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T05:21:27.306 INFO:teuthology.orchestra.run.smithi062.stdout:Reading package lists... 2024-09-17T05:21:27.494 INFO:teuthology.orchestra.run.smithi062.stdout:Building dependency tree... 2024-09-17T05:21:27.494 INFO:teuthology.orchestra.run.smithi062.stdout:Reading state information... 2024-09-17T05:21:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:21:27.690 INFO:teuthology.orchestra.run.smithi062.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:21:27.691 INFO:teuthology.orchestra.run.smithi062.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:21:27.691 INFO:teuthology.orchestra.run.smithi062.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:21:27.691 INFO:teuthology.orchestra.run.smithi062.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:21:27.691 INFO:teuthology.orchestra.run.smithi062.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:21:27.692 INFO:teuthology.orchestra.run.smithi062.stdout:The following additional packages will be installed: 2024-09-17T05:21:27.692 INFO:teuthology.orchestra.run.smithi062.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T05:21:27.692 INFO:teuthology.orchestra.run.smithi062.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T05:21:27.693 INFO:teuthology.orchestra.run.smithi062.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T05:21:27.693 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T05:21:27.693 INFO:teuthology.orchestra.run.smithi062.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:21:27.694 INFO:teuthology.orchestra.run.smithi062.stdout:Suggested packages: 2024-09-17T05:21:27.694 INFO:teuthology.orchestra.run.smithi062.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T05:21:27.695 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T05:21:27.695 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T05:21:27.751 INFO:teuthology.orchestra.run.smithi062.stdout:The following NEW packages will be installed: 2024-09-17T05:21:27.752 INFO:teuthology.orchestra.run.smithi062.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T05:21:27.752 INFO:teuthology.orchestra.run.smithi062.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T05:21:27.752 INFO:teuthology.orchestra.run.smithi062.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T05:21:27.752 INFO:teuthology.orchestra.run.smithi062.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T05:21:27.752 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T05:21:27.753 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T05:21:27.753 INFO:teuthology.orchestra.run.smithi062.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:21:27.754 INFO:teuthology.orchestra.run.smithi062.stdout:The following packages will be upgraded: 2024-09-17T05:21:27.755 INFO:teuthology.orchestra.run.smithi062.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T05:21:27.941 INFO:teuthology.orchestra.run.smithi062.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T05:21:27.941 INFO:teuthology.orchestra.run.smithi062.stdout:Need to get 55.8 MB of archives. 2024-09-17T05:21:27.941 INFO:teuthology.orchestra.run.smithi062.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T05:21:27.941 INFO:teuthology.orchestra.run.smithi062.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T05:21:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:21:28.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T05:21:28.410 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-17T05:21:28.565 INFO:teuthology.orchestra.run.smithi062.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T05:21:28.582 INFO:teuthology.orchestra.run.smithi062.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T05:21:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-17T05:21:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-17T05:21:28.707 INFO:teuthology.orchestra.run.smithi062.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T05:21:28.708 INFO:teuthology.orchestra.run.smithi062.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T05:21:28.709 INFO:teuthology.orchestra.run.smithi062.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T05:21:28.737 INFO:teuthology.orchestra.run.smithi062.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T05:21:28.738 INFO:teuthology.orchestra.run.smithi062.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T05:21:28.741 INFO:teuthology.orchestra.run.smithi062.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T05:21:28.744 INFO:teuthology.orchestra.run.smithi062.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T05:21:28.745 INFO:teuthology.orchestra.run.smithi062.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T05:21:28.746 INFO:teuthology.orchestra.run.smithi062.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:21:28.793 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:21:28.794 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:21:28.794 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:21:28.794 INFO:teuthology.orchestra.run.smithi043.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:21:28.794 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:21:28.795 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-09-17T05:21:28.795 INFO:teuthology.orchestra.run.smithi043.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T05:21:28.795 INFO:teuthology.orchestra.run.smithi043.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T05:21:28.796 INFO:teuthology.orchestra.run.smithi043.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T05:21:28.796 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T05:21:28.796 INFO:teuthology.orchestra.run.smithi043.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:21:28.797 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-09-17T05:21:28.797 INFO:teuthology.orchestra.run.smithi043.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T05:21:28.797 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T05:21:28.797 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T05:21:28.810 INFO:teuthology.orchestra.run.smithi062.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T05:21:28.810 INFO:teuthology.orchestra.run.smithi062.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T05:21:28.811 INFO:teuthology.orchestra.run.smithi062.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T05:21:28.855 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-09-17T05:21:28.855 INFO:teuthology.orchestra.run.smithi043.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T05:21:28.855 INFO:teuthology.orchestra.run.smithi043.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T05:21:28.856 INFO:teuthology.orchestra.run.smithi043.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T05:21:28.856 INFO:teuthology.orchestra.run.smithi043.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T05:21:28.856 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T05:21:28.856 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T05:21:28.856 INFO:teuthology.orchestra.run.smithi043.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:21:28.857 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-09-17T05:21:28.858 INFO:teuthology.orchestra.run.smithi043.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T05:21:28.892 INFO:teuthology.orchestra.run.smithi062.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T05:21:29.043 INFO:teuthology.orchestra.run.smithi043.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T05:21:29.043 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 55.8 MB of archives. 2024-09-17T05:21:29.044 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T05:21:29.044 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T05:21:29.573 INFO:teuthology.orchestra.run.smithi062.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T05:21:29.574 INFO:teuthology.orchestra.run.smithi062.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T05:21:29.575 INFO:teuthology.orchestra.run.smithi062.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T05:21:29.578 INFO:teuthology.orchestra.run.smithi062.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T05:21:29.578 INFO:teuthology.orchestra.run.smithi062.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T05:21:29.582 INFO:teuthology.orchestra.run.smithi062.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T05:21:29.584 INFO:teuthology.orchestra.run.smithi062.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T05:21:29.635 INFO:teuthology.orchestra.run.smithi062.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T05:21:29.635 INFO:teuthology.orchestra.run.smithi062.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T05:21:29.662 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T05:21:29.699 INFO:teuthology.orchestra.run.smithi062.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T05:21:29.699 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T05:21:29.720 INFO:teuthology.orchestra.run.smithi062.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T05:21:29.729 INFO:teuthology.orchestra.run.smithi062.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T05:21:29.803 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T05:21:29.803 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T05:21:29.804 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T05:21:29.805 INFO:teuthology.orchestra.run.smithi062.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T05:21:29.808 INFO:teuthology.orchestra.run.smithi062.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T05:21:29.809 INFO:teuthology.orchestra.run.smithi062.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T05:21:29.810 INFO:teuthology.orchestra.run.smithi062.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T05:21:29.810 INFO:teuthology.orchestra.run.smithi062.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T05:21:29.824 INFO:teuthology.orchestra.run.smithi062.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T05:21:29.826 INFO:teuthology.orchestra.run.smithi062.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T05:21:29.833 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T05:21:29.834 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T05:21:29.837 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T05:21:29.840 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T05:21:29.840 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T05:21:29.842 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T05:21:29.906 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T05:21:29.907 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T05:21:29.907 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T05:21:29.988 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T05:21:30.311 INFO:teuthology.orchestra.run.smithi062.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T05:21:30.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T05:21:30.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T05:21:30.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T05:21:30.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T05:21:30.312 INFO:teuthology.orchestra.run.smithi062.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T05:21:30.374 INFO:teuthology.orchestra.run.smithi062.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T05:21:30.375 INFO:teuthology.orchestra.run.smithi062.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T05:21:30.668 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T05:21:30.669 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T05:21:30.670 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T05:21:30.673 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T05:21:30.673 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T05:21:30.677 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T05:21:30.679 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T05:21:30.730 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T05:21:30.730 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T05:21:30.763 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T05:21:30.774 INFO:teuthology.orchestra.run.smithi062.stdout:Preconfiguring packages ... 2024-09-17T05:21:30.814 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T05:21:30.825 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T05:21:30.901 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T05:21:30.904 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T05:21:30.905 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T05:21:30.906 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T05:21:30.906 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T05:21:30.919 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T05:21:30.921 INFO:teuthology.orchestra.run.smithi062.stdout:Fetched 55.8 MB in 3s (21.4 MB/s) 2024-09-17T05:21:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T05:21:31.201 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T05:21:31.205 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:31.370 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T05:21:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T05:21:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T05:21:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T05:21:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T05:21:31.409 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T05:21:31.422 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T05:21:31.423 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T05:21:31.828 INFO:teuthology.orchestra.run.smithi043.stdout:Preconfiguring packages ... 2024-09-17T05:21:32.004 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:32.017 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 55.8 MB in 3s (21.8 MB/s) 2024-09-17T05:21:32.264 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T05:21:32.268 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:32.407 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:33.059 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:33.087 INFO:teuthology.orchestra.run.smithi062.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T05:21:33.316 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package flex. 2024-09-17T05:21:33.390 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T05:21:33.393 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:33.439 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T05:21:33.902 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:21:34.088 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T05:21:34.202 INFO:teuthology.orchestra.run.smithi043.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T05:21:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:34.882 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package flex. 2024-09-17T05:21:34.959 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T05:21:34.963 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:35.005 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T05:21:35.029 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T05:21:35.032 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:35.161 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:35.551 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:21:35.617 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:35.719 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T05:21:35.743 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:36.035 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:36.170 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:36.453 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package bison. 2024-09-17T05:21:36.532 INFO:teuthology.orchestra.run.smithi062.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T05:21:36.536 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T05:21:36.581 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:21:36.718 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T05:21:36.721 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:36.869 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:36.990 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T05:21:37.006 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T05:21:37.048 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:21:37.274 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:21:37.408 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:21:37.476 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package dump. 2024-09-17T05:21:37.493 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T05:21:37.618 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T05:21:37.758 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:37.914 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package indent. 2024-09-17T05:21:37.931 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:37.972 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:21:38.134 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package bison. 2024-09-17T05:21:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T05:21:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T05:21:38.233 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T05:21:38.250 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T05:21:38.252 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:21:38.316 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:21:38.621 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T05:21:38.637 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T05:21:38.670 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T05:21:38.679 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:21:38.686 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T05:21:38.728 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:21:38.932 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T05:21:38.949 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T05:21:38.990 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:21:39.082 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package dump. 2024-09-17T05:21:39.099 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T05:21:39.140 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T05:21:39.327 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T05:21:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:39.410 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:21:39.452 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package indent. 2024-09-17T05:21:39.469 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:39.502 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:21:39.646 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T05:21:39.663 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:39.704 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:21:39.797 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T05:21:39.814 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T05:21:39.855 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:21:39.957 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T05:21:39.974 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T05:21:40.015 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T05:21:40.193 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T05:21:40.209 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T05:21:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:21:40.327 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T05:21:40.344 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T05:21:40.393 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:21:40.503 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T05:21:40.520 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T05:21:40.553 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:21:40.907 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T05:21:40.924 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:40.973 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:21:41.243 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T05:21:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T05:21:41.301 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:21:41.570 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T05:21:41.587 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T05:21:41.629 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T05:21:41.791 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T05:21:41.807 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:21:41.849 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:21:41.923 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T05:21:41.940 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T05:21:41.981 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:21:42.143 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T05:21:42.160 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:21:42.210 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:21:42.596 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T05:21:42.613 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:21:42.688 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:21:43.066 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T05:21:43.083 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T05:21:43.124 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:21:43.385 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T05:21:43.402 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:21:43.443 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:21:43.493 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T05:21:43.510 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:21:43.560 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:21:43.754 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T05:21:43.770 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:21:43.812 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:21:43.988 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T05:21:44.005 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:44.055 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:44.115 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T05:21:44.132 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:21:44.173 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:21:44.441 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T05:21:44.459 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:44.500 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:21:44.535 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T05:21:44.552 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T05:21:44.593 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:21:44.861 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T05:21:44.877 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:44.927 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:21:45.072 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T05:21:45.093 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:21:45.122 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:21:45.230 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T05:21:45.247 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:21:45.296 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:45.467 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T05:21:45.484 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:45.525 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:45.825 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T05:21:45.842 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T05:21:45.883 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:21:45.886 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T05:21:45.903 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:45.944 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:21:46.221 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package modemmanager. 2024-09-17T05:21:46.222 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T05:21:46.238 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:21:46.239 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:46.272 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:21:46.279 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:21:46.542 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T05:21:46.558 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:21:46.600 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:46.961 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package network-manager. 2024-09-17T05:21:46.978 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:21:47.037 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:21:47.162 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T05:21:47.178 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T05:21:47.220 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:21:47.556 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package modemmanager. 2024-09-17T05:21:47.573 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:21:47.614 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:21:47.644 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package ppp. 2024-09-17T05:21:47.660 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T05:21:48.029 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:21:48.313 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager. 2024-09-17T05:21:48.330 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:21:48.374 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T05:21:48.379 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:21:48.389 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T05:21:48.432 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T05:21:48.710 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T05:21:48.725 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T05:21:48.777 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T05:21:49.187 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ppp. 2024-09-17T05:21:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T05:21:49.203 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T05:21:49.207 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T05:21:49.255 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T05:21:49.569 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:21:49.626 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T05:21:49.641 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:49.684 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:49.917 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T05:21:49.932 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T05:21:49.983 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T05:21:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T05:21:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T05:21:50.269 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T05:21:50.285 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T05:21:50.318 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T05:21:50.352 INFO:teuthology.orchestra.run.smithi062.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T05:21:50.381 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-common (238) ... 2024-09-17T05:21:50.614 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T05:21:50.630 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T05:21:50.672 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T05:21:50.840 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T05:21:50.855 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:50.901 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:50.993 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T05:21:51.009 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:51.043 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:51.581 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T05:21:51.598 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T05:21:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T05:21:51.722 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-common (238) ... 2024-09-17T05:21:52.205 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T05:21:52.222 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:21:52.266 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:52.576 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql. 2024-09-17T05:21:52.592 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T05:21:52.676 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql (14+238) ... 2024-09-17T05:21:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T05:21:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T05:21:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T05:21:53.180 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T05:21:53.196 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T05:21:53.238 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T05:21:53.391 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql. 2024-09-17T05:21:53.406 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T05:21:53.441 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql (14+238) ... 2024-09-17T05:21:53.566 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T05:21:53.582 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T05:21:53.624 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:21:53.710 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T05:21:53.726 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T05:21:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T05:21:53.927 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package tcl. 2024-09-17T05:21:53.944 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T05:21:53.990 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T05:21:54.012 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T05:21:54.028 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T05:21:54.070 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T05:21:54.296 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T05:21:54.314 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T05:21:54.346 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:21:54.414 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T05:21:54.431 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T05:21:54.473 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:21:54.692 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T05:21:54.710 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T05:21:54.725 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package tcl. 2024-09-17T05:21:54.742 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T05:21:54.750 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T05:21:54.780 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T05:21:55.145 INFO:teuthology.orchestra.run.smithi062.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T05:21:55.161 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T05:21:55.162 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T05:21:55.178 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T05:21:55.212 INFO:teuthology.orchestra.run.smithi062.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:21:55.219 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:21:55.498 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T05:21:55.591 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T05:21:55.607 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T05:21:55.657 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T05:21:55.701 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T05:21:55.827 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:21:55.978 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:56.027 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T05:21:56.044 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T05:21:56.093 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:21:56.104 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:21:56.230 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:21:56.357 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:56.397 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T05:21:56.582 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T05:21:56.709 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:21:56.835 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:56.994 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:21:57.120 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:21:57.246 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:21:57.313 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T05:21:57.771 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:21:58.167 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:21:58.167 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:21:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T05:21:58.326 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:21:58.712 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:21:58.713 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:21:58.949 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:59.049 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:21:59.192 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:59.318 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T05:21:59.437 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:21:59.444 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:21:59.538 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:21:59.695 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:21:59.698 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:21:59.821 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:21:59.823 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T05:21:59.947 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:21:59.950 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:22:00.076 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:22:00.202 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:22:00.586 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:22:00.711 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:22:00.712 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T05:22:00.807 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T05:22:00.872 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:22:00.998 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:22:01.107 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T05:22:01.208 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T05:22:01.339 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:22:01.375 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:22:01.432 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T05:22:01.492 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T05:22:01.502 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:22:01.588 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T05:22:01.628 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:22:01.652 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:22:01.779 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:22:01.805 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:22:01.930 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T05:22:01.930 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:22:02.055 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T05:22:02.065 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T05:22:02.181 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:22:02.209 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T05:22:02.271 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T05:22:02.309 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:22:02.350 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:22:02.451 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:22:02.668 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:22:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:22:02.920 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:which means they are currently configured by ifupdown: 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:- enp3s0f1 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T05:22:02.951 INFO:teuthology.orchestra.run.smithi062.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T05:22:02.952 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:03.064 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T05:22:03.092 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T05:22:03.181 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:22:03.525 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout:which means they are currently configured by ifupdown: 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout:- ens1f0 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T05:22:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:03.959 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T05:22:03.965 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T05:22:04.409 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T05:22:04.712 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:22:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T05:22:04.855 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:22:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T05:22:05.124 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-common (238) ... 2024-09-17T05:22:05.750 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:22:05.893 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:22:06.044 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T05:22:06.101 INFO:teuthology.orchestra.run.smithi062.stdout:Adding user postgres to group ssl-cert 2024-09-17T05:22:06.170 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-common (238) ... 2024-09-17T05:22:06.355 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:06.356 INFO:teuthology.orchestra.run.smithi062.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T05:22:06.603 INFO:teuthology.orchestra.run.smithi062.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T05:22:06.604 INFO:teuthology.orchestra.run.smithi062.stdout:Removing obsolete dictionary files: 2024-09-17T05:22:07.142 INFO:teuthology.orchestra.run.smithi043.stdout:Adding user postgres to group ssl-cert 2024-09-17T05:22:07.353 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T05:22:07.420 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:07.420 INFO:teuthology.orchestra.run.smithi043.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T05:22:07.686 INFO:teuthology.orchestra.run.smithi043.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T05:22:07.686 INFO:teuthology.orchestra.run.smithi043.stdout:Removing obsolete dictionary files: 2024-09-17T05:22:07.975 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:22:08.305 INFO:teuthology.orchestra.run.smithi062.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T05:22:08.325 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:This user must also own the server process. 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:The default text search configuration will be set to "english". 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:08.374 INFO:teuthology.orchestra.run.smithi062.stdout:Data page checksums are disabled. 2024-09-17T05:22:08.375 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:08.375 INFO:teuthology.orchestra.run.smithi062.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T05:22:08.375 INFO:teuthology.orchestra.run.smithi062.stdout:creating subdirectories ... ok 2024-09-17T05:22:08.375 INFO:teuthology.orchestra.run.smithi062.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T05:22:08.400 INFO:teuthology.orchestra.run.smithi062.stdout:selecting default max_connections ... 100 2024-09-17T05:22:08.434 INFO:teuthology.orchestra.run.smithi062.stdout:selecting default shared_buffers ... 128MB 2024-09-17T05:22:08.445 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T05:22:08.448 INFO:teuthology.orchestra.run.smithi062.stdout:selecting default time zone ... Etc/UTC 2024-09-17T05:22:08.449 INFO:teuthology.orchestra.run.smithi062.stdout:creating configuration files ... ok 2024-09-17T05:22:08.619 INFO:teuthology.orchestra.run.smithi062.stdout:running bootstrap script ... ok 2024-09-17T05:22:08.929 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:22:09.236 INFO:teuthology.orchestra.run.smithi062.stdout:performing post-bootstrap initialization ... ok 2024-09-17T05:22:09.273 INFO:teuthology.orchestra.run.smithi043.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T05:22:09.292 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T05:22:09.349 INFO:teuthology.orchestra.run.smithi043.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T05:22:09.349 INFO:teuthology.orchestra.run.smithi043.stdout:This user must also own the server process. 2024-09-17T05:22:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout:The default text search configuration will be set to "english". 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout:Data page checksums are disabled. 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:09.350 INFO:teuthology.orchestra.run.smithi043.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T05:22:09.351 INFO:teuthology.orchestra.run.smithi043.stdout:creating subdirectories ... ok 2024-09-17T05:22:09.351 INFO:teuthology.orchestra.run.smithi043.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T05:22:09.376 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default max_connections ... 100 2024-09-17T05:22:09.410 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default shared_buffers ... 128MB 2024-09-17T05:22:09.424 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default time zone ... Etc/UTC 2024-09-17T05:22:09.425 INFO:teuthology.orchestra.run.smithi043.stdout:creating configuration files ... ok 2024-09-17T05:22:09.595 INFO:teuthology.orchestra.run.smithi043.stdout:running bootstrap script ... ok 2024-09-17T05:22:09.742 INFO:teuthology.orchestra.run.smithi062.stdout:syncing data to disk ... ok 2024-09-17T05:22:10.252 INFO:teuthology.orchestra.run.smithi043.stdout:performing post-bootstrap initialization ... ok 2024-09-17T05:22:10.287 INFO:teuthology.orchestra.run.smithi062.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T05:22:10.814 INFO:teuthology.orchestra.run.smithi043.stdout:syncing data to disk ... ok 2024-09-17T05:22:11.321 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T05:22:13.383 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:22:13.483 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T05:22:13.609 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up postgresql (14+238) ... 2024-09-17T05:22:13.735 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:22:13.861 INFO:teuthology.orchestra.run.smithi062.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:22:14.130 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:22:14.130 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:22:14.412 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:22:14.562 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T05:22:14.688 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql (14+238) ... 2024-09-17T05:22:14.814 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:22:14.924 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:22:14.930 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:22:15.031 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T05:22:15.185 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:22:15.185 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:22:15.317 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:22:15.452 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:22:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:22:16.110 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T05:22:16.420 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:22:16.614 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:22:16.921 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T05:22:17.021 INFO:teuthology.orchestra.run.smithi062.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:22:17.576 INFO:teuthology.orchestra.run.smithi062.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:22:17.584 DEBUG:teuthology.parallel:result is None 2024-09-17T05:22:18.127 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T05:22:18.227 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:22:18.800 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:22:18.806 DEBUG:teuthology.parallel:result is None 2024-09-17T05:22:18.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T05:22:18.953 DEBUG:teuthology.orchestra.run.smithi043:> dpkg-query -W -f '${Version}' ceph 2024-09-17T05:22:18.976 INFO:teuthology.orchestra.run.smithi043.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:22:18.976 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:22:18.977 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T05:22:18.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T05:22:19.120 DEBUG:teuthology.orchestra.run.smithi062:> dpkg-query -W -f '${Version}' ceph 2024-09-17T05:22:19.145 INFO:teuthology.orchestra.run.smithi062.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:22:19.145 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T05:22:19.145 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T05:22:19.147 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T05:22:19.147 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:19.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T05:22:19.167 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:19.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T05:22:19.210 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T05:22:19.211 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:19.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T05:22:19.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T05:22:19.291 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:19.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T05:22:19.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T05:22:19.377 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T05:22:19.377 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:19.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T05:22:19.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T05:22:19.464 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:19.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T05:22:19.485 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T05:22:19.549 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T05:22:19.550 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:19.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T05:22:19.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T05:22:19.631 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:19.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T05:22:19.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T05:22:19.717 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T05:22:19.832 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T05:22:19.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-09-17T05:22:19.835 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-09-17T05:22:19.856 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T05:22:19.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T05:22:19.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T05:22:19.924 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T05:22:19.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T05:22:19.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T05:22:20.000 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T05:22:20.000 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T05:22:20.000 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T05:22:20.000 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:20.000 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T05:22:20.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T05:22:20.136 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:20.136 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T05:22:20.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T05:22:20.243 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T05:22:20.243 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T05:22:20.244 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908606', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVFijXA5oXIMS5FWWjKJ9ohT3FkjrQ9Vc8KgrQHB6dt15v45lXgasZEpSRcX4qRXbu9IBK2dNxx/qfXG9Eke58=', 'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM7aXWy+vrWX7qJlh0bvpeZLqz9g1SKXWHriE99HsvuoBb9GXfoViAHONezhGTA6L7u3H9/AAM81o1t6PIyfiw0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T05:22:20.244 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T05:22:20.249 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T05:22:20.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T05:22:20.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T05:22:20.320 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:20.320 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T05:22:20.369 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T05:22:20.369 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-17T05:22:20.420 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T05:22:20.420 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:20.421 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-09-17T05:22:20.421 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:20.421 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:15:35.522440770 +0000 2024-09-17T05:22:20.421 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:15:35.518440805 +0000 2024-09-17T05:22:20.421 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:15:35.518440805 +0000 2024-09-17T05:22:20.422 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:15:35.518440805 +0000 2024-09-17T05:22:20.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T05:22:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-17T05:22:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-17T05:22:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000250797 s, 2.0 MB/s 2024-09-17T05:22:20.488 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T05:22:20.543 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 915 Links: 1 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:15:36.358433302 +0000 2024-09-17T05:22:20.597 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:15:36.078435803 +0000 2024-09-17T05:22:20.598 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:15:36.078435803 +0000 2024-09-17T05:22:20.598 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:15:36.078435803 +0000 2024-09-17T05:22:20.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:22:20.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-17T05:22:20.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-17T05:22:20.659 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000253901 s, 2.0 MB/s 2024-09-17T05:22:20.661 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:22:20.719 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 932 Links: 1 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:15:36.966427871 +0000 2024-09-17T05:22:20.773 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:15:36.678430445 +0000 2024-09-17T05:22:20.774 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:15:36.678430445 +0000 2024-09-17T05:22:20.774 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:15:36.678430445 +0000 2024-09-17T05:22:20.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:22:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-17T05:22:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-17T05:22:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000269361 s, 1.9 MB/s 2024-09-17T05:22:20.838 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:22:20.891 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-17T05:22:20.941 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 948 Links: 1 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:15:37.538422763 +0000 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:15:37.270425157 +0000 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:15:37.270425157 +0000 2024-09-17T05:22:20.942 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:15:37.270425157 +0000 2024-09-17T05:22:20.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:22:21.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-17T05:22:21.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-17T05:22:21.004 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000264621 s, 1.9 MB/s 2024-09-17T05:22:21.006 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:22:21.059 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:21.059 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:21.059 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T05:22:21.068 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T05:22:21.069 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-09-17T05:22:21.121 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T05:22:21.121 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:21.121 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 902 Links: 1 2024-09-17T05:22:21.121 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:21.122 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-17 05:15:36.691097406 +0000 2024-09-17T05:22:21.122 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-17 05:15:36.431099730 +0000 2024-09-17T05:22:21.122 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-17 05:15:36.431099730 +0000 2024-09-17T05:22:21.122 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-17 05:15:36.431099730 +0000 2024-09-17T05:22:21.122 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T05:22:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-17T05:22:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-17T05:22:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00038 s, 1.3 MB/s 2024-09-17T05:22:21.186 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T05:22:21.239 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-09-17T05:22:21.289 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T05:22:21.289 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:21.289 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-09-17T05:22:21.289 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:21.289 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-17 05:15:37.331091691 +0000 2024-09-17T05:22:21.290 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-17 05:15:37.083093906 +0000 2024-09-17T05:22:21.290 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-17 05:15:37.083093906 +0000 2024-09-17T05:22:21.290 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-17 05:15:37.083093906 +0000 2024-09-17T05:22:21.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:22:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-17T05:22:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-17T05:22:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000371831 s, 1.4 MB/s 2024-09-17T05:22:21.354 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:22:21.411 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-17 05:15:37.891086688 +0000 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-17 05:15:37.615089153 +0000 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-17 05:15:37.615089153 +0000 2024-09-17T05:22:21.461 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-17 05:15:37.615089153 +0000 2024-09-17T05:22:21.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:22:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-17T05:22:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-17T05:22:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000371939 s, 1.4 MB/s 2024-09-17T05:22:21.530 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:22:21.587 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-09-17T05:22:21.637 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:22:21.637 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:22:21.637 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 954 Links: 1 2024-09-17T05:22:21.637 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:22:21.638 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-09-17 05:15:38.487081366 +0000 2024-09-17T05:22:21.638 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-09-17 05:15:38.191084009 +0000 2024-09-17T05:22:21.638 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-09-17 05:15:38.191084009 +0000 2024-09-17T05:22:21.638 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-09-17 05:15:38.191084009 +0000 2024-09-17T05:22:21.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:22:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-09-17T05:22:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-09-17T05:22:21.703 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000388717 s, 1.3 MB/s 2024-09-17T05:22:21.705 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:22:21.759 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:21.759 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:22:21.759 INFO:tasks.ceph:Generating config... 2024-09-17T05:22:21.761 INFO:tasks.ceph:[client] pid file = 2024-09-17T05:22:21.761 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T05:22:21.761 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T05:22:21.761 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T05:22:21.761 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T05:22:21.761 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T05:22:21.762 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T05:22:21.762 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T05:22:21.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T05:22:21.825 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T05:22:21.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T05:22:21.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:22:21.930 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.62'), ('mon.c', '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]')] 2024-09-17T05:22:21.930 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T05:22:21.931 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:21.931 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:22:21.984 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.62 --addv c '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:22.088 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2024-09-17T05:22:22.092022+0000 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-17T05:22:22.092022+0000 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 17 (quincy) 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon.c 2024-09-17T05:22:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T05:22:22.092 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:22:22.100 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1cac0027-7a1b-472a-9fc8-e0027668aa6d... 2024-09-17T05:22:22.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:22:22.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-09-17T05:22:22.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:22:22.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-09-17T05:22:22.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on skipped message = False 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.190 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-09-17T05:22:22.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-09-17T05:22:22.192 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: bdev async discard = True 2024-09-17T05:22:22.193 INFO:teuthology.orchestra.run.smithi043.stdout: bdev enable discard = True 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore allocator = bitmap 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 1/20 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 1/20 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 4/10 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:22:22.194 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-17T05:22:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:22:22.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: mon down mkfs grace = 300 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:22:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-09-17T05:22:22.198 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.199 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-09-17T05:22:22.200 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-09-17T05:22:22.201 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:22:22.202 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.203 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.43,172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790]" 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:22:22.204 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-09-17T05:22:22.205 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug trim objects = true 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.206 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-09-17T05:22:22.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2024-09-17T05:22:22.208 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:22:22.209 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.210 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:22:22.211 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: mon down mkfs grace = 300 2024-09-17T05:22:22.212 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-09-17T05:22:22.213 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-09-17T05:22:22.214 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-09-17T05:22:22.214 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-09-17T05:22:22.214 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-09-17T05:22:22.214 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-09-17T05:22:22.215 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T05:22:22.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T05:22:22.329 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T05:22:22.329 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:22.329 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T05:22:22.338 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:22.339 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T05:22:22.388 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:22:22.388 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:22.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:22:22.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:22:22.464 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:22.464 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:22.521 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:22:22.521 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:22.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:22:22.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:22:22.558 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:22.558 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:22.613 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T05:22:22.613 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T05:22:22.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:22:22.679 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:22:22.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:22:22.759 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:22:22.772 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T05:22:22.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:22:22.837 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:22:22.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T05:22:22.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:22:22.974 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:22:22.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T05:22:22.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:22:23.074 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:22:23.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T05:22:23.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:22:23.206 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:22:23.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T05:22:23.231 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T05:22:23.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T05:22:23.286 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T05:22:23.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T05:22:23.356 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T05:22:23.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T05:22:23.427 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T05:22:23.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T05:22:23.497 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T05:22:23.515 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T05:22:23.515 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:22:23.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T05:22:23.535 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:23.536 INFO:tasks.ceph:role: osd.0 2024-09-17T05:22:23.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:22:23.536 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:23.596 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:23.597 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:23.597 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:23.597 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:23.620 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-17T05:22:23.622 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-17T05:22:23.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T05:22:23.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T05:22:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:23.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:23.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T05:22:23.815 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:23.816 INFO:tasks.ceph:role: osd.1 2024-09-17T05:22:23.816 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:22:23.816 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:23.875 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:23.876 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:23.876 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:23.900 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-17T05:22:23.903 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-17T05:22:23.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T05:22:23.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T05:22:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:24.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:24.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T05:22:24.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:24.095 INFO:tasks.ceph:role: osd.2 2024-09-17T05:22:24.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:22:24.096 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:24.159 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:24.160 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:24.160 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:24.160 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:24.160 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:24.184 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-17T05:22:24.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-17T05:22:24.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T05:22:24.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T05:22:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:24.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:24.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T05:22:24.379 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:24.379 INFO:tasks.ceph:role: osd.3 2024-09-17T05:22:24.379 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:22:24.380 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:24.439 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:24.440 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:24.463 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-17T05:22:24.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-17T05:22:24.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T05:22:24.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T05:22:24.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:24.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:24.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:24.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:24.725+0000 7ff53fe127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T05:22:24.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:24.725+0000 7ff53fe127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T05:22:24.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:24.725+0000 7ff53fe127c0 -1 bdev(0x55ae36242a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:24.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:24.725+0000 7ff53fe127c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T05:22:27.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T05:22:27.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:28.109+0000 7f6ff6e957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T05:22:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:28.109+0000 7f6ff6e957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T05:22:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:28.109+0000 7f6ff6e957c0 -1 bdev(0x558b37ff2a00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:28.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:28.109+0000 7f6ff6e957c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T05:22:31.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T05:22:31.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:31.749+0000 7f641710b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T05:22:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:31.753+0000 7f641710b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T05:22:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:31.753+0000 7f641710b7c0 -1 bdev(0x55fe215d4a00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:31.753+0000 7f641710b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T05:22:35.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T05:22:35.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:35.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:35.397+0000 7fbf45a607c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T05:22:35.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:35.397+0000 7fbf45a607c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T05:22:35.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:35.397+0000 7fbf45a607c0 -1 bdev(0x55ca958f0a00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:35.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:35.397+0000 7fbf45a607c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T05:22:38.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T05:22:38.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T05:22:38.654 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:38.654 INFO:tasks.ceph:role: osd.4 2024-09-17T05:22:38.654 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:22:38.654 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:22:38.729 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:38.730 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:39.125 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-09-17T05:22:39.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-09-17T05:22:39.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T05:22:39.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T05:22:39.277 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:39.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:39.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T05:22:39.344 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:39.345 INFO:tasks.ceph:role: osd.5 2024-09-17T05:22:39.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:22:39.345 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:22:39.416 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:39.416 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:39.416 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:39.416 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:39.416 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:39.417 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:39.417 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:39.417 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:39.417 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:39.417 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:39.829 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-09-17T05:22:39.831 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-09-17T05:22:39.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T05:22:39.944 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T05:22:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:39.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:39.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T05:22:40.029 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:40.029 INFO:tasks.ceph:role: osd.6 2024-09-17T05:22:40.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:22:40.031 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:40.104 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:40.105 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:40.105 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:40.105 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:40.105 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:40.511 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-09-17T05:22:40.515 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-09-17T05:22:40.515 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T05:22:40.639 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T05:22:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:40.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:40.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T05:22:40.720 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:22:40.721 INFO:tasks.ceph:role: osd.7 2024-09-17T05:22:40.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:22:40.721 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:22:40.792 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:22:41.147 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-09-17T05:22:41.150 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-09-17T05:22:41.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T05:22:41.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T05:22:41.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:22:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:22:41.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:41.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:41.441+0000 7f3abe8c97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T05:22:41.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:41.445+0000 7f3abe8c97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T05:22:41.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:41.445+0000 7f3abe8c97c0 -1 bdev(0x557888b0ca00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:41.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:41.445+0000 7f3abe8c97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T05:22:44.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T05:22:44.654 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:44.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:44.761+0000 7f2ba750b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T05:22:44.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:44.765+0000 7f2ba750b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T05:22:44.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:44.765+0000 7f2ba750b7c0 -1 bdev(0x55a7dd60ea00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:44.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:44.765+0000 7f2ba750b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T05:22:47.981 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T05:22:48.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:48.108+0000 7f1e916ab7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T05:22:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:48.108+0000 7f1e916ab7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T05:22:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:48.108+0000 7f1e916ab7c0 -1 bdev(0x5615e5216a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:48.108+0000 7f1e916ab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T05:22:50.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:22:50.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:22:51.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T05:22:51.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:51.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:51.700+0000 7f47e997c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T05:22:51.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:51.700+0000 7f47e997c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T05:22:51.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:51.700+0000 7f47e997c7c0 -1 bdev(0x563d8ea4ca00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T05:22:51.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:22:51.700+0000 7f47e997c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T05:22:55.014 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T05:22:55.036 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T05:22:55.036 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T05:22:55.059 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T05:22:55.122 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T05:22:55.187 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T05:22:55.249 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T05:22:55.317 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T05:22:55.381 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T05:22:55.445 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T05:22:55.468 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T05:22:55.537 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T05:22:55.605 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T05:22:55.672 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T05:22:55.737 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T05:22:55.801 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T05:22:55.865 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.865 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T05:22:55.875 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:22:55.875 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T05:22:55.929 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.929 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T05:22:55.938 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:22:55.938 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T05:22:55.988 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T05:22:55.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:22:55.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:22:56.039 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-09-17T05:22:56.039 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg== 2024-09-17T05:22:56.039 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg== 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-09-17T05:22:56.040 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA== 2024-09-17T05:22:56.041 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ== 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg== 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2024-09-17T05:22:56.042 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw== 2024-09-17T05:22:56.054 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-09-17T05:22:56.054 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg== 2024-09-17T05:22:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg== 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-09-17T05:22:56.056 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA== 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ== 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg== 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2024-09-17T05:22:56.057 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw== 2024-09-17T05:22:56.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:22:56.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:22:56.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:22:56.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:22:56.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:22:56.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:56.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:57.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:57.026 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:22:57.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:22:57.396 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T05:22:57.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T05:22:57.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:22:57.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T05:22:57.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T05:22:57.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:22:57.722 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T05:22:57.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T05:22:57.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:22:57.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T05:22:57.946 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:57.950 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:22:57.995 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T05:22:57.995 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T05:22:57.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T05:22:57.998 INFO:tasks.ceph.mon.a:Started 2024-09-17T05:22:57.998 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T05:22:57.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T05:22:58.038 INFO:tasks.ceph.mon.b:Started 2024-09-17T05:22:58.038 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T05:22:58.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T05:22:58.042 INFO:tasks.ceph.mon.c:Started 2024-09-17T05:22:58.042 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T05:22:58.042 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T05:22:58.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T05:22:58.046 INFO:tasks.ceph.mgr.y:Started 2024-09-17T05:22:58.046 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T05:22:58.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T05:22:58.049 INFO:tasks.ceph.mgr.x:Started 2024-09-17T05:22:58.049 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T05:22:58.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T05:22:58.053 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T05:22:58.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T05:22:58.102 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T05:22:58.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T05:22:58.147 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T05:22:58.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T05:22:58.196 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T05:22:58.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T05:22:58.244 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T05:22:58.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T05:22:58.296 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T05:22:58.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T05:22:58.343 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T05:22:58.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T05:22:58.388 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T05:22:58.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T05:22:58.435 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T05:22:58.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T05:22:58.487 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T05:22:58.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T05:22:58.543 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T05:22:58.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T05:22:58.608 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T05:22:58.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T05:22:58.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.648+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f879c005320 con 0x7f87b0004810 2024-09-17T05:22:58.654 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T05:22:58.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T05:22:58.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.648+0000 7f296a000640 1 -- 172.21.15.43:0/545937758 <== mon.2 v2:172.21.15.62:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7f2964010720 con 0x7f295800ddd0 2024-09-17T05:22:58.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.656+0000 7f6033a00640 1 -- 172.21.15.43:0/3846635310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 msgr2=0x7f602c0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.656+0000 7f6033a00640 1 --2- 172.21.15.43:0/3846635310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c0b3a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6024009bc0 tx=0x7f602401cba0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.661 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T05:22:58.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T05:22:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.660+0000 7f5f77400640 1 -- 172.21.15.43:0/92107355 <== mon.1 v2:172.21.15.62:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f5f700d0020 con 0x7f5f64003b90 2024-09-17T05:22:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.660+0000 7f5f77400640 1 -- 172.21.15.43:0/92107355 <== mon.1 v2:172.21.15.62:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7f5f70003c20 con 0x7f5f64003b90 2024-09-17T05:22:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 -- 172.21.15.43:0/92107355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f64003b90 msgr2=0x7f5f801e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 --2- 172.21.15.43:0/92107355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f64003b90 0x7f5f801e1d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f7001dd10 tx=0x7f5f64005840 comp rx=0 tx=0).stop 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 -- 172.21.15.43:0/92107355 shutdown_connections 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 --2- 172.21.15.43:0/92107355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f801f0a50 0x7f5f8009ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 --2- 172.21.15.43:0/92107355 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f8015af80 0x7f5f8014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 --2- 172.21.15.43:0/92107355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f64003b90 0x7f5f801e1d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 -- 172.21.15.43:0/92107355 >> 172.21.15.43:0/92107355 conn(0x7f5f800b2f80 msgr2=0x7f5f800cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 -- 172.21.15.43:0/92107355 shutdown_connections 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.668+0000 7f5f86800640 1 -- 172.21.15.43:0/92107355 wait complete. 2024-09-17T05:22:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.660+0000 7f2977200640 1 -- 172.21.15.43:0/545937758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f295800ddd0 msgr2=0x7f297015ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.660+0000 7f2977200640 1 --2- 172.21.15.43:0/545937758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f295800ddd0 0x7f297015ef80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2964003d90 tx=0x7f2964001830 comp rx=0 tx=0).stop 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 -- 172.21.15.43:0/545937758 shutdown_connections 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 --2- 172.21.15.43:0/545937758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f297014f290 0x7f29701520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 --2- 172.21.15.43:0/545937758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f297014f6a0 0x7f2970152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 --2- 172.21.15.43:0/545937758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f295800ddd0 0x7f297015ef80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 -- 172.21.15.43:0/545937758 >> 172.21.15.43:0/545937758 conn(0x7f29700b6590 msgr2=0x7f2970159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 -- 172.21.15.43:0/545937758 shutdown_connections 2024-09-17T05:22:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f2977200640 1 -- 172.21.15.43:0/545937758 wait complete. 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.660+0000 7f87c3e00640 1 -- 172.21.15.43:0/2246239553 <== mon.2 v2:172.21.15.62:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f87bc0ca020 con 0x7f87b0004810 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f7f69600640 1 -- 172.21.15.43:0/2753427844 <== mon.1 v2:172.21.15.62:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f7f60005020 con 0x7f7f6c0aead0 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 -- 172.21.15.43:0/3846635310 shutdown_connections 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 --2- 172.21.15.43:0/3846635310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c0b3a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 --2- 172.21.15.43:0/3846635310 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 -- 172.21.15.43:0/3846635310 >> 172.21.15.43:0/3846635310 conn(0x7f602c0b1dd0 msgr2=0x7f602c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 -- 172.21.15.43:0/3846635310 shutdown_connections 2024-09-17T05:22:58.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.664+0000 7f6033a00640 1 -- 172.21.15.43:0/3846635310 wait complete. 2024-09-17T05:22:58.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.672+0000 7f87c3e00640 1 -- 172.21.15.43:0/2246239553 <== mon.2 v2:172.21.15.62:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87bc0bcb30 con 0x7f87b0004810 2024-09-17T05:22:58.681 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T05:22:58.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T05:22:58.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.672+0000 7f6033a00640 1 Processor -- start 2024-09-17T05:22:58.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.672+0000 7f6033a00640 1 -- start start 2024-09-17T05:22:58.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f602c0b3610 0x7f602c0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f602c166f60 con 0x7f602c0b3610 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f602c166de0 con 0x7f602c0b52a0 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6033a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f602c1670e0 con 0x7f602c0cf390 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f602be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38208/0 (socket says 172.21.15.43:38208) 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 -- 172.21.15.43:0/3724808127 learned_addr learned my addr 172.21.15.43:0/3724808127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 -- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 msgr2=0x7f602c15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 -- 172.21.15.43:0/3724808127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f602c0b3610 msgr2=0x7f602c0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f602c0b3610 0x7f602c0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 -- 172.21.15.43:0/3724808127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c0c8c40 con 0x7f602c0cf390 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.676+0000 7f6031a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f602401b250 tx=0x7f6024003a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.680+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240bc070 con 0x7f602c0cf390 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.680+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60240ad730 con 0x7f602c0cf390 2024-09-17T05:22:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.680+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240b2040 con 0x7f602c0cf390 2024-09-17T05:22:58.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.680+0000 7f87c3e00640 1 -- 172.21.15.43:0/2246239553 <== mon.2 v2:172.21.15.62:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f87bc01acf0 con 0x7f87b0004810 2024-09-17T05:22:58.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.680+0000 7f7f69600640 1 -- 172.21.15.43:0/2753427844 <== mon.1 v2:172.21.15.62:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f600052c0 con 0x7f7f6c0aead0 2024-09-17T05:22:58.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.684+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c0c8f00 con 0x7f602c0cf390 2024-09-17T05:22:58.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.684+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c0c94c0 con 0x7f602c0cf390 2024-09-17T05:22:58.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.684+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c0b3a20 con 0x7f602c0cf390 2024-09-17T05:22:58.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.684+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60240adb40 con 0x7f602c0cf390 2024-09-17T05:22:58.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.684+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f60240c0a80 con 0x7f602c0cf390 2024-09-17T05:22:58.694 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T05:22:58.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T05:22:58.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.692+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87b0004810 msgr2=0x7f87cc1f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.692+0000 7f87d2000640 1 --2- 172.21.15.43:0/2246239553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87b0004810 0x7f87cc1f1a10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c2c0 tx=0x7f87b0005840 comp rx=0 tx=0).stop 2024-09-17T05:22:58.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.688+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f60240c6020 con 0x7f602c0cf390 2024-09-17T05:22:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 shutdown_connections 2024-09-17T05:22:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f87d2000640 1 --2- 172.21.15.43:0/2246239553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc15b0e0 0x7f87cc13e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f87d2000640 1 --2- 172.21.15.43:0/2246239553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87cc13ef10 0x7f87cc13f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f87d2000640 1 --2- 172.21.15.43:0/2246239553 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87b0004810 0x7f87cc1f1a10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 >> 172.21.15.43:0/2246239553 conn(0x7f87cc06d030 msgr2=0x7f87cc15e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 -- 172.21.15.43:0/2384076224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc15f910 msgr2=0x7f77bc15fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2384076224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc15f910 0x7f77bc15fd20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77bc071a50 tx=0x7f77a401c900 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 -- 172.21.15.43:0/2384076224 shutdown_connections 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2384076224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc15f910 0x7f77bc15fd20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2384076224 >> v2:172.21.15.62:3300/0 conn(0x7f77bc0b3610 0x7f77bc0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2384076224 >> v2:172.21.15.43:3300/0 conn(0x7f77bc0cf390 0x7f77bc0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.696+0000 7f77c2e00640 1 -- 172.21.15.43:0/2384076224 >> 172.21.15.43:0/2384076224 conn(0x7f77bc0b1dd0 msgr2=0x7f77bc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.706 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T05:22:58.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T05:22:58.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.704+0000 7f77c2e00640 1 -- 172.21.15.43:0/2384076224 shutdown_connections 2024-09-17T05:22:58.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.704+0000 7f77c2e00640 1 -- 172.21.15.43:0/2384076224 wait complete. 2024-09-17T05:22:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.708+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 shutdown_connections 2024-09-17T05:22:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.708+0000 7f87d2000640 1 -- 172.21.15.43:0/2246239553 wait complete. 2024-09-17T05:22:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.716+0000 7f77c2e00640 1 Processor -- start 2024-09-17T05:22:58.720 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T05:22:58.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77c2e00640 1 -- start start 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77c2e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77c2e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77bc0cf390 0x7f77bc1fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77bbe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77bbe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38212/0 (socket says 172.21.15.43:38212) 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77c2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc15f910 0x7f77bc0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.720+0000 7f77c2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc0b4c20 con 0x7f77bc15f910 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77c2e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc0b4aa0 con 0x7f77bc0cf390 2024-09-17T05:22:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77c2e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc0b4da0 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 -- 172.21.15.43:0/2279295482 learned_addr learned my addr 172.21.15.43:0/2279295482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bb400640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77bc0cf390 0x7f77bc1fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 -- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77bc0cf390 msgr2=0x7f77bc1fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77bc0cf390 0x7f77bc1fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 -- 172.21.15.43:0/2279295482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc15f910 msgr2=0x7f77bc0ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc15f910 0x7f77bc0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 -- 172.21.15.43:0/2279295482 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc201cb0 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77bbe00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b450 tx=0x7f77ac00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac015020 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ac011940 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac011d90 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc2019d0 con 0x7f77bc0b3610 2024-09-17T05:22:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc1fdf10 con 0x7f77bc0b3610 2024-09-17T05:22:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77ac018020 con 0x7f77bc0b3610 2024-09-17T05:22:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.724+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77ac01a5f0 con 0x7f77bc0b3610 2024-09-17T05:22:58.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.728+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc1fdd30 con 0x7f77bc0b3610 2024-09-17T05:22:58.732 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T05:22:58.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T05:22:58.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.728+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f77bc1fdd30 con 0x7f77bc0b3610 2024-09-17T05:22:58.765 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T05:22:58.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T05:22:58.834 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T05:22:58.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T05:22:58.881 DEBUG:tasks.ceph:set 22 configs 2024-09-17T05:22:58.881 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.881 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.881 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.881 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.882 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.882 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi043') 2024-09-17T05:22:58.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 -- 172.21.15.43:0/2948462102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 msgr2=0x7f821c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 --2- 172.21.15.43:0/2948462102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c073420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f821c09e2e0 tx=0x7f821401cae0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 -- 172.21.15.43:0/2948462102 shutdown_connections 2024-09-17T05:22:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 --2- 172.21.15.43:0/2948462102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c073420 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 --2- 172.21.15.43:0/2948462102 >> v2:172.21.15.43:3300/0 conn(0x7f821c0cd870 0x7f821c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.916+0000 7f8220e00640 1 -- 172.21.15.43:0/2948462102 >> 172.21.15.43:0/2948462102 conn(0x7f821c0b1cd0 msgr2=0x7f821c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.932+0000 7f8220e00640 1 -- 172.21.15.43:0/2948462102 shutdown_connections 2024-09-17T05:22:58.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.936+0000 7f8220e00640 1 -- 172.21.15.43:0/2948462102 wait complete. 2024-09-17T05:22:58.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.936+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f7f48001700 con 0x7f7f6c0aead0 2024-09-17T05:22:58.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.940+0000 7f8220e00640 1 Processor -- start 2024-09-17T05:22:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.944+0000 7f8220e00640 1 -- start start 2024-09-17T05:22:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.944+0000 7f8220e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 0x7f821c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.948+0000 7f7f69600640 1 -- 172.21.15.43:0/2753427844 <== mon.1 v2:172.21.15.62:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f7f60004020 con 0x7f7f6c0aead0 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8220e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8220e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f821c0cd870 0x7f821c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8220e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f821c06fad0 con 0x7f821c0cd870 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8220e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f821c06f950 con 0x7f821c073010 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8220e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f821c06fc50 con 0x7f821c070040 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 0x7f821c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 0x7f821c1533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38242/0 (socket says 172.21.15.43:38242) 2024-09-17T05:22:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219e00640 1 -- 172.21.15.43:0/1215365267 learned_addr learned my addr 172.21.15.43:0/1215365267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 -- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 msgr2=0x7f821c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 0x7f821c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 -- 172.21.15.43:0/1215365267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f821c0cd870 msgr2=0x7f821c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f821a800640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f821c0cd870 0x7f821c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f821c0cd870 0x7f821c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 -- 172.21.15.43:0/1215365267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c158020 con 0x7f821c073010 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.952+0000 7f8219400640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c142b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f821c070f60 tx=0x7f820c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c015020 con 0x7f821c073010 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f820c011920 con 0x7f821c073010 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c011d70 con 0x7f821c073010 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c157260 con 0x7f821c073010 2024-09-17T05:22:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c157820 con 0x7f821c073010 2024-09-17T05:22:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f6c0aead0 msgr2=0x7f7f6c160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f820c018020 con 0x7f821c073010 2024-09-17T05:22:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f820c0197f0 con 0x7f821c073010 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 --2- 172.21.15.43:0/2753427844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f6c0aead0 0x7f7f6c160280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e7d0 tx=0x7f7f6000ec90 comp rx=0 tx=0).stop 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 shutdown_connections 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 --2- 172.21.15.43:0/2753427844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c0cfc00 0x7f7f6c0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 --2- 172.21.15.43:0/2753427844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c0aeee0 0x7f7f6c0c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 --2- 172.21.15.43:0/2753427844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f6c0aead0 0x7f7f6c160280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.956+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 >> 172.21.15.43:0/2753427844 conn(0x7f7f6c0b1dd0 msgr2=0x7f7f6c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 shutdown_connections 2024-09-17T05:22:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c069c20 con 0x7f821c073010 2024-09-17T05:22:58.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f7f5ea00640 1 -- 172.21.15.43:0/2753427844 wait complete. 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 -- 172.21.15.43:0/764992443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 msgr2=0x7f6fdc0b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 --2- 172.21.15.43:0/764992443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc0b47d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00bbe0 tx=0x7f6fcc01c7a0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 -- 172.21.15.43:0/764992443 shutdown_connections 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 --2- 172.21.15.43:0/764992443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc0b47d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 --2- 172.21.15.43:0/764992443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc0b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f6fe3800640 1 -- 172.21.15.43:0/764992443 >> 172.21.15.43:0/764992443 conn(0x7f6fdc0b1dd0 msgr2=0x7f6fdc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:58.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.960+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f820c01f020 con 0x7f821c073010 2024-09-17T05:22:58.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.972+0000 7f6fe3800640 1 -- 172.21.15.43:0/764992443 shutdown_connections 2024-09-17T05:22:58.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.972+0000 7f6fe3800640 1 -- 172.21.15.43:0/764992443 wait complete. 2024-09-17T05:22:58.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.984+0000 7f6fe3800640 1 Processor -- start 2024-09-17T05:22:58.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.984+0000 7f6fe3800640 1 -- start start 2024-09-17T05:22:58.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.984+0000 7f6fe3800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc1fb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 0x7f6fdc0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:58.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b0210 con 0x7f6fdc0b3bf0 2024-09-17T05:22:58.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b0090 con 0x7f6fdc0cf390 2024-09-17T05:22:58.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b0390 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 0x7f6fdc0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 0x7f6fdc0aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47850/0 (socket says 172.21.15.43:47850) 2024-09-17T05:22:58.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38258/0 (socket says 172.21.15.43:38258) 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe1800640 1 -- 172.21.15.43:0/3418265645 learned_addr learned my addr 172.21.15.43:0/3418265645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fdbe00640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc1fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 -- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 msgr2=0x7f6fdc0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 0x7f6fdc0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 -- 172.21.15.43:0/3418265645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 msgr2=0x7f6fdc1fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc1fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 -- 172.21.15.43:0/3418265645 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc09d320 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe0e00640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400b410 tx=0x7f6fd400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4015020 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd4011940 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc09d600 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4011d90 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc09dbc0 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fd4018020 con 0x7f6fdc0b37e0 2024-09-17T05:22:58.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:58.992+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fd40043b0 con 0x7f6fdc0b37e0 2024-09-17T05:22:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.000+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc09dfa0 con 0x7f6fdc0b37e0 2024-09-17T05:22:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.020+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6fdc09dfa0 con 0x7f6fdc0b37e0 2024-09-17T05:22:59.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.028+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f602c06ec90 con 0x7f602c0cf390 2024-09-17T05:22:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.032+0000 7f9c7c400640 1 -- 172.21.15.43:0/2416837031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b5b20 msgr2=0x7f9c740ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.032+0000 7f9c7c400640 1 --2- 172.21.15.43:0/2416837031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b5b20 0x7f9c740ce3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009900 tx=0x7f9c6801c850 comp rx=0 tx=0).stop 2024-09-17T05:22:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.032+0000 7f9c7c400640 1 -- 172.21.15.43:0/2416837031 shutdown_connections 2024-09-17T05:22:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.032+0000 7f9c7c400640 1 --2- 172.21.15.43:0/2416837031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b5b20 0x7f9c740ce3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.032+0000 7f9c7c400640 1 -- 172.21.15.43:0/2416837031 >> 172.21.15.43:0/2416837031 conn(0x7f9c740b1dd0 msgr2=0x7f9c740b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.036+0000 7f9c7c400640 1 -- 172.21.15.43:0/2416837031 shutdown_connections 2024-09-17T05:22:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.036+0000 7f9c7c400640 1 -- 172.21.15.43:0/2416837031 wait complete. 2024-09-17T05:22:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c7c400640 1 Processor -- start 2024-09-17T05:22:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c7c400640 1 -- start start 2024-09-17T05:22:59.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c7c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c7c400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 0x7f9c740c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c7c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7415e800 0x7f9c740c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.040+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47866/0 (socket says 172.21.15.43:47866) 2024-09-17T05:22:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 -- 172.21.15.43:0/456136808 learned_addr learned my addr 172.21.15.43:0/456136808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74171c20 con 0x7f9c7415e800 2024-09-17T05:22:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741631c0 con 0x7f9c740b30d0 2024-09-17T05:22:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9c74171da0 con 0x7f9c740b5b20 2024-09-17T05:22:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79000640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 0x7f9c740c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 -- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 msgr2=0x7f9c740c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 0x7f9c740c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 -- 172.21.15.43:0/456136808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7415e800 msgr2=0x7f9c740c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7415e800 0x7f9c740c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c740c5de0 con 0x7f9c740b30d0 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79000640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 0x7f9c740c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c79a00640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c6801a480 tx=0x7f9c680a5680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680bb070 con 0x7f9c740b30d0 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c68004470 con 0x7f9c740b30d0 2024-09-17T05:22:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c740c6080 con 0x7f9c740b30d0 2024-09-17T05:22:59.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680b2040 con 0x7f9c740b30d0 2024-09-17T05:22:59.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c74070c80 con 0x7f9c740b30d0 2024-09-17T05:22:59.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c68003820 con 0x7f9c740b30d0 2024-09-17T05:22:59.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.048+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c680ad8d0 con 0x7f9c740b30d0 2024-09-17T05:22:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.056+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c740ce3e0 con 0x7f9c740b30d0 2024-09-17T05:22:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.060+0000 7f6028c00640 1 -- 172.21.15.43:0/3724808127 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f6024016790 con 0x7f602c0cf390 2024-09-17T05:22:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.060+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9c680c4020 con 0x7f9c740b30d0 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 msgr2=0x7f602c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f602401b250 tx=0x7f6024003a70 comp rx=0 tx=0).stop 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 shutdown_connections 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602c0b52a0 0x7f602c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f602c0cf390 0x7f602c0c4830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 --2- 172.21.15.43:0/3724808127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f602c0b3610 0x7f602c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.072+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 >> 172.21.15.43:0/3724808127 conn(0x7f602c0b1dd0 msgr2=0x7f602c15d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.076+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 shutdown_connections 2024-09-17T05:22:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.076+0000 7f6033a00640 1 -- 172.21.15.43:0/3724808127 wait complete. 2024-09-17T05:22:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.080+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f77bc1fe170 con 0x7f77bc0b3610 2024-09-17T05:22:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.092+0000 7f2b08200640 1 -- 172.21.15.43:0/2154177474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ae8004c90 msgr2=0x7f2ae8005120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.092+0000 7f2b08200640 1 --2- 172.21.15.43:0/2154177474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ae8004c90 0x7f2ae8005120 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2af400c6c0 tx=0x7f2af401d2c0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 -- 172.21.15.43:0/854046911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40d0590 msgr2=0x7feae40b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 --2- 172.21.15.43:0/854046911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40d0590 0x7feae40b4a80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feae40cfd40 tx=0x7feadc01c850 comp rx=0 tx=0).stop 2024-09-17T05:22:59.106 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 -- 172.21.15.43:0/854046911 shutdown_connections 2024-09-17T05:22:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 --2- 172.21.15.43:0/854046911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40d0590 0x7feae40b4a80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 --2- 172.21.15.43:0/854046911 >> v2:172.21.15.43:3300/0 conn(0x7feae40ce050 0x7feae40d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.104+0000 7feaeae00640 1 -- 172.21.15.43:0/854046911 >> 172.21.15.43:0/854046911 conn(0x7feae4074480 msgr2=0x7feae4074890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.108+0000 7f77b3e00640 1 -- 172.21.15.43:0/2279295482 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f77ac01f070 con 0x7f77bc0b3610 2024-09-17T05:22:59.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.108+0000 7feaeae00640 1 -- 172.21.15.43:0/854046911 shutdown_connections 2024-09-17T05:22:59.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.108+0000 7feaeae00640 1 -- 172.21.15.43:0/854046911 wait complete. 2024-09-17T05:22:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.112+0000 7feaeae00640 1 Processor -- start 2024-09-17T05:22:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.112+0000 7feaeae00640 1 -- start start 2024-09-17T05:22:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7f2b08200640 1 -- 172.21.15.43:0/2154177474 shutdown_connections 2024-09-17T05:22:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7f2b08200640 1 --2- 172.21.15.43:0/2154177474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ae8004c90 0x7f2ae8005120 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7f2b08200640 1 -- 172.21.15.43:0/2154177474 >> 172.21.15.43:0/2154177474 conn(0x7f2b00074480 msgr2=0x7f2b00074890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae40b4fc0 0x7feae414d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7f2b08200640 1 -- 172.21.15.43:0/2154177474 shutdown_connections 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feae40d0590 0x7feae41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feae415fd40 con 0x7feae40b4fc0 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feae415fbc0 con 0x7feae40ce050 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feaeae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7feae415fec0 con 0x7feae40d0590 2024-09-17T05:22:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47892/0 (socket says 172.21.15.43:47892) 2024-09-17T05:22:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 -- 172.21.15.43:0/46703561 learned_addr learned my addr 172.21.15.43:0/46703561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae3e00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feae40d0590 0x7feae41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 -- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feae40d0590 msgr2=0x7feae41524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feae40d0590 0x7feae41524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 -- 172.21.15.43:0/46703561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae40b4fc0 msgr2=0x7feae414d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae40b4fc0 0x7feae414d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 -- 172.21.15.43:0/46703561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae4158e40 con 0x7feae40ce050 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7feae2a00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c40 tx=0x7feadc005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 msgr2=0x7f77bc09db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b450 tx=0x7f77ac00b910 comp rx=0 tx=0).stop 2024-09-17T05:22:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 shutdown_connections 2024-09-17T05:22:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77bc0cf390 0x7f77bc1fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc15f910 0x7f77bc0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 --2- 172.21.15.43:0/2279295482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc0b3610 0x7f77bc09db60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.124+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 >> 172.21.15.43:0/2279295482 conn(0x7f77bc0b1dd0 msgr2=0x7f77bc1599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.120+0000 7f2b08200640 1 -- 172.21.15.43:0/2154177474 wait complete. 2024-09-17T05:22:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0b2040 con 0x7feae40ce050 2024-09-17T05:22:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feadc0055e0 con 0x7feae40ce050 2024-09-17T05:22:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7f2b08200640 1 Processor -- start 2024-09-17T05:22:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004250 con 0x7feae40ce050 2024-09-17T05:22:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7f2b08200640 1 -- start start 2024-09-17T05:22:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41fca50 con 0x7feae40ce050 2024-09-17T05:22:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41fd010 con 0x7feae40ce050 2024-09-17T05:22:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feadc004a90 con 0x7feae40ce050 2024-09-17T05:22:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.128+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feadc003970 con 0x7feae40ce050 2024-09-17T05:22:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.132+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 shutdown_connections 2024-09-17T05:22:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.132+0000 7f77c2e00640 1 -- 172.21.15.43:0/2279295482 wait complete. 2024-09-17T05:22:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.136+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae40d0050 con 0x7feae40ce050 2024-09-17T05:22:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.136+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7feadc0164d0 con 0x7feae40ce050 2024-09-17T05:22:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b001fe670 0x7f2b00200fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000ae040 0x7f2b000ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000b04d0 con 0x7f2b000ae040 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000b0350 con 0x7f2b001fe670 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b08200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000b0650 con 0x7f2ae8004c90 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38292/0 (socket says 172.21.15.43:38292) 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 -- 172.21.15.43:0/4285076573 learned_addr learned my addr 172.21.15.43:0/4285076573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 -- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b001fe670 msgr2=0x7f2b00200fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:22:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b001fe670 0x7f2b00200fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 -- 172.21.15.43:0/4285076573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000ae040 msgr2=0x7f2b000ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000ae040 0x7f2b000ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 -- 172.21.15.43:0/4285076573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000ae990 con 0x7f2ae8004c90 2024-09-17T05:22:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.140+0000 7f2b04e00640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2af401cd30 tx=0x7f2af4006490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.144+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4001820 con 0x7f2ae8004c90 2024-09-17T05:22:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.144+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af4003980 con 0x7f2ae8004c90 2024-09-17T05:22:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.144+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4003dd0 con 0x7f2ae8004c90 2024-09-17T05:22:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.148+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b00157a00 con 0x7f2ae8004c90 2024-09-17T05:22:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.148+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001fdb90 con 0x7f2ae8004c90 2024-09-17T05:22:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.152+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2af40b7070 con 0x7f2ae8004c90 2024-09-17T05:22:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.152+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2af40c2730 con 0x7f2ae8004c90 2024-09-17T05:22:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.152+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b001fd9c0 con 0x7f2ae8004c90 2024-09-17T05:22:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.152+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2af40c7020 con 0x7f2ae8004c90 2024-09-17T05:22:59.192 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.237 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f821c153c80 con 0x7f821c073010 2024-09-17T05:22:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 -- 172.21.15.43:0/1526866497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195804efc0 msgr2=0x7f1958052540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 --2- 172.21.15.43:0/1526866497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195804efc0 0x7f1958052540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1960066410 tx=0x7f19600989f0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 -- 172.21.15.43:0/1526866497 shutdown_connections 2024-09-17T05:22:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 --2- 172.21.15.43:0/1526866497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958039230 0x7f19580eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 --2- 172.21.15.43:0/1526866497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195804efc0 0x7f1958052540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 -- 172.21.15.43:0/1526866497 >> 172.21.15.43:0/1526866497 conn(0x7f1958054970 msgr2=0x7f1958054d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 -- 172.21.15.43:0/1526866497 shutdown_connections 2024-09-17T05:22:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.244+0000 7f1965000640 1 -- 172.21.15.43:0/1526866497 wait complete. 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 Processor -- start 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 -- start start 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958039230 0x7f19580da630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958052a80 0x7f1958181690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1958053400 con 0x7f1958052a80 2024-09-17T05:22:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1958053280 con 0x7f1958039230 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f1965000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f195801c790 con 0x7f195804efc0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f195ea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f195ea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38308/0 (socket says 172.21.15.43:38308) 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.256+0000 7f195ea00640 1 -- 172.21.15.43:0/3514271361 learned_addr learned my addr 172.21.15.43:0/3514271361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 -- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958039230 msgr2=0x7f19580da630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958039230 0x7f19580da630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 -- 172.21.15.43:0/3514271361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958052a80 msgr2=0x7f1958181690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958052a80 0x7f1958181690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 -- 172.21.15.43:0/3514271361 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958182550 con 0x7f195804efc0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f195ea00640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f194c00c730 tx=0x7f194c00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c016020 con 0x7f195804efc0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f194c00a8d0 con 0x7f195804efc0 2024-09-17T05:22:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c00ad20 con 0x7f195804efc0 2024-09-17T05:22:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1958184f20 con 0x7f195804efc0 2024-09-17T05:22:59.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.264+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19580d9b50 con 0x7f195804efc0 2024-09-17T05:22:59.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.260+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f194c019020 con 0x7f195804efc0 2024-09-17T05:22:59.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.264+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f194c01b840 con 0x7f195804efc0 2024-09-17T05:22:59.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.268+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958052540 con 0x7f195804efc0 2024-09-17T05:22:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.272+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f194c020020 con 0x7f195804efc0 2024-09-17T05:22:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.276+0000 7f8212000640 1 -- 172.21.15.43:0/1215365267 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f820c019a40 con 0x7f821c073010 2024-09-17T05:22:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.284+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 msgr2=0x7f821c142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.284+0000 7f8220e00640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c142b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f821c070f60 tx=0x7f820c00b870 comp rx=0 tx=0).stop 2024-09-17T05:22:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 shutdown_connections 2024-09-17T05:22:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f821c073010 0x7f821c142b90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f821c0cd870 0x7f821c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 --2- 172.21.15.43:0/1215365267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f821c070040 0x7f821c1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 >> 172.21.15.43:0/1215365267 conn(0x7f821c0b1cd0 msgr2=0x7f821c158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 -- 172.21.15.43:0/909307359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e409c3c0 msgr2=0x7f06e40b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 --2- 172.21.15.43:0/909307359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e409c3c0 0x7f06e40b5ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009bc0 tx=0x7f06d401cb90 comp rx=0 tx=0).stop 2024-09-17T05:22:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 -- 172.21.15.43:0/909307359 shutdown_connections 2024-09-17T05:22:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 --2- 172.21.15.43:0/909307359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e409c3c0 0x7f06e40b5ee0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 --2- 172.21.15.43:0/909307359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e406fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 -- 172.21.15.43:0/909307359 >> 172.21.15.43:0/909307359 conn(0x7f06e40691b0 msgr2=0x7f06e40695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 shutdown_connections 2024-09-17T05:22:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f8220e00640 1 -- 172.21.15.43:0/1215365267 wait complete. 2024-09-17T05:22:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 -- 172.21.15.43:0/909307359 shutdown_connections 2024-09-17T05:22:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.296+0000 7f06ea000640 1 -- 172.21.15.43:0/909307359 wait complete. 2024-09-17T05:22:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 -- 172.21.15.43:0/1601762397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c06f5a0 msgr2=0x7f7d8c0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1601762397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c06f5a0 0x7f7d8c0cef70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009bc0 tx=0x7f7d8401cb50 comp rx=0 tx=0).stop 2024-09-17T05:22:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 -- 172.21.15.43:0/1601762397 shutdown_connections 2024-09-17T05:22:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1601762397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c06f5a0 0x7f7d8c0cef70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1601762397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c0b3ae0 0x7f7d8c152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.300+0000 7f7d91e00640 1 -- 172.21.15.43:0/1601762397 >> 172.21.15.43:0/1601762397 conn(0x7f7d8c0b1c60 msgr2=0x7f7d8c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.304+0000 7f06ea000640 1 Processor -- start 2024-09-17T05:22:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.304+0000 7f06ea000640 1 -- start start 2024-09-17T05:22:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f7d91e00640 1 -- 172.21.15.43:0/1601762397 shutdown_connections 2024-09-17T05:22:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e40b6420 0x7f06e4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f7d91e00640 1 -- 172.21.15.43:0/1601762397 wait complete. 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47952/0 (socket says 172.21.15.43:47952) 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 -- 172.21.15.43:0/2760189115 learned_addr learned my addr 172.21.15.43:0/2760189115 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e40b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4170490 con 0x7f06e40b6420 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4170310 con 0x7f06e409c3c0 2024-09-17T05:22:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f06e4170610 con 0x7f06e4162f20 2024-09-17T05:22:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e9600640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e40b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 -- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 msgr2=0x7f06e40b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e40b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 -- 172.21.15.43:0/1149261067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab04005360 msgr2=0x7fab0400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 --2- 172.21.15.43:0/1149261067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab04005360 0x7fab0400b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c5c0 tx=0x7fab1801d330 comp rx=0 tx=0).stop 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 -- 172.21.15.43:0/1149261067 shutdown_connections 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 --2- 172.21.15.43:0/1149261067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab04005360 0x7fab0400b9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 -- 172.21.15.43:0/1149261067 >> 172.21.15.43:0/1149261067 conn(0x7fab200b1cd0 msgr2=0x7fab200b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 -- 172.21.15.43:0/1149261067 shutdown_connections 2024-09-17T05:22:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7fab27600640 1 -- 172.21.15.43:0/1149261067 wait complete. 2024-09-17T05:22:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 -- 172.21.15.43:0/2760189115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e40b6420 msgr2=0x7f06e4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e40b6420 0x7f06e4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e415a660 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e9600640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e40b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e8c00640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b420 tx=0x7f06cc00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc011830 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06cc016070 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.308+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc011d80 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e40d4a60 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41538d0 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06cc01b460 con 0x7f06e409c3c0 2024-09-17T05:22:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06cc01a8b0 con 0x7f06e409c3c0 2024-09-17T05:22:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f7d91e00640 1 Processor -- start 2024-09-17T05:22:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f7d91e00640 1 -- start start 2024-09-17T05:22:59.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.312+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e415ea50 con 0x7f06e409c3c0 2024-09-17T05:22:59.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 0x7f7d8c2062f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d8c146dd0 0x7f7d8c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 0x7f7d8c2062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 0x7f7d8c2062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47968/0 (socket says 172.21.15.43:47968) 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0aec90 con 0x7f7d8c146dd0 2024-09-17T05:22:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c0aeb10 con 0x7f7d8c0b3ae0 2024-09-17T05:22:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d91e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c169050 con 0x7f7d8c15c210 2024-09-17T05:22:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38350/0 (socket says 172.21.15.43:38350) 2024-09-17T05:22:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 -- 172.21.15.43:0/1189411915 learned_addr learned my addr 172.21.15.43:0/1189411915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f06cc01a3c0 con 0x7f06e409c3c0 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 -- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 msgr2=0x7f7d8c2062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 0x7f7d8c2062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 -- 172.21.15.43:0/1189411915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d8c146dd0 msgr2=0x7f7d8c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d8c146dd0 0x7f7d8c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.316+0000 7f7d8be00640 1 -- 172.21.15.43:0/1189411915 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c2098d0 con 0x7f7d8c15c210 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f7d8be00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c420 tx=0x7f7d7800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 -- 172.21.15.43:0/3890270856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380af1a0 msgr2=0x7f563815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 --2- 172.21.15.43:0/3890270856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380af1a0 0x7f563815edf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5630009840 tx=0x7f563001c790 comp rx=0 tx=0).stop 2024-09-17T05:22:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7fab27600640 1 Processor -- start 2024-09-17T05:22:59.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7fab27600640 1 -- start start 2024-09-17T05:22:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 -- 172.21.15.43:0/3890270856 shutdown_connections 2024-09-17T05:22:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 --2- 172.21.15.43:0/3890270856 >> v2:172.21.15.62:3300/0 conn(0x7f56380af5b0 0x7f563815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 --2- 172.21.15.43:0/3890270856 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380af1a0 0x7f563815edf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 -- 172.21.15.43:0/3890270856 >> 172.21.15.43:0/3890270856 conn(0x7f56380b1cd0 msgr2=0x7f56380b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f5640400640 1 -- 172.21.15.43:0/3890270856 shutdown_connections 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78011790 con 0x7f7d8c15c210 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d78017070 con 0x7f7d8c15c210 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.320+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78011ce0 con 0x7f7d8c15c210 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.324+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c209bb0 con 0x7f7d8c15c210 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.324+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c205af0 con 0x7f7d8c15c210 2024-09-17T05:22:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.324+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d7801c460 con 0x7f7d8c15c210 2024-09-17T05:22:59.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.324+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d780127e0 con 0x7f7d8c15c210 2024-09-17T05:22:59.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab04005360 0x7fab2009d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.324+0000 7f5640400640 1 -- 172.21.15.43:0/3890270856 wait complete. 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015c280 0x7fab2009bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab200b44d0 con 0x7fab04005360 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab2015f050 con 0x7fab201fd970 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab27600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fab200b4650 con 0x7fab2015c280 2024-09-17T05:22:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47974/0 (socket says 172.21.15.43:47974) 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 -- 172.21.15.43:0/83142114 learned_addr learned my addr 172.21.15.43:0/83142114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab25600640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015c280 0x7fab2009bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 -- 172.21.15.43:0/271527037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e98070620 msgr2=0x7f2e980b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 --2- 172.21.15.43:0/271527037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e98070620 0x7f2e980b30d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800ad00 tx=0x7f2e8801cda0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 -- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015c280 msgr2=0x7fab2009bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015c280 0x7fab2009bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 -- 172.21.15.43:0/83142114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab04005360 msgr2=0x7fab2009d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab04005360 0x7fab2009d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7fab1fe00640 1 -- 172.21.15.43:0/83142114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2009ab80 con 0x7fab201fd970 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 -- 172.21.15.43:0/271527037 shutdown_connections 2024-09-17T05:22:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c209620 con 0x7f7d8c15c210 2024-09-17T05:22:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 --2- 172.21.15.43:0/271527037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e98070620 0x7f2e980b30d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 -- 172.21.15.43:0/271527037 >> 172.21.15.43:0/271527037 conn(0x7f2e980b23d0 msgr2=0x7f2e980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 -- 172.21.15.43:0/271527037 shutdown_connections 2024-09-17T05:22:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.332+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7d78012a30 con 0x7f7d8c15c210 2024-09-17T05:22:59.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.328+0000 7f2e9da00640 1 -- 172.21.15.43:0/271527037 wait complete. 2024-09-17T05:22:59.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.332+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7feae4069300 con 0x7feae40ce050 2024-09-17T05:22:59.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.332+0000 7f2e9da00640 1 Processor -- start 2024-09-17T05:22:59.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.332+0000 7f2e9da00640 1 -- start start 2024-09-17T05:22:59.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab1fe00640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c410 tx=0x7fab0c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f5640400640 1 Processor -- start 2024-09-17T05:22:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017070 con 0x7fab201fd970 2024-09-17T05:22:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab0c003600 con 0x7fab201fd970 2024-09-17T05:22:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c003a50 con 0x7fab201fd970 2024-09-17T05:22:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab27600640 1 -- 172.21.15.43:0/83142114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2009ae60 con 0x7fab201fd970 2024-09-17T05:22:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab27600640 1 -- 172.21.15.43:0/83142114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2009b420 con 0x7fab201fd970 2024-09-17T05:22:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f2e9da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf40052a0 con 0x7fab201fd970 2024-09-17T05:22:59.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f2e9da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 0x7f2e980c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f2e96a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f2e96a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38358/0 (socket says 172.21.15.43:38358) 2024-09-17T05:22:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab0c01a070 con 0x7fab201fd970 2024-09-17T05:22:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f5640400640 1 -- start start 2024-09-17T05:22:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.336+0000 7f5640400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f2e96a00640 1 -- 172.21.15.43:0/1548323882 learned_addr learned my addr 172.21.15.43:0/1548323882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f2e9da00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980cb860 0x7f2e9809d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab0c01c780 con 0x7fab201fd970 2024-09-17T05:22:59.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e980ce9a0 con 0x7f2e98070620 2024-09-17T05:22:59.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47990/0 (socket says 172.21.15.43:47990) 2024-09-17T05:22:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 -- 172.21.15.43:0/1039688236 learned_addr learned my addr 172.21.15.43:0/1039688236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56380af5b0 0x7f5638151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e980ce820 con 0x7f2e980cb860 2024-09-17T05:22:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2e980ceb20 con 0x7f2e9806ce60 2024-09-17T05:22:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380b44d0 0x7f56381520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5638170320 con 0x7f56380af5b0 2024-09-17T05:22:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563e400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380b44d0 0x7f56381520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56381701a0 con 0x7f56380af1a0 2024-09-17T05:22:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f56381704a0 con 0x7f56380b44d0 2024-09-17T05:22:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fab0c003bf0 con 0x7fab201fd970 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 -- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380b44d0 msgr2=0x7f56381520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380b44d0 0x7f56381520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 -- 172.21.15.43:0/1039688236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56380af5b0 msgr2=0x7f5638151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f563da00640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56380af5b0 0x7f5638151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e97400640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980cb860 0x7f2e9809d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f563da00640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5638158950 con 0x7f56380af1a0 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e8e000640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 0x7f2e980c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 -- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980cb860 msgr2=0x7f2e9809d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980cb860 0x7f2e9809d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 -- 172.21.15.43:0/1548323882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 msgr2=0x7f2e980c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 0x7f2e980c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e980c3ef0 con 0x7f2e9806ce60 2024-09-17T05:22:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e8e000640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 0x7f2e980c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:22:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.340+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f9c740d0a80 con 0x7f9c740b30d0 2024-09-17T05:22:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e96a00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2e88000f90 tx=0x7f2e880a5a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.350 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e880bc070 con 0x7f2e9806ce60 2024-09-17T05:22:59.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f6fdc0b08f0 con 0x7f6fdc0b37e0 2024-09-17T05:22:59.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.344+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e880043b0 con 0x7f2e9806ce60 2024-09-17T05:22:59.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e880b2040 con 0x7f2e9806ce60 2024-09-17T05:22:59.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e981f85a0 con 0x7f2e9806ce60 2024-09-17T05:22:59.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981f8b60 con 0x7f2e9806ce60 2024-09-17T05:22:59.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e980b30d0 con 0x7f2e9806ce60 2024-09-17T05:22:59.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e880bf070 con 0x7f2e9806ce60 2024-09-17T05:22:59.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e880ad440 con 0x7f2e9806ce60 2024-09-17T05:22:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f563da00640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5630009790 tx=0x7f5630005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.348+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300b2040 con 0x7f56380af1a0 2024-09-17T05:22:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56300043e0 con 0x7f56380af1a0 2024-09-17T05:22:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300052d0 con 0x7f56380af1a0 2024-09-17T05:22:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5638205bf0 con 0x7f56380af1a0 2024-09-17T05:22:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5638157d90 con 0x7f56380af1a0 2024-09-17T05:22:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7fead7e00640 1 -- 172.21.15.43:0/46703561 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v11)=0 v11) ==== 119+0+0 (secure 0 0 0) 0x7feadc004d80 con 0x7feae40ce050 2024-09-17T05:22:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2e880c6080 con 0x7f2e9806ce60 2024-09-17T05:22:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5638069c20 con 0x7f56380af1a0 2024-09-17T05:22:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56300046d0 con 0x7f56380af1a0 2024-09-17T05:22:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.352+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5630003c00 con 0x7f56380af1a0 2024-09-17T05:22:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.356+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f56300bd060 con 0x7f56380af1a0 2024-09-17T05:22:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 msgr2=0x7feae41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c40 tx=0x7feadc005040 comp rx=0 tx=0).stop 2024-09-17T05:22:59.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 shutdown_connections 2024-09-17T05:22:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feae40ce050 0x7feae41590a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feae40d0590 0x7feae41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 --2- 172.21.15.43:0/46703561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feae40b4fc0 0x7feae414d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 >> 172.21.15.43:0/46703561 conn(0x7feae4074480 msgr2=0x7feae41645c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 shutdown_connections 2024-09-17T05:22:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.368+0000 7feaeae00640 1 -- 172.21.15.43:0/46703561 wait complete. 2024-09-17T05:22:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.372+0000 7f6fd8c00640 1 -- 172.21.15.43:0/3418265645 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f6fd4003600 con 0x7f6fdc0b37e0 2024-09-17T05:22:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 msgr2=0x7f6fdc09ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400b410 tx=0x7f6fd400b8d0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 shutdown_connections 2024-09-17T05:22:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc0cf390 0x7f6fdc0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fdc0b3bf0 0x7f6fdc1fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 --2- 172.21.15.43:0/3418265645 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fdc0b37e0 0x7f6fdc09ca60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 >> 172.21.15.43:0/3418265645 conn(0x7f6fdc0b1dd0 msgr2=0x7f6fdc0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f9c71600640 1 -- 172.21.15.43:0/456136808 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f9c680164d0 con 0x7f9c740b30d0 2024-09-17T05:22:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 shutdown_connections 2024-09-17T05:22:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.376+0000 7f6fe3800640 1 -- 172.21.15.43:0/3418265645 wait complete. 2024-09-17T05:22:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.384+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f2b000b0b40 con 0x7f2ae8004c90 2024-09-17T05:22:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.384+0000 7fd50a200640 1 -- 172.21.15.43:0/1743366726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5040730a0 msgr2=0x7fd5040734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.384+0000 7fd50a200640 1 --2- 172.21.15.43:0/1743366726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5040730a0 0x7fd5040734b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009bc0 tx=0x7fd4f401cae0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 msgr2=0x7f9c740c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7f9c7c400640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c6801a480 tx=0x7f9c680a5680 comp rx=0 tx=0).stop 2024-09-17T05:22:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.400+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 shutdown_connections 2024-09-17T05:22:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.400+0000 7f9c7c400640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7415e800 0x7f9c740c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.400+0000 7f9c7c400640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c740b5b20 0x7f9c740c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 -- 172.21.15.43:0/1743366726 shutdown_connections 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 --2- 172.21.15.43:0/1743366726 >> v2:172.21.15.62:3300/0 conn(0x7fd50406ee90 0x7fd50406f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 --2- 172.21.15.43:0/1743366726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd5040730a0 0x7fd5040734b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 -- 172.21.15.43:0/1743366726 >> 172.21.15.43:0/1743366726 conn(0x7fd5040b1c60 msgr2=0x7fd5040b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 -- 172.21.15.43:0/1743366726 shutdown_connections 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.396+0000 7fd50a200640 1 -- 172.21.15.43:0/1743366726 wait complete. 2024-09-17T05:22:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.400+0000 7f9c7c400640 1 --2- 172.21.15.43:0/456136808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c740b30d0 0x7f9c740c9650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.400+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 >> 172.21.15.43:0/456136808 conn(0x7f9c740b1dd0 msgr2=0x7f9c74161190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.404+0000 7fd50a200640 1 Processor -- start 2024-09-17T05:22:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.404+0000 7fd50a200640 1 -- start start 2024-09-17T05:22:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.408+0000 7f2afd600640 1 -- 172.21.15.43:0/4285076573 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7f2b000ae990 con 0x7f2ae8004c90 2024-09-17T05:22:59.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 shutdown_connections 2024-09-17T05:22:59.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7f9c7c400640 1 -- 172.21.15.43:0/456136808 wait complete. 2024-09-17T05:22:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd50a200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 0x7fd5040b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd50a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 0x7fd5041503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd50a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 0x7fd50414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 0x7fd5041503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd509800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 0x7fd50414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 0x7fd5041503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:48014/0 (socket says 172.21.15.43:48014) 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd509800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 0x7fd50414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34128/0 (socket says 172.21.15.43:34128) 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.412+0000 7fd508e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 0x7fd5040b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd4fbe00640 1 -- 172.21.15.43:0/316274935 learned_addr learned my addr 172.21.15.43:0/316274935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd504157790 con 0x7fd5040cef70 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd504157610 con 0x7fd5040730a0 2024-09-17T05:22:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd504157910 con 0x7fd50406ee90 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 -- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 msgr2=0x7fd5041503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 0x7fd5041503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 -- 172.21.15.43:0/316274935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 msgr2=0x7fd50414afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 0x7fd50414afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50414f380 con 0x7fd50406ee90 2024-09-17T05:22:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.416+0000 7fd508e00640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 0x7fd5040b0160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd504070f60 tx=0x7fd4f4009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 msgr2=0x7f2b001574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40bb070 con 0x7fd50406ee90 2024-09-17T05:22:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7f2b08200640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2af401cd30 tx=0x7f2af4006490 comp rx=0 tx=0).stop 2024-09-17T05:22:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4f40a5e50 con 0x7fd50406ee90 2024-09-17T05:22:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5040af980 con 0x7fd50406ee90 2024-09-17T05:22:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40b2040 con 0x7fd50406ee90 2024-09-17T05:22:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50414c330 con 0x7fd50406ee90 2024-09-17T05:22:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4f40be070 con 0x7fd50406ee90 2024-09-17T05:22:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4f4005370 con 0x7fd50406ee90 2024-09-17T05:22:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.420+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50414c590 con 0x7fd50406ee90 2024-09-17T05:22:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.424+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd4f4016790 con 0x7fd50406ee90 2024-09-17T05:22:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.432+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 shutdown_connections 2024-09-17T05:22:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.432+0000 7f2b08200640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b001fe670 0x7f2b00200fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.432+0000 7f2b08200640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000ae040 0x7f2b000ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.432+0000 7f2b08200640 1 --2- 172.21.15.43:0/4285076573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ae8004c90 0x7f2b001574c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.432+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 >> 172.21.15.43:0/4285076573 conn(0x7f2b00074480 msgr2=0x7f2b00074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.436+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 shutdown_connections 2024-09-17T05:22:59.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.436+0000 7f2b08200640 1 -- 172.21.15.43:0/4285076573 wait complete. 2024-09-17T05:22:59.446 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.484+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7faaf4005590 con 0x7fab201fd970 2024-09-17T05:22:59.488 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.492+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f1958048280 con 0x7f195804efc0 2024-09-17T05:22:59.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.496+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f06e406ce60 con 0x7f06e409c3c0 2024-09-17T05:22:59.503 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 -- 172.21.15.43:0/1141485988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f400af390 msgr2=0x7f5f4015eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 --2- 172.21.15.43:0/1141485988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f400af390 0x7f5f4015eb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5f400b5470 tx=0x7f5f3801cb80 comp rx=0 tx=0).stop 2024-09-17T05:22:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 -- 172.21.15.43:0/1141485988 shutdown_connections 2024-09-17T05:22:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 --2- 172.21.15.43:0/1141485988 >> v2:172.21.15.62:3300/0 conn(0x7f5f4015f050 0x7f5f4006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 --2- 172.21.15.43:0/1141485988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f400af390 0x7f5f4015eb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 -- 172.21.15.43:0/1141485988 >> 172.21.15.43:0/1141485988 conn(0x7f5f400b1cd0 msgr2=0x7f5f400b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 -- 172.21.15.43:0/1141485988 shutdown_connections 2024-09-17T05:22:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.500+0000 7f5f47800640 1 -- 172.21.15.43:0/1141485988 wait complete. 2024-09-17T05:22:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab1cc00640 1 -- 172.21.15.43:0/83142114 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7fab0c003dd0 con 0x7fab201fd970 2024-09-17T05:22:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f7d8c09de40 con 0x7f7d8c15c210 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 msgr2=0x7fab201fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c410 tx=0x7fab0c00c8d0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 shutdown_connections 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab201fd970 0x7fab201fe020 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2015c280 0x7fab2009bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 --2- 172.21.15.43:0/83142114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab04005360 0x7fab2009d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 >> 172.21.15.43:0/83142114 conn(0x7fab200b1cd0 msgr2=0x7fab2015ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 shutdown_connections 2024-09-17T05:22:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.504+0000 7fab16000640 1 -- 172.21.15.43:0/83142114 wait complete. 2024-09-17T05:22:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f5f47800640 1 Processor -- start 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 -- 172.21.15.43:0/310340094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183409c3c0 msgr2=0x7f18340b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 --2- 172.21.15.43:0/310340094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183409c3c0 0x7f18340b5ee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f18340d1520 tx=0x7f182801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f5f47800640 1 -- start start 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 -- 172.21.15.43:0/310340094 shutdown_connections 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 --2- 172.21.15.43:0/310340094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18340b6420 0x7f18340b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 --2- 172.21.15.43:0/310340094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183409c3c0 0x7f18340b5ee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 --2- 172.21.15.43:0/310340094 >> v2:172.21.15.43:3300/0 conn(0x7f1834162f20 0x7f183406fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f183b000640 1 -- 172.21.15.43:0/310340094 >> 172.21.15.43:0/310340094 conn(0x7f18340691b0 msgr2=0x7f18340695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.508+0000 7f5f47800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f183b000640 1 -- 172.21.15.43:0/310340094 shutdown_connections 2024-09-17T05:22:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f47800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 0x7f5f4014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f06e0c00640 1 -- 172.21.15.43:0/2760189115 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) ==== 106+0+0 (secure 0 0 0) 0x7f06cc01ab80 con 0x7f06e409c3c0 2024-09-17T05:22:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f47800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4015f050 0x7f5f40147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f47800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400ccfc0 con 0x7f5f4015a9c0 2024-09-17T05:22:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f47800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400cce40 con 0x7f5f4015f050 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f47800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f400cd140 con 0x7f5f400af390 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38400/0 (socket says 172.21.15.43:38400) 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 -- 172.21.15.43:0/1977803763 learned_addr learned my addr 172.21.15.43:0/1977803763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 -- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4015f050 msgr2=0x7f5f40147120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f3fe00640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 0x7f5f4014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f183b000640 1 -- 172.21.15.43:0/310340094 wait complete. 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4015f050 0x7f5f40147120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 -- 172.21.15.43:0/1977803763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 msgr2=0x7f5f4014c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 0x7f5f4014c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 -- 172.21.15.43:0/1977803763 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4015c900 con 0x7f5f400af390 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f3fe00640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 0x7f5f4014c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:22:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f5f44e00640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5f38000f90 tx=0x7f5f38009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 msgr2=0x7f06e4162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.512+0000 7f06ea000640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00b420 tx=0x7f06cc00b8e0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 shutdown_connections 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06e4162f20 0x7f06e40b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06e40b6420 0x7f06e4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 --2- 172.21.15.43:0/2760189115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e409c3c0 0x7f06e4162020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 >> 172.21.15.43:0/2760189115 conn(0x7f06e40691b0 msgr2=0x7f06e4076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 shutdown_connections 2024-09-17T05:22:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f1953400640 1 -- 172.21.15.43:0/3514271361 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7f1958052e90 con 0x7f195804efc0 2024-09-17T05:22:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380bb070 con 0x7f5f400af390 2024-09-17T05:22:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f38004b40 con 0x7f5f400af390 2024-09-17T05:22:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b2040 con 0x7f5f400af390 2024-09-17T05:22:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4009de00 con 0x7f5f400af390 2024-09-17T05:22:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f06ea000640 1 -- 172.21.15.43:0/2760189115 wait complete. 2024-09-17T05:22:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f2e980b1600 con 0x7f2e9806ce60 2024-09-17T05:22:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4015cbd0 con 0x7f5f400af390 2024-09-17T05:22:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f380be070 con 0x7f5f400af390 2024-09-17T05:22:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.516+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f38004340 con 0x7f5f400af390 2024-09-17T05:22:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 Processor -- start 2024-09-17T05:22:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f40069c20 con 0x7f5f400af390 2024-09-17T05:22:59.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 msgr2=0x7f19581856f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f194c00c730 tx=0x7f194c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 shutdown_connections 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958052a80 0x7f1958181690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f195804efc0 0x7f19581856f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 --2- 172.21.15.43:0/3514271361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958039230 0x7f19580da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 >> 172.21.15.43:0/3514271361 conn(0x7f1958054970 msgr2=0x7f19580df990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 shutdown_connections 2024-09-17T05:22:59.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5f38016770 con 0x7f5f400af390 2024-09-17T05:22:59.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 -- start start 2024-09-17T05:22:59.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340b6420 0x7f1834154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834162f20 0x7f18341554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18340b7e50 con 0x7f18340b6420 2024-09-17T05:22:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18340b7cd0 con 0x7f1834162f20 2024-09-17T05:22:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f183b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f18340b7fd0 con 0x7f183409c3c0 2024-09-17T05:22:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.520+0000 7f1965000640 1 -- 172.21.15.43:0/3514271361 wait complete. 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:38412/0 (socket says 172.21.15.43:38412) 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 -- 172.21.15.43:0/345430641 learned_addr learned my addr 172.21.15.43:0/345430641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f183a600640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834162f20 0x7f18341554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 -- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834162f20 msgr2=0x7f18341554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834162f20 0x7f18341554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 -- 172.21.15.43:0/345430641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340b6420 msgr2=0x7f1834154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340b6420 0x7f1834154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 -- 172.21.15.43:0/345430641 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183406ea00 con 0x7f183409c3c0 2024-09-17T05:22:59.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f1839c00640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f182000e7a0 tx=0x7f182000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.524+0000 7f7d88c00640 1 -- 172.21.15.43:0/1189411915 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f7d78012c10 con 0x7f7d8c15c210 2024-09-17T05:22:59.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.528+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182000ccc0 con 0x7f183409c3c0 2024-09-17T05:22:59.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.528+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1820010430 con 0x7f183409c3c0 2024-09-17T05:22:59.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.528+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820010880 con 0x7f183409c3c0 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 msgr2=0x7f7d8c208cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c420 tx=0x7f7d7800c8e0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 shutdown_connections 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d8c146dd0 0x7f7d8c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d8c15c210 0x7f7d8c208cf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 --2- 172.21.15.43:0/1189411915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d8c0b3ae0 0x7f7d8c2062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 >> 172.21.15.43:0/1189411915 conn(0x7f7d8c0b1c60 msgr2=0x7f7d8c151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f2e8f400640 1 -- 172.21.15.43:0/1548323882 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v19)=0 v19) ==== 108+0+0 (secure 0 0 0) 0x7f2e88016a20 con 0x7f2e9806ce60 2024-09-17T05:22:59.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.532+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 shutdown_connections 2024-09-17T05:22:59.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.536+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183406ece0 con 0x7f183409c3c0 2024-09-17T05:22:59.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.536+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f183406f2a0 con 0x7f183409c3c0 2024-09-17T05:22:59.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.536+0000 7f7d91e00640 1 -- 172.21.15.43:0/1189411915 wait complete. 2024-09-17T05:22:59.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.536+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1820010a20 con 0x7f183409c3c0 2024-09-17T05:22:59.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.536+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1820005130 con 0x7f183409c3c0 2024-09-17T05:22:59.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.540+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183406f6e0 con 0x7f183409c3c0 2024-09-17T05:22:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 msgr2=0x7f2e9814a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2e88000f90 tx=0x7f2e880a5a70 comp rx=0 tx=0).stop 2024-09-17T05:22:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 shutdown_connections 2024-09-17T05:22:59.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1820026070 con 0x7f183409c3c0 2024-09-17T05:22:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980cb860 0x7f2e9809d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e98070620 0x7f2e980c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 --2- 172.21.15.43:0/1548323882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e9806ce60 0x7f2e9814a200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 >> 172.21.15.43:0/1548323882 conn(0x7f2e980b23d0 msgr2=0x7f2e980af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 shutdown_connections 2024-09-17T05:22:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.544+0000 7f2e9da00640 1 -- 172.21.15.43:0/1548323882 wait complete. 2024-09-17T05:22:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.548+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f56382059e0 con 0x7f56380af1a0 2024-09-17T05:22:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.556+0000 7f5635600640 1 -- 172.21.15.43:0/1039688236 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v20)=0 v20) ==== 111+0+0 (secure 0 0 0) 0x7f56300bd060 con 0x7f56380af1a0 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 msgr2=0x7f563814f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5630009790 tx=0x7f5630005db0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 shutdown_connections 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56380b44d0 0x7f56381520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56380af5b0 0x7f5638151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 --2- 172.21.15.43:0/1039688236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56380af1a0 0x7f563814f9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 >> 172.21.15.43:0/1039688236 conn(0x7f56380b1cd0 msgr2=0x7f5638153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 shutdown_connections 2024-09-17T05:22:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.560+0000 7f5640400640 1 -- 172.21.15.43:0/1039688236 wait complete. 2024-09-17T05:22:59.577 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.577 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.577 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.577 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.588 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.589 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.589 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.588+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fd50406f2a0 con 0x7fd50406ee90 2024-09-17T05:22:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd501600640 1 -- 172.21.15.43:0/316274935 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v21)=0 v21) ==== 125+0+0 (secure 0 0 0) 0x7fd4f4005570 con 0x7fd50406ee90 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 msgr2=0x7fd5040b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 0x7fd5040b0160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd504070f60 tx=0x7fd4f4009200 comp rx=0 tx=0).stop 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 shutdown_connections 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5040730a0 0x7fd5041503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5040cef70 0x7fd50414afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 --2- 172.21.15.43:0/316274935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd50406ee90 0x7fd5040b0160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 >> 172.21.15.43:0/316274935 conn(0x7fd5040b1c60 msgr2=0x7fd504161380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 shutdown_connections 2024-09-17T05:22:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.592+0000 7fd50a200640 1 -- 172.21.15.43:0/316274935 wait complete. 2024-09-17T05:22:59.618 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.636+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f5f400cd660 con 0x7f5f400af390 2024-09-17T05:22:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.636+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f18340b1ff0 con 0x7f183409c3c0 2024-09-17T05:22:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f3cc00640 1 -- 172.21.15.43:0/1977803763 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f5f38004540 con 0x7f5f400af390 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 msgr2=0x7f5f4015d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5f38000f90 tx=0x7f5f38009280 comp rx=0 tx=0).stop 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 shutdown_connections 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4015f050 0x7f5f40147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4015a9c0 0x7f5f4014c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 --2- 172.21.15.43:0/1977803763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f400af390 0x7f5f4015d3d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 >> 172.21.15.43:0/1977803763 conn(0x7f5f400b1cd0 msgr2=0x7f5f400cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 shutdown_connections 2024-09-17T05:22:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.640+0000 7f5f47800640 1 -- 172.21.15.43:0/1977803763 wait complete. 2024-09-17T05:22:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f1831600640 1 -- 172.21.15.43:0/345430641 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f1820005380 con 0x7f183409c3c0 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 msgr2=0x7f183415cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f182000e7a0 tx=0x7f182000ec60 comp rx=0 tx=0).stop 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 shutdown_connections 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1834162f20 0x7f18341554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340b6420 0x7f1834154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 --2- 172.21.15.43:0/345430641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f183409c3c0 0x7f183415cea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 >> 172.21.15.43:0/345430641 conn(0x7f18340691b0 msgr2=0x7f183415e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 shutdown_connections 2024-09-17T05:22:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.644+0000 7f183b000640 1 -- 172.21.15.43:0/345430641 wait complete. 2024-09-17T05:22:59.678 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi043') 2024-09-17T05:22:59.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config dump 2024-09-17T05:22:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 -- 172.21.15.43:0/2051490603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 msgr2=0x7f20ec0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 --2- 172.21.15.43:0/2051490603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0cd040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009900 tx=0x7f20d801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 -- 172.21.15.43:0/2051490603 shutdown_connections 2024-09-17T05:22:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 --2- 172.21.15.43:0/2051490603 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 0x7f20ec0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 --2- 172.21.15.43:0/2051490603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0cd040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 -- 172.21.15.43:0/2051490603 >> 172.21.15.43:0/2051490603 conn(0x7f20ec0cee60 msgr2=0x7f20ec0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 -- 172.21.15.43:0/2051490603 shutdown_connections 2024-09-17T05:22:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.876+0000 7f20f4400640 1 -- 172.21.15.43:0/2051490603 wait complete. 2024-09-17T05:22:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 Processor -- start 2024-09-17T05:22:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 -- start start 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20ec09ba60 0x7f20ec154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 0x7f20ec14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0b2b60 con 0x7f20ec0d0240 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0b29e0 con 0x7f20ec09ba60 2024-09-17T05:22:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.880+0000 7f20f4400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec169e30 con 0x7f20ec0d0650 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34168/0 (socket says 172.21.15.43:34168) 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 -- 172.21.15.43:0/3604752889 learned_addr learned my addr 172.21.15.43:0/3604752889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1a00640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20ec09ba60 0x7f20ec154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f2400640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 0x7f20ec14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 -- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 msgr2=0x7f20ec14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 0x7f20ec14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 -- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20ec09ba60 msgr2=0x7f20ec154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20ec09ba60 0x7f20ec154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 -- 172.21.15.43:0/3604752889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec155510 con 0x7f20ec0d0240 2024-09-17T05:22:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f1000640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009850 tx=0x7f20d80a6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:22:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80bd070 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20d8005030 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec155990 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8005480 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec152b10 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20d80c0070 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f20d8004e00 con 0x7f20ec0d0240 2024-09-17T05:22:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.884+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec1556e0 con 0x7f20ec0d0240 2024-09-17T05:22:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.888+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f20d80da860 con 0x7f20ec0d0240 2024-09-17T05:22:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.948+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f20ec0b3130 con 0x7f20ec0d0240 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.948+0000 7f20e9600640 1 -- 172.21.15.43:0/3604752889 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f20d80c8030 con 0x7f20ec0d0240 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:22:59.952 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:22:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:22:59.954 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:22:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 msgr2=0x7f20ec0aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:22:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009850 tx=0x7f20d80a6720 comp rx=0 tx=0).stop 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 shutdown_connections 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20ec0d0650 0x7f20ec14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0d0240 0x7f20ec0aef70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 --2- 172.21.15.43:0/3604752889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20ec09ba60 0x7f20ec154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 >> 172.21.15.43:0/3604752889 conn(0x7f20ec0cee60 msgr2=0x7f20ec0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 shutdown_connections 2024-09-17T05:22:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:22:59.952+0000 7f20f4400640 1 -- 172.21.15.43:0/3604752889 wait complete. 2024-09-17T05:22:59.972 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T05:22:59.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 -- 172.21.15.43:0/2095143668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 msgr2=0x7fc2a40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/2095143668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a40cd040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc290009840 tx=0x7fc29001c820 comp rx=0 tx=0).stop 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 -- 172.21.15.43:0/2095143668 shutdown_connections 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/2095143668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a40cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/2095143668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a40cd040 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 -- 172.21.15.43:0/2095143668 >> 172.21.15.43:0/2095143668 conn(0x7fc2a40cee60 msgr2=0x7fc2a40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 -- 172.21.15.43:0/2095143668 shutdown_connections 2024-09-17T05:23:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.128+0000 7fc2a8e00640 1 -- 172.21.15.43:0/2095143668 wait complete. 2024-09-17T05:23:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 Processor -- start 2024-09-17T05:23:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 -- start start 2024-09-17T05:23:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2a409ba60 0x7fc2a4150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4169e30 con 0x7fc2a40d0240 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a40b2a60 con 0x7fc2a409ba60 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a8e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a4169fb0 con 0x7fc2a40d0650 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34194/0 (socket says 172.21.15.43:34194) 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 -- 172.21.15.43:0/1606254433 learned_addr learned my addr 172.21.15.43:0/1606254433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a2800640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1e00640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2a409ba60 0x7fc2a4150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 -- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 msgr2=0x7fc2a415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 -- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2a409ba60 msgr2=0x7fc2a4150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2a409ba60 0x7fc2a4150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a1400640 1 -- 172.21.15.43:0/1606254433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a40c6120 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.132+0000 7fc2a2800640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc2a1400640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc290000c10 tx=0x7fc2900a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900a8ca0 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2900a8e40 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290005270 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a4158d50 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a4159310 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2900b8030 con 0x7fc2a40d0240 2024-09-17T05:23:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2900aa370 con 0x7fc2a40d0240 2024-09-17T05:23:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.136+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a40c62f0 con 0x7fc2a40d0240 2024-09-17T05:23:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.140+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc2900ac030 con 0x7fc2a40d0240 2024-09-17T05:23:00.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.204+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc2a40b3130 con 0x7fc2a40d0240 2024-09-17T05:23:00.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc29a000640 1 -- 172.21.15.43:0/1606254433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc290016410 con 0x7fc2a40d0240 2024-09-17T05:23:00.210 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-17T05:23:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 msgr2=0x7fc2a41f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc290000c10 tx=0x7fc2900a8590 comp rx=0 tx=0).stop 2024-09-17T05:23:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 shutdown_connections 2024-09-17T05:23:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2a40d0650 0x7fc2a415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a40d0240 0x7fc2a41f98f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 --2- 172.21.15.43:0/1606254433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2a409ba60 0x7fc2a4150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 >> 172.21.15.43:0/1606254433 conn(0x7fc2a40cee60 msgr2=0x7fc2a4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:00.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 shutdown_connections 2024-09-17T05:23:00.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:00.208+0000 7fc2a8e00640 1 -- 172.21.15.43:0/1606254433 wait complete. 2024-09-17T05:23:00.230 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T05:23:00.230 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T05:23:00.231 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:00.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T05:23:00.253 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:00.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T05:23:00.315 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:00.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T05:23:00.376 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:00.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T05:23:00.444 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:00.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T05:23:00.464 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:00.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T05:23:00.527 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:00.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T05:23:00.588 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:00.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T05:23:00.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new e58bb03d-6df4-4384-9392-7f75c166a6e2 0 2024-09-17T05:23:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- 172.21.15.62:0/1110036633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087406d7b0 msgr2=0x7f08740b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 --2- 172.21.15.62:0/1110036633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087406d7b0 0x7f08740b1180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f08740cdc60 tx=0x7f086c01caf0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- 172.21.15.62:0/1110036633 shutdown_connections 2024-09-17T05:23:00.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 --2- 172.21.15.62:0/1110036633 >> v2:172.21.15.43:3300/0 conn(0x7f087406ce60 0x7f087406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 --2- 172.21.15.62:0/1110036633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f087406d7b0 0x7f08740b1180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 --2- 172.21.15.62:0/1110036633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08740d03d0 0x7f08740d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- 172.21.15.62:0/1110036633 >> 172.21.15.62:0/1110036633 conn(0x7f08740b2f80 msgr2=0x7f08740b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- 172.21.15.62:0/1110036633 shutdown_connections 2024-09-17T05:23:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- 172.21.15.62:0/1110036633 wait complete. 2024-09-17T05:23:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 Processor -- start 2024-09-17T05:23:00.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.848+0000 7f0879400640 1 -- start start 2024-09-17T05:23:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 0x7f0874154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 0x7f08740af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 0x7f08740afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08740b2050 con 0x7f087406ce60 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08740b1ed0 con 0x7f08740d03d0 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f08740b21d0 con 0x7f087406d7b0 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 0x7f08740af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0872800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 0x7f0874154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0873200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 0x7f08740afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0872800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 0x7f0874154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.62:44366/0 (socket says 172.21.15.62:44366) 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0872800640 1 -- 172.21.15.62:0/317355870 learned_addr learned my addr 172.21.15.62:0/317355870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 -- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 msgr2=0x7f08740afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 0x7f08740afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 -- 172.21.15.62:0/317355870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 msgr2=0x7f0874154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 0x7f0874154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 -- 172.21.15.62:0/317355870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08740c90d0 con 0x7f087406d7b0 2024-09-17T05:23:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0873200640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 0x7f08740afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086be00640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 0x7f08740af5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0874073f90 tx=0x7f085c00da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c017070 con 0x7f087406d7b0 2024-09-17T05:23:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f085c004310 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c005280 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08740c93b0 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08740c9970 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f085c0047b0 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f085c00fda0 con 0x7f087406d7b0 2024-09-17T05:23:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.852+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740b1180 con 0x7f087406d7b0 2024-09-17T05:23:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.856+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f085c020030 con 0x7f087406d7b0 2024-09-17T05:23:00.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.928+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e58bb03d-6df4-4384-9392-7f75c166a6e2", "id": 0} v 0) -- 0x7f08740d1ff0 con 0x7f087406d7b0 2024-09-17T05:23:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.932+0000 7f086aa00640 1 -- 172.21.15.62:0/317355870 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e58bb03d-6df4-4384-9392-7f75c166a6e2", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f085c004cd0 con 0x7f087406d7b0 2024-09-17T05:23:00.935 INFO:teuthology.orchestra.run.smithi062.stdout:0 2024-09-17T05:23:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 msgr2=0x7f08740af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 0x7f08740af5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0874073f90 tx=0x7f085c00da20 comp rx=0 tx=0).stop 2024-09-17T05:23:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 shutdown_connections 2024-09-17T05:23:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08740d03d0 0x7f08740afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f087406d7b0 0x7f08740af5d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 --2- 172.21.15.62:0/317355870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087406ce60 0x7f0874154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 >> 172.21.15.62:0/317355870 conn(0x7f08740b2f80 msgr2=0x7f08740ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 shutdown_connections 2024-09-17T05:23:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:00.936+0000 7f0879400640 1 -- 172.21.15.62:0/317355870 wait complete. 2024-09-17T05:23:00.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 14750228-325c-43bb-bd26-84cfbf867ab3 1 2024-09-17T05:23:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 -- 172.21.15.62:0/1029703014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba80b3af0 msgr2=0x7f0ba815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 --2- 172.21.15.62:0/1029703014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba80b3af0 0x7f0ba815c7e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009840 tx=0x7f0b9001c760 comp rx=0 tx=0).stop 2024-09-17T05:23:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 -- 172.21.15.62:0/1029703014 shutdown_connections 2024-09-17T05:23:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 --2- 172.21.15.62:0/1029703014 >> v2:172.21.15.43:3300/0 conn(0x7f0ba806f110 0x7f0ba80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 --2- 172.21.15.62:0/1029703014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba80b3af0 0x7f0ba815c7e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 -- 172.21.15.62:0/1029703014 >> 172.21.15.62:0/1029703014 conn(0x7f0ba80b23d0 msgr2=0x7f0ba80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 -- 172.21.15.62:0/1029703014 shutdown_connections 2024-09-17T05:23:01.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.108+0000 7f0baf000640 1 -- 172.21.15.62:0/1029703014 wait complete. 2024-09-17T05:23:01.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 Processor -- start 2024-09-17T05:23:01.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 -- start start 2024-09-17T05:23:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba806f110 0x7f0ba80cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba80cfb70 0x7f0ba80cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba8168cb0 con 0x7f0ba80cfb70 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba8168b30 con 0x7f0ba8155d70 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0baf000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba8168e30 con 0x7f0ba806f110 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0b9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0ba7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba806f110 0x7f0ba80cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0b9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56826/0 (socket says 172.21.15.62:56826) 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.112+0000 7f0b9f400640 1 -- 172.21.15.62:0/3214674840 learned_addr learned my addr 172.21.15.62:0/3214674840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0bad000640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba80cfb70 0x7f0ba80cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 -- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba806f110 msgr2=0x7f0ba80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba806f110 0x7f0ba80cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 -- 172.21.15.62:0/3214674840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba80cfb70 msgr2=0x7f0ba80cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba80cfb70 0x7f0ba80cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 -- 172.21.15.62:0/3214674840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80d04c0 con 0x7f0ba8155d70 2024-09-17T05:23:01.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0b9f400640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009790 tx=0x7f0b900a56a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900bb070 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b90004470 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba8073090 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900b2040 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba81fca50 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b900be070 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f0b90003600 con 0x7f0ba8155d70 2024-09-17T05:23:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.116+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba80ad0b0 con 0x7f0ba8155d70 2024-09-17T05:23:01.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.120+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0b900c6020 con 0x7f0ba8155d70 2024-09-17T05:23:01.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.188+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14750228-325c-43bb-bd26-84cfbf867ab3", "id": 1} v 0) -- 0x7f0ba81fc490 con 0x7f0ba8155d70 2024-09-17T05:23:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0ba4c00640 1 -- 172.21.15.62:0/3214674840 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14750228-325c-43bb-bd26-84cfbf867ab3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0ba80d04c0 con 0x7f0ba8155d70 2024-09-17T05:23:01.197 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-09-17T05:23:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 msgr2=0x7f0ba8074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009790 tx=0x7f0b900a56a0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 shutdown_connections 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ba8155d70 0x7f0ba8074b00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ba80cfb70 0x7f0ba80cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 --2- 172.21.15.62:0/3214674840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ba806f110 0x7f0ba80cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 >> 172.21.15.62:0/3214674840 conn(0x7f0ba80b23d0 msgr2=0x7f0ba80ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 shutdown_connections 2024-09-17T05:23:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.196+0000 7f0baf000640 1 -- 172.21.15.62:0/3214674840 wait complete. 2024-09-17T05:23:01.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new a10f45b8-705a-48e6-8862-8f92b0bc1225 2 2024-09-17T05:23:01.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 -- 172.21.15.62:0/350707808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec0d0240 msgr2=0x7f0dec0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 --2- 172.21.15.62:0/350707808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec0d0240 0x7f0dec0cd040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009900 tx=0x7f0dd801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 -- 172.21.15.62:0/350707808 shutdown_connections 2024-09-17T05:23:01.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 --2- 172.21.15.62:0/350707808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec0d0240 0x7f0dec0cd040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 --2- 172.21.15.62:0/350707808 >> v2:172.21.15.43:3300/0 conn(0x7f0dec09ba60 0x7f0dec0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 -- 172.21.15.62:0/350707808 >> 172.21.15.62:0/350707808 conn(0x7f0dec0cee60 msgr2=0x7f0dec0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 -- 172.21.15.62:0/350707808 shutdown_connections 2024-09-17T05:23:01.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.404+0000 7f0df2000640 1 -- 172.21.15.62:0/350707808 wait complete. 2024-09-17T05:23:01.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 Processor -- start 2024-09-17T05:23:01.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 -- start start 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dec0d0240 0x7f0dec1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dec0d0650 0x7f0dec15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0b2ae0 con 0x7f0dec0d0650 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec0b2960 con 0x7f0dec0d0240 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0df2000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec169e30 con 0x7f0dec09ba60 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:41352/0 (socket says 172.21.15.62:41352) 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 -- 172.21.15.62:0/3134694671 learned_addr learned my addr 172.21.15.62:0/3134694671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0debe00640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dec0d0650 0x7f0dec15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deaa00640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dec0d0240 0x7f0dec1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 -- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dec0d0240 msgr2=0x7f0dec1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dec0d0240 0x7f0dec1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 -- 172.21.15.62:0/3134694671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dec0d0650 msgr2=0x7f0dec15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dec0d0650 0x7f0dec15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.408+0000 7f0deb400640 1 -- 172.21.15.62:0/3134694671 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec1566e0 con 0x7f0dec09ba60 2024-09-17T05:23:01.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0deb400640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e7a0 tx=0x7f0ddc00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00ccc0 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ddc010430 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc010860 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec1fa890 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec0ca770 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ddc010a00 con 0x7f0dec09ba60 2024-09-17T05:23:01.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0ddc010cf0 con 0x7f0dec09ba60 2024-09-17T05:23:01.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.412+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dec1fadb0 con 0x7f0dec09ba60 2024-09-17T05:23:01.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.416+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0dec1fadb0 con 0x7f0dec09ba60 2024-09-17T05:23:01.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.484+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a10f45b8-705a-48e6-8862-8f92b0bc1225", "id": 2} v 0) -- 0x7f0dec0b3130 con 0x7f0dec09ba60 2024-09-17T05:23:01.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0de3400640 1 -- 172.21.15.62:0/3134694671 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a10f45b8-705a-48e6-8862-8f92b0bc1225", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0ddc018c40 con 0x7f0dec09ba60 2024-09-17T05:23:01.493 INFO:teuthology.orchestra.run.smithi062.stdout:2 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 msgr2=0x7f0dec1f9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e7a0 tx=0x7f0ddc00ec60 comp rx=0 tx=0).stop 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 shutdown_connections 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dec0d0240 0x7f0dec1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dec0d0650 0x7f0dec15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 --2- 172.21.15.62:0/3134694671 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0dec09ba60 0x7f0dec1f9eb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 >> 172.21.15.62:0/3134694671 conn(0x7f0dec0cee60 msgr2=0x7f0dec0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 shutdown_connections 2024-09-17T05:23:01.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.492+0000 7f0df2000640 1 -- 172.21.15.62:0/3134694671 wait complete. 2024-09-17T05:23:01.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 359bdccb-91ae-428a-864b-50613ad48610 3 2024-09-17T05:23:01.595 INFO:tasks.ceph.mgr.x.smithi062.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:23:01.596 INFO:tasks.ceph.mgr.x.smithi062.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:23:01.596 INFO:tasks.ceph.mgr.x.smithi062.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- 172.21.15.62:0/804957247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 msgr2=0x7f1e2c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 --2- 172.21.15.62:0/804957247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0cd750 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009b70 tx=0x7f1e2401caa0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- 172.21.15.62:0/804957247 shutdown_connections 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 --2- 172.21.15.62:0/804957247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0cd750 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- 172.21.15.62:0/804957247 >> 172.21.15.62:0/804957247 conn(0x7f1e2c0cee60 msgr2=0x7f1e2c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- 172.21.15.62:0/804957247 shutdown_connections 2024-09-17T05:23:01.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- 172.21.15.62:0/804957247 wait complete. 2024-09-17T05:23:01.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 Processor -- start 2024-09-17T05:23:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.668+0000 7f1e32200640 1 -- start start 2024-09-17T05:23:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2c09ba60 0x7f1e2c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c16abd0 con 0x7f1e2c09ba60 2024-09-17T05:23:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c16aa50 con 0x7f1e2c0d0240 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2c16ad50 con 0x7f1e2c0d0650 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e2be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2c09ba60 0x7f1e2c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56836/0 (socket says 172.21.15.62:56836) 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e2be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:41380/0 (socket says 172.21.15.62:41380) 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 -- 172.21.15.62:0/369826845 learned_addr learned my addr 172.21.15.62:0/369826845 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 -- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 msgr2=0x7f1e2c0c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 -- 172.21.15.62:0/369826845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2c09ba60 msgr2=0x7f1e2c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2c09ba60 0x7f1e2c14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 -- 172.21.15.62:0/369826845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c167890 con 0x7f1e2c0d0240 2024-09-17T05:23:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e2be00640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23e00640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400c760 tx=0x7f1e1400cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14017070 con 0x7f1e2c0d0240 2024-09-17T05:23:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e14005360 con 0x7f1e2c0d0240 2024-09-17T05:23:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400a8a0 con 0x7f1e2c0d0240 2024-09-17T05:23:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c1672d0 con 0x7f1e2c0d0240 2024-09-17T05:23:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c0c7160 con 0x7f1e2c0d0240 2024-09-17T05:23:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e1401a070 con 0x7f1e2c0d0240 2024-09-17T05:23:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1e14003620 con 0x7f1e2c0d0240 2024-09-17T05:23:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.672+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c0b3130 con 0x7f1e2c0d0240 2024-09-17T05:23:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.676+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1e14003b20 con 0x7f1e2c0d0240 2024-09-17T05:23:01.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.748+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "359bdccb-91ae-428a-864b-50613ad48610", "id": 3} v 0) -- 0x7f1e2c076480 con 0x7f1e2c0d0240 2024-09-17T05:23:01.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e23400640 1 -- 172.21.15.62:0/369826845 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "359bdccb-91ae-428a-864b-50613ad48610", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1e14003d00 con 0x7f1e2c0d0240 2024-09-17T05:23:01.755 INFO:teuthology.orchestra.run.smithi062.stdout:3 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 msgr2=0x7f1e2c0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400c760 tx=0x7f1e1400cc20 comp rx=0 tx=0).stop 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 shutdown_connections 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e2c0d0240 0x7f1e2c0c7960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e2c0d0650 0x7f1e2c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 --2- 172.21.15.62:0/369826845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2c09ba60 0x7f1e2c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.752+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 >> 172.21.15.62:0/369826845 conn(0x7f1e2c0cee60 msgr2=0x7f1e2c0b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.756+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 shutdown_connections 2024-09-17T05:23:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.756+0000 7f1e32200640 1 -- 172.21.15.62:0/369826845 wait complete. 2024-09-17T05:23:01.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 5fcdbf08-53ee-4331-b1a9-c868a69c40f6 4 2024-09-17T05:23:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 -- 172.21.15.62:0/1506878238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040b0df0 msgr2=0x7f67040b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 --2- 172.21.15.62:0/1506878238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040b0df0 0x7f67040b1200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009c80 tx=0x7f66fc01cda0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 -- 172.21.15.62:0/1506878238 shutdown_connections 2024-09-17T05:23:01.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 --2- 172.21.15.62:0/1506878238 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040b0df0 0x7f67040b1200 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 --2- 172.21.15.62:0/1506878238 >> v2:172.21.15.43:3300/0 conn(0x7f67040d0730 0x7f67040ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 -- 172.21.15.62:0/1506878238 >> 172.21.15.62:0/1506878238 conn(0x7f67040b2f80 msgr2=0x7f67040b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 -- 172.21.15.62:0/1506878238 shutdown_connections 2024-09-17T05:23:01.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.928+0000 7f670ac00640 1 -- 172.21.15.62:0/1506878238 wait complete. 2024-09-17T05:23:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 Processor -- start 2024-09-17T05:23:01.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 -- start start 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67040c67a0 0x7f670415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67041690c0 con 0x7f67040b0df0 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6704168f40 con 0x7f67040c67a0 2024-09-17T05:23:01.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f670ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6704169240 con 0x7f67040d0730 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f6708c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f66fb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67040c67a0 0x7f670415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f6703e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f6708c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:41396/0 (socket says 172.21.15.62:41396) 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f6703e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.62:44448/0 (socket says 172.21.15.62:44448) 2024-09-17T05:23:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.932+0000 7f6708c00640 1 -- 172.21.15.62:0/1572443496 learned_addr learned my addr 172.21.15.62:0/1572443496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 -- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67040c67a0 msgr2=0x7f670415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67040c67a0 0x7f670415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 -- 172.21.15.62:0/1572443496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 msgr2=0x7f6704150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 -- 172.21.15.62:0/1572443496 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67040b6440 con 0x7f67040d0730 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6703e00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6708c00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009bd0 tx=0x7f66fc0092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0bb070 con 0x7f67040d0730 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66fc004d40 con 0x7f67040d0730 2024-09-17T05:23:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670415ed00 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0b2040 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f670406da20 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66fc0be070 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f66fc004220 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.936+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67040b1200 con 0x7f67040d0730 2024-09-17T05:23:01.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:01.940+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f66fc016990 con 0x7f67040d0730 2024-09-17T05:23:02.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.008+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5fcdbf08-53ee-4331-b1a9-c868a69c40f6", "id": 4} v 0) -- 0x7f670406f180 con 0x7f67040d0730 2024-09-17T05:23:02.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f6700c00640 1 -- 172.21.15.62:0/1572443496 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5fcdbf08-53ee-4331-b1a9-c868a69c40f6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f66fc004420 con 0x7f67040d0730 2024-09-17T05:23:02.018 INFO:teuthology.orchestra.run.smithi062.stdout:4 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 msgr2=0x7f670415e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009bd0 tx=0x7f66fc0092c0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 shutdown_connections 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67040c67a0 0x7f670415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67040d0730 0x7f670415e7c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 --2- 172.21.15.62:0/1572443496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67040b0df0 0x7f6704150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 >> 172.21.15.62:0/1572443496 conn(0x7f67040b2f80 msgr2=0x7f6704071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 shutdown_connections 2024-09-17T05:23:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.016+0000 7f670ac00640 1 -- 172.21.15.62:0/1572443496 wait complete. 2024-09-17T05:23:02.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 64597b7a-54bb-46e0-8d71-aebe6fc6a6b1 5 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- 172.21.15.62:0/3981489351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0240 msgr2=0x7f01280cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 --2- 172.21.15.62:0/3981489351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0240 0x7f01280cd040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0118009840 tx=0x7f011801c820 comp rx=0 tx=0).stop 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- 172.21.15.62:0/3981489351 shutdown_connections 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 --2- 172.21.15.62:0/3981489351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0240 0x7f01280cd040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- 172.21.15.62:0/3981489351 >> 172.21.15.62:0/3981489351 conn(0x7f01280cee60 msgr2=0x7f01280cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- 172.21.15.62:0/3981489351 shutdown_connections 2024-09-17T05:23:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- 172.21.15.62:0/3981489351 wait complete. 2024-09-17T05:23:02.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 Processor -- start 2024-09-17T05:23:02.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.196+0000 7f0130800640 1 -- start start 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f012809ba60 0x7f01280b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01280d0240 0x7f012814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f012816aae0 con 0x7f01280d0240 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f012816a960 con 0x7f01280d0650 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f012816ac60 con 0x7f012809ba60 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012de00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f012809ba60 0x7f01280b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56860/0 (socket says 172.21.15.62:56860) 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01280d0240 0x7f012814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 -- 172.21.15.62:0/3889489911 learned_addr learned my addr 172.21.15.62:0/3889489911 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 -- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f012809ba60 msgr2=0x7f01280b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f012809ba60 0x7f01280b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 -- 172.21.15.62:0/3889489911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01280d0240 msgr2=0x7f012814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01280d0240 0x7f012814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 -- 172.21.15.62:0/3889489911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01280b1170 con 0x7f01280d0650 2024-09-17T05:23:02.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f012e800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f012400cb40 tx=0x7f012400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:02.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124016020 con 0x7f01280d0650 2024-09-17T05:23:02.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012400ab40 con 0x7f01280d0650 2024-09-17T05:23:02.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01280c64f0 con 0x7f01280d0650 2024-09-17T05:23:02.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124012420 con 0x7f01280d0650 2024-09-17T05:23:02.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01280c6ab0 con 0x7f01280d0650 2024-09-17T05:23:02.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0124019020 con 0x7f01280d0650 2024-09-17T05:23:02.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f012401b400 con 0x7f01280d0650 2024-09-17T05:23:02.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.200+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01280b3130 con 0x7f01280d0650 2024-09-17T05:23:02.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.204+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0124020030 con 0x7f01280d0650 2024-09-17T05:23:02.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.276+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "64597b7a-54bb-46e0-8d71-aebe6fc6a6b1", "id": 5} v 0) -- 0x7f01280c6210 con 0x7f01280d0650 2024-09-17T05:23:02.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0122000640 1 -- 172.21.15.62:0/3889489911 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64597b7a-54bb-46e0-8d71-aebe6fc6a6b1", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f012401b9a0 con 0x7f01280d0650 2024-09-17T05:23:02.283 INFO:teuthology.orchestra.run.smithi062.stdout:5 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 msgr2=0x7f01281664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f012400cb40 tx=0x7f012400cc30 comp rx=0 tx=0).stop 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 shutdown_connections 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01280d0650 0x7f01281664f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01280d0240 0x7f012814e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 --2- 172.21.15.62:0/3889489911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f012809ba60 0x7f01280b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 >> 172.21.15.62:0/3889489911 conn(0x7f01280cee60 msgr2=0x7f01280b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 shutdown_connections 2024-09-17T05:23:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.280+0000 7f0130800640 1 -- 172.21.15.62:0/3889489911 wait complete. 2024-09-17T05:23:02.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new 42791e8e-8675-4f6d-b6ac-c5f4525fcb4b 6 2024-09-17T05:23:02.308 INFO:tasks.ceph.mgr.y.smithi043.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:23:02.308 INFO:tasks.ceph.mgr.y.smithi043.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:23:02.308 INFO:tasks.ceph.mgr.y.smithi043.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:23:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 -- 172.21.15.62:0/1819091849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 msgr2=0x7f20dc0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 --2- 172.21.15.62:0/1819091849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc0cd040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 -- 172.21.15.62:0/1819091849 shutdown_connections 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 --2- 172.21.15.62:0/1819091849 >> v2:172.21.15.43:3300/0 conn(0x7f20dc0d0650 0x7f20dc0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 --2- 172.21.15.62:0/1819091849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc0cd040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 -- 172.21.15.62:0/1819091849 >> 172.21.15.62:0/1819091849 conn(0x7f20dc0cee60 msgr2=0x7f20dc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 -- 172.21.15.62:0/1819091849 shutdown_connections 2024-09-17T05:23:02.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.464+0000 7f20e4000640 1 -- 172.21.15.62:0/1819091849 wait complete. 2024-09-17T05:23:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 Processor -- start 2024-09-17T05:23:02.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- start start 2024-09-17T05:23:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20dc0d0650 0x7f20dc1fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0b2ad0 con 0x7f20dc0d0650 2024-09-17T05:23:02.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0b2950 con 0x7f20dc09ba60 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc169e30 con 0x7f20dc0d0240 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.62:45694/0 (socket says 172.21.15.62:45694) 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40452/0 (socket says 172.21.15.62:40452) 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 -- 172.21.15.62:0/760950497 learned_addr learned my addr 172.21.15.62:0/760950497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e2000640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20dc0d0650 0x7f20dc1fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 -- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 msgr2=0x7f20dc0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 -- 172.21.15.62:0/760950497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20dc0d0650 msgr2=0x7f20dc1fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20dc0d0650 0x7f20dc1fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 -- 172.21.15.62:0/760950497 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc15cc90 con 0x7f20dc0d0240 2024-09-17T05:23:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e1600640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e0c00640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009850 tx=0x7f20c8005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80bc020 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c8005020 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8005450 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc15cf70 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1fc510 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.468+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20c80bf070 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.472+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f20c8004db0 con 0x7f20dc0d0240 2024-09-17T05:23:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.472+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc0b3130 con 0x7f20dc0d0240 2024-09-17T05:23:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.472+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f20c80164d0 con 0x7f20dc0d0240 2024-09-17T05:23:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.544+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd new", "uuid": "42791e8e-8675-4f6d-b6ac-c5f4525fcb4b", "id": 6} v 0) -- 0x7f20dc0b75f0 con 0x7f20dc0d0240 2024-09-17T05:23:02.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20d9600640 1 -- 172.21.15.62:0/760950497 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42791e8e-8675-4f6d-b6ac-c5f4525fcb4b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f20c80046d0 con 0x7f20dc0d0240 2024-09-17T05:23:02.552 INFO:teuthology.orchestra.run.smithi062.stdout:6 2024-09-17T05:23:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 msgr2=0x7f20dc1fca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009850 tx=0x7f20c8005e70 comp rx=0 tx=0).stop 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 shutdown_connections 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20dc0d0650 0x7f20dc1fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20dc0d0240 0x7f20dc1fca30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 --2- 172.21.15.62:0/760950497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20dc09ba60 0x7f20dc0b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 >> 172.21.15.62:0/760950497 conn(0x7f20dc0cee60 msgr2=0x7f20dc0ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 shutdown_connections 2024-09-17T05:23:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.552+0000 7f20e4000640 1 -- 172.21.15.62:0/760950497 wait complete. 2024-09-17T05:23:02.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd new a3e0f337-0f35-4867-b65c-47703989edb3 7 2024-09-17T05:23:02.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 -- 172.21.15.62:0/1162391991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 msgr2=0x7faf140cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 --2- 172.21.15.62:0/1162391991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf140cd750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf140cc8c0 tx=0x7faf0c01cb80 comp rx=0 tx=0).stop 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 -- 172.21.15.62:0/1162391991 shutdown_connections 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 --2- 172.21.15.62:0/1162391991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf140cd750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 --2- 172.21.15.62:0/1162391991 >> v2:172.21.15.43:3300/0 conn(0x7faf1409ba60 0x7faf140b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 -- 172.21.15.62:0/1162391991 >> 172.21.15.62:0/1162391991 conn(0x7faf140cee60 msgr2=0x7faf140cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 -- 172.21.15.62:0/1162391991 shutdown_connections 2024-09-17T05:23:02.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.728+0000 7faf19200640 1 -- 172.21.15.62:0/1162391991 wait complete. 2024-09-17T05:23:02.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 Processor -- start 2024-09-17T05:23:02.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- start start 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1409ba60 0x7faf14151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf140d0240 0x7faf14155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf140b2ad0 con 0x7faf1409ba60 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf140b2950 con 0x7faf140d0650 2024-09-17T05:23:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faf14169e30 con 0x7faf140d0240 2024-09-17T05:23:02.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf09400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf140d0240 0x7faf14155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf11e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1409ba60 0x7faf14151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:02.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40464/0 (socket says 172.21.15.62:40464) 2024-09-17T05:23:02.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 -- 172.21.15.62:0/2931896128 learned_addr learned my addr 172.21.15.62:0/2931896128 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-09-17T05:23:02.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 -- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf140d0240 msgr2=0x7faf14155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf140d0240 0x7faf14155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 -- 172.21.15.62:0/2931896128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1409ba60 msgr2=0x7faf14151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1409ba60 0x7faf14151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 -- 172.21.15.62:0/2931896128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141524d0 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf12800640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009b10 tx=0x7faf0c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0bc070 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf0c0042f0 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b2040 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141f9a80 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141fa040 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf0c0bf070 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.732+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7faf0c004720 con 0x7faf140d0650 2024-09-17T05:23:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.736+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf140b3130 con 0x7faf140d0650 2024-09-17T05:23:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.736+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faf0c016770 con 0x7faf140d0650 2024-09-17T05:23:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.808+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a3e0f337-0f35-4867-b65c-47703989edb3", "id": 7} v 0) -- 0x7faf14072e40 con 0x7faf140d0650 2024-09-17T05:23:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.812+0000 7faf0aa00640 1 -- 172.21.15.62:0/2931896128 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3e0f337-0f35-4867-b65c-47703989edb3", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7faf0c0c80b0 con 0x7faf140d0650 2024-09-17T05:23:02.815 INFO:teuthology.orchestra.run.smithi062.stdout:7 2024-09-17T05:23:02.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 msgr2=0x7faf14155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009b10 tx=0x7faf0c009260 comp rx=0 tx=0).stop 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 shutdown_connections 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf140d0650 0x7faf14155bb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf140d0240 0x7faf14155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 --2- 172.21.15.62:0/2931896128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1409ba60 0x7faf14151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 >> 172.21.15.62:0/2931896128 conn(0x7faf140cee60 msgr2=0x7faf140712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 shutdown_connections 2024-09-17T05:23:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:02.816+0000 7faf19200640 1 -- 172.21.15.62:0/2931896128 wait complete. 2024-09-17T05:23:02.836 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T05:23:02.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T05:23:02.840 INFO:tasks.ceph.osd.0:Started 2024-09-17T05:23:02.840 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T05:23:02.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T05:23:02.843 INFO:tasks.ceph.osd.1:Started 2024-09-17T05:23:02.843 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T05:23:02.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T05:23:02.846 INFO:tasks.ceph.osd.2:Started 2024-09-17T05:23:02.847 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T05:23:02.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T05:23:02.851 INFO:tasks.ceph.osd.3:Started 2024-09-17T05:23:02.851 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T05:23:02.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T05:23:02.854 INFO:tasks.ceph.osd.4:Started 2024-09-17T05:23:02.854 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T05:23:02.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T05:23:02.857 INFO:tasks.ceph.osd.5:Started 2024-09-17T05:23:02.858 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T05:23:02.858 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T05:23:02.861 INFO:tasks.ceph.osd.6:Started 2024-09-17T05:23:02.861 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T05:23:02.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T05:23:02.865 INFO:tasks.ceph.osd.7:Started 2024-09-17T05:23:02.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 -- 172.21.15.43:0/2520315715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00b6840 msgr2=0x7f98b00ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 --2- 172.21.15.43:0/2520315715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00b6840 0x7f98b00ccfa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f989c009840 tx=0x7f989c01c820 comp rx=0 tx=0).stop 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 -- 172.21.15.43:0/2520315715 shutdown_connections 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 --2- 172.21.15.43:0/2520315715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00cd4e0 0x7f98b00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 --2- 172.21.15.43:0/2520315715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00b6840 0x7f98b00ccfa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 -- 172.21.15.43:0/2520315715 >> 172.21.15.43:0/2520315715 conn(0x7f98b00b2e80 msgr2=0x7f98b00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:03.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 -- 172.21.15.43:0/2520315715 shutdown_connections 2024-09-17T05:23:03.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.072+0000 7f98b7200640 1 -- 172.21.15.43:0/2520315715 wait complete. 2024-09-17T05:23:03.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 Processor -- start 2024-09-17T05:23:03.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 -- start start 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 0x7f98b0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00cd4e0 0x7f98b0153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0168ce0 con 0x7f98b00b6840 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0168b60 con 0x7f98b00cd4e0 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b7200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f98b0168e60 con 0x7f98b00b45a0 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:03.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 0x7f98b0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:03.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34210/0 (socket says 172.21.15.43:34210) 2024-09-17T05:23:03.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98af400640 1 -- 172.21.15.43:0/3630154882 learned_addr learned my addr 172.21.15.43:0/3630154882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:03.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98b5200640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00cd4e0 0x7f98b0153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 -- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00cd4e0 msgr2=0x7f98b0153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00cd4e0 0x7f98b0153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 -- 172.21.15.43:0/3630154882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 msgr2=0x7f98b014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98afe00640 1 -- 172.21.15.43:0/3630154882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b0150820 con 0x7f98b00b45a0 2024-09-17T05:23:03.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.076+0000 7f98af400640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98afe00640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 0x7f98b0150a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c3c0 tx=0x7f98a000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0011790 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a0017070 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b00ae200 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0011ce0 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b00b18b0 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98a001c460 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f98a001b3c0 con 0x7f98b00b45a0 2024-09-17T05:23:03.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.080+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b00ccfa0 con 0x7f98b00b45a0 2024-09-17T05:23:03.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.084+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f98b0150820 con 0x7f98b00b45a0 2024-09-17T05:23:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.144+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98b00b7120 con 0x7f98b00b45a0 2024-09-17T05:23:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.144+0000 7f98a7e00640 1 -- 172.21.15.43:0/3630154882 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f98a0012740 con 0x7f98b00b45a0 2024-09-17T05:23:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:03.147 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:02.815265+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 msgr2=0x7f98b0150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 0x7f98b0150a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98a000c3c0 tx=0x7f98a000c880 comp rx=0 tx=0).stop 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 shutdown_connections 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00cd4e0 0x7f98b0153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98b00b6840 0x7f98b014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 --2- 172.21.15.43:0/3630154882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f98b00b45a0 0x7f98b0150a40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:03.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 >> 172.21.15.43:0/3630154882 conn(0x7f98b00b2e80 msgr2=0x7f98b0071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:03.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 shutdown_connections 2024-09-17T05:23:03.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:03.148+0000 7f98b7200640 1 -- 172.21.15.43:0/3630154882 wait complete. 2024-09-17T05:23:03.177 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T05:23:03.177 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T05:23:03.984 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:04.134 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 -- 172.21.15.43:0/1747484370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c06f410 msgr2=0x7f385c072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 --2- 172.21.15.43:0/1747484370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c06f410 0x7f385c072b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3844009840 tx=0x7f384401c6b0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 -- 172.21.15.43:0/1747484370 shutdown_connections 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 --2- 172.21.15.43:0/1747484370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c06f410 0x7f385c072b60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 -- 172.21.15.43:0/1747484370 >> 172.21.15.43:0/1747484370 conn(0x7f385c0d2460 msgr2=0x7f385c0d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 -- 172.21.15.43:0/1747484370 shutdown_connections 2024-09-17T05:23:04.135 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 -- 172.21.15.43:0/1747484370 wait complete. 2024-09-17T05:23:04.136 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.132+0000 7f3861400640 1 Processor -- start 2024-09-17T05:23:04.136 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- start start 2024-09-17T05:23:04.138 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c06f410 0x7f385c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:04.138 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c0730a0 0x7f385c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f385c160030 con 0x7f385c1627d0 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f385c15feb0 con 0x7f385c0730a0 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f385c1601b0 con 0x7f385c06f410 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49886/0 (socket says 172.21.15.43:49886) 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 -- 172.21.15.43:0/2390098015 learned_addr learned my addr 172.21.15.43:0/2390098015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385a800640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c06f410 0x7f385c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:04.139 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3851e00640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c0730a0 0x7f385c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:04.140 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 -- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c06f410 msgr2=0x7f385c157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:04.140 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c06f410 0x7f385c157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.140 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 -- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c0730a0 msgr2=0x7f385c15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:04.140 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c0730a0 0x7f385c15b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.141 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 -- 172.21.15.43:0/2390098015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0cb780 con 0x7f385c1627d0 2024-09-17T05:23:04.141 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f385b200640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f385400fc70 tx=0x7f385400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:04.142 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385401a070 con 0x7f385c1627d0 2024-09-17T05:23:04.142 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3854006aa0 con 0x7f385c1627d0 2024-09-17T05:23:04.142 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c0cba60 con 0x7f385c1627d0 2024-09-17T05:23:04.142 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854008070 con 0x7f385c1627d0 2024-09-17T05:23:04.142 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.136+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c0b5570 con 0x7f385c1627d0 2024-09-17T05:23:04.144 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.140+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7f385401d070 con 0x7f385c1627d0 2024-09-17T05:23:04.144 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.140+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c0ce160 con 0x7f385c1627d0 2024-09-17T05:23:04.145 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.140+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3854051a90 con 0x7f385c1627d0 2024-09-17T05:23:04.145 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.144+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3854036860 con 0x7f385c1627d0 2024-09-17T05:23:04.207 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.204+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f385c070180 con 0x7f385c1627d0 2024-09-17T05:23:04.208 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.204+0000 7f3853400640 1 -- 172.21.15.43:0/2390098015 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f385c0734b0 con 0x7f385c1627d0 2024-09-17T05:23:04.208 INFO:teuthology.misc.health.smithi043.stdout: 2024-09-17T05:23:04.208 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:02.815265+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 msgr2=0x7f385c0b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f385400fc70 tx=0x7f385400d430 comp rx=0 tx=0).stop 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 shutdown_connections 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f385c0730a0 0x7f385c15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f385c1627d0 0x7f385c0b3800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.209 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 --2- 172.21.15.43:0/2390098015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f385c06f410 0x7f385c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:04.210 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 >> 172.21.15.43:0/2390098015 conn(0x7f385c0d2460 msgr2=0x7f385c0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:04.210 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 shutdown_connections 2024-09-17T05:23:04.210 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:04.208+0000 7f3861400640 1 -- 172.21.15.43:0/2390098015 wait complete. 2024-09-17T05:23:04.226 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T05:23:04.486 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-09-17T05:23:04.484+0000 7fcdecb0b7c0 -1 Falling back to public interface 2024-09-17T05:23:04.538 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-09-17T05:23:04.536+0000 7f998855f7c0 -1 Falling back to public interface 2024-09-17T05:23:04.614 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-09-17T05:23:04.612+0000 7fd58a5fd7c0 -1 Falling back to public interface 2024-09-17T05:23:04.634 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-09-17T05:23:04.632+0000 7f93858047c0 -1 Falling back to public interface 2024-09-17T05:23:04.649 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-09-17T05:23:04.648+0000 7fc07193c7c0 -1 Falling back to public interface 2024-09-17T05:23:04.654 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-09-17T05:23:04.652+0000 7f94495a17c0 -1 Falling back to public interface 2024-09-17T05:23:04.822 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-17T05:23:04.820+0000 7ff1b20a27c0 -1 Falling back to public interface 2024-09-17T05:23:04.822 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-09-17T05:23:04.820+0000 7f83e3cf77c0 -1 Falling back to public interface 2024-09-17T05:23:04.846 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-09-17T05:23:04.844+0000 7ff01c200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T05:23:04.846 INFO:tasks.ceph.mgr.x.smithi062.stderr:2024-09-17T05:23:04.844+0000 7ff01c200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T05:23:08.709 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-09-17T05:23:08.708+0000 7f998855f7c0 -1 osd.6 0 log_to_monitors true 2024-09-17T05:23:08.843 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-09-17T05:23:08.840+0000 7fcdecb0b7c0 -1 osd.0 0 log_to_monitors true 2024-09-17T05:23:08.856 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-09-17T05:23:08.852+0000 7f94495a17c0 -1 osd.2 0 log_to_monitors true 2024-09-17T05:23:08.918 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-09-17T05:23:08.916+0000 7fc07193c7c0 -1 osd.5 0 log_to_monitors true 2024-09-17T05:23:08.974 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-17T05:23:08.972+0000 7ff1b20a27c0 -1 osd.3 0 log_to_monitors true 2024-09-17T05:23:09.050 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-09-17T05:23:09.048+0000 7f93858047c0 -1 osd.4 0 log_to_monitors true 2024-09-17T05:23:09.056 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-09-17T05:23:09.052+0000 7f83e3cf77c0 -1 osd.1 0 log_to_monitors true 2024-09-17T05:23:09.351 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-09-17T05:23:09.348+0000 7fd58a5fd7c0 -1 osd.7 0 log_to_monitors true 2024-09-17T05:23:10.956 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-09-17T05:23:10.952+0000 7f9983400640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T05:23:11.018 INFO:tasks.ceph.osd.6.smithi062.stderr:2024-09-17T05:23:11.016+0000 7f9979400640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:11.033 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:11.270 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.268+0000 7f30d2000640 1 -- 172.21.15.43:0/970996408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 msgr2=0x7f30cc0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:11.270 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.268+0000 7f30d2000640 1 --2- 172.21.15.43:0/970996408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc0ccad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009870 tx=0x7f30bc01c610 comp rx=0 tx=0).stop 2024-09-17T05:23:11.271 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.268+0000 7f30d2000640 1 -- 172.21.15.43:0/970996408 shutdown_connections 2024-09-17T05:23:11.271 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.268+0000 7f30d2000640 1 --2- 172.21.15.43:0/970996408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc0ccad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.271 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.268+0000 7f30d2000640 1 -- 172.21.15.43:0/970996408 >> 172.21.15.43:0/970996408 conn(0x7f30cc0694d0 msgr2=0x7f30cc0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:11.273 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.272+0000 7f30d2000640 1 -- 172.21.15.43:0/970996408 shutdown_connections 2024-09-17T05:23:11.273 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.272+0000 7f30d2000640 1 -- 172.21.15.43:0/970996408 wait complete. 2024-09-17T05:23:11.277 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 Processor -- start 2024-09-17T05:23:11.277 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 -- start start 2024-09-17T05:23:11.279 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:11.280 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc1655f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:11.281 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 0x7f30cc165b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:11.281 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc16ac40 con 0x7f30cc0d0c10 2024-09-17T05:23:11.281 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc16aac0 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.281 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d2000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc16adc0 con 0x7f30cc0c86f0 2024-09-17T05:23:11.281 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc1655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:60428/0 (socket says 172.21.15.43:60428) 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 -- 172.21.15.43:0/918742220 learned_addr learned my addr 172.21.15.43:0/918742220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d1600640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 0x7f30cc165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 -- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 msgr2=0x7f30cc1655f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc1655f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 -- 172.21.15.43:0/918742220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 msgr2=0x7f30cc165b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 0x7f30cc165b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30d0c00640 1 -- 172.21.15.43:0/918742220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc167110 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.276+0000 7f30c3e00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc1655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30d1600640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 0x7f30cc165b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:11.282 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30d0c00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc019220 tx=0x7f30bc0b16b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:11.283 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0b1b60 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.283 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30bc0bd070 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.283 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc003600 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.283 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1673f0 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.284 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.280+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1679b0 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.285 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.284+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a40052a0 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.290 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.288+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f30bc0b7070 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.291 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.288+0000 7f30c9600640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 0x7f30a80756d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:11.291 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.288+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f30bc12d6e0 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.292 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.288+0000 7f30c3e00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 0x7f30a80756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:11.292 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.288+0000 7f30c3e00640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 0x7f30a80756d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000fc0 tx=0x7f30b4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:11.304 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.300+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30bc0c7020 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.601 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30a4005590 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.602 INFO:teuthology.misc.health.smithi043.stdout: 2024-09-17T05:23:11.603 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":13,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:10.781937+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:11.603 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30c9600640 1 -- 172.21.15.43:0/918742220 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f30bc0ac030 con 0x7f30cc0b3cf0 2024-09-17T05:23:11.605 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 msgr2=0x7f30a80756d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:11.605 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 0x7f30a80756d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000fc0 tx=0x7f30b4009040 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 msgr2=0x7f30cc06f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30bc019220 tx=0x7f30bc0b16b0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 shutdown_connections 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a80731c0 0x7f30a80756d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30cc0d0c10 0x7f30cc165b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30cc0c86f0 0x7f30cc1655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 --2- 172.21.15.43:0/918742220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30cc0b3cf0 0x7f30cc06f240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.600+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 >> 172.21.15.43:0/918742220 conn(0x7f30cc0694d0 msgr2=0x7f30cc15ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.604+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 shutdown_connections 2024-09-17T05:23:11.606 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:11.604+0000 7f30d2000640 1 -- 172.21.15.43:0/918742220 wait complete. 2024-09-17T05:23:11.632 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T05:23:12.575 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-09-17T05:23:12.572+0000 7f83df600640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T05:23:12.606 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-09-17T05:23:12.604+0000 7f83d4c00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.632 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-09-17T05:23:12.628+0000 7f9444e00640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T05:23:12.640 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-09-17T05:23:12.636+0000 7fcde7a00640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T05:23:12.654 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-09-17T05:23:12.652+0000 7f943a400640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.654 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-17T05:23:12.652+0000 7ff1ad000640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T05:23:12.686 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-09-17T05:23:12.684+0000 7fcddda00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.688 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-09-17T05:23:12.684+0000 7f9380600640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T05:23:12.698 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-09-17T05:23:12.696+0000 7ff1a3000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.713 INFO:tasks.ceph.osd.4.smithi062.stderr:2024-09-17T05:23:12.712+0000 7f9376600640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.717 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-09-17T05:23:12.716+0000 7fc06c800640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T05:23:12.726 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-09-17T05:23:12.724+0000 7fd585e00640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T05:23:12.746 INFO:tasks.ceph.osd.5.smithi062.stderr:2024-09-17T05:23:12.744+0000 7fc062800640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:12.746 INFO:tasks.ceph.osd.7.smithi062.stderr:2024-09-17T05:23:12.744+0000 7fd57b400640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:23:18.440 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:18.603 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 -- 172.21.15.43:0/3577401287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 msgr2=0x7f4c60004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.603 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 --2- 172.21.15.43:0/3577401287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c60004070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c001890 tx=0x7f4c6c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.604 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c81a00640 1 -- 172.21.15.43:0/3577401287 reap_dead start 2024-09-17T05:23:18.604 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 -- 172.21.15.43:0/3577401287 shutdown_connections 2024-09-17T05:23:18.604 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 -- 172.21.15.43:0/3577401287 >> 172.21.15.43:0/3577401287 conn(0x7f4c7c06d490 msgr2=0x7f4c7c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:18.604 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 -- 172.21.15.43:0/3577401287 shutdown_connections 2024-09-17T05:23:18.604 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.600+0000 7f4c83a00640 1 -- 172.21.15.43:0/3577401287 wait complete. 2024-09-17T05:23:18.605 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 Processor -- start 2024-09-17T05:23:18.605 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- start start 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c14b010 0x7f4c7c0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c14a450 0x7f4c7c14a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1678a0 con 0x7f4c60003c20 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c167720 con 0x7f4c7c14b010 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c167a20 con 0x7f4c7c14a450 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44596/0 (socket says 172.21.15.43:44596) 2024-09-17T05:23:18.608 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 -- 172.21.15.43:0/129617955 learned_addr learned my addr 172.21.15.43:0/129617955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c14a450 0x7f4c7c14a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c73e00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c14b010 0x7f4c7c0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 -- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c14a450 msgr2=0x7f4c7c14a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c14a450 0x7f4c7c14a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 -- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c14b010 msgr2=0x7f4c7c0b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c14b010 0x7f4c7c0b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 -- 172.21.15.43:0/129617955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c073b70 con 0x7f4c60003c20 2024-09-17T05:23:18.609 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c81000640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c01cda0 tx=0x7f4c6c0a5da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0014b0 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c6c0b4030 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c006920 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c14ada0 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.604+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c15f300 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.608+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7c0b32f0 con 0x7f4c60003c20 2024-09-17T05:23:18.611 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.608+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4c6c006150 con 0x7f4c60003c20 2024-09-17T05:23:18.612 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.608+0000 7f4c79600640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 0x7f4c58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.612 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.608+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f4c6c12e630 con 0x7f4c60003c20 2024-09-17T05:23:18.612 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.608+0000 7f4c73e00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 0x7f4c58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:18.613 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.612+0000 7f4c73e00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 0x7f4c58075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c64000c10 tx=0x7f4c64001b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:18.616 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.612+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4c7c14ada0 con 0x7f4c60003c20 2024-09-17T05:23:18.797 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c7c0cf360 con 0x7f4c60003c20 2024-09-17T05:23:18.798 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c79600640 1 -- 172.21.15.43:0/129617955 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12836 (secure 0 0 0) 0x7f4c6c016670 con 0x7f4c60003c20 2024-09-17T05:23:18.798 INFO:teuthology.misc.health.smithi043.stdout: 2024-09-17T05:23:18.799 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":18,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:16.871155+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 msgr2=0x7f4c58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 0x7f4c58075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c64000c10 tx=0x7f4c64001b70 comp rx=0 tx=0).stop 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 msgr2=0x7f4c7c15edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c01cda0 tx=0x7f4c6c0a5da0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 shutdown_connections 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c58073200 0x7f4c58075710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.801 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c14b010 0x7f4c7c0b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.802 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c14a450 0x7f4c7c14a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.802 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 --2- 172.21.15.43:0/129617955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60003c20 0x7f4c7c15edc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.802 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 >> 172.21.15.43:0/129617955 conn(0x7f4c7c06d490 msgr2=0x7f4c7c073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:18.802 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 shutdown_connections 2024-09-17T05:23:18.802 INFO:teuthology.misc.health.smithi043.stderr:2024-09-17T05:23:18.796+0000 7f4c83a00640 1 -- 172.21.15.43:0/129617955 wait complete. 2024-09-17T05:23:18.819 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T05:23:18.819 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T05:23:18.820 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:23:18.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:23:18.823 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:23:18.823 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:23:18.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:23:18.826 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:23:18.826 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:23:18.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:23:18.830 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:23:18.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:23:18.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:23:18.833 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:23:18.833 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T05:23:18.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:18.956 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:23:18.956 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:23:18.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 -- 172.21.15.43:0/1774161946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 msgr2=0x7f1c9c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/1774161946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c0cd750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009840 tx=0x7f1c9401c820 comp rx=0 tx=0).stop 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 -- 172.21.15.43:0/1774161946 shutdown_connections 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/1774161946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c0cd750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 -- 172.21.15.43:0/1774161946 >> 172.21.15.43:0/1774161946 conn(0x7f1c9c0cee60 msgr2=0x7f1c9c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 -- 172.21.15.43:0/1774161946 shutdown_connections 2024-09-17T05:23:18.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 -- 172.21.15.43:0/1774161946 wait complete. 2024-09-17T05:23:18.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.992+0000 7f1ca2c00640 1 Processor -- start 2024-09-17T05:23:18.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- start start 2024-09-17T05:23:18.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9c09ba60 0x7f1c9c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:18.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c16aae0 con 0x7f1c9c0d0240 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c16a960 con 0x7f1c9c0d0650 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c16ac60 con 0x7f1c9c09ba60 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9c09ba60 0x7f1c9c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44606/0 (socket says 172.21.15.43:44606) 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 -- 172.21.15.43:0/595806292 learned_addr learned my addr 172.21.15.43:0/595806292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca0c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 -- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9c09ba60 msgr2=0x7f1c9c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9c09ba60 0x7f1c9c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 -- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 msgr2=0x7f1c9c15ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c15ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 -- 172.21.15.43:0/595806292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c165150 con 0x7f1c9c0d0240 2024-09-17T05:23:19.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c93400640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c390 tx=0x7f1c8400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:19.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84017070 con 0x7f1c9c0d0240 2024-09-17T05:23:19.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c840053c0 con 0x7f1c9c0d0240 2024-09-17T05:23:19.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84003840 con 0x7f1c9c0d0240 2024-09-17T05:23:19.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c165430 con 0x7f1c9c0d0240 2024-09-17T05:23:19.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:18.996+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1fbb20 con 0x7f1c9c0d0240 2024-09-17T05:23:19.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c0b3130 con 0x7f1c9c0d0240 2024-09-17T05:23:19.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1c8401a070 con 0x7f1c9c0d0240 2024-09-17T05:23:19.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1c98c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 0x7f1c78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:19.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1c840931c0 con 0x7f1c9c0d0240 2024-09-17T05:23:19.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1c9be00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 0x7f1c78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:19.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.000+0000 7f1c9be00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 0x7f1c78075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c004a40 tx=0x7f1c8c001a60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:19.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.008+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1c8405ded0 con 0x7f1c9c0d0240 2024-09-17T05:23:19.051 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:23:19.051 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:23:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.224+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c9c0c7f30 con 0x7f1c9c0d0240 2024-09-17T05:23:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.224+0000 7f1c98c00640 1 -- 172.21.15.43:0/595806292 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f1c84062900 con 0x7f1c9c0d0240 2024-09-17T05:23:19.228 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:19.228 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"btime":"2024-09-17T05:23:18:998378+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T05:23:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 msgr2=0x7f1c78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 0x7f1c78075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c004a40 tx=0x7f1c8c001a60 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 msgr2=0x7f1c9c0c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c390 tx=0x7f1c8400c850 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 shutdown_connections 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1c78073200 0x7f1c78075710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c9c0d0650 0x7f1c9c15ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c9c0d0240 0x7f1c9c0c74e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 --2- 172.21.15.43:0/595806292 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1c9c09ba60 0x7f1c9c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 >> 172.21.15.43:0/595806292 conn(0x7f1c9c0cee60 msgr2=0x7f1c9c0ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 shutdown_connections 2024-09-17T05:23:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:19.228+0000 7f1ca2c00640 1 -- 172.21.15.43:0/595806292 wait complete. 2024-09-17T05:23:20.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:23:20.330 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:23:21.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- 172.21.15.43:0/2766073625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d40730b0 msgr2=0x7f08d40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 --2- 172.21.15.43:0/2766073625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d40730b0 0x7f08d40734c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009bc0 tx=0x7f08cc01ca80 comp rx=0 tx=0).stop 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- 172.21.15.43:0/2766073625 shutdown_connections 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 --2- 172.21.15.43:0/2766073625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d40730b0 0x7f08d40734c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 --2- 172.21.15.43:0/2766073625 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d406fc90 0x7f08d40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- 172.21.15.43:0/2766073625 >> 172.21.15.43:0/2766073625 conn(0x7f08d40b23d0 msgr2=0x7f08d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- 172.21.15.43:0/2766073625 shutdown_connections 2024-09-17T05:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- 172.21.15.43:0/2766073625 wait complete. 2024-09-17T05:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 Processor -- start 2024-09-17T05:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.420+0000 7f08daa00640 1 -- start start 2024-09-17T05:23:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 0x7f08d40989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d415fe10 0x7f08d415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4169f20 con 0x7f08d40730b0 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4169da0 con 0x7f08d406fc90 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f08d416a0a0 con 0x7f08d415fe10 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44634/0 (socket says 172.21.15.43:44634) 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 -- 172.21.15.43:0/1738334163 learned_addr learned my addr 172.21.15.43:0/1738334163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d3e00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 0x7f08d40989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d8a00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d415fe10 0x7f08d415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 -- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d415fe10 msgr2=0x7f08d415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d415fe10 0x7f08d415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 -- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 msgr2=0x7f08d40989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 0x7f08d40989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 -- 172.21.15.43:0/1738334163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d4099500 con 0x7f08d40730b0 2024-09-17T05:23:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d3e00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 0x7f08d40989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08cbe00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e970 tx=0x7f08bc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00cd80 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d4099950 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08bc004020 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc004470 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d4156f50 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.424+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d40700a0 con 0x7f08d40730b0 2024-09-17T05:23:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.428+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08bc003600 con 0x7f08d40730b0 2024-09-17T05:23:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.428+0000 7f08d0c00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 0x7f08b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.428+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f08bc094b30 con 0x7f08d40730b0 2024-09-17T05:23:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.428+0000 7f08d3e00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 0x7f08b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.428+0000 7f08d3e00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 0x7f08b0075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08c4001400 tx=0x7f08c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.432+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08bc05fe00 con 0x7f08d40730b0 2024-09-17T05:23:21.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.640+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08d4159ff0 con 0x7f08d40730b0 2024-09-17T05:23:21.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.640+0000 7f08d0c00640 1 -- 172.21.15.43:0/1738334163 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7f08bc05f700 con 0x7f08d40730b0 2024-09-17T05:23:21.645 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:21.645 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"btime":"2024-09-17T05:23:20:000074+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T05:23:21.645 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 msgr2=0x7f08b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 0x7f08b0075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08c4001400 tx=0x7f08c4009040 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 msgr2=0x7f08d415f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e970 tx=0x7f08bc00ee30 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 shutdown_connections 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08b0073200 0x7f08b0075710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08d415fe10 0x7f08d415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d40730b0 0x7f08d415f110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 --2- 172.21.15.43:0/1738334163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08d406fc90 0x7f08d40989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 >> 172.21.15.43:0/1738334163 conn(0x7f08d40b23d0 msgr2=0x7f08d40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:21.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 shutdown_connections 2024-09-17T05:23:21.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.644+0000 7f08daa00640 1 -- 172.21.15.43:0/1738334163 wait complete. 2024-09-17T05:23:21.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- 172.21.15.43:0/1630094298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 msgr2=0x7f319806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 --2- 172.21.15.43:0/1630094298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f319806d270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3184009900 tx=0x7f318401c670 comp rx=0 tx=0).stop 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- 172.21.15.43:0/1630094298 shutdown_connections 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 --2- 172.21.15.43:0/1630094298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 0x7f31980cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 --2- 172.21.15.43:0/1630094298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f319806d270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- 172.21.15.43:0/1630094298 >> 172.21.15.43:0/1630094298 conn(0x7f31980b23d0 msgr2=0x7f31980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- 172.21.15.43:0/1630094298 shutdown_connections 2024-09-17T05:23:21.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- 172.21.15.43:0/1630094298 wait complete. 2024-09-17T05:23:21.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 Processor -- start 2024-09-17T05:23:21.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.824+0000 7f31a0400640 1 -- start start 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3198070620 0x7f319809db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 0x7f31980c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31980ce630 con 0x7f319806ce60 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31980ce4b0 con 0x7f3198070620 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f31980ce7b0 con 0x7f31980cb860 2024-09-17T05:23:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44652/0 (socket says 172.21.15.43:44652) 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 -- 172.21.15.43:0/3157981862 learned_addr learned my addr 172.21.15.43:0/3157981862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319e400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 0x7f31980c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319d000640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3198070620 0x7f319809db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 -- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 msgr2=0x7f31980c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 0x7f31980c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 -- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3198070620 msgr2=0x7f319809db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3198070620 0x7f319809db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 -- 172.21.15.43:0/3157981862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31980c5a80 con 0x7f319806ce60 2024-09-17T05:23:21.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f319da00640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f318c00b470 tx=0x7f318c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c015030 con 0x7f319806ce60 2024-09-17T05:23:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f318c004020 con 0x7f319806ce60 2024-09-17T05:23:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c004470 con 0x7f319806ce60 2024-09-17T05:23:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31980c44a0 con 0x7f319806ce60 2024-09-17T05:23:21.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31980c4a60 con 0x7f319806ce60 2024-09-17T05:23:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.828+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31980ced80 con 0x7f319806ce60 2024-09-17T05:23:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.832+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f318c018070 con 0x7f319806ce60 2024-09-17T05:23:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.832+0000 7f3195600640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 0x7f3178075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.832+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f318c093b90 con 0x7f319806ce60 2024-09-17T05:23:21.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.832+0000 7f319d000640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 0x7f3178075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:21.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.836+0000 7f319d000640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 0x7f3178075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31840af000 tx=0x7f318401aeb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:21.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:21.836+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f318c00d040 con 0x7f319806ce60 2024-09-17T05:23:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.012+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f31980cfcf0 con 0x7f319806ce60 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f3195600640 1 -- 172.21.15.43:0/3157981862 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f318c05e8a0 con 0x7f319806ce60 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:23:22.017 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:23:22.018 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:23:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 msgr2=0x7f3178075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 0x7f3178075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31840af000 tx=0x7f318401aeb0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 msgr2=0x7f31980cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f318c00b470 tx=0x7f318c00b930 comp rx=0 tx=0).stop 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 shutdown_connections 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3178073230 0x7f3178075740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3198070620 0x7f319809db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31980cb860 0x7f31980c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 --2- 172.21.15.43:0/3157981862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f319806ce60 0x7f31980cd560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 >> 172.21.15.43:0/3157981862 conn(0x7f31980b23d0 msgr2=0x7f31980b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 shutdown_connections 2024-09-17T05:23:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.016+0000 7f31a0400640 1 -- 172.21.15.43:0/3157981862 wait complete. 2024-09-17T05:23:22.039 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:23:22.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:23:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- 172.21.15.43:0/3712334489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea900cf980 msgr2=0x7fea900b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 --2- 172.21.15.43:0/3712334489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea900cf980 0x7fea900b3b80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- 172.21.15.43:0/3712334489 shutdown_connections 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 --2- 172.21.15.43:0/3712334489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9015b7e0 0x7fea900731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 --2- 172.21.15.43:0/3712334489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea900cf980 0x7fea900b3b80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- 172.21.15.43:0/3712334489 >> 172.21.15.43:0/3712334489 conn(0x7fea9006ce60 msgr2=0x7fea9006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- 172.21.15.43:0/3712334489 shutdown_connections 2024-09-17T05:23:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- 172.21.15.43:0/3712334489 wait complete. 2024-09-17T05:23:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 Processor -- start 2024-09-17T05:23:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.200+0000 7fea95000640 1 -- start start 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90074c60 0x7fea90154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9014bb30 0x7fea9015f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea90168ef0 con 0x7fea9015b7e0 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fea90168d70 con 0x7fea90074c60 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fea90169070 con 0x7fea9014bb30 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44678/0 (socket says 172.21.15.43:44678) 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 -- 172.21.15.43:0/2315211848 learned_addr learned my addr 172.21.15.43:0/2315211848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea8de00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90074c60 0x7fea90154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea8e800640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9014bb30 0x7fea9015f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 -- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9014bb30 msgr2=0x7fea9015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9014bb30 0x7fea9015f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 -- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90074c60 msgr2=0x7fea90154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90074c60 0x7fea90154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 -- 172.21.15.43:0/2315211848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9014b860 con 0x7fea9015b7e0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea87e00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea7801af90 tx=0x7fea780a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780bc070 con 0x7fea9015b7e0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea78004430 con 0x7fea9015b7e0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901647f0 con 0x7fea9015b7e0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780b2040 con 0x7fea9015b7e0 2024-09-17T05:23:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901551e0 con 0x7fea9015b7e0 2024-09-17T05:23:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.204+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9015bdc0 con 0x7fea9015b7e0 2024-09-17T05:23:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.208+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea78005280 con 0x7fea9015b7e0 2024-09-17T05:23:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.208+0000 7fea86000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 0x7fea6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.208+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fea7812dfb0 con 0x7fea9015b7e0 2024-09-17T05:23:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.208+0000 7fea8de00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 0x7fea6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.208+0000 7fea8de00640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 0x7fea6c075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:22.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.212+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea9015bdc0 con 0x7fea9015b7e0 2024-09-17T05:23:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:22.388+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fea900b3b60 con 0x7fea9015b7e0 2024-09-17T05:23:23.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.016+0000 7fea86000640 1 -- 172.21.15.43:0/2315211848 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fea780164d0 con 0x7fea9015b7e0 2024-09-17T05:23:23.021 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:23:23.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 msgr2=0x7fea6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 0x7fea6c075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea80007fd0 tx=0x7fea80009040 comp rx=0 tx=0).stop 2024-09-17T05:23:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 msgr2=0x7fea9014b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea7801af90 tx=0x7fea780a5780 comp rx=0 tx=0).stop 2024-09-17T05:23:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 shutdown_connections 2024-09-17T05:23:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea6c073200 0x7fea6c075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea9014bb30 0x7fea9015f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea9015b7e0 0x7fea9014b320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 --2- 172.21.15.43:0/2315211848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea90074c60 0x7fea90154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 >> 172.21.15.43:0/2315211848 conn(0x7fea9006ce60 msgr2=0x7fea900ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 shutdown_connections 2024-09-17T05:23:23.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.020+0000 7fea95000640 1 -- 172.21.15.43:0/2315211848 wait complete. 2024-09-17T05:23:23.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:23:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 -- 172.21.15.43:0/1621748715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b4155d70 msgr2=0x7fe3b415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 --2- 172.21.15.43:0/1621748715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b4155d70 0x7fe3b415c7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009900 tx=0x7fe3a401c820 comp rx=0 tx=0).stop 2024-09-17T05:23:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 -- 172.21.15.43:0/1621748715 shutdown_connections 2024-09-17T05:23:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 --2- 172.21.15.43:0/1621748715 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b406f110 0x7fe3b40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 --2- 172.21.15.43:0/1621748715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b4155d70 0x7fe3b415c7e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 -- 172.21.15.43:0/1621748715 >> 172.21.15.43:0/1621748715 conn(0x7fe3b40b23d0 msgr2=0x7fe3b40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 -- 172.21.15.43:0/1621748715 shutdown_connections 2024-09-17T05:23:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.216+0000 7fe3b9000640 1 -- 172.21.15.43:0/1621748715 wait complete. 2024-09-17T05:23:23.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 Processor -- start 2024-09-17T05:23:23.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- start start 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 0x7fe3b4153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b4155d70 0x7fe3b4154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4168a00 con 0x7fe3b406f110 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4168880 con 0x7fe3b40b3920 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b4168b80 con 0x7fe3b4155d70 2024-09-17T05:23:23.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44688/0 (socket says 172.21.15.43:44688) 2024-09-17T05:23:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 -- 172.21.15.43:0/3733360829 learned_addr learned my addr 172.21.15.43:0/3733360829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b2800640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b4155d70 0x7fe3b4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3a9400640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 0x7fe3b4153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:23.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 -- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b4155d70 msgr2=0x7fe3b4154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:23.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b4155d70 0x7fe3b4154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 -- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 msgr2=0x7fe3b4153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 0x7fe3b4153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 -- 172.21.15.43:0/3733360829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b409bff0 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3a9400640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 0x7fe3b4153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b1e00640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40b0040 tx=0x7fe3a40a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a40b2040 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3a4004230 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4005020 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b40cee90 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.220+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b40ca010 con 0x7fe3b406f110 2024-09-17T05:23:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b40b33e0 con 0x7fe3b406f110 2024-09-17T05:23:23.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3a40bd030 con 0x7fe3b406f110 2024-09-17T05:23:23.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3aaa00640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 0x7fe390075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:23.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fe3a412dd00 con 0x7fe3b406f110 2024-09-17T05:23:23.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3a9400640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 0x7fe390075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:23.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.224+0000 7fe3a9400640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 0x7fe390075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4153d00 tx=0x7fe39c00b5d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.228+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3a40c4060 con 0x7fe3b406f110 2024-09-17T05:23:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:23.404+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3b415d160 con 0x7fe3b406f110 2024-09-17T05:23:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.024+0000 7fe3aaa00640 1 -- 172.21.15.43:0/3733360829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7fe3a40164d0 con 0x7fe3b406f110 2024-09-17T05:23:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:23:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.028+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 msgr2=0x7fe390075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.028+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 0x7fe390075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4153d00 tx=0x7fe39c00b5d0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.028+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 msgr2=0x7fe3b409bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.028+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40b0040 tx=0x7fe3a40a57b0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 shutdown_connections 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe390073230 0x7fe390075740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b40b3920 0x7fe3b4153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b4155d70 0x7fe3b4154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 --2- 172.21.15.43:0/3733360829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b406f110 0x7fe3b409bab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 >> 172.21.15.43:0/3733360829 conn(0x7fe3b40b23d0 msgr2=0x7fe3b415ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 shutdown_connections 2024-09-17T05:23:24.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.032+0000 7fe3b9000640 1 -- 172.21.15.43:0/3733360829 wait complete. 2024-09-17T05:23:24.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- 172.21.15.43:0/2683311637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840d0f50 msgr2=0x7fc7840cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 --2- 172.21.15.43:0/2683311637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840d0f50 0x7fc7840cce70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc78409bbe0 tx=0x7fc76c01c820 comp rx=0 tx=0).stop 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- 172.21.15.43:0/2683311637 shutdown_connections 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 --2- 172.21.15.43:0/2683311637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840d0f50 0x7fc7840cce70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- 172.21.15.43:0/2683311637 >> 172.21.15.43:0/2683311637 conn(0x7fc7840cee80 msgr2=0x7fc7840cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- 172.21.15.43:0/2683311637 shutdown_connections 2024-09-17T05:23:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- 172.21.15.43:0/2683311637 wait complete. 2024-09-17T05:23:24.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 Processor -- start 2024-09-17T05:23:24.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.240+0000 7fc78ae00640 1 -- start start 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840cd3b0 0x7fc7840cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7840d0f50 0x7fc7840cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc78416b030 con 0x7fc7840b7ce0 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc78416aeb0 con 0x7fc7840cd3b0 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc78416b1b0 con 0x7fc7840d0f50 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49252/0 (socket says 172.21.15.43:49252) 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 -- 172.21.15.43:0/1532033085 learned_addr learned my addr 172.21.15.43:0/1532033085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc77be00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840cd3b0 0x7fc7840cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc788e00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7840d0f50 0x7fc7840cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 -- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7840d0f50 msgr2=0x7fc7840cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7840d0f50 0x7fc7840cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 -- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840cd3b0 msgr2=0x7fc7840cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840cd3b0 0x7fc7840cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 -- 172.21.15.43:0/1532033085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc784157a10 con 0x7fc7840b7ce0 2024-09-17T05:23:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc783e00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc77400c470 tx=0x7fc774001ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774017070 con 0x7fc7840b7ce0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7740053c0 con 0x7fc7840b7ce0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7840caed0 con 0x7fc7840b7ce0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740038b0 con 0x7fc7840b7ce0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc784154910 con 0x7fc7840b7ce0 2024-09-17T05:23:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.244+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7840cce70 con 0x7fc7840b7ce0 2024-09-17T05:23:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.248+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc77401a070 con 0x7fc7840b7ce0 2024-09-17T05:23:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.248+0000 7fc780c00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 0x7fc760075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.248+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fc7740938c0 con 0x7fc7840b7ce0 2024-09-17T05:23:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.248+0000 7fc77be00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 0x7fc760075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.248+0000 7fc77be00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 0x7fc760075740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0af000 tx=0x7fc76c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.252+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc774021020 con 0x7fc7840b7ce0 2024-09-17T05:23:24.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:24.464+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc784072180 con 0x7fc7840b7ce0 2024-09-17T05:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.028+0000 7fc780c00640 1 -- 172.21.15.43:0/1532033085 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fc77405e360 con 0x7fc7840b7ce0 2024-09-17T05:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T05:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 msgr2=0x7fc760075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 0x7fc760075740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0af000 tx=0x7fc76c0b2040 comp rx=0 tx=0).stop 2024-09-17T05:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 msgr2=0x7fc7841543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc77400c470 tx=0x7fc774001ed0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 shutdown_connections 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc760073230 0x7fc760075740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7840cd3b0 0x7fc7840cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc7840d0f50 0x7fc7840cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 --2- 172.21.15.43:0/1532033085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7840b7ce0 0x7fc7841543d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 >> 172.21.15.43:0/1532033085 conn(0x7fc7840cee80 msgr2=0x7fc784155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.032+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 shutdown_connections 2024-09-17T05:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.036+0000 7fc78ae00640 1 -- 172.21.15.43:0/1532033085 wait complete. 2024-09-17T05:23:25.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- 172.21.15.43:0/1192733903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c0155d70 msgr2=0x7fd1c015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 --2- 172.21.15.43:0/1192733903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c0155d70 0x7fd1c015c7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009bc0 tx=0x7fd1b001cae0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- 172.21.15.43:0/1192733903 shutdown_connections 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 --2- 172.21.15.43:0/1192733903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c00b3af0 0x7fd1c00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 --2- 172.21.15.43:0/1192733903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c0155d70 0x7fd1c015c7e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- 172.21.15.43:0/1192733903 >> 172.21.15.43:0/1192733903 conn(0x7fd1c00b23d0 msgr2=0x7fd1c00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- 172.21.15.43:0/1192733903 shutdown_connections 2024-09-17T05:23:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- 172.21.15.43:0/1192733903 wait complete. 2024-09-17T05:23:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 Processor -- start 2024-09-17T05:23:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.240+0000 7fd1c5000640 1 -- start start 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 0x7fd1c0165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c009d760 0x7fd1c00c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0168bc0 con 0x7fd1c0155d70 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0168a40 con 0x7fd1c006f110 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c0168d40 con 0x7fd1c009d760 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1bde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 0x7fd1c0165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49256/0 (socket says 172.21.15.43:49256) 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 -- 172.21.15.43:0/673556512 learned_addr learned my addr 172.21.15.43:0/673556512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1be800640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c009d760 0x7fd1c00c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 -- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c009d760 msgr2=0x7fd1c00c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c009d760 0x7fd1c00c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 -- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 msgr2=0x7fd1c0165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 0x7fd1c0165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 -- 172.21.15.43:0/673556512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c00c78d0 con 0x7fd1c0155d70 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1bde00640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 0x7fd1c0165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b5400640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c430 tx=0x7fd1a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011790 con 0x7fd1c0155d70 2024-09-17T05:23:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c00c9af0 con 0x7fd1c0155d70 2024-09-17T05:23:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a8017070 con 0x7fd1c0155d70 2024-09-17T05:23:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011cc0 con 0x7fd1c0155d70 2024-09-17T05:23:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c00ca0b0 con 0x7fd1c0155d70 2024-09-17T05:23:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.244+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c00c7640 con 0x7fd1c0155d70 2024-09-17T05:23:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.248+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1a801c460 con 0x7fd1c0155d70 2024-09-17T05:23:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.248+0000 7fd1b6a00640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 0x7fd19c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.248+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd1a8093ce0 con 0x7fd1c0155d70 2024-09-17T05:23:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.248+0000 7fd1bde00640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 0x7fd19c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.248+0000 7fd1bde00640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 0x7fd19c075740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00af000 tx=0x7fd1b001b320 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.252+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1a8021020 con 0x7fd1c0155d70 2024-09-17T05:23:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.428+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1c00ad0b0 con 0x7fd1c0155d70 2024-09-17T05:23:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.428+0000 7fd1b6a00640 1 -- 172.21.15.43:0/673556512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16622 (secure 0 0 0) 0x7fd1a805e620 con 0x7fd1c0155d70 2024-09-17T05:23:25.433 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:25.433 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:24.470677+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 msgr2=0x7fd19c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 0x7fd19c075740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00af000 tx=0x7fd1b001b320 comp rx=0 tx=0).stop 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 msgr2=0x7fd1c00c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c430 tx=0x7fd1a800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 shutdown_connections 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd19c073230 0x7fd19c075740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1c009d760 0x7fd1c00c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0155d70 0x7fd1c00c5f00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 --2- 172.21.15.43:0/673556512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1c006f110 0x7fd1c0165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 >> 172.21.15.43:0/673556512 conn(0x7fd1c00b23d0 msgr2=0x7fd1c015d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 shutdown_connections 2024-09-17T05:23:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.432+0000 7fd1c5000640 1 -- 172.21.15.43:0/673556512 wait complete. 2024-09-17T05:23:25.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- 172.21.15.43:0/3144157621 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640737c0 msgr2=0x7fd8640700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 --2- 172.21.15.43:0/3144157621 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640737c0 0x7fd8640700a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009c20 tx=0x7fd85c01caa0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- 172.21.15.43:0/3144157621 shutdown_connections 2024-09-17T05:23:25.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 --2- 172.21.15.43:0/3144157621 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640737c0 0x7fd8640700a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- 172.21.15.43:0/3144157621 >> 172.21.15.43:0/3144157621 conn(0x7fd8640b23d0 msgr2=0x7fd8640d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- 172.21.15.43:0/3144157621 shutdown_connections 2024-09-17T05:23:25.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- 172.21.15.43:0/3144157621 wait complete. 2024-09-17T05:23:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 Processor -- start 2024-09-17T05:23:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.612+0000 7fd86b000640 1 -- start start 2024-09-17T05:23:25.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86415fe10 0x7fd864158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640c0c30 0x7fd864159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd86416b640 con 0x7fd86406fc90 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd86416b4c0 con 0x7fd86415fe10 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd86416b7c0 con 0x7fd8640c0c30 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49264/0 (socket says 172.21.15.43:49264) 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 -- 172.21.15.43:0/3584578986 learned_addr learned my addr 172.21.15.43:0/3584578986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd869000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640c0c30 0x7fd864159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd85b400640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86415fe10 0x7fd864158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 -- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640c0c30 msgr2=0x7fd864159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640c0c30 0x7fd864159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 -- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86415fe10 msgr2=0x7fd864158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86415fe10 0x7fd864158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 -- 172.21.15.43:0/3584578986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8640c3580 con 0x7fd86406fc90 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd863e00640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c400 tx=0x7fd85400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854017070 con 0x7fd86406fc90 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8640c27c0 con 0x7fd86406fc90 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd854005400 con 0x7fd86406fc90 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854003860 con 0x7fd86406fc90 2024-09-17T05:23:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8640c2d80 con 0x7fd86406fc90 2024-09-17T05:23:25.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.616+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8640aeb70 con 0x7fd86406fc90 2024-09-17T05:23:25.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.620+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd85401a070 con 0x7fd86406fc90 2024-09-17T05:23:25.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.620+0000 7fd860c00640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 0x7fd840075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.620+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd854093e90 con 0x7fd86406fc90 2024-09-17T05:23:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.620+0000 7fd85b400640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 0x7fd840075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.620+0000 7fd85b400640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 0x7fd840075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.624+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd85405e840 con 0x7fd86406fc90 2024-09-17T05:23:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8640ae890 con 0x7fd86406fc90 2024-09-17T05:23:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd860c00640 1 -- 172.21.15.43:0/3584578986 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16622 (secure 0 0 0) 0x7fd854063270 con 0x7fd86406fc90 2024-09-17T05:23:25.803 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:25.804 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:24.470677+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:25.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 msgr2=0x7fd840075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 0x7fd840075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2024-09-17T05:23:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 msgr2=0x7fd86415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c400 tx=0x7fd85400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 shutdown_connections 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd840073200 0x7fd840075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd86415fe10 0x7fd864158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8640c0c30 0x7fd864159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 --2- 172.21.15.43:0/3584578986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd86406fc90 0x7fd86415a860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 >> 172.21.15.43:0/3584578986 conn(0x7fd8640b23d0 msgr2=0x7fd8640b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.800+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 shutdown_connections 2024-09-17T05:23:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.804+0000 7fd86b000640 1 -- 172.21.15.43:0/3584578986 wait complete. 2024-09-17T05:23:25.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 -- 172.21.15.43:0/3545081345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 msgr2=0x7fd4880d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 --2- 172.21.15.43:0/3545081345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd4880d0a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4880ce010 tx=0x7fd48001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 -- 172.21.15.43:0/3545081345 shutdown_connections 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 --2- 172.21.15.43:0/3545081345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd4880d0a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 --2- 172.21.15.43:0/3545081345 >> v2:172.21.15.62:3300/0 conn(0x7fd4880d28a0 0x7fd4880d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 -- 172.21.15.43:0/3545081345 >> 172.21.15.43:0/3545081345 conn(0x7fd4880691b0 msgr2=0x7fd4880695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 -- 172.21.15.43:0/3545081345 shutdown_connections 2024-09-17T05:23:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.984+0000 7fd487e00640 1 -- 172.21.15.43:0/3545081345 wait complete. 2024-09-17T05:23:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 Processor -- start 2024-09-17T05:23:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- start start 2024-09-17T05:23:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 0x7fd48815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488154960 0x7fd48813f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd488169a50 con 0x7fd4880d05f0 2024-09-17T05:23:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880b80d0 con 0x7fd488154960 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd488169bd0 con 0x7fd48815c610 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49284/0 (socket says 172.21.15.43:49284) 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 -- 172.21.15.43:0/2926364835 learned_addr learned my addr 172.21.15.43:0/2926364835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487400640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488154960 0x7fd48813f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd47e000640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 0x7fd48815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 -- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 msgr2=0x7fd48815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 0x7fd48815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 -- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488154960 msgr2=0x7fd48813f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488154960 0x7fd48813f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 -- 172.21.15.43:0/2926364835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4880b45f0 con 0x7fd4880d05f0 2024-09-17T05:23:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd47e000640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 0x7fd48815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd486a00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c3f0 tx=0x7fd47800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780052d0 con 0x7fd4880d05f0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd478007020 con 0x7fd4880d05f0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478007470 con 0x7fd4880d05f0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4880b48d0 con 0x7fd4880d05f0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.988+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48815a6e0 con 0x7fd4880d05f0 2024-09-17T05:23:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.992+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4880d0a00 con 0x7fd4880d05f0 2024-09-17T05:23:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.992+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4780067e0 con 0x7fd4880d05f0 2024-09-17T05:23:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.992+0000 7fd47f400640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 0x7fd464075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.992+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd478093720 con 0x7fd4880d05f0 2024-09-17T05:23:25.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.992+0000 7fd47e000640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 0x7fd464075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.996+0000 7fd47e000640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 0x7fd464075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b6370 tx=0x7fd4700091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:26.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:25.996+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd478001090 con 0x7fd4880d05f0 2024-09-17T05:23:26.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:26.212+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd48806a1e0 con 0x7fd4880d05f0 2024-09-17T05:23:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd47f400640 1 -- 172.21.15.43:0/2926364835 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fd47805e050 con 0x7fd4880d05f0 2024-09-17T05:23:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 msgr2=0x7fd464075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 0x7fd464075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b6370 tx=0x7fd4700091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 msgr2=0x7fd488152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c3f0 tx=0x7fd47800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 shutdown_connections 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd464073200 0x7fd464075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd488154960 0x7fd48813f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd48815c610 0x7fd48815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 --2- 172.21.15.43:0/2926364835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4880d05f0 0x7fd488152d10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 >> 172.21.15.43:0/2926364835 conn(0x7fd4880691b0 msgr2=0x7fd488158570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 shutdown_connections 2024-09-17T05:23:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.052+0000 7fd487e00640 1 -- 172.21.15.43:0/2926364835 wait complete. 2024-09-17T05:23:27.081 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:23:27.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:23:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 -- 172.21.15.43:0/827651969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e415d4c0 msgr2=0x7f55e40d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 --2- 172.21.15.43:0/827651969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e415d4c0 0x7f55e40d21a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009900 tx=0x7f55cc01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 -- 172.21.15.43:0/827651969 shutdown_connections 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 --2- 172.21.15.43:0/827651969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e415d4c0 0x7f55e40d21a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 --2- 172.21.15.43:0/827651969 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e40ccf00 0x7f55e41601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 -- 172.21.15.43:0/827651969 >> 172.21.15.43:0/827651969 conn(0x7f55e40691b0 msgr2=0x7f55e40695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 -- 172.21.15.43:0/827651969 shutdown_connections 2024-09-17T05:23:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.244+0000 7f55e9600640 1 -- 172.21.15.43:0/827651969 wait complete. 2024-09-17T05:23:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 Processor -- start 2024-09-17T05:23:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- start start 2024-09-17T05:23:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 0x7f55e414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 0x7f55e414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55e416a340 con 0x7f55e40ccf00 2024-09-17T05:23:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4153a40 con 0x7f55e415d4c0 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f55e416a4c0 con 0x7f55e415d8d0 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49310/0 (socket says 172.21.15.43:49310) 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 -- 172.21.15.43:0/360705793 learned_addr learned my addr 172.21.15.43:0/360705793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55dbe00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 0x7f55e414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 -- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 msgr2=0x7f55e414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e8c00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 0x7f55e414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 0x7f55e414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 -- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 msgr2=0x7f55e414cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 0x7f55e414cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 -- 172.21.15.43:0/360705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e40b51c0 con 0x7f55e40ccf00 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55dbe00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 0x7f55e414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e8c00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 0x7f55e414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e3e00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d400d750 tx=0x7f55d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4017070 con 0x7f55e40ccf00 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55d4009cc0 con 0x7f55e40ccf00 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e40b54a0 con 0x7f55e40ccf00 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005280 con 0x7f55e40ccf00 2024-09-17T05:23:27.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.248+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e40c6120 con 0x7f55e40ccf00 2024-09-17T05:23:27.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e41601a0 con 0x7f55e40ccf00 2024-09-17T05:23:27.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55d4005420 con 0x7f55e40ccf00 2024-09-17T05:23:27.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55e0c00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:27.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55d4094310 con 0x7f55e40ccf00 2024-09-17T05:23:27.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55dbe00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:27.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.252+0000 7f55dbe00640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 0x7f55c0075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0af000 tx=0x7f55cc01b0f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:27.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.256+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f55d400de60 con 0x7f55e40ccf00 2024-09-17T05:23:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:27.464+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f55e406a1e0 con 0x7f55e40ccf00 2024-09-17T05:23:28.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.056+0000 7f55e0c00640 1 -- 172.21.15.43:0/360705793 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f55d405f320 con 0x7f55e40ccf00 2024-09-17T05:23:28.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 msgr2=0x7f55c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 0x7f55c0075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0af000 tx=0x7f55cc01b0f0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 msgr2=0x7f55e415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d400d750 tx=0x7f55d400dc10 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 shutdown_connections 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55c0073200 0x7f55c0075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55e415d4c0 0x7f55e414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55e415d8d0 0x7f55e414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 --2- 172.21.15.43:0/360705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55e40ccf00 0x7f55e415df20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 >> 172.21.15.43:0/360705793 conn(0x7f55e40691b0 msgr2=0x7f55e40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:28.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 shutdown_connections 2024-09-17T05:23:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.060+0000 7f55e9600640 1 -- 172.21.15.43:0/360705793 wait complete. 2024-09-17T05:23:28.087 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:23:28.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 -- 172.21.15.43:0/1868425375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b24155d70 msgr2=0x7f8b2415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 --2- 172.21.15.43:0/1868425375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b24155d70 0x7f8b2415c7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b240af0a0 tx=0x7f8b1401c620 comp rx=0 tx=0).stop 2024-09-17T05:23:28.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 -- 172.21.15.43:0/1868425375 shutdown_connections 2024-09-17T05:23:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 --2- 172.21.15.43:0/1868425375 >> v2:172.21.15.62:3300/0 conn(0x7f8b2406f110 0x7f8b240b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 --2- 172.21.15.43:0/1868425375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b24155d70 0x7f8b2415c7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 -- 172.21.15.43:0/1868425375 >> 172.21.15.43:0/1868425375 conn(0x7f8b240b23d0 msgr2=0x7f8b240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 -- 172.21.15.43:0/1868425375 shutdown_connections 2024-09-17T05:23:28.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.252+0000 7f8b29400640 1 -- 172.21.15.43:0/1868425375 wait complete. 2024-09-17T05:23:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 Processor -- start 2024-09-17T05:23:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- start start 2024-09-17T05:23:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b240b3920 0x7f8b2408e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b24155d70 0x7f8b2415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b241689c0 con 0x7f8b2406f110 2024-09-17T05:23:28.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24168840 con 0x7f8b24155d70 2024-09-17T05:23:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8b24168b40 con 0x7f8b240b3920 2024-09-17T05:23:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49328/0 (socket says 172.21.15.43:49328) 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 -- 172.21.15.43:0/349164175 learned_addr learned my addr 172.21.15.43:0/349164175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b1be00640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b240b3920 0x7f8b2408e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b23200640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b24155d70 0x7f8b2415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 -- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b240b3920 msgr2=0x7f8b2408e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b240b3920 0x7f8b2408e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 -- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b24155d70 msgr2=0x7f8b2415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b24155d70 0x7f8b2415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 -- 172.21.15.43:0/349164175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2408ed30 con 0x7f8b2406f110 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b22800640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009850 tx=0x7f8b140a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140ac070 con 0x7f8b2406f110 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b14004400 con 0x7f8b2406f110 2024-09-17T05:23:28.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2408c110 con 0x7f8b2406f110 2024-09-17T05:23:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b140053a0 con 0x7f8b2406f110 2024-09-17T05:23:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.256+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2408c6d0 con 0x7f8b2406f110 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.260+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240b33e0 con 0x7f8b2406f110 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.260+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b140af070 con 0x7f8b2406f110 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.260+0000 7f8b1aa00640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 0x7f8b00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.260+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8b1412e810 con 0x7f8b2406f110 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.260+0000 7f8b1be00640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 0x7f8b00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.264+0000 7f8b1be00640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 0x7f8b00075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b24075270 tx=0x7f8b0c00a250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.264+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b240b3d30 con 0x7f8b2406f110 2024-09-17T05:23:28.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.472+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b2415ea40 con 0x7f8b2406f110 2024-09-17T05:23:28.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b1aa00640 1 -- 172.21.15.43:0/349164175 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f8b140f9250 con 0x7f8b2406f110 2024-09-17T05:23:28.474 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:28.474 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:28.474 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 msgr2=0x7f8b00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 0x7f8b00075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b24075270 tx=0x7f8b0c00a250 comp rx=0 tx=0).stop 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 msgr2=0x7f8b24164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009850 tx=0x7f8b140a6580 comp rx=0 tx=0).stop 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 shutdown_connections 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b00073200 0x7f8b00075710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b24155d70 0x7f8b2415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b240b3920 0x7f8b2408e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 --2- 172.21.15.43:0/349164175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b2406f110 0x7f8b24164b60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 >> 172.21.15.43:0/349164175 conn(0x7f8b240b23d0 msgr2=0x7f8b2415df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:28.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 shutdown_connections 2024-09-17T05:23:28.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.476+0000 7f8b29400640 1 -- 172.21.15.43:0/349164175 wait complete. 2024-09-17T05:23:28.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 -- 172.21.15.43:0/1387432144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 msgr2=0x7fedb00cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 --2- 172.21.15.43:0/1387432144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00cbc70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feda8009840 tx=0x7feda801c790 comp rx=0 tx=0).stop 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 -- 172.21.15.43:0/1387432144 shutdown_connections 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 --2- 172.21.15.43:0/1387432144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00cbc70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 --2- 172.21.15.43:0/1387432144 >> v2:172.21.15.62:3300/0 conn(0x7fedb006ce60 0x7fedb006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 -- 172.21.15.43:0/1387432144 >> 172.21.15.43:0/1387432144 conn(0x7fedb00b23d0 msgr2=0x7fedb00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 -- 172.21.15.43:0/1387432144 shutdown_connections 2024-09-17T05:23:28.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.656+0000 7fedb5800640 1 -- 172.21.15.43:0/1387432144 wait complete. 2024-09-17T05:23:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 Processor -- start 2024-09-17T05:23:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- start start 2024-09-17T05:23:28.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb0070620 0x7fedb00ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00ce5a0 con 0x7fedb006ce60 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00ce420 con 0x7fedb0070620 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fedb00ce720 con 0x7fedb00cb860 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49342/0 (socket says 172.21.15.43:49342) 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 -- 172.21.15.43:0/1459548210 learned_addr learned my addr 172.21.15.43:0/1459548210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaf400640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7feda6000640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb0070620 0x7fedb00ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 -- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 msgr2=0x7fedb00ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 -- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb0070620 msgr2=0x7fedb00ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb0070620 0x7fedb00ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 -- 172.21.15.43:0/1459548210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb00c92f0 con 0x7fedb006ce60 2024-09-17T05:23:28.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedaea00640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feda000cb80 tx=0x7feda000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0016020 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feda000ab40 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb00c95d0 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0012420 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.660+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb00c9b90 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb00cecd0 con 0x7fedb006ce60 2024-09-17T05:23:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feda0019020 con 0x7fedb006ce60 2024-09-17T05:23:28.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7feda7400640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 0x7fed8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:28.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feda0094530 con 0x7fedb006ce60 2024-09-17T05:23:28.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7feda6000640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 0x7fed8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:28.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.664+0000 7feda6000640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 0x7fed8c075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fedb00c73b0 tx=0x7fed980091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:28.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.668+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feda0020020 con 0x7fedb006ce60 2024-09-17T05:23:28.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.848+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedb009e190 con 0x7fedb006ce60 2024-09-17T05:23:28.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7feda7400640 1 -- 172.21.15.43:0/1459548210 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fedb006d7b0 con 0x7fedb006ce60 2024-09-17T05:23:28.850 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:28.850 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 msgr2=0x7fed8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 0x7fed8c075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fedb00c73b0 tx=0x7fed980091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 msgr2=0x7fedb009c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feda000cb80 tx=0x7feda000cc70 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 shutdown_connections 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed8c073200 0x7fed8c075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb0070620 0x7fedb00ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fedb00cb860 0x7fedb00ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 --2- 172.21.15.43:0/1459548210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedb006ce60 0x7fedb009c6d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 >> 172.21.15.43:0/1459548210 conn(0x7fedb00b23d0 msgr2=0x7fedb00ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 shutdown_connections 2024-09-17T05:23:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:28.852+0000 7fedb5800640 1 -- 172.21.15.43:0/1459548210 wait complete. 2024-09-17T05:23:28.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:23:28.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:29.004 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:23:29.000+0000 7f2e8ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 -- 172.21.15.43:0/929871084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 msgr2=0x7fd0f4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 --2- 172.21.15.43:0/929871084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd0f4004c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009b30 tx=0x7fd0f801d250 comp rx=0 tx=0).stop 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd10e800640 1 -- 172.21.15.43:0/929871084 reap_dead start 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 -- 172.21.15.43:0/929871084 shutdown_connections 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 -- 172.21.15.43:0/929871084 >> 172.21.15.43:0/929871084 conn(0x7fd1100cee80 msgr2=0x7fd1100cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 -- 172.21.15.43:0/929871084 shutdown_connections 2024-09-17T05:23:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.032+0000 7fd115000640 1 -- 172.21.15.43:0/929871084 wait complete. 2024-09-17T05:23:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 Processor -- start 2024-09-17T05:23:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 -- start start 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 0x7fd1101f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1101f8f30 0x7fd1101f9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1100cd940 con 0x7fd0f4004810 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1100cd7c0 con 0x7fd1101f5b70 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd115000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd1100cce70 con 0x7fd1101f8f30 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49364/0 (socket says 172.21.15.43:49364) 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 -- 172.21.15.43:0/3777099129 learned_addr learned my addr 172.21.15.43:0/3777099129 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd107e00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 0x7fd1101f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10e800640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1101f8f30 0x7fd1101f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 -- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1101f8f30 msgr2=0x7fd1101f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1101f8f30 0x7fd1101f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 -- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 msgr2=0x7fd1101f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 0x7fd1101f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 -- 172.21.15.43:0/3777099129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1101f9880 con 0x7fd0f4004810 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd107e00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 0x7fd1101f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd10de00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd100006c80 tx=0x7fd100010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.036+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100017030 con 0x7fd0f4004810 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd100010e40 con 0x7fd0f4004810 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1100ca470 con 0x7fd0f4004810 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100006310 con 0x7fd0f4004810 2024-09-17T05:23:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1100caa30 con 0x7fd0f4004810 2024-09-17T05:23:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1100d0840 con 0x7fd0f4004810 2024-09-17T05:23:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd10001a070 con 0x7fd0f4004810 2024-09-17T05:23:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd106000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 0x7fd0ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd100096d50 con 0x7fd0f4004810 2024-09-17T05:23:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.040+0000 7fd107e00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 0x7fd0ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.044+0000 7fd107e00640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 0x7fd0ec075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80ae000 tx=0x7fd0f8019ff0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.044+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd10009a030 con 0x7fd0f4004810 2024-09-17T05:23:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.220+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1101f68e0 con 0x7fd0ec073200 2024-09-17T05:23:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.224+0000 7fd106000640 1 -- 172.21.15.43:0/3777099129 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241330 (secure 0 0 0) 0x7fd1101f68e0 con 0x7fd0ec073200 2024-09-17T05:23:29.227 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:29.229 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:29.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 msgr2=0x7fd0ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 0x7fd0ec075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80ae000 tx=0x7fd0f8019ff0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 msgr2=0x7fd11009faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd100006c80 tx=0x7fd100010730 comp rx=0 tx=0).stop 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 shutdown_connections 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd0ec073200 0x7fd0ec075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1101f5b70 0x7fd1101f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1101f8f30 0x7fd1101f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 --2- 172.21.15.43:0/3777099129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f4004810 0x7fd11009faa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 >> 172.21.15.43:0/3777099129 conn(0x7fd1100cee80 msgr2=0x7fd1100b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 shutdown_connections 2024-09-17T05:23:29.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.228+0000 7fd115000640 1 -- 172.21.15.43:0/3777099129 wait complete. 2024-09-17T05:23:29.248 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T05:23:27.839372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":2588,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":2650112,"data_stored":1632290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.816164"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030501+0000","last_change":"2024-09-17T05:23:23.030501+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030501+0000","last_undegraded":"2024-09-17T05:23:23.030501+0000","last_fullsized":"2024-09-17T05:23:23.030501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.036407+0000","last_change":"2024-09-17T05:23:23.036407+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.036407+0000","last_undegraded":"2024-09-17T05:23:23.036407+0000","last_fullsized":"2024-09-17T05:23:23.036407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.035749+0000","last_change":"2024-09-17T05:23:23.035749+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.035749+0000","last_undegraded":"2024-09-17T05:23:23.035749+0000","last_fullsized":"2024-09-17T05:23:23.035749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.032827+0000","last_change":"2024-09-17T05:23:23.032827+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.032827+0000","last_undegraded":"2024-09-17T05:23:23.032827+0000","last_fullsized":"2024-09-17T05:23:23.032827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028726+0000","last_change":"2024-09-17T05:23:23.028726+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028726+0000","last_undegraded":"2024-09-17T05:23:23.028726+0000","last_fullsized":"2024-09-17T05:23:23.028726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030689+0000","last_change":"2024-09-17T05:23:23.030689+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030689+0000","last_undegraded":"2024-09-17T05:23:23.030689+0000","last_fullsized":"2024-09-17T05:23:23.030689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.027712+0000","last_change":"2024-09-17T05:23:23.027712+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.027712+0000","last_undegraded":"2024-09-17T05:23:23.027712+0000","last_fullsized":"2024-09-17T05:23:23.027712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030480+0000","last_change":"2024-09-17T05:23:23.030480+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030480+0000","last_undegraded":"2024-09-17T05:23:23.030480+0000","last_fullsized":"2024-09-17T05:23:23.030480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.035692+0000","last_change":"2024-09-17T05:23:23.035692+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.035692+0000","last_undegraded":"2024-09-17T05:23:23.035692+0000","last_fullsized":"2024-09-17T05:23:23.035692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028853+0000","last_change":"2024-09-17T05:23:23.028853+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028853+0000","last_undegraded":"2024-09-17T05:23:23.028853+0000","last_fullsized":"2024-09-17T05:23:23.028853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026494+0000","last_change":"2024-09-17T05:23:23.026494+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026494+0000","last_undegraded":"2024-09-17T05:23:23.026494+0000","last_fullsized":"2024-09-17T05:23:23.026494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028742+0000","last_change":"2024-09-17T05:23:23.028742+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028742+0000","last_undegraded":"2024-09-17T05:23:23.028742+0000","last_fullsized":"2024-09-17T05:23:23.028742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.027672+0000","last_change":"2024-09-17T05:23:23.027672+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.027672+0000","last_undegraded":"2024-09-17T05:23:23.027672+0000","last_fullsized":"2024-09-17T05:23:23.027672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031158+0000","last_change":"2024-09-17T05:23:23.031158+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031158+0000","last_undegraded":"2024-09-17T05:23:23.031158+0000","last_fullsized":"2024-09-17T05:23:23.031158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T05:23:23.032774+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:23.032774+0000","last_peered":"2024-09-17T05:23:23.032774+0000","last_clean":"2024-09-17T05:23:23.032774+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:23.032774+0000","last_undegraded":"2024-09-17T05:23:23.032774+0000","last_fullsized":"2024-09-17T05:23:23.032774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.033445+0000","last_change":"2024-09-17T05:23:23.033445+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.033445+0000","last_undegraded":"2024-09-17T05:23:23.033445+0000","last_fullsized":"2024-09-17T05:23:23.033445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031896+0000","last_change":"2024-09-17T05:23:23.031896+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031896+0000","last_undegraded":"2024-09-17T05:23:23.031896+0000","last_fullsized":"2024-09-17T05:23:23.031896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030501+0000","last_change":"2024-09-17T05:23:23.030501+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030501+0000","last_undegraded":"2024-09-17T05:23:23.030501+0000","last_fullsized":"2024-09-17T05:23:23.030501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.034941+0000","last_change":"2024-09-17T05:23:23.034941+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.034941+0000","last_undegraded":"2024-09-17T05:23:23.034941+0000","last_fullsized":"2024-09-17T05:23:23.034941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031474+0000","last_change":"2024-09-17T05:23:23.031474+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031474+0000","last_undegraded":"2024-09-17T05:23:23.031474+0000","last_fullsized":"2024-09-17T05:23:23.031474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.032106+0000","last_change":"2024-09-17T05:23:23.032106+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.032106+0000","last_undegraded":"2024-09-17T05:23:23.032106+0000","last_fullsized":"2024-09-17T05:23:23.032106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026424+0000","last_change":"2024-09-17T05:23:23.026424+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026424+0000","last_undegraded":"2024-09-17T05:23:23.026424+0000","last_fullsized":"2024-09-17T05:23:23.026424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031979+0000","last_change":"2024-09-17T05:23:23.031979+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031979+0000","last_undegraded":"2024-09-17T05:23:23.031979+0000","last_fullsized":"2024-09-17T05:23:23.031979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028442+0000","last_change":"2024-09-17T05:23:23.028442+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028442+0000","last_undegraded":"2024-09-17T05:23:23.028442+0000","last_fullsized":"2024-09-17T05:23:23.028442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026607+0000","last_change":"2024-09-17T05:23:23.026607+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026607+0000","last_undegraded":"2024-09-17T05:23:23.026607+0000","last_fullsized":"2024-09-17T05:23:23.026607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.034116+0000","last_change":"2024-09-17T05:23:23.034116+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.034116+0000","last_undegraded":"2024-09-17T05:23:23.034116+0000","last_fullsized":"2024-09-17T05:23:23.034116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.029223+0000","last_change":"2024-09-17T05:23:23.029223+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.029223+0000","last_undegraded":"2024-09-17T05:23:23.029223+0000","last_fullsized":"2024-09-17T05:23:23.029223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028410+0000","last_change":"2024-09-17T05:23:23.028410+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028410+0000","last_undegraded":"2024-09-17T05:23:23.028410+0000","last_fullsized":"2024-09-17T05:23:23.028410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030617+0000","last_change":"2024-09-17T05:23:23.030617+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030617+0000","last_undegraded":"2024-09-17T05:23:23.030617+0000","last_fullsized":"2024-09-17T05:23:23.030617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026568+0000","last_change":"2024-09-17T05:23:23.026568+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026568+0000","last_undegraded":"2024-09-17T05:23:23.026568+0000","last_fullsized":"2024-09-17T05:23:23.026568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:29.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 -- 172.21.15.43:0/1717073655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 msgr2=0x7efc30004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 --2- 172.21.15.43:0/1717073655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc30004c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efc34009b10 tx=0x7efc3401d250 comp rx=0 tx=0).stop 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc51000640 1 -- 172.21.15.43:0/1717073655 reap_dead start 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 -- 172.21.15.43:0/1717073655 shutdown_connections 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 -- 172.21.15.43:0/1717073655 >> 172.21.15.43:0/1717073655 conn(0x7efc4c0cee80 msgr2=0x7efc4c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 -- 172.21.15.43:0/1717073655 shutdown_connections 2024-09-17T05:23:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.448+0000 7efc53000640 1 -- 172.21.15.43:0/1717073655 wait complete. 2024-09-17T05:23:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 Processor -- start 2024-09-17T05:23:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 -- start start 2024-09-17T05:23:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc4c1f63d0 0x7efc4c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc4c0cc220 0x7efc4c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c070be0 con 0x7efc30004810 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c070a60 con 0x7efc4c0cc220 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc53000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c0cd7c0 con 0x7efc4c1f63d0 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49376/0 (socket says 172.21.15.43:49376) 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 -- 172.21.15.43:0/1638154452 learned_addr learned my addr 172.21.15.43:0/1638154452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc43e00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc4c1f63d0 0x7efc4c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc51000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc4c0cc220 0x7efc4c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 -- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc4c1f63d0 msgr2=0x7efc4c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc4c1f63d0 0x7efc4c06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 -- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc4c0cc220 msgr2=0x7efc4c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc4c0cc220 0x7efc4c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.452+0000 7efc4be00640 1 -- 172.21.15.43:0/1638154452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0caeb0 con 0x7efc30004810 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc4be00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc3c011ae0 tx=0x7efc3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c01e070 con 0x7efc30004810 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc3c00fc10 con 0x7efc30004810 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c019410 con 0x7efc30004810 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c0cb180 con 0x7efc30004810 2024-09-17T05:23:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c0cb740 con 0x7efc30004810 2024-09-17T05:23:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c0ccfe0 con 0x7efc30004810 2024-09-17T05:23:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc3c021070 con 0x7efc30004810 2024-09-17T05:23:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 0x7efc28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.456+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efc3c094b80 con 0x7efc30004810 2024-09-17T05:23:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.460+0000 7efc43e00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 0x7efc28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.460+0000 7efc43e00640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 0x7efc28075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc340ad000 tx=0x7efc34019ff0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.464+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc3c05f4b0 con 0x7efc30004810 2024-09-17T05:23:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.636+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc4c071a50 con 0x7efc28073200 2024-09-17T05:23:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.640+0000 7efc48c00640 1 -- 172.21.15.43:0/1638154452 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241330 (secure 0 0 0) 0x7efc4c071a50 con 0x7efc28073200 2024-09-17T05:23:29.641 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 msgr2=0x7efc28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 0x7efc28075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc340ad000 tx=0x7efc34019ff0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 msgr2=0x7efc4c1600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc3c011ae0 tx=0x7efc3c009e30 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 shutdown_connections 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc28073200 0x7efc28075710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc4c0cc220 0x7efc4c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc4c1f63d0 0x7efc4c06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 --2- 172.21.15.43:0/1638154452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc30004810 0x7efc4c1600a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 >> 172.21.15.43:0/1638154452 conn(0x7efc4c0cee80 msgr2=0x7efc4c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 shutdown_connections 2024-09-17T05:23:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.644+0000 7efc53000640 1 -- 172.21.15.43:0/1638154452 wait complete. 2024-09-17T05:23:29.663 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T05:23:27.839372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":2588,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":2650112,"data_stored":1632290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.816164"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030501+0000","last_change":"2024-09-17T05:23:23.030501+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030501+0000","last_undegraded":"2024-09-17T05:23:23.030501+0000","last_fullsized":"2024-09-17T05:23:23.030501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.036407+0000","last_change":"2024-09-17T05:23:23.036407+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.036407+0000","last_undegraded":"2024-09-17T05:23:23.036407+0000","last_fullsized":"2024-09-17T05:23:23.036407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.035749+0000","last_change":"2024-09-17T05:23:23.035749+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.035749+0000","last_undegraded":"2024-09-17T05:23:23.035749+0000","last_fullsized":"2024-09-17T05:23:23.035749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.032827+0000","last_change":"2024-09-17T05:23:23.032827+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.032827+0000","last_undegraded":"2024-09-17T05:23:23.032827+0000","last_fullsized":"2024-09-17T05:23:23.032827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028726+0000","last_change":"2024-09-17T05:23:23.028726+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028726+0000","last_undegraded":"2024-09-17T05:23:23.028726+0000","last_fullsized":"2024-09-17T05:23:23.028726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030689+0000","last_change":"2024-09-17T05:23:23.030689+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030689+0000","last_undegraded":"2024-09-17T05:23:23.030689+0000","last_fullsized":"2024-09-17T05:23:23.030689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.027712+0000","last_change":"2024-09-17T05:23:23.027712+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.027712+0000","last_undegraded":"2024-09-17T05:23:23.027712+0000","last_fullsized":"2024-09-17T05:23:23.027712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030480+0000","last_change":"2024-09-17T05:23:23.030480+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030480+0000","last_undegraded":"2024-09-17T05:23:23.030480+0000","last_fullsized":"2024-09-17T05:23:23.030480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.035692+0000","last_change":"2024-09-17T05:23:23.035692+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.035692+0000","last_undegraded":"2024-09-17T05:23:23.035692+0000","last_fullsized":"2024-09-17T05:23:23.035692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028853+0000","last_change":"2024-09-17T05:23:23.028853+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028853+0000","last_undegraded":"2024-09-17T05:23:23.028853+0000","last_fullsized":"2024-09-17T05:23:23.028853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026494+0000","last_change":"2024-09-17T05:23:23.026494+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026494+0000","last_undegraded":"2024-09-17T05:23:23.026494+0000","last_fullsized":"2024-09-17T05:23:23.026494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028742+0000","last_change":"2024-09-17T05:23:23.028742+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028742+0000","last_undegraded":"2024-09-17T05:23:23.028742+0000","last_fullsized":"2024-09-17T05:23:23.028742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.027672+0000","last_change":"2024-09-17T05:23:23.027672+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.027672+0000","last_undegraded":"2024-09-17T05:23:23.027672+0000","last_fullsized":"2024-09-17T05:23:23.027672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031158+0000","last_change":"2024-09-17T05:23:23.031158+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031158+0000","last_undegraded":"2024-09-17T05:23:23.031158+0000","last_fullsized":"2024-09-17T05:23:23.031158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T05:23:23.032774+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:23.032774+0000","last_peered":"2024-09-17T05:23:23.032774+0000","last_clean":"2024-09-17T05:23:23.032774+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:23.032774+0000","last_undegraded":"2024-09-17T05:23:23.032774+0000","last_fullsized":"2024-09-17T05:23:23.032774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.033445+0000","last_change":"2024-09-17T05:23:23.033445+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.033445+0000","last_undegraded":"2024-09-17T05:23:23.033445+0000","last_fullsized":"2024-09-17T05:23:23.033445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031896+0000","last_change":"2024-09-17T05:23:23.031896+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031896+0000","last_undegraded":"2024-09-17T05:23:23.031896+0000","last_fullsized":"2024-09-17T05:23:23.031896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030501+0000","last_change":"2024-09-17T05:23:23.030501+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030501+0000","last_undegraded":"2024-09-17T05:23:23.030501+0000","last_fullsized":"2024-09-17T05:23:23.030501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.034941+0000","last_change":"2024-09-17T05:23:23.034941+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.034941+0000","last_undegraded":"2024-09-17T05:23:23.034941+0000","last_fullsized":"2024-09-17T05:23:23.034941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031474+0000","last_change":"2024-09-17T05:23:23.031474+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031474+0000","last_undegraded":"2024-09-17T05:23:23.031474+0000","last_fullsized":"2024-09-17T05:23:23.031474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.032106+0000","last_change":"2024-09-17T05:23:23.032106+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.032106+0000","last_undegraded":"2024-09-17T05:23:23.032106+0000","last_fullsized":"2024-09-17T05:23:23.032106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026424+0000","last_change":"2024-09-17T05:23:23.026424+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026424+0000","last_undegraded":"2024-09-17T05:23:23.026424+0000","last_fullsized":"2024-09-17T05:23:23.026424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.031979+0000","last_change":"2024-09-17T05:23:23.031979+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.031979+0000","last_undegraded":"2024-09-17T05:23:23.031979+0000","last_fullsized":"2024-09-17T05:23:23.031979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028442+0000","last_change":"2024-09-17T05:23:23.028442+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028442+0000","last_undegraded":"2024-09-17T05:23:23.028442+0000","last_fullsized":"2024-09-17T05:23:23.028442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026607+0000","last_change":"2024-09-17T05:23:23.026607+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026607+0000","last_undegraded":"2024-09-17T05:23:23.026607+0000","last_fullsized":"2024-09-17T05:23:23.026607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.034116+0000","last_change":"2024-09-17T05:23:23.034116+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.034116+0000","last_undegraded":"2024-09-17T05:23:23.034116+0000","last_fullsized":"2024-09-17T05:23:23.034116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.029223+0000","last_change":"2024-09-17T05:23:23.029223+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.029223+0000","last_undegraded":"2024-09-17T05:23:23.029223+0000","last_fullsized":"2024-09-17T05:23:23.029223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.028410+0000","last_change":"2024-09-17T05:23:23.028410+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.028410+0000","last_undegraded":"2024-09-17T05:23:23.028410+0000","last_fullsized":"2024-09-17T05:23:23.028410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.030617+0000","last_change":"2024-09-17T05:23:23.030617+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.030617+0000","last_undegraded":"2024-09-17T05:23:23.030617+0000","last_fullsized":"2024-09-17T05:23:23.030617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:23:24.024374+0000","last_change":"2024-09-17T05:23:24.024374+0000","last_active":"2024-09-17T05:23:24.024374+0000","last_peered":"2024-09-17T05:23:24.024374+0000","last_clean":"2024-09-17T05:23:24.024374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:24.024374+0000","last_undegraded":"2024-09-17T05:23:24.024374+0000","last_fullsized":"2024-09-17T05:23:24.024374+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:23:23.026568+0000","last_change":"2024-09-17T05:23:23.026568+0000","last_active":"2024-09-17T05:23:23.016587+0000","last_peered":"2024-09-17T05:23:23.016587+0000","last_clean":"2024-09-17T05:23:23.016587+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:23:23.026568+0000","last_undegraded":"2024-09-17T05:23:23.026568+0000","last_fullsized":"2024-09-17T05:23:23.026568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31360,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340480,"statfs":{"total":96636764160,"available":96604651520,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:29.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 -- 172.21.15.43:0/2992468042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 msgr2=0x7f0ac80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 --2- 172.21.15.43:0/2992468042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac80ad0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80af0a0 tx=0x7f0ac001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 -- 172.21.15.43:0/2992468042 shutdown_connections 2024-09-17T05:23:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 --2- 172.21.15.43:0/2992468042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac80ad0b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 --2- 172.21.15.43:0/2992468042 >> v2:172.21.15.62:3300/0 conn(0x7f0ac80b3af0 0x7f0ac815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 -- 172.21.15.43:0/2992468042 >> 172.21.15.43:0/2992468042 conn(0x7f0ac80b23d0 msgr2=0x7f0ac80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 -- 172.21.15.43:0/2992468042 shutdown_connections 2024-09-17T05:23:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.868+0000 7f0acdc00640 1 -- 172.21.15.43:0/2992468042 wait complete. 2024-09-17T05:23:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 Processor -- start 2024-09-17T05:23:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- start start 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 0x7f0ac80c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 0x7f0ac8164a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8168c30 con 0x7f0ac806f110 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8168ab0 con 0x7f0ac815c7e0 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac8168db0 con 0x7f0ac8155d70 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49390/0 (socket says 172.21.15.43:49390) 2024-09-17T05:23:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 -- 172.21.15.43:0/3068476907 learned_addr learned my addr 172.21.15.43:0/3068476907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac7400640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 0x7f0ac8164a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6000640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 0x7f0ac80c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 -- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 msgr2=0x7f0ac80c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 0x7f0ac80c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 -- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 msgr2=0x7f0ac8164a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 0x7f0ac8164a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 -- 172.21.15.43:0/3068476907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac8074620 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6000640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 0x7f0ac80c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac7400640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 0x7f0ac8164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c3f0 tx=0x7f0ab800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8017070 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ab80053c0 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003860 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac8074c50 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.872+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac80c5650 con 0x7f0ac806f110 2024-09-17T05:23:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.876+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac80c58b0 con 0x7f0ac806f110 2024-09-17T05:23:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.876+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ab801a070 con 0x7f0ac806f110 2024-09-17T05:23:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.876+0000 7f0abea00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 0x7f0aa8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.876+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0ab8093580 con 0x7f0ac806f110 2024-09-17T05:23:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.876+0000 7f0ac6000640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 0x7f0aa8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.880+0000 7f0ac6000640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 0x7f0aa8075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005fd0 tx=0x7f0ab4005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:29.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:29.880+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ab805e590 con 0x7f0ac806f110 2024-09-17T05:23:30.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.052+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ac8165fd0 con 0x7f0aa8073200 2024-09-17T05:23:30.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.060+0000 7f0abea00640 1 -- 172.21.15.43:0/3068476907 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7f0ac8165fd0 con 0x7f0aa8073200 2024-09-17T05:23:30.061 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 msgr2=0x7f0aa8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 0x7f0aa8075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005fd0 tx=0x7f0ab4005de0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 msgr2=0x7f0ac815e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c3f0 tx=0x7f0ab800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 shutdown_connections 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa8073200 0x7f0aa8075710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac815c7e0 0x7f0ac8164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0ac8155d70 0x7f0ac80c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 --2- 172.21.15.43:0/3068476907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac806f110 0x7f0ac815e270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 >> 172.21.15.43:0/3068476907 conn(0x7f0ac80b23d0 msgr2=0x7f0ac80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 shutdown_connections 2024-09-17T05:23:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.064+0000 7f0acdc00640 1 -- 172.21.15.43:0/3068476907 wait complete. 2024-09-17T05:23:30.085 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T05:23:29.839861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235860,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738860,"statfs":{"total":773094113280,"available":772852592640,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053057+0000","last_change":"2024-09-17T05:23:25.072214+0000","last_active":"2024-09-17T05:23:26.053057+0000","last_peered":"2024-09-17T05:23:26.053057+0000","last_clean":"2024-09-17T05:23:26.053057+0000","last_became_active":"2024-09-17T05:23:25.071839+0000","last_became_peered":"2024-09-17T05:23:25.071839+0000","last_unstale":"2024-09-17T05:23:26.053057+0000","last_undegraded":"2024-09-17T05:23:26.053057+0000","last_fullsized":"2024-09-17T05:23:26.053057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052684+0000","last_change":"2024-09-17T05:23:25.068245+0000","last_active":"2024-09-17T05:23:26.052684+0000","last_peered":"2024-09-17T05:23:26.052684+0000","last_clean":"2024-09-17T05:23:26.052684+0000","last_became_active":"2024-09-17T05:23:25.063871+0000","last_became_peered":"2024-09-17T05:23:25.063871+0000","last_unstale":"2024-09-17T05:23:26.052684+0000","last_undegraded":"2024-09-17T05:23:26.052684+0000","last_fullsized":"2024-09-17T05:23:26.052684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:20.196005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052943+0000","last_change":"2024-09-17T05:23:25.077572+0000","last_active":"2024-09-17T05:23:26.052943+0000","last_peered":"2024-09-17T05:23:26.052943+0000","last_clean":"2024-09-17T05:23:26.052943+0000","last_became_active":"2024-09-17T05:23:25.077232+0000","last_became_peered":"2024-09-17T05:23:25.077232+0000","last_unstale":"2024-09-17T05:23:26.052943+0000","last_undegraded":"2024-09-17T05:23:26.052943+0000","last_fullsized":"2024-09-17T05:23:26.052943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070147+0000","last_change":"2024-09-17T05:23:25.080315+0000","last_active":"2024-09-17T05:23:26.070147+0000","last_peered":"2024-09-17T05:23:26.070147+0000","last_clean":"2024-09-17T05:23:26.070147+0000","last_became_active":"2024-09-17T05:23:25.079991+0000","last_became_peered":"2024-09-17T05:23:25.079991+0000","last_unstale":"2024-09-17T05:23:26.070147+0000","last_undegraded":"2024-09-17T05:23:26.070147+0000","last_fullsized":"2024-09-17T05:23:26.070147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:58.988827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052636+0000","last_change":"2024-09-17T05:23:25.062672+0000","last_active":"2024-09-17T05:23:26.052636+0000","last_peered":"2024-09-17T05:23:26.052636+0000","last_clean":"2024-09-17T05:23:26.052636+0000","last_became_active":"2024-09-17T05:23:25.060858+0000","last_became_peered":"2024-09-17T05:23:25.060858+0000","last_unstale":"2024-09-17T05:23:26.052636+0000","last_undegraded":"2024-09-17T05:23:26.052636+0000","last_fullsized":"2024-09-17T05:23:26.052636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.102130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069891+0000","last_change":"2024-09-17T05:23:25.065376+0000","last_active":"2024-09-17T05:23:26.069891+0000","last_peered":"2024-09-17T05:23:26.069891+0000","last_clean":"2024-09-17T05:23:26.069891+0000","last_became_active":"2024-09-17T05:23:25.064806+0000","last_became_peered":"2024-09-17T05:23:25.064806+0000","last_unstale":"2024-09-17T05:23:26.069891+0000","last_undegraded":"2024-09-17T05:23:26.069891+0000","last_fullsized":"2024-09-17T05:23:26.069891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052148+0000","last_change":"2024-09-17T05:23:25.053855+0000","last_active":"2024-09-17T05:23:26.052148+0000","last_peered":"2024-09-17T05:23:26.052148+0000","last_clean":"2024-09-17T05:23:26.052148+0000","last_became_active":"2024-09-17T05:23:25.053569+0000","last_became_peered":"2024-09-17T05:23:25.053569+0000","last_unstale":"2024-09-17T05:23:26.052148+0000","last_undegraded":"2024-09-17T05:23:26.052148+0000","last_fullsized":"2024-09-17T05:23:26.052148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052419+0000","last_change":"2024-09-17T05:23:25.056925+0000","last_active":"2024-09-17T05:23:26.052419+0000","last_peered":"2024-09-17T05:23:26.052419+0000","last_clean":"2024-09-17T05:23:26.052419+0000","last_became_active":"2024-09-17T05:23:25.056619+0000","last_became_peered":"2024-09-17T05:23:25.056619+0000","last_unstale":"2024-09-17T05:23:26.052419+0000","last_undegraded":"2024-09-17T05:23:26.052419+0000","last_fullsized":"2024-09-17T05:23:26.052419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054267+0000","last_change":"2024-09-17T05:23:25.061010+0000","last_active":"2024-09-17T05:23:26.054267+0000","last_peered":"2024-09-17T05:23:26.054267+0000","last_clean":"2024-09-17T05:23:26.054267+0000","last_became_active":"2024-09-17T05:23:25.060714+0000","last_became_peered":"2024-09-17T05:23:25.060714+0000","last_unstale":"2024-09-17T05:23:26.054267+0000","last_undegraded":"2024-09-17T05:23:26.054267+0000","last_fullsized":"2024-09-17T05:23:26.054267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.664289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052921+0000","last_change":"2024-09-17T05:23:25.079732+0000","last_active":"2024-09-17T05:23:26.052921+0000","last_peered":"2024-09-17T05:23:26.052921+0000","last_clean":"2024-09-17T05:23:26.052921+0000","last_became_active":"2024-09-17T05:23:25.079100+0000","last_became_peered":"2024-09-17T05:23:25.079100+0000","last_unstale":"2024-09-17T05:23:26.052921+0000","last_undegraded":"2024-09-17T05:23:26.052921+0000","last_fullsized":"2024-09-17T05:23:26.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:01.901521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052498+0000","last_change":"2024-09-17T05:23:25.075619+0000","last_active":"2024-09-17T05:23:26.052498+0000","last_peered":"2024-09-17T05:23:26.052498+0000","last_clean":"2024-09-17T05:23:26.052498+0000","last_became_active":"2024-09-17T05:23:25.071655+0000","last_became_peered":"2024-09-17T05:23:25.071655+0000","last_unstale":"2024-09-17T05:23:26.052498+0000","last_undegraded":"2024-09-17T05:23:26.052498+0000","last_fullsized":"2024-09-17T05:23:26.052498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052932+0000","last_change":"2024-09-17T05:23:25.077778+0000","last_active":"2024-09-17T05:23:26.052932+0000","last_peered":"2024-09-17T05:23:26.052932+0000","last_clean":"2024-09-17T05:23:26.052932+0000","last_became_active":"2024-09-17T05:23:25.077040+0000","last_became_peered":"2024-09-17T05:23:25.077040+0000","last_unstale":"2024-09-17T05:23:26.052932+0000","last_undegraded":"2024-09-17T05:23:26.052932+0000","last_fullsized":"2024-09-17T05:23:26.052932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052584+0000","last_change":"2024-09-17T05:23:25.067070+0000","last_active":"2024-09-17T05:23:26.052584+0000","last_peered":"2024-09-17T05:23:26.052584+0000","last_clean":"2024-09-17T05:23:26.052584+0000","last_became_active":"2024-09-17T05:23:25.066734+0000","last_became_peered":"2024-09-17T05:23:25.066734+0000","last_unstale":"2024-09-17T05:23:26.052584+0000","last_undegraded":"2024-09-17T05:23:26.052584+0000","last_fullsized":"2024-09-17T05:23:26.052584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:58.712013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070882+0000","last_change":"2024-09-17T05:23:25.066014+0000","last_active":"2024-09-17T05:23:26.070882+0000","last_peered":"2024-09-17T05:23:26.070882+0000","last_clean":"2024-09-17T05:23:26.070882+0000","last_became_active":"2024-09-17T05:23:25.063873+0000","last_became_peered":"2024-09-17T05:23:25.063873+0000","last_unstale":"2024-09-17T05:23:26.070882+0000","last_undegraded":"2024-09-17T05:23:26.070882+0000","last_fullsized":"2024-09-17T05:23:26.070882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071924+0000","last_change":"2024-09-17T05:23:25.055365+0000","last_active":"2024-09-17T05:23:26.071924+0000","last_peered":"2024-09-17T05:23:26.071924+0000","last_clean":"2024-09-17T05:23:26.071924+0000","last_became_active":"2024-09-17T05:23:25.055046+0000","last_became_peered":"2024-09-17T05:23:25.055046+0000","last_unstale":"2024-09-17T05:23:26.071924+0000","last_undegraded":"2024-09-17T05:23:26.071924+0000","last_fullsized":"2024-09-17T05:23:26.071924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:48.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070072+0000","last_change":"2024-09-17T05:23:25.076193+0000","last_active":"2024-09-17T05:23:26.070072+0000","last_peered":"2024-09-17T05:23:26.070072+0000","last_clean":"2024-09-17T05:23:26.070072+0000","last_became_active":"2024-09-17T05:23:25.075582+0000","last_became_peered":"2024-09-17T05:23:25.075582+0000","last_unstale":"2024-09-17T05:23:26.070072+0000","last_undegraded":"2024-09-17T05:23:26.070072+0000","last_fullsized":"2024-09-17T05:23:26.070072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071700+0000","last_change":"2024-09-17T05:23:25.053491+0000","last_active":"2024-09-17T05:23:26.071700+0000","last_peered":"2024-09-17T05:23:26.071700+0000","last_clean":"2024-09-17T05:23:26.071700+0000","last_became_active":"2024-09-17T05:23:25.053209+0000","last_became_peered":"2024-09-17T05:23:25.053209+0000","last_unstale":"2024-09-17T05:23:26.071700+0000","last_undegraded":"2024-09-17T05:23:26.071700+0000","last_fullsized":"2024-09-17T05:23:26.071700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:33.151198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053156+0000","last_change":"2024-09-17T05:23:25.068427+0000","last_active":"2024-09-17T05:23:26.053156+0000","last_peered":"2024-09-17T05:23:26.053156+0000","last_clean":"2024-09-17T05:23:26.053156+0000","last_became_active":"2024-09-17T05:23:25.068043+0000","last_became_peered":"2024-09-17T05:23:25.068043+0000","last_unstale":"2024-09-17T05:23:26.053156+0000","last_undegraded":"2024-09-17T05:23:26.053156+0000","last_fullsized":"2024-09-17T05:23:26.053156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070102+0000","last_change":"2024-09-17T05:23:25.057241+0000","last_active":"2024-09-17T05:23:26.070102+0000","last_peered":"2024-09-17T05:23:26.070102+0000","last_clean":"2024-09-17T05:23:26.070102+0000","last_became_active":"2024-09-17T05:23:25.056934+0000","last_became_peered":"2024-09-17T05:23:25.056934+0000","last_unstale":"2024-09-17T05:23:26.070102+0000","last_undegraded":"2024-09-17T05:23:26.070102+0000","last_fullsized":"2024-09-17T05:23:26.070102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054089+0000","last_change":"2024-09-17T05:23:25.064365+0000","last_active":"2024-09-17T05:23:26.054089+0000","last_peered":"2024-09-17T05:23:26.054089+0000","last_clean":"2024-09-17T05:23:26.054089+0000","last_became_active":"2024-09-17T05:23:25.064064+0000","last_became_peered":"2024-09-17T05:23:25.064064+0000","last_unstale":"2024-09-17T05:23:26.054089+0000","last_undegraded":"2024-09-17T05:23:26.054089+0000","last_fullsized":"2024-09-17T05:23:26.054089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052827+0000","last_change":"2024-09-17T05:23:25.059278+0000","last_active":"2024-09-17T05:23:26.052827+0000","last_peered":"2024-09-17T05:23:26.052827+0000","last_clean":"2024-09-17T05:23:26.052827+0000","last_became_active":"2024-09-17T05:23:25.058926+0000","last_became_peered":"2024-09-17T05:23:25.058926+0000","last_unstale":"2024-09-17T05:23:26.052827+0000","last_undegraded":"2024-09-17T05:23:26.052827+0000","last_fullsized":"2024-09-17T05:23:26.052827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070891+0000","last_change":"2024-09-17T05:23:25.064634+0000","last_active":"2024-09-17T05:23:26.070891+0000","last_peered":"2024-09-17T05:23:26.070891+0000","last_clean":"2024-09-17T05:23:26.070891+0000","last_became_active":"2024-09-17T05:23:25.064250+0000","last_became_peered":"2024-09-17T05:23:25.064250+0000","last_unstale":"2024-09-17T05:23:26.070891+0000","last_undegraded":"2024-09-17T05:23:26.070891+0000","last_fullsized":"2024-09-17T05:23:26.070891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053318+0000","last_change":"2024-09-17T05:23:25.079930+0000","last_active":"2024-09-17T05:23:26.053318+0000","last_peered":"2024-09-17T05:23:26.053318+0000","last_clean":"2024-09-17T05:23:26.053318+0000","last_became_active":"2024-09-17T05:23:25.079276+0000","last_became_peered":"2024-09-17T05:23:25.079276+0000","last_unstale":"2024-09-17T05:23:26.053318+0000","last_undegraded":"2024-09-17T05:23:26.053318+0000","last_fullsized":"2024-09-17T05:23:26.053318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070179+0000","last_change":"2024-09-17T05:23:25.083629+0000","last_active":"2024-09-17T05:23:26.070179+0000","last_peered":"2024-09-17T05:23:26.070179+0000","last_clean":"2024-09-17T05:23:26.070179+0000","last_became_active":"2024-09-17T05:23:25.083394+0000","last_became_peered":"2024-09-17T05:23:25.083394+0000","last_unstale":"2024-09-17T05:23:26.070179+0000","last_undegraded":"2024-09-17T05:23:26.070179+0000","last_fullsized":"2024-09-17T05:23:26.070179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:22.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053114+0000","last_change":"2024-09-17T05:23:25.075408+0000","last_active":"2024-09-17T05:23:26.053114+0000","last_peered":"2024-09-17T05:23:26.053114+0000","last_clean":"2024-09-17T05:23:26.053114+0000","last_became_active":"2024-09-17T05:23:25.075101+0000","last_became_peered":"2024-09-17T05:23:25.075101+0000","last_unstale":"2024-09-17T05:23:26.053114+0000","last_undegraded":"2024-09-17T05:23:26.053114+0000","last_fullsized":"2024-09-17T05:23:26.053114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071351+0000","last_change":"2024-09-17T05:23:25.068874+0000","last_active":"2024-09-17T05:23:26.071351+0000","last_peered":"2024-09-17T05:23:26.071351+0000","last_clean":"2024-09-17T05:23:26.071351+0000","last_became_active":"2024-09-17T05:23:25.064181+0000","last_became_peered":"2024-09-17T05:23:25.064181+0000","last_unstale":"2024-09-17T05:23:26.071351+0000","last_undegraded":"2024-09-17T05:23:26.071351+0000","last_fullsized":"2024-09-17T05:23:26.071351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:43.495535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068665+0000","last_change":"2024-09-17T05:23:25.064004+0000","last_active":"2024-09-17T05:23:26.068665+0000","last_peered":"2024-09-17T05:23:26.068665+0000","last_clean":"2024-09-17T05:23:26.068665+0000","last_became_active":"2024-09-17T05:23:25.063599+0000","last_became_peered":"2024-09-17T05:23:25.063599+0000","last_unstale":"2024-09-17T05:23:26.068665+0000","last_undegraded":"2024-09-17T05:23:26.068665+0000","last_fullsized":"2024-09-17T05:23:26.068665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.655466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069668+0000","last_change":"2024-09-17T05:23:25.071820+0000","last_active":"2024-09-17T05:23:26.069668+0000","last_peered":"2024-09-17T05:23:26.069668+0000","last_clean":"2024-09-17T05:23:26.069668+0000","last_became_active":"2024-09-17T05:23:25.070565+0000","last_became_peered":"2024-09-17T05:23:25.070565+0000","last_unstale":"2024-09-17T05:23:26.069668+0000","last_undegraded":"2024-09-17T05:23:26.069668+0000","last_fullsized":"2024-09-17T05:23:26.069668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:53.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054175+0000","last_change":"2024-09-17T05:23:25.064336+0000","last_active":"2024-09-17T05:23:26.054175+0000","last_peered":"2024-09-17T05:23:26.054175+0000","last_clean":"2024-09-17T05:23:26.054175+0000","last_became_active":"2024-09-17T05:23:25.063789+0000","last_became_peered":"2024-09-17T05:23:25.063789+0000","last_unstale":"2024-09-17T05:23:26.054175+0000","last_undegraded":"2024-09-17T05:23:26.054175+0000","last_fullsized":"2024-09-17T05:23:26.054175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:27.415615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052848+0000","last_change":"2024-09-17T05:23:24.057092+0000","last_active":"2024-09-17T05:23:26.052848+0000","last_peered":"2024-09-17T05:23:26.052848+0000","last_clean":"2024-09-17T05:23:26.052848+0000","last_became_active":"2024-09-17T05:23:24.056017+0000","last_became_peered":"2024-09-17T05:23:24.056017+0000","last_unstale":"2024-09-17T05:23:26.052848+0000","last_undegraded":"2024-09-17T05:23:26.052848+0000","last_fullsized":"2024-09-17T05:23:26.052848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:21.643758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070140+0000","last_change":"2024-09-17T05:23:25.050194+0000","last_active":"2024-09-17T05:23:26.070140+0000","last_peered":"2024-09-17T05:23:26.070140+0000","last_clean":"2024-09-17T05:23:26.070140+0000","last_became_active":"2024-09-17T05:23:25.049834+0000","last_became_peered":"2024-09-17T05:23:25.049834+0000","last_unstale":"2024-09-17T05:23:26.070140+0000","last_undegraded":"2024-09-17T05:23:26.070140+0000","last_fullsized":"2024-09-17T05:23:26.070140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071221+0000","last_change":"2024-09-17T05:23:24.060034+0000","last_active":"2024-09-17T05:23:26.071221+0000","last_peered":"2024-09-17T05:23:26.071221+0000","last_clean":"2024-09-17T05:23:26.071221+0000","last_became_active":"2024-09-17T05:23:24.059693+0000","last_became_peered":"2024-09-17T05:23:24.059693+0000","last_unstale":"2024-09-17T05:23:26.071221+0000","last_undegraded":"2024-09-17T05:23:26.071221+0000","last_fullsized":"2024-09-17T05:23:26.071221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:09.436373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052733+0000","last_change":"2024-09-17T05:23:25.066266+0000","last_active":"2024-09-17T05:23:26.052733+0000","last_peered":"2024-09-17T05:23:26.052733+0000","last_clean":"2024-09-17T05:23:26.052733+0000","last_became_active":"2024-09-17T05:23:25.064050+0000","last_became_peered":"2024-09-17T05:23:25.064050+0000","last_unstale":"2024-09-17T05:23:26.052733+0000","last_undegraded":"2024-09-17T05:23:26.052733+0000","last_fullsized":"2024-09-17T05:23:26.052733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070980+0000","last_change":"2024-09-17T05:23:24.052942+0000","last_active":"2024-09-17T05:23:26.070980+0000","last_peered":"2024-09-17T05:23:26.070980+0000","last_clean":"2024-09-17T05:23:26.070980+0000","last_became_active":"2024-09-17T05:23:24.052628+0000","last_became_peered":"2024-09-17T05:23:24.052628+0000","last_unstale":"2024-09-17T05:23:26.070980+0000","last_undegraded":"2024-09-17T05:23:26.070980+0000","last_fullsized":"2024-09-17T05:23:26.070980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.647679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069032+0000","last_change":"2024-09-17T05:23:25.062501+0000","last_active":"2024-09-17T05:23:26.069032+0000","last_peered":"2024-09-17T05:23:26.069032+0000","last_clean":"2024-09-17T05:23:26.069032+0000","last_became_active":"2024-09-17T05:23:25.062195+0000","last_became_peered":"2024-09-17T05:23:25.062195+0000","last_unstale":"2024-09-17T05:23:26.069032+0000","last_undegraded":"2024-09-17T05:23:26.069032+0000","last_fullsized":"2024-09-17T05:23:26.069032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:28.768705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054381+0000","last_change":"2024-09-17T05:23:24.051900+0000","last_active":"2024-09-17T05:23:26.054381+0000","last_peered":"2024-09-17T05:23:26.054381+0000","last_clean":"2024-09-17T05:23:26.054381+0000","last_became_active":"2024-09-17T05:23:24.051525+0000","last_became_peered":"2024-09-17T05:23:24.051525+0000","last_unstale":"2024-09-17T05:23:26.054381+0000","last_undegraded":"2024-09-17T05:23:26.054381+0000","last_fullsized":"2024-09-17T05:23:26.054381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070296+0000","last_change":"2024-09-17T05:23:25.074116+0000","last_active":"2024-09-17T05:23:26.070296+0000","last_peered":"2024-09-17T05:23:26.070296+0000","last_clean":"2024-09-17T05:23:26.070296+0000","last_became_active":"2024-09-17T05:23:25.073750+0000","last_became_peered":"2024-09-17T05:23:25.073750+0000","last_unstale":"2024-09-17T05:23:26.070296+0000","last_undegraded":"2024-09-17T05:23:26.070296+0000","last_fullsized":"2024-09-17T05:23:26.070296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:23.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052816+0000","last_change":"2024-09-17T05:23:24.058892+0000","last_active":"2024-09-17T05:23:26.052816+0000","last_peered":"2024-09-17T05:23:26.052816+0000","last_clean":"2024-09-17T05:23:26.052816+0000","last_became_active":"2024-09-17T05:23:24.057777+0000","last_became_peered":"2024-09-17T05:23:24.057777+0000","last_unstale":"2024-09-17T05:23:26.052816+0000","last_undegraded":"2024-09-17T05:23:26.052816+0000","last_fullsized":"2024-09-17T05:23:26.052816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:52.022043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053572+0000","last_change":"2024-09-17T05:23:25.065444+0000","last_active":"2024-09-17T05:23:26.053572+0000","last_peered":"2024-09-17T05:23:26.053572+0000","last_clean":"2024-09-17T05:23:26.053572+0000","last_became_active":"2024-09-17T05:23:25.064932+0000","last_became_peered":"2024-09-17T05:23:25.064932+0000","last_unstale":"2024-09-17T05:23:26.053572+0000","last_undegraded":"2024-09-17T05:23:26.053572+0000","last_fullsized":"2024-09-17T05:23:26.053572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069641+0000","last_change":"2024-09-17T05:23:24.058300+0000","last_active":"2024-09-17T05:23:26.069641+0000","last_peered":"2024-09-17T05:23:26.069641+0000","last_clean":"2024-09-17T05:23:26.069641+0000","last_became_active":"2024-09-17T05:23:24.057127+0000","last_became_peered":"2024-09-17T05:23:24.057127+0000","last_unstale":"2024-09-17T05:23:26.069641+0000","last_undegraded":"2024-09-17T05:23:26.069641+0000","last_fullsized":"2024-09-17T05:23:26.069641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071142+0000","last_change":"2024-09-17T05:23:25.067068+0000","last_active":"2024-09-17T05:23:26.071142+0000","last_peered":"2024-09-17T05:23:26.071142+0000","last_clean":"2024-09-17T05:23:26.071142+0000","last_became_active":"2024-09-17T05:23:25.066755+0000","last_became_peered":"2024-09-17T05:23:25.066755+0000","last_unstale":"2024-09-17T05:23:26.071142+0000","last_undegraded":"2024-09-17T05:23:26.071142+0000","last_fullsized":"2024-09-17T05:23:26.071142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:11.227030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069906+0000","last_change":"2024-09-17T05:23:24.053733+0000","last_active":"2024-09-17T05:23:26.069906+0000","last_peered":"2024-09-17T05:23:26.069906+0000","last_clean":"2024-09-17T05:23:26.069906+0000","last_became_active":"2024-09-17T05:23:24.053370+0000","last_became_peered":"2024-09-17T05:23:24.053370+0000","last_unstale":"2024-09-17T05:23:26.069906+0000","last_undegraded":"2024-09-17T05:23:26.069906+0000","last_fullsized":"2024-09-17T05:23:26.069906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.457450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052622+0000","last_change":"2024-09-17T05:23:25.069241+0000","last_active":"2024-09-17T05:23:26.052622+0000","last_peered":"2024-09-17T05:23:26.052622+0000","last_clean":"2024-09-17T05:23:26.052622+0000","last_became_active":"2024-09-17T05:23:25.067530+0000","last_became_peered":"2024-09-17T05:23:25.067530+0000","last_unstale":"2024-09-17T05:23:26.052622+0000","last_undegraded":"2024-09-17T05:23:26.052622+0000","last_fullsized":"2024-09-17T05:23:26.052622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069422+0000","last_change":"2024-09-17T05:23:24.069042+0000","last_active":"2024-09-17T05:23:26.069422+0000","last_peered":"2024-09-17T05:23:26.069422+0000","last_clean":"2024-09-17T05:23:26.069422+0000","last_became_active":"2024-09-17T05:23:24.068629+0000","last_became_peered":"2024-09-17T05:23:24.068629+0000","last_unstale":"2024-09-17T05:23:26.069422+0000","last_undegraded":"2024-09-17T05:23:26.069422+0000","last_fullsized":"2024-09-17T05:23:26.069422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:54.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071355+0000","last_change":"2024-09-17T05:23:24.059071+0000","last_active":"2024-09-17T05:23:26.071355+0000","last_peered":"2024-09-17T05:23:26.071355+0000","last_clean":"2024-09-17T05:23:26.071355+0000","last_became_active":"2024-09-17T05:23:24.058788+0000","last_became_peered":"2024-09-17T05:23:24.058788+0000","last_unstale":"2024-09-17T05:23:26.071355+0000","last_undegraded":"2024-09-17T05:23:26.071355+0000","last_fullsized":"2024-09-17T05:23:26.071355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:26.407828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071163+0000","last_change":"2024-09-17T05:23:25.070863+0000","last_active":"2024-09-17T05:23:26.071163+0000","last_peered":"2024-09-17T05:23:26.071163+0000","last_clean":"2024-09-17T05:23:26.071163+0000","last_became_active":"2024-09-17T05:23:25.070508+0000","last_became_peered":"2024-09-17T05:23:25.070508+0000","last_unstale":"2024-09-17T05:23:26.071163+0000","last_undegraded":"2024-09-17T05:23:26.071163+0000","last_fullsized":"2024-09-17T05:23:26.071163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:08.540151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052361+0000","last_change":"2024-09-17T05:23:24.054907+0000","last_active":"2024-09-17T05:23:26.052361+0000","last_peered":"2024-09-17T05:23:26.052361+0000","last_clean":"2024-09-17T05:23:26.052361+0000","last_became_active":"2024-09-17T05:23:24.054532+0000","last_became_peered":"2024-09-17T05:23:24.054532+0000","last_unstale":"2024-09-17T05:23:26.052361+0000","last_undegraded":"2024-09-17T05:23:26.052361+0000","last_fullsized":"2024-09-17T05:23:26.052361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053550+0000","last_change":"2024-09-17T05:23:25.072781+0000","last_active":"2024-09-17T05:23:26.053550+0000","last_peered":"2024-09-17T05:23:26.053550+0000","last_clean":"2024-09-17T05:23:26.053550+0000","last_became_active":"2024-09-17T05:23:25.072303+0000","last_became_peered":"2024-09-17T05:23:25.072303+0000","last_unstale":"2024-09-17T05:23:26.053550+0000","last_undegraded":"2024-09-17T05:23:26.053550+0000","last_fullsized":"2024-09-17T05:23:26.053550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068751+0000","last_change":"2024-09-17T05:23:24.050270+0000","last_active":"2024-09-17T05:23:26.068751+0000","last_peered":"2024-09-17T05:23:26.068751+0000","last_clean":"2024-09-17T05:23:26.068751+0000","last_became_active":"2024-09-17T05:23:24.049787+0000","last_became_peered":"2024-09-17T05:23:24.049787+0000","last_unstale":"2024-09-17T05:23:26.068751+0000","last_undegraded":"2024-09-17T05:23:26.068751+0000","last_fullsized":"2024-09-17T05:23:26.068751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071171+0000","last_change":"2024-09-17T05:23:25.065390+0000","last_active":"2024-09-17T05:23:26.071171+0000","last_peered":"2024-09-17T05:23:26.071171+0000","last_clean":"2024-09-17T05:23:26.071171+0000","last_became_active":"2024-09-17T05:23:25.065054+0000","last_became_peered":"2024-09-17T05:23:25.065054+0000","last_unstale":"2024-09-17T05:23:26.071171+0000","last_undegraded":"2024-09-17T05:23:26.071171+0000","last_fullsized":"2024-09-17T05:23:26.071171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:55.059902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053136+0000","last_change":"2024-09-17T05:23:24.066652+0000","last_active":"2024-09-17T05:23:26.053136+0000","last_peered":"2024-09-17T05:23:26.053136+0000","last_clean":"2024-09-17T05:23:26.053136+0000","last_became_active":"2024-09-17T05:23:24.065549+0000","last_became_peered":"2024-09-17T05:23:24.065549+0000","last_unstale":"2024-09-17T05:23:26.053136+0000","last_undegraded":"2024-09-17T05:23:26.053136+0000","last_fullsized":"2024-09-17T05:23:26.053136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:44.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070411+0000","last_change":"2024-09-17T05:23:25.074083+0000","last_active":"2024-09-17T05:23:26.070411+0000","last_peered":"2024-09-17T05:23:26.070411+0000","last_clean":"2024-09-17T05:23:26.070411+0000","last_became_active":"2024-09-17T05:23:25.073740+0000","last_became_peered":"2024-09-17T05:23:25.073740+0000","last_unstale":"2024-09-17T05:23:26.070411+0000","last_undegraded":"2024-09-17T05:23:26.070411+0000","last_fullsized":"2024-09-17T05:23:26.070411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:15.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070325+0000","last_change":"2024-09-17T05:23:24.060840+0000","last_active":"2024-09-17T05:23:26.070325+0000","last_peered":"2024-09-17T05:23:26.070325+0000","last_clean":"2024-09-17T05:23:26.070325+0000","last_became_active":"2024-09-17T05:23:24.060518+0000","last_became_peered":"2024-09-17T05:23:24.060518+0000","last_unstale":"2024-09-17T05:23:26.070325+0000","last_undegraded":"2024-09-17T05:23:26.070325+0000","last_fullsized":"2024-09-17T05:23:26.070325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:47.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053110+0000","last_change":"2024-09-17T05:23:25.067357+0000","last_active":"2024-09-17T05:23:26.053110+0000","last_peered":"2024-09-17T05:23:26.053110+0000","last_clean":"2024-09-17T05:23:26.053110+0000","last_became_active":"2024-09-17T05:23:25.067017+0000","last_became_peered":"2024-09-17T05:23:25.067017+0000","last_unstale":"2024-09-17T05:23:26.053110+0000","last_undegraded":"2024-09-17T05:23:26.053110+0000","last_fullsized":"2024-09-17T05:23:26.053110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:05.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052624+0000","last_change":"2024-09-17T05:23:24.058023+0000","last_active":"2024-09-17T05:23:26.052624+0000","last_peered":"2024-09-17T05:23:26.052624+0000","last_clean":"2024-09-17T05:23:26.052624+0000","last_became_active":"2024-09-17T05:23:24.057266+0000","last_became_peered":"2024-09-17T05:23:24.057266+0000","last_unstale":"2024-09-17T05:23:26.052624+0000","last_undegraded":"2024-09-17T05:23:26.052624+0000","last_fullsized":"2024-09-17T05:23:26.052624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053458+0000","last_change":"2024-09-17T05:23:25.051518+0000","last_active":"2024-09-17T05:23:26.053458+0000","last_peered":"2024-09-17T05:23:26.053458+0000","last_clean":"2024-09-17T05:23:26.053458+0000","last_became_active":"2024-09-17T05:23:25.051182+0000","last_became_peered":"2024-09-17T05:23:25.051182+0000","last_unstale":"2024-09-17T05:23:26.053458+0000","last_undegraded":"2024-09-17T05:23:26.053458+0000","last_fullsized":"2024-09-17T05:23:26.053458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071578+0000","last_change":"2024-09-17T05:23:25.085148+0000","last_active":"2024-09-17T05:23:26.071578+0000","last_peered":"2024-09-17T05:23:26.071578+0000","last_clean":"2024-09-17T05:23:26.071578+0000","last_became_active":"2024-09-17T05:23:25.083807+0000","last_became_peered":"2024-09-17T05:23:25.083807+0000","last_unstale":"2024-09-17T05:23:26.071578+0000","last_undegraded":"2024-09-17T05:23:26.071578+0000","last_fullsized":"2024-09-17T05:23:26.071578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:32.917183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053650+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:26.053650+0000","last_peered":"2024-09-17T05:23:26.053650+0000","last_clean":"2024-09-17T05:23:26.053650+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:26.053650+0000","last_undegraded":"2024-09-17T05:23:26.053650+0000","last_fullsized":"2024-09-17T05:23:26.053650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071284+0000","last_change":"2024-09-17T05:23:24.056449+0000","last_active":"2024-09-17T05:23:26.071284+0000","last_peered":"2024-09-17T05:23:26.071284+0000","last_clean":"2024-09-17T05:23:26.071284+0000","last_became_active":"2024-09-17T05:23:24.056055+0000","last_became_peered":"2024-09-17T05:23:24.056055+0000","last_unstale":"2024-09-17T05:23:26.071284+0000","last_undegraded":"2024-09-17T05:23:26.071284+0000","last_fullsized":"2024-09-17T05:23:26.071284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.624751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069696+0000","last_change":"2024-09-17T05:23:25.055754+0000","last_active":"2024-09-17T05:23:26.069696+0000","last_peered":"2024-09-17T05:23:26.069696+0000","last_clean":"2024-09-17T05:23:26.069696+0000","last_became_active":"2024-09-17T05:23:25.055368+0000","last_became_peered":"2024-09-17T05:23:25.055368+0000","last_unstale":"2024-09-17T05:23:26.069696+0000","last_undegraded":"2024-09-17T05:23:26.069696+0000","last_fullsized":"2024-09-17T05:23:26.069696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:35.911660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070994+0000","last_change":"2024-09-17T05:23:24.065017+0000","last_active":"2024-09-17T05:23:26.070994+0000","last_peered":"2024-09-17T05:23:26.070994+0000","last_clean":"2024-09-17T05:23:26.070994+0000","last_became_active":"2024-09-17T05:23:24.064688+0000","last_became_peered":"2024-09-17T05:23:24.064688+0000","last_unstale":"2024-09-17T05:23:26.070994+0000","last_undegraded":"2024-09-17T05:23:26.070994+0000","last_fullsized":"2024-09-17T05:23:26.070994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:38.132272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068472+0000","last_change":"2024-09-17T05:23:24.065406+0000","last_active":"2024-09-17T05:23:26.068472+0000","last_peered":"2024-09-17T05:23:26.068472+0000","last_clean":"2024-09-17T05:23:26.068472+0000","last_became_active":"2024-09-17T05:23:24.065058+0000","last_became_peered":"2024-09-17T05:23:24.065058+0000","last_unstale":"2024-09-17T05:23:26.068472+0000","last_undegraded":"2024-09-17T05:23:26.068472+0000","last_fullsized":"2024-09-17T05:23:26.068472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053208+0000","last_change":"2024-09-17T05:23:25.074915+0000","last_active":"2024-09-17T05:23:26.053208+0000","last_peered":"2024-09-17T05:23:26.053208+0000","last_clean":"2024-09-17T05:23:26.053208+0000","last_became_active":"2024-09-17T05:23:25.074603+0000","last_became_peered":"2024-09-17T05:23:25.074603+0000","last_unstale":"2024-09-17T05:23:26.053208+0000","last_undegraded":"2024-09-17T05:23:26.053208+0000","last_fullsized":"2024-09-17T05:23:26.053208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071423+0000","last_change":"2024-09-17T05:23:24.058441+0000","last_active":"2024-09-17T05:23:26.071423+0000","last_peered":"2024-09-17T05:23:26.071423+0000","last_clean":"2024-09-17T05:23:26.071423+0000","last_became_active":"2024-09-17T05:23:24.054951+0000","last_became_peered":"2024-09-17T05:23:24.054951+0000","last_unstale":"2024-09-17T05:23:26.071423+0000","last_undegraded":"2024-09-17T05:23:26.071423+0000","last_fullsized":"2024-09-17T05:23:26.071423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:11.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052393+0000","last_change":"2024-09-17T05:23:25.056791+0000","last_active":"2024-09-17T05:23:26.052393+0000","last_peered":"2024-09-17T05:23:26.052393+0000","last_clean":"2024-09-17T05:23:26.052393+0000","last_became_active":"2024-09-17T05:23:25.056445+0000","last_became_peered":"2024-09-17T05:23:25.056445+0000","last_unstale":"2024-09-17T05:23:26.052393+0000","last_undegraded":"2024-09-17T05:23:26.052393+0000","last_fullsized":"2024-09-17T05:23:26.052393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052571+0000","last_change":"2024-09-17T05:23:24.058736+0000","last_active":"2024-09-17T05:23:26.052571+0000","last_peered":"2024-09-17T05:23:26.052571+0000","last_clean":"2024-09-17T05:23:26.052571+0000","last_became_active":"2024-09-17T05:23:24.056996+0000","last_became_peered":"2024-09-17T05:23:24.056996+0000","last_unstale":"2024-09-17T05:23:26.052571+0000","last_undegraded":"2024-09-17T05:23:26.052571+0000","last_fullsized":"2024-09-17T05:23:26.052571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:09.319432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070023+0000","last_change":"2024-09-17T05:23:25.072427+0000","last_active":"2024-09-17T05:23:26.070023+0000","last_peered":"2024-09-17T05:23:26.070023+0000","last_clean":"2024-09-17T05:23:26.070023+0000","last_became_active":"2024-09-17T05:23:25.071951+0000","last_became_peered":"2024-09-17T05:23:25.071951+0000","last_unstale":"2024-09-17T05:23:26.070023+0000","last_undegraded":"2024-09-17T05:23:26.070023+0000","last_fullsized":"2024-09-17T05:23:26.070023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:45.608054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071621+0000","last_change":"2024-09-17T05:23:24.064058+0000","last_active":"2024-09-17T05:23:26.071621+0000","last_peered":"2024-09-17T05:23:26.071621+0000","last_clean":"2024-09-17T05:23:26.071621+0000","last_became_active":"2024-09-17T05:23:24.063686+0000","last_became_peered":"2024-09-17T05:23:24.063686+0000","last_unstale":"2024-09-17T05:23:26.071621+0000","last_undegraded":"2024-09-17T05:23:26.071621+0000","last_fullsized":"2024-09-17T05:23:26.071621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:03.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070421+0000","last_change":"2024-09-17T05:23:24.061808+0000","last_active":"2024-09-17T05:23:26.070421+0000","last_peered":"2024-09-17T05:23:26.070421+0000","last_clean":"2024-09-17T05:23:26.070421+0000","last_became_active":"2024-09-17T05:23:24.061393+0000","last_became_peered":"2024-09-17T05:23:24.061393+0000","last_unstale":"2024-09-17T05:23:26.070421+0000","last_undegraded":"2024-09-17T05:23:26.070421+0000","last_fullsized":"2024-09-17T05:23:26.070421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:10.321148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070599+0000","last_change":"2024-09-17T05:23:25.065038+0000","last_active":"2024-09-17T05:23:26.070599+0000","last_peered":"2024-09-17T05:23:26.070599+0000","last_clean":"2024-09-17T05:23:26.070599+0000","last_became_active":"2024-09-17T05:23:25.063557+0000","last_became_peered":"2024-09-17T05:23:25.063557+0000","last_unstale":"2024-09-17T05:23:26.070599+0000","last_undegraded":"2024-09-17T05:23:26.070599+0000","last_fullsized":"2024-09-17T05:23:26.070599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:46.016555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071788+0000","last_change":"2024-09-17T05:23:24.061738+0000","last_active":"2024-09-17T05:23:26.071788+0000","last_peered":"2024-09-17T05:23:26.071788+0000","last_clean":"2024-09-17T05:23:26.071788+0000","last_became_active":"2024-09-17T05:23:24.061344+0000","last_became_peered":"2024-09-17T05:23:24.061344+0000","last_unstale":"2024-09-17T05:23:26.071788+0000","last_undegraded":"2024-09-17T05:23:26.071788+0000","last_fullsized":"2024-09-17T05:23:26.071788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:00.795235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070211+0000","last_change":"2024-09-17T05:23:25.059622+0000","last_active":"2024-09-17T05:23:26.070211+0000","last_peered":"2024-09-17T05:23:26.070211+0000","last_clean":"2024-09-17T05:23:26.070211+0000","last_became_active":"2024-09-17T05:23:25.059267+0000","last_became_peered":"2024-09-17T05:23:25.059267+0000","last_unstale":"2024-09-17T05:23:26.070211+0000","last_undegraded":"2024-09-17T05:23:26.070211+0000","last_fullsized":"2024-09-17T05:23:26.070211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:58.979778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.051888+0000","last_change":"2024-09-17T05:23:24.053175+0000","last_active":"2024-09-17T05:23:26.051888+0000","last_peered":"2024-09-17T05:23:26.051888+0000","last_clean":"2024-09-17T05:23:26.051888+0000","last_became_active":"2024-09-17T05:23:24.052859+0000","last_became_peered":"2024-09-17T05:23:24.052859+0000","last_unstale":"2024-09-17T05:23:26.051888+0000","last_undegraded":"2024-09-17T05:23:26.051888+0000","last_fullsized":"2024-09-17T05:23:26.051888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071095+0000","last_change":"2024-09-17T05:23:25.057826+0000","last_active":"2024-09-17T05:23:26.071095+0000","last_peered":"2024-09-17T05:23:26.071095+0000","last_clean":"2024-09-17T05:23:26.071095+0000","last_became_active":"2024-09-17T05:23:25.057507+0000","last_became_peered":"2024-09-17T05:23:25.057507+0000","last_unstale":"2024-09-17T05:23:26.071095+0000","last_undegraded":"2024-09-17T05:23:26.071095+0000","last_fullsized":"2024-09-17T05:23:26.071095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068904+0000","last_change":"2024-09-17T05:23:24.068858+0000","last_active":"2024-09-17T05:23:26.068904+0000","last_peered":"2024-09-17T05:23:26.068904+0000","last_clean":"2024-09-17T05:23:26.068904+0000","last_became_active":"2024-09-17T05:23:24.068387+0000","last_became_peered":"2024-09-17T05:23:24.068387+0000","last_unstale":"2024-09-17T05:23:26.068904+0000","last_undegraded":"2024-09-17T05:23:26.068904+0000","last_fullsized":"2024-09-17T05:23:26.068904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068658+0000","last_change":"2024-09-17T05:23:25.049296+0000","last_active":"2024-09-17T05:23:26.068658+0000","last_peered":"2024-09-17T05:23:26.068658+0000","last_clean":"2024-09-17T05:23:26.068658+0000","last_became_active":"2024-09-17T05:23:25.048948+0000","last_became_peered":"2024-09-17T05:23:25.048948+0000","last_unstale":"2024-09-17T05:23:26.068658+0000","last_undegraded":"2024-09-17T05:23:26.068658+0000","last_fullsized":"2024-09-17T05:23:26.068658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:49.995105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069337+0000","last_change":"2024-09-17T05:23:25.067211+0000","last_active":"2024-09-17T05:23:26.069337+0000","last_peered":"2024-09-17T05:23:26.069337+0000","last_clean":"2024-09-17T05:23:26.069337+0000","last_became_active":"2024-09-17T05:23:25.066883+0000","last_became_peered":"2024-09-17T05:23:25.066883+0000","last_unstale":"2024-09-17T05:23:26.069337+0000","last_undegraded":"2024-09-17T05:23:26.069337+0000","last_fullsized":"2024-09-17T05:23:26.069337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071212+0000","last_change":"2024-09-17T05:23:24.061156+0000","last_active":"2024-09-17T05:23:26.071212+0000","last_peered":"2024-09-17T05:23:26.071212+0000","last_clean":"2024-09-17T05:23:26.071212+0000","last_became_active":"2024-09-17T05:23:24.060751+0000","last_became_peered":"2024-09-17T05:23:24.060751+0000","last_unstale":"2024-09-17T05:23:26.071212+0000","last_undegraded":"2024-09-17T05:23:26.071212+0000","last_fullsized":"2024-09-17T05:23:26.071212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.506573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071131+0000","last_change":"2024-09-17T05:23:25.085054+0000","last_active":"2024-09-17T05:23:26.071131+0000","last_peered":"2024-09-17T05:23:26.071131+0000","last_clean":"2024-09-17T05:23:26.071131+0000","last_became_active":"2024-09-17T05:23:25.084307+0000","last_became_peered":"2024-09-17T05:23:25.084307+0000","last_unstale":"2024-09-17T05:23:26.071131+0000","last_undegraded":"2024-09-17T05:23:26.071131+0000","last_fullsized":"2024-09-17T05:23:26.071131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071557+0000","last_change":"2024-09-17T05:23:25.052805+0000","last_active":"2024-09-17T05:23:26.071557+0000","last_peered":"2024-09-17T05:23:26.071557+0000","last_clean":"2024-09-17T05:23:26.071557+0000","last_became_active":"2024-09-17T05:23:25.052440+0000","last_became_peered":"2024-09-17T05:23:25.052440+0000","last_unstale":"2024-09-17T05:23:26.071557+0000","last_undegraded":"2024-09-17T05:23:26.071557+0000","last_fullsized":"2024-09-17T05:23:26.071557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:51.739703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053334+0000","last_change":"2024-09-17T05:23:24.061930+0000","last_active":"2024-09-17T05:23:26.053334+0000","last_peered":"2024-09-17T05:23:26.053334+0000","last_clean":"2024-09-17T05:23:26.053334+0000","last_became_active":"2024-09-17T05:23:24.060988+0000","last_became_peered":"2024-09-17T05:23:24.060988+0000","last_unstale":"2024-09-17T05:23:26.053334+0000","last_undegraded":"2024-09-17T05:23:26.053334+0000","last_fullsized":"2024-09-17T05:23:26.053334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:47.573508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054561+0000","last_change":"2024-09-17T05:23:25.065743+0000","last_active":"2024-09-17T05:23:26.054561+0000","last_peered":"2024-09-17T05:23:26.054561+0000","last_clean":"2024-09-17T05:23:26.054561+0000","last_became_active":"2024-09-17T05:23:25.062840+0000","last_became_peered":"2024-09-17T05:23:25.062840+0000","last_unstale":"2024-09-17T05:23:26.054561+0000","last_undegraded":"2024-09-17T05:23:26.054561+0000","last_fullsized":"2024-09-17T05:23:26.054561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:01.803022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052543+0000","last_change":"2024-09-17T05:23:24.060124+0000","last_active":"2024-09-17T05:23:26.052543+0000","last_peered":"2024-09-17T05:23:26.052543+0000","last_clean":"2024-09-17T05:23:26.052543+0000","last_became_active":"2024-09-17T05:23:24.057538+0000","last_became_peered":"2024-09-17T05:23:24.057538+0000","last_unstale":"2024-09-17T05:23:26.052543+0000","last_undegraded":"2024-09-17T05:23:26.052543+0000","last_fullsized":"2024-09-17T05:23:26.052543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:41.202615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053337+0000","last_change":"2024-09-17T05:23:25.079567+0000","last_active":"2024-09-17T05:23:26.053337+0000","last_peered":"2024-09-17T05:23:26.053337+0000","last_clean":"2024-09-17T05:23:26.053337+0000","last_became_active":"2024-09-17T05:23:25.078899+0000","last_became_peered":"2024-09-17T05:23:25.078899+0000","last_unstale":"2024-09-17T05:23:26.053337+0000","last_undegraded":"2024-09-17T05:23:26.053337+0000","last_fullsized":"2024-09-17T05:23:26.053337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:39.680169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053486+0000","last_change":"2024-09-17T05:23:24.061891+0000","last_active":"2024-09-17T05:23:26.053486+0000","last_peered":"2024-09-17T05:23:26.053486+0000","last_clean":"2024-09-17T05:23:26.053486+0000","last_became_active":"2024-09-17T05:23:24.055527+0000","last_became_peered":"2024-09-17T05:23:24.055527+0000","last_unstale":"2024-09-17T05:23:26.053486+0000","last_undegraded":"2024-09-17T05:23:26.053486+0000","last_fullsized":"2024-09-17T05:23:26.053486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053220+0000","last_change":"2024-09-17T05:23:25.076838+0000","last_active":"2024-09-17T05:23:26.053220+0000","last_peered":"2024-09-17T05:23:26.053220+0000","last_clean":"2024-09-17T05:23:26.053220+0000","last_became_active":"2024-09-17T05:23:25.076547+0000","last_became_peered":"2024-09-17T05:23:25.076547+0000","last_unstale":"2024-09-17T05:23:26.053220+0000","last_undegraded":"2024-09-17T05:23:26.053220+0000","last_fullsized":"2024-09-17T05:23:26.053220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.142652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070483+0000","last_change":"2024-09-17T05:23:24.059741+0000","last_active":"2024-09-17T05:23:26.070483+0000","last_peered":"2024-09-17T05:23:26.070483+0000","last_clean":"2024-09-17T05:23:26.070483+0000","last_became_active":"2024-09-17T05:23:24.058336+0000","last_became_peered":"2024-09-17T05:23:24.058336+0000","last_unstale":"2024-09-17T05:23:26.070483+0000","last_undegraded":"2024-09-17T05:23:26.070483+0000","last_fullsized":"2024-09-17T05:23:26.070483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:01.640938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:30.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- 172.21.15.43:0/1081760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300d0d80 msgr2=0x7f86300cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 --2- 172.21.15.43:0/1081760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300d0d80 0x7f86300cce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f861c009900 tx=0x7f861c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- 172.21.15.43:0/1081760 shutdown_connections 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 --2- 172.21.15.43:0/1081760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300d0d80 0x7f86300cce70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 --2- 172.21.15.43:0/1081760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300b7ce0 0x7f86300d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- 172.21.15.43:0/1081760 >> 172.21.15.43:0/1081760 conn(0x7f86300cee80 msgr2=0x7f86300cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- 172.21.15.43:0/1081760 shutdown_connections 2024-09-17T05:23:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- 172.21.15.43:0/1081760 wait complete. 2024-09-17T05:23:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 Processor -- start 2024-09-17T05:23:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.288+0000 7f8638000640 1 -- start start 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300cd3b0 0x7f8630152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 0x7f8630153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8630169e10 con 0x7f86300b7ce0 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8630169c90 con 0x7f86300d0d80 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8630169f90 con 0x7f86300cd3b0 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49412/0 (socket says 172.21.15.43:49412) 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 -- 172.21.15.43:0/2270501139 learned_addr learned my addr 172.21.15.43:0/2270501139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8634c00640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300cd3b0 0x7f8630152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8636000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 0x7f8630153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 -- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300cd3b0 msgr2=0x7f8630152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300cd3b0 0x7f8630152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 -- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 msgr2=0x7f8630153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 0x7f8630153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 -- 172.21.15.43:0/2270501139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86301f9630 con 0x7f86300b7ce0 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8636000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 0x7f8630153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8635600640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f862400c970 tx=0x7f862400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624007bb0 con 0x7f86300b7ce0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8624007e40 con 0x7f86300b7ce0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624004a50 con 0x7f86300b7ce0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86301f8870 con 0x7f86300b7ce0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301f8e30 con 0x7f86300b7ce0 2024-09-17T05:23:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.292+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86300d0840 con 0x7f86300b7ce0 2024-09-17T05:23:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.296+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8624004bf0 con 0x7f86300b7ce0 2024-09-17T05:23:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.296+0000 7f862d600640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 0x7f8610075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.296+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8624094770 con 0x7f86300b7ce0 2024-09-17T05:23:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.296+0000 7f8634c00640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 0x7f8610075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.296+0000 7f8634c00640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 0x7f8610075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f861c0af000 tx=0x7f861c01b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:30.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.300+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8624014030 con 0x7f86300b7ce0 2024-09-17T05:23:30.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.536+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8630157d40 con 0x7f86300b7ce0 2024-09-17T05:23:30.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.536+0000 7f862d600640 1 -- 172.21.15.43:0/2270501139 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1053 (secure 0 0 0) 0x7f862405f790 con 0x7f86300b7ce0 2024-09-17T05:23:30.537 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:30.537 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":32,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550593,"num_in_osds":8,"osd_in_since":1726550582,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241520640,"bytes_avail":772852592640,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4384}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T05:22:58.376604+0000","services":{}},"progress_events":{}} 2024-09-17T05:23:30.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 msgr2=0x7f8610075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 0x7f8610075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f861c0af000 tx=0x7f861c01b120 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 msgr2=0x7f863014e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f862400c970 tx=0x7f862400ce30 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 shutdown_connections 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8610073200 0x7f8610075710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86300d0d80 0x7f8630153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f86300cd3b0 0x7f8630152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 --2- 172.21.15.43:0/2270501139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86300b7ce0 0x7f863014e140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 >> 172.21.15.43:0/2270501139 conn(0x7f86300cee80 msgr2=0x7f86300ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 shutdown_connections 2024-09-17T05:23:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.540+0000 7f8638000640 1 -- 172.21.15.43:0/2270501139 wait complete. 2024-09-17T05:23:30.562 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:23:30.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 -- 172.21.15.43:0/1004072435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8003ad0 msgr2=0x7fd3d8003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 --2- 172.21.15.43:0/1004072435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3d8003ad0 0x7fd3d8003f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009b10 tx=0x7fd3e401d250 comp rx=0 tx=0).stop 2024-09-17T05:23:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f3400640 1 -- 172.21.15.43:0/1004072435 reap_dead start 2024-09-17T05:23:30.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 -- 172.21.15.43:0/1004072435 shutdown_connections 2024-09-17T05:23:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 -- 172.21.15.43:0/1004072435 >> 172.21.15.43:0/1004072435 conn(0x7fd3f40b23d0 msgr2=0x7fd3f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 -- 172.21.15.43:0/1004072435 shutdown_connections 2024-09-17T05:23:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.724+0000 7fd3f9600640 1 -- 172.21.15.43:0/1004072435 wait complete. 2024-09-17T05:23:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 Processor -- start 2024-09-17T05:23:30.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- start start 2024-09-17T05:23:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3d8003ad0 0x7fd3f409cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3f40748b0 0x7fd3f40c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4167a80 con 0x7fd3f40744a0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4167900 con 0x7fd3d8003ad0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f4167c00 con 0x7fd3f40748b0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49422/0 (socket says 172.21.15.43:49422) 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 -- 172.21.15.43:0/2527270540 learned_addr learned my addr 172.21.15.43:0/2527270540 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f2a00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3d8003ad0 0x7fd3f409cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f3400640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3f40748b0 0x7fd3f40c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 -- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3f40748b0 msgr2=0x7fd3f40c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3f40748b0 0x7fd3f40c31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 -- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3d8003ad0 msgr2=0x7fd3f409cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3d8003ad0 0x7fd3f409cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 -- 172.21.15.43:0/2527270540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41fc0c0 con 0x7fd3f40744a0 2024-09-17T05:23:30.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3ebe00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc004fd0 tx=0x7fd3dc00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc015020 con 0x7fd3f40744a0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3dc010980 con 0x7fd3f40744a0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc010db0 con 0x7fd3f40744a0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41fc3a0 con 0x7fd3f40744a0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.728+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41fc960 con 0x7fd3f40744a0 2024-09-17T05:23:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f40ad0b0 con 0x7fd3f40744a0 2024-09-17T05:23:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd3dc018020 con 0x7fd3f40744a0 2024-09-17T05:23:30.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3eaa00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 0x7fd3d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd3dc0946f0 con 0x7fd3f40744a0 2024-09-17T05:23:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3f2a00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 0x7fd3d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:30.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.732+0000 7fd3f2a00640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 0x7fd3d0075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40ae000 tx=0x7fd3e4019ff0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:30.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.736+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd3dc01f020 con 0x7fd3f40744a0 2024-09-17T05:23:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.908+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3f41fbe70 con 0x7fd3d0073200 2024-09-17T05:23:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.916+0000 7fd3eaa00640 1 -- 172.21.15.43:0/2527270540 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7fd3f41fbe70 con 0x7fd3d0073200 2024-09-17T05:23:30.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 msgr2=0x7fd3d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 0x7fd3d0075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40ae000 tx=0x7fd3e4019ff0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 msgr2=0x7fd3f40c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc004fd0 tx=0x7fd3dc00ea20 comp rx=0 tx=0).stop 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 shutdown_connections 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd3d0073200 0x7fd3d0075710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3f40748b0 0x7fd3f40c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3f40744a0 0x7fd3f40c2c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 --2- 172.21.15.43:0/2527270540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3d8003ad0 0x7fd3f409cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 >> 172.21.15.43:0/2527270540 conn(0x7fd3f40b23d0 msgr2=0x7fd3f40b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 shutdown_connections 2024-09-17T05:23:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:30.920+0000 7fd3f9600640 1 -- 172.21.15.43:0/2527270540 wait complete. 2024-09-17T05:23:30.937 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T05:23:29.839861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235860,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738860,"statfs":{"total":773094113280,"available":772852592640,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053057+0000","last_change":"2024-09-17T05:23:25.072214+0000","last_active":"2024-09-17T05:23:26.053057+0000","last_peered":"2024-09-17T05:23:26.053057+0000","last_clean":"2024-09-17T05:23:26.053057+0000","last_became_active":"2024-09-17T05:23:25.071839+0000","last_became_peered":"2024-09-17T05:23:25.071839+0000","last_unstale":"2024-09-17T05:23:26.053057+0000","last_undegraded":"2024-09-17T05:23:26.053057+0000","last_fullsized":"2024-09-17T05:23:26.053057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052684+0000","last_change":"2024-09-17T05:23:25.068245+0000","last_active":"2024-09-17T05:23:26.052684+0000","last_peered":"2024-09-17T05:23:26.052684+0000","last_clean":"2024-09-17T05:23:26.052684+0000","last_became_active":"2024-09-17T05:23:25.063871+0000","last_became_peered":"2024-09-17T05:23:25.063871+0000","last_unstale":"2024-09-17T05:23:26.052684+0000","last_undegraded":"2024-09-17T05:23:26.052684+0000","last_fullsized":"2024-09-17T05:23:26.052684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:20.196005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052943+0000","last_change":"2024-09-17T05:23:25.077572+0000","last_active":"2024-09-17T05:23:26.052943+0000","last_peered":"2024-09-17T05:23:26.052943+0000","last_clean":"2024-09-17T05:23:26.052943+0000","last_became_active":"2024-09-17T05:23:25.077232+0000","last_became_peered":"2024-09-17T05:23:25.077232+0000","last_unstale":"2024-09-17T05:23:26.052943+0000","last_undegraded":"2024-09-17T05:23:26.052943+0000","last_fullsized":"2024-09-17T05:23:26.052943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070147+0000","last_change":"2024-09-17T05:23:25.080315+0000","last_active":"2024-09-17T05:23:26.070147+0000","last_peered":"2024-09-17T05:23:26.070147+0000","last_clean":"2024-09-17T05:23:26.070147+0000","last_became_active":"2024-09-17T05:23:25.079991+0000","last_became_peered":"2024-09-17T05:23:25.079991+0000","last_unstale":"2024-09-17T05:23:26.070147+0000","last_undegraded":"2024-09-17T05:23:26.070147+0000","last_fullsized":"2024-09-17T05:23:26.070147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:58.988827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052636+0000","last_change":"2024-09-17T05:23:25.062672+0000","last_active":"2024-09-17T05:23:26.052636+0000","last_peered":"2024-09-17T05:23:26.052636+0000","last_clean":"2024-09-17T05:23:26.052636+0000","last_became_active":"2024-09-17T05:23:25.060858+0000","last_became_peered":"2024-09-17T05:23:25.060858+0000","last_unstale":"2024-09-17T05:23:26.052636+0000","last_undegraded":"2024-09-17T05:23:26.052636+0000","last_fullsized":"2024-09-17T05:23:26.052636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.102130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069891+0000","last_change":"2024-09-17T05:23:25.065376+0000","last_active":"2024-09-17T05:23:26.069891+0000","last_peered":"2024-09-17T05:23:26.069891+0000","last_clean":"2024-09-17T05:23:26.069891+0000","last_became_active":"2024-09-17T05:23:25.064806+0000","last_became_peered":"2024-09-17T05:23:25.064806+0000","last_unstale":"2024-09-17T05:23:26.069891+0000","last_undegraded":"2024-09-17T05:23:26.069891+0000","last_fullsized":"2024-09-17T05:23:26.069891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052148+0000","last_change":"2024-09-17T05:23:25.053855+0000","last_active":"2024-09-17T05:23:26.052148+0000","last_peered":"2024-09-17T05:23:26.052148+0000","last_clean":"2024-09-17T05:23:26.052148+0000","last_became_active":"2024-09-17T05:23:25.053569+0000","last_became_peered":"2024-09-17T05:23:25.053569+0000","last_unstale":"2024-09-17T05:23:26.052148+0000","last_undegraded":"2024-09-17T05:23:26.052148+0000","last_fullsized":"2024-09-17T05:23:26.052148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052419+0000","last_change":"2024-09-17T05:23:25.056925+0000","last_active":"2024-09-17T05:23:26.052419+0000","last_peered":"2024-09-17T05:23:26.052419+0000","last_clean":"2024-09-17T05:23:26.052419+0000","last_became_active":"2024-09-17T05:23:25.056619+0000","last_became_peered":"2024-09-17T05:23:25.056619+0000","last_unstale":"2024-09-17T05:23:26.052419+0000","last_undegraded":"2024-09-17T05:23:26.052419+0000","last_fullsized":"2024-09-17T05:23:26.052419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054267+0000","last_change":"2024-09-17T05:23:25.061010+0000","last_active":"2024-09-17T05:23:26.054267+0000","last_peered":"2024-09-17T05:23:26.054267+0000","last_clean":"2024-09-17T05:23:26.054267+0000","last_became_active":"2024-09-17T05:23:25.060714+0000","last_became_peered":"2024-09-17T05:23:25.060714+0000","last_unstale":"2024-09-17T05:23:26.054267+0000","last_undegraded":"2024-09-17T05:23:26.054267+0000","last_fullsized":"2024-09-17T05:23:26.054267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.664289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052921+0000","last_change":"2024-09-17T05:23:25.079732+0000","last_active":"2024-09-17T05:23:26.052921+0000","last_peered":"2024-09-17T05:23:26.052921+0000","last_clean":"2024-09-17T05:23:26.052921+0000","last_became_active":"2024-09-17T05:23:25.079100+0000","last_became_peered":"2024-09-17T05:23:25.079100+0000","last_unstale":"2024-09-17T05:23:26.052921+0000","last_undegraded":"2024-09-17T05:23:26.052921+0000","last_fullsized":"2024-09-17T05:23:26.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:01.901521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052498+0000","last_change":"2024-09-17T05:23:25.075619+0000","last_active":"2024-09-17T05:23:26.052498+0000","last_peered":"2024-09-17T05:23:26.052498+0000","last_clean":"2024-09-17T05:23:26.052498+0000","last_became_active":"2024-09-17T05:23:25.071655+0000","last_became_peered":"2024-09-17T05:23:25.071655+0000","last_unstale":"2024-09-17T05:23:26.052498+0000","last_undegraded":"2024-09-17T05:23:26.052498+0000","last_fullsized":"2024-09-17T05:23:26.052498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052932+0000","last_change":"2024-09-17T05:23:25.077778+0000","last_active":"2024-09-17T05:23:26.052932+0000","last_peered":"2024-09-17T05:23:26.052932+0000","last_clean":"2024-09-17T05:23:26.052932+0000","last_became_active":"2024-09-17T05:23:25.077040+0000","last_became_peered":"2024-09-17T05:23:25.077040+0000","last_unstale":"2024-09-17T05:23:26.052932+0000","last_undegraded":"2024-09-17T05:23:26.052932+0000","last_fullsized":"2024-09-17T05:23:26.052932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052584+0000","last_change":"2024-09-17T05:23:25.067070+0000","last_active":"2024-09-17T05:23:26.052584+0000","last_peered":"2024-09-17T05:23:26.052584+0000","last_clean":"2024-09-17T05:23:26.052584+0000","last_became_active":"2024-09-17T05:23:25.066734+0000","last_became_peered":"2024-09-17T05:23:25.066734+0000","last_unstale":"2024-09-17T05:23:26.052584+0000","last_undegraded":"2024-09-17T05:23:26.052584+0000","last_fullsized":"2024-09-17T05:23:26.052584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:58.712013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070882+0000","last_change":"2024-09-17T05:23:25.066014+0000","last_active":"2024-09-17T05:23:26.070882+0000","last_peered":"2024-09-17T05:23:26.070882+0000","last_clean":"2024-09-17T05:23:26.070882+0000","last_became_active":"2024-09-17T05:23:25.063873+0000","last_became_peered":"2024-09-17T05:23:25.063873+0000","last_unstale":"2024-09-17T05:23:26.070882+0000","last_undegraded":"2024-09-17T05:23:26.070882+0000","last_fullsized":"2024-09-17T05:23:26.070882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071924+0000","last_change":"2024-09-17T05:23:25.055365+0000","last_active":"2024-09-17T05:23:26.071924+0000","last_peered":"2024-09-17T05:23:26.071924+0000","last_clean":"2024-09-17T05:23:26.071924+0000","last_became_active":"2024-09-17T05:23:25.055046+0000","last_became_peered":"2024-09-17T05:23:25.055046+0000","last_unstale":"2024-09-17T05:23:26.071924+0000","last_undegraded":"2024-09-17T05:23:26.071924+0000","last_fullsized":"2024-09-17T05:23:26.071924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:48.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070072+0000","last_change":"2024-09-17T05:23:25.076193+0000","last_active":"2024-09-17T05:23:26.070072+0000","last_peered":"2024-09-17T05:23:26.070072+0000","last_clean":"2024-09-17T05:23:26.070072+0000","last_became_active":"2024-09-17T05:23:25.075582+0000","last_became_peered":"2024-09-17T05:23:25.075582+0000","last_unstale":"2024-09-17T05:23:26.070072+0000","last_undegraded":"2024-09-17T05:23:26.070072+0000","last_fullsized":"2024-09-17T05:23:26.070072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071700+0000","last_change":"2024-09-17T05:23:25.053491+0000","last_active":"2024-09-17T05:23:26.071700+0000","last_peered":"2024-09-17T05:23:26.071700+0000","last_clean":"2024-09-17T05:23:26.071700+0000","last_became_active":"2024-09-17T05:23:25.053209+0000","last_became_peered":"2024-09-17T05:23:25.053209+0000","last_unstale":"2024-09-17T05:23:26.071700+0000","last_undegraded":"2024-09-17T05:23:26.071700+0000","last_fullsized":"2024-09-17T05:23:26.071700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:33.151198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053156+0000","last_change":"2024-09-17T05:23:25.068427+0000","last_active":"2024-09-17T05:23:26.053156+0000","last_peered":"2024-09-17T05:23:26.053156+0000","last_clean":"2024-09-17T05:23:26.053156+0000","last_became_active":"2024-09-17T05:23:25.068043+0000","last_became_peered":"2024-09-17T05:23:25.068043+0000","last_unstale":"2024-09-17T05:23:26.053156+0000","last_undegraded":"2024-09-17T05:23:26.053156+0000","last_fullsized":"2024-09-17T05:23:26.053156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070102+0000","last_change":"2024-09-17T05:23:25.057241+0000","last_active":"2024-09-17T05:23:26.070102+0000","last_peered":"2024-09-17T05:23:26.070102+0000","last_clean":"2024-09-17T05:23:26.070102+0000","last_became_active":"2024-09-17T05:23:25.056934+0000","last_became_peered":"2024-09-17T05:23:25.056934+0000","last_unstale":"2024-09-17T05:23:26.070102+0000","last_undegraded":"2024-09-17T05:23:26.070102+0000","last_fullsized":"2024-09-17T05:23:26.070102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054089+0000","last_change":"2024-09-17T05:23:25.064365+0000","last_active":"2024-09-17T05:23:26.054089+0000","last_peered":"2024-09-17T05:23:26.054089+0000","last_clean":"2024-09-17T05:23:26.054089+0000","last_became_active":"2024-09-17T05:23:25.064064+0000","last_became_peered":"2024-09-17T05:23:25.064064+0000","last_unstale":"2024-09-17T05:23:26.054089+0000","last_undegraded":"2024-09-17T05:23:26.054089+0000","last_fullsized":"2024-09-17T05:23:26.054089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052827+0000","last_change":"2024-09-17T05:23:25.059278+0000","last_active":"2024-09-17T05:23:26.052827+0000","last_peered":"2024-09-17T05:23:26.052827+0000","last_clean":"2024-09-17T05:23:26.052827+0000","last_became_active":"2024-09-17T05:23:25.058926+0000","last_became_peered":"2024-09-17T05:23:25.058926+0000","last_unstale":"2024-09-17T05:23:26.052827+0000","last_undegraded":"2024-09-17T05:23:26.052827+0000","last_fullsized":"2024-09-17T05:23:26.052827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070891+0000","last_change":"2024-09-17T05:23:25.064634+0000","last_active":"2024-09-17T05:23:26.070891+0000","last_peered":"2024-09-17T05:23:26.070891+0000","last_clean":"2024-09-17T05:23:26.070891+0000","last_became_active":"2024-09-17T05:23:25.064250+0000","last_became_peered":"2024-09-17T05:23:25.064250+0000","last_unstale":"2024-09-17T05:23:26.070891+0000","last_undegraded":"2024-09-17T05:23:26.070891+0000","last_fullsized":"2024-09-17T05:23:26.070891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053318+0000","last_change":"2024-09-17T05:23:25.079930+0000","last_active":"2024-09-17T05:23:26.053318+0000","last_peered":"2024-09-17T05:23:26.053318+0000","last_clean":"2024-09-17T05:23:26.053318+0000","last_became_active":"2024-09-17T05:23:25.079276+0000","last_became_peered":"2024-09-17T05:23:25.079276+0000","last_unstale":"2024-09-17T05:23:26.053318+0000","last_undegraded":"2024-09-17T05:23:26.053318+0000","last_fullsized":"2024-09-17T05:23:26.053318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070179+0000","last_change":"2024-09-17T05:23:25.083629+0000","last_active":"2024-09-17T05:23:26.070179+0000","last_peered":"2024-09-17T05:23:26.070179+0000","last_clean":"2024-09-17T05:23:26.070179+0000","last_became_active":"2024-09-17T05:23:25.083394+0000","last_became_peered":"2024-09-17T05:23:25.083394+0000","last_unstale":"2024-09-17T05:23:26.070179+0000","last_undegraded":"2024-09-17T05:23:26.070179+0000","last_fullsized":"2024-09-17T05:23:26.070179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:22.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053114+0000","last_change":"2024-09-17T05:23:25.075408+0000","last_active":"2024-09-17T05:23:26.053114+0000","last_peered":"2024-09-17T05:23:26.053114+0000","last_clean":"2024-09-17T05:23:26.053114+0000","last_became_active":"2024-09-17T05:23:25.075101+0000","last_became_peered":"2024-09-17T05:23:25.075101+0000","last_unstale":"2024-09-17T05:23:26.053114+0000","last_undegraded":"2024-09-17T05:23:26.053114+0000","last_fullsized":"2024-09-17T05:23:26.053114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071351+0000","last_change":"2024-09-17T05:23:25.068874+0000","last_active":"2024-09-17T05:23:26.071351+0000","last_peered":"2024-09-17T05:23:26.071351+0000","last_clean":"2024-09-17T05:23:26.071351+0000","last_became_active":"2024-09-17T05:23:25.064181+0000","last_became_peered":"2024-09-17T05:23:25.064181+0000","last_unstale":"2024-09-17T05:23:26.071351+0000","last_undegraded":"2024-09-17T05:23:26.071351+0000","last_fullsized":"2024-09-17T05:23:26.071351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:43.495535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068665+0000","last_change":"2024-09-17T05:23:25.064004+0000","last_active":"2024-09-17T05:23:26.068665+0000","last_peered":"2024-09-17T05:23:26.068665+0000","last_clean":"2024-09-17T05:23:26.068665+0000","last_became_active":"2024-09-17T05:23:25.063599+0000","last_became_peered":"2024-09-17T05:23:25.063599+0000","last_unstale":"2024-09-17T05:23:26.068665+0000","last_undegraded":"2024-09-17T05:23:26.068665+0000","last_fullsized":"2024-09-17T05:23:26.068665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.655466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069668+0000","last_change":"2024-09-17T05:23:25.071820+0000","last_active":"2024-09-17T05:23:26.069668+0000","last_peered":"2024-09-17T05:23:26.069668+0000","last_clean":"2024-09-17T05:23:26.069668+0000","last_became_active":"2024-09-17T05:23:25.070565+0000","last_became_peered":"2024-09-17T05:23:25.070565+0000","last_unstale":"2024-09-17T05:23:26.069668+0000","last_undegraded":"2024-09-17T05:23:26.069668+0000","last_fullsized":"2024-09-17T05:23:26.069668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:53.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054175+0000","last_change":"2024-09-17T05:23:25.064336+0000","last_active":"2024-09-17T05:23:26.054175+0000","last_peered":"2024-09-17T05:23:26.054175+0000","last_clean":"2024-09-17T05:23:26.054175+0000","last_became_active":"2024-09-17T05:23:25.063789+0000","last_became_peered":"2024-09-17T05:23:25.063789+0000","last_unstale":"2024-09-17T05:23:26.054175+0000","last_undegraded":"2024-09-17T05:23:26.054175+0000","last_fullsized":"2024-09-17T05:23:26.054175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:27.415615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052848+0000","last_change":"2024-09-17T05:23:24.057092+0000","last_active":"2024-09-17T05:23:26.052848+0000","last_peered":"2024-09-17T05:23:26.052848+0000","last_clean":"2024-09-17T05:23:26.052848+0000","last_became_active":"2024-09-17T05:23:24.056017+0000","last_became_peered":"2024-09-17T05:23:24.056017+0000","last_unstale":"2024-09-17T05:23:26.052848+0000","last_undegraded":"2024-09-17T05:23:26.052848+0000","last_fullsized":"2024-09-17T05:23:26.052848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:21.643758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070140+0000","last_change":"2024-09-17T05:23:25.050194+0000","last_active":"2024-09-17T05:23:26.070140+0000","last_peered":"2024-09-17T05:23:26.070140+0000","last_clean":"2024-09-17T05:23:26.070140+0000","last_became_active":"2024-09-17T05:23:25.049834+0000","last_became_peered":"2024-09-17T05:23:25.049834+0000","last_unstale":"2024-09-17T05:23:26.070140+0000","last_undegraded":"2024-09-17T05:23:26.070140+0000","last_fullsized":"2024-09-17T05:23:26.070140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071221+0000","last_change":"2024-09-17T05:23:24.060034+0000","last_active":"2024-09-17T05:23:26.071221+0000","last_peered":"2024-09-17T05:23:26.071221+0000","last_clean":"2024-09-17T05:23:26.071221+0000","last_became_active":"2024-09-17T05:23:24.059693+0000","last_became_peered":"2024-09-17T05:23:24.059693+0000","last_unstale":"2024-09-17T05:23:26.071221+0000","last_undegraded":"2024-09-17T05:23:26.071221+0000","last_fullsized":"2024-09-17T05:23:26.071221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:09.436373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052733+0000","last_change":"2024-09-17T05:23:25.066266+0000","last_active":"2024-09-17T05:23:26.052733+0000","last_peered":"2024-09-17T05:23:26.052733+0000","last_clean":"2024-09-17T05:23:26.052733+0000","last_became_active":"2024-09-17T05:23:25.064050+0000","last_became_peered":"2024-09-17T05:23:25.064050+0000","last_unstale":"2024-09-17T05:23:26.052733+0000","last_undegraded":"2024-09-17T05:23:26.052733+0000","last_fullsized":"2024-09-17T05:23:26.052733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070980+0000","last_change":"2024-09-17T05:23:24.052942+0000","last_active":"2024-09-17T05:23:26.070980+0000","last_peered":"2024-09-17T05:23:26.070980+0000","last_clean":"2024-09-17T05:23:26.070980+0000","last_became_active":"2024-09-17T05:23:24.052628+0000","last_became_peered":"2024-09-17T05:23:24.052628+0000","last_unstale":"2024-09-17T05:23:26.070980+0000","last_undegraded":"2024-09-17T05:23:26.070980+0000","last_fullsized":"2024-09-17T05:23:26.070980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.647679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069032+0000","last_change":"2024-09-17T05:23:25.062501+0000","last_active":"2024-09-17T05:23:26.069032+0000","last_peered":"2024-09-17T05:23:26.069032+0000","last_clean":"2024-09-17T05:23:26.069032+0000","last_became_active":"2024-09-17T05:23:25.062195+0000","last_became_peered":"2024-09-17T05:23:25.062195+0000","last_unstale":"2024-09-17T05:23:26.069032+0000","last_undegraded":"2024-09-17T05:23:26.069032+0000","last_fullsized":"2024-09-17T05:23:26.069032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:28.768705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054381+0000","last_change":"2024-09-17T05:23:24.051900+0000","last_active":"2024-09-17T05:23:26.054381+0000","last_peered":"2024-09-17T05:23:26.054381+0000","last_clean":"2024-09-17T05:23:26.054381+0000","last_became_active":"2024-09-17T05:23:24.051525+0000","last_became_peered":"2024-09-17T05:23:24.051525+0000","last_unstale":"2024-09-17T05:23:26.054381+0000","last_undegraded":"2024-09-17T05:23:26.054381+0000","last_fullsized":"2024-09-17T05:23:26.054381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070296+0000","last_change":"2024-09-17T05:23:25.074116+0000","last_active":"2024-09-17T05:23:26.070296+0000","last_peered":"2024-09-17T05:23:26.070296+0000","last_clean":"2024-09-17T05:23:26.070296+0000","last_became_active":"2024-09-17T05:23:25.073750+0000","last_became_peered":"2024-09-17T05:23:25.073750+0000","last_unstale":"2024-09-17T05:23:26.070296+0000","last_undegraded":"2024-09-17T05:23:26.070296+0000","last_fullsized":"2024-09-17T05:23:26.070296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:23.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052816+0000","last_change":"2024-09-17T05:23:24.058892+0000","last_active":"2024-09-17T05:23:26.052816+0000","last_peered":"2024-09-17T05:23:26.052816+0000","last_clean":"2024-09-17T05:23:26.052816+0000","last_became_active":"2024-09-17T05:23:24.057777+0000","last_became_peered":"2024-09-17T05:23:24.057777+0000","last_unstale":"2024-09-17T05:23:26.052816+0000","last_undegraded":"2024-09-17T05:23:26.052816+0000","last_fullsized":"2024-09-17T05:23:26.052816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:52.022043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053572+0000","last_change":"2024-09-17T05:23:25.065444+0000","last_active":"2024-09-17T05:23:26.053572+0000","last_peered":"2024-09-17T05:23:26.053572+0000","last_clean":"2024-09-17T05:23:26.053572+0000","last_became_active":"2024-09-17T05:23:25.064932+0000","last_became_peered":"2024-09-17T05:23:25.064932+0000","last_unstale":"2024-09-17T05:23:26.053572+0000","last_undegraded":"2024-09-17T05:23:26.053572+0000","last_fullsized":"2024-09-17T05:23:26.053572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069641+0000","last_change":"2024-09-17T05:23:24.058300+0000","last_active":"2024-09-17T05:23:26.069641+0000","last_peered":"2024-09-17T05:23:26.069641+0000","last_clean":"2024-09-17T05:23:26.069641+0000","last_became_active":"2024-09-17T05:23:24.057127+0000","last_became_peered":"2024-09-17T05:23:24.057127+0000","last_unstale":"2024-09-17T05:23:26.069641+0000","last_undegraded":"2024-09-17T05:23:26.069641+0000","last_fullsized":"2024-09-17T05:23:26.069641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071142+0000","last_change":"2024-09-17T05:23:25.067068+0000","last_active":"2024-09-17T05:23:26.071142+0000","last_peered":"2024-09-17T05:23:26.071142+0000","last_clean":"2024-09-17T05:23:26.071142+0000","last_became_active":"2024-09-17T05:23:25.066755+0000","last_became_peered":"2024-09-17T05:23:25.066755+0000","last_unstale":"2024-09-17T05:23:26.071142+0000","last_undegraded":"2024-09-17T05:23:26.071142+0000","last_fullsized":"2024-09-17T05:23:26.071142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:11.227030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069906+0000","last_change":"2024-09-17T05:23:24.053733+0000","last_active":"2024-09-17T05:23:26.069906+0000","last_peered":"2024-09-17T05:23:26.069906+0000","last_clean":"2024-09-17T05:23:26.069906+0000","last_became_active":"2024-09-17T05:23:24.053370+0000","last_became_peered":"2024-09-17T05:23:24.053370+0000","last_unstale":"2024-09-17T05:23:26.069906+0000","last_undegraded":"2024-09-17T05:23:26.069906+0000","last_fullsized":"2024-09-17T05:23:26.069906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.457450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052622+0000","last_change":"2024-09-17T05:23:25.069241+0000","last_active":"2024-09-17T05:23:26.052622+0000","last_peered":"2024-09-17T05:23:26.052622+0000","last_clean":"2024-09-17T05:23:26.052622+0000","last_became_active":"2024-09-17T05:23:25.067530+0000","last_became_peered":"2024-09-17T05:23:25.067530+0000","last_unstale":"2024-09-17T05:23:26.052622+0000","last_undegraded":"2024-09-17T05:23:26.052622+0000","last_fullsized":"2024-09-17T05:23:26.052622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069422+0000","last_change":"2024-09-17T05:23:24.069042+0000","last_active":"2024-09-17T05:23:26.069422+0000","last_peered":"2024-09-17T05:23:26.069422+0000","last_clean":"2024-09-17T05:23:26.069422+0000","last_became_active":"2024-09-17T05:23:24.068629+0000","last_became_peered":"2024-09-17T05:23:24.068629+0000","last_unstale":"2024-09-17T05:23:26.069422+0000","last_undegraded":"2024-09-17T05:23:26.069422+0000","last_fullsized":"2024-09-17T05:23:26.069422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:54.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071355+0000","last_change":"2024-09-17T05:23:24.059071+0000","last_active":"2024-09-17T05:23:26.071355+0000","last_peered":"2024-09-17T05:23:26.071355+0000","last_clean":"2024-09-17T05:23:26.071355+0000","last_became_active":"2024-09-17T05:23:24.058788+0000","last_became_peered":"2024-09-17T05:23:24.058788+0000","last_unstale":"2024-09-17T05:23:26.071355+0000","last_undegraded":"2024-09-17T05:23:26.071355+0000","last_fullsized":"2024-09-17T05:23:26.071355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:26.407828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071163+0000","last_change":"2024-09-17T05:23:25.070863+0000","last_active":"2024-09-17T05:23:26.071163+0000","last_peered":"2024-09-17T05:23:26.071163+0000","last_clean":"2024-09-17T05:23:26.071163+0000","last_became_active":"2024-09-17T05:23:25.070508+0000","last_became_peered":"2024-09-17T05:23:25.070508+0000","last_unstale":"2024-09-17T05:23:26.071163+0000","last_undegraded":"2024-09-17T05:23:26.071163+0000","last_fullsized":"2024-09-17T05:23:26.071163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:08.540151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052361+0000","last_change":"2024-09-17T05:23:24.054907+0000","last_active":"2024-09-17T05:23:26.052361+0000","last_peered":"2024-09-17T05:23:26.052361+0000","last_clean":"2024-09-17T05:23:26.052361+0000","last_became_active":"2024-09-17T05:23:24.054532+0000","last_became_peered":"2024-09-17T05:23:24.054532+0000","last_unstale":"2024-09-17T05:23:26.052361+0000","last_undegraded":"2024-09-17T05:23:26.052361+0000","last_fullsized":"2024-09-17T05:23:26.052361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053550+0000","last_change":"2024-09-17T05:23:25.072781+0000","last_active":"2024-09-17T05:23:26.053550+0000","last_peered":"2024-09-17T05:23:26.053550+0000","last_clean":"2024-09-17T05:23:26.053550+0000","last_became_active":"2024-09-17T05:23:25.072303+0000","last_became_peered":"2024-09-17T05:23:25.072303+0000","last_unstale":"2024-09-17T05:23:26.053550+0000","last_undegraded":"2024-09-17T05:23:26.053550+0000","last_fullsized":"2024-09-17T05:23:26.053550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068751+0000","last_change":"2024-09-17T05:23:24.050270+0000","last_active":"2024-09-17T05:23:26.068751+0000","last_peered":"2024-09-17T05:23:26.068751+0000","last_clean":"2024-09-17T05:23:26.068751+0000","last_became_active":"2024-09-17T05:23:24.049787+0000","last_became_peered":"2024-09-17T05:23:24.049787+0000","last_unstale":"2024-09-17T05:23:26.068751+0000","last_undegraded":"2024-09-17T05:23:26.068751+0000","last_fullsized":"2024-09-17T05:23:26.068751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071171+0000","last_change":"2024-09-17T05:23:25.065390+0000","last_active":"2024-09-17T05:23:26.071171+0000","last_peered":"2024-09-17T05:23:26.071171+0000","last_clean":"2024-09-17T05:23:26.071171+0000","last_became_active":"2024-09-17T05:23:25.065054+0000","last_became_peered":"2024-09-17T05:23:25.065054+0000","last_unstale":"2024-09-17T05:23:26.071171+0000","last_undegraded":"2024-09-17T05:23:26.071171+0000","last_fullsized":"2024-09-17T05:23:26.071171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:55.059902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053136+0000","last_change":"2024-09-17T05:23:24.066652+0000","last_active":"2024-09-17T05:23:26.053136+0000","last_peered":"2024-09-17T05:23:26.053136+0000","last_clean":"2024-09-17T05:23:26.053136+0000","last_became_active":"2024-09-17T05:23:24.065549+0000","last_became_peered":"2024-09-17T05:23:24.065549+0000","last_unstale":"2024-09-17T05:23:26.053136+0000","last_undegraded":"2024-09-17T05:23:26.053136+0000","last_fullsized":"2024-09-17T05:23:26.053136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:44.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070411+0000","last_change":"2024-09-17T05:23:25.074083+0000","last_active":"2024-09-17T05:23:26.070411+0000","last_peered":"2024-09-17T05:23:26.070411+0000","last_clean":"2024-09-17T05:23:26.070411+0000","last_became_active":"2024-09-17T05:23:25.073740+0000","last_became_peered":"2024-09-17T05:23:25.073740+0000","last_unstale":"2024-09-17T05:23:26.070411+0000","last_undegraded":"2024-09-17T05:23:26.070411+0000","last_fullsized":"2024-09-17T05:23:26.070411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:15.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070325+0000","last_change":"2024-09-17T05:23:24.060840+0000","last_active":"2024-09-17T05:23:26.070325+0000","last_peered":"2024-09-17T05:23:26.070325+0000","last_clean":"2024-09-17T05:23:26.070325+0000","last_became_active":"2024-09-17T05:23:24.060518+0000","last_became_peered":"2024-09-17T05:23:24.060518+0000","last_unstale":"2024-09-17T05:23:26.070325+0000","last_undegraded":"2024-09-17T05:23:26.070325+0000","last_fullsized":"2024-09-17T05:23:26.070325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:47.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053110+0000","last_change":"2024-09-17T05:23:25.067357+0000","last_active":"2024-09-17T05:23:26.053110+0000","last_peered":"2024-09-17T05:23:26.053110+0000","last_clean":"2024-09-17T05:23:26.053110+0000","last_became_active":"2024-09-17T05:23:25.067017+0000","last_became_peered":"2024-09-17T05:23:25.067017+0000","last_unstale":"2024-09-17T05:23:26.053110+0000","last_undegraded":"2024-09-17T05:23:26.053110+0000","last_fullsized":"2024-09-17T05:23:26.053110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:05.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052624+0000","last_change":"2024-09-17T05:23:24.058023+0000","last_active":"2024-09-17T05:23:26.052624+0000","last_peered":"2024-09-17T05:23:26.052624+0000","last_clean":"2024-09-17T05:23:26.052624+0000","last_became_active":"2024-09-17T05:23:24.057266+0000","last_became_peered":"2024-09-17T05:23:24.057266+0000","last_unstale":"2024-09-17T05:23:26.052624+0000","last_undegraded":"2024-09-17T05:23:26.052624+0000","last_fullsized":"2024-09-17T05:23:26.052624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053458+0000","last_change":"2024-09-17T05:23:25.051518+0000","last_active":"2024-09-17T05:23:26.053458+0000","last_peered":"2024-09-17T05:23:26.053458+0000","last_clean":"2024-09-17T05:23:26.053458+0000","last_became_active":"2024-09-17T05:23:25.051182+0000","last_became_peered":"2024-09-17T05:23:25.051182+0000","last_unstale":"2024-09-17T05:23:26.053458+0000","last_undegraded":"2024-09-17T05:23:26.053458+0000","last_fullsized":"2024-09-17T05:23:26.053458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071578+0000","last_change":"2024-09-17T05:23:25.085148+0000","last_active":"2024-09-17T05:23:26.071578+0000","last_peered":"2024-09-17T05:23:26.071578+0000","last_clean":"2024-09-17T05:23:26.071578+0000","last_became_active":"2024-09-17T05:23:25.083807+0000","last_became_peered":"2024-09-17T05:23:25.083807+0000","last_unstale":"2024-09-17T05:23:26.071578+0000","last_undegraded":"2024-09-17T05:23:26.071578+0000","last_fullsized":"2024-09-17T05:23:26.071578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:32.917183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053650+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:26.053650+0000","last_peered":"2024-09-17T05:23:26.053650+0000","last_clean":"2024-09-17T05:23:26.053650+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:26.053650+0000","last_undegraded":"2024-09-17T05:23:26.053650+0000","last_fullsized":"2024-09-17T05:23:26.053650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071284+0000","last_change":"2024-09-17T05:23:24.056449+0000","last_active":"2024-09-17T05:23:26.071284+0000","last_peered":"2024-09-17T05:23:26.071284+0000","last_clean":"2024-09-17T05:23:26.071284+0000","last_became_active":"2024-09-17T05:23:24.056055+0000","last_became_peered":"2024-09-17T05:23:24.056055+0000","last_unstale":"2024-09-17T05:23:26.071284+0000","last_undegraded":"2024-09-17T05:23:26.071284+0000","last_fullsized":"2024-09-17T05:23:26.071284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.624751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069696+0000","last_change":"2024-09-17T05:23:25.055754+0000","last_active":"2024-09-17T05:23:26.069696+0000","last_peered":"2024-09-17T05:23:26.069696+0000","last_clean":"2024-09-17T05:23:26.069696+0000","last_became_active":"2024-09-17T05:23:25.055368+0000","last_became_peered":"2024-09-17T05:23:25.055368+0000","last_unstale":"2024-09-17T05:23:26.069696+0000","last_undegraded":"2024-09-17T05:23:26.069696+0000","last_fullsized":"2024-09-17T05:23:26.069696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:35.911660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070994+0000","last_change":"2024-09-17T05:23:24.065017+0000","last_active":"2024-09-17T05:23:26.070994+0000","last_peered":"2024-09-17T05:23:26.070994+0000","last_clean":"2024-09-17T05:23:26.070994+0000","last_became_active":"2024-09-17T05:23:24.064688+0000","last_became_peered":"2024-09-17T05:23:24.064688+0000","last_unstale":"2024-09-17T05:23:26.070994+0000","last_undegraded":"2024-09-17T05:23:26.070994+0000","last_fullsized":"2024-09-17T05:23:26.070994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:38.132272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068472+0000","last_change":"2024-09-17T05:23:24.065406+0000","last_active":"2024-09-17T05:23:26.068472+0000","last_peered":"2024-09-17T05:23:26.068472+0000","last_clean":"2024-09-17T05:23:26.068472+0000","last_became_active":"2024-09-17T05:23:24.065058+0000","last_became_peered":"2024-09-17T05:23:24.065058+0000","last_unstale":"2024-09-17T05:23:26.068472+0000","last_undegraded":"2024-09-17T05:23:26.068472+0000","last_fullsized":"2024-09-17T05:23:26.068472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053208+0000","last_change":"2024-09-17T05:23:25.074915+0000","last_active":"2024-09-17T05:23:26.053208+0000","last_peered":"2024-09-17T05:23:26.053208+0000","last_clean":"2024-09-17T05:23:26.053208+0000","last_became_active":"2024-09-17T05:23:25.074603+0000","last_became_peered":"2024-09-17T05:23:25.074603+0000","last_unstale":"2024-09-17T05:23:26.053208+0000","last_undegraded":"2024-09-17T05:23:26.053208+0000","last_fullsized":"2024-09-17T05:23:26.053208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071423+0000","last_change":"2024-09-17T05:23:24.058441+0000","last_active":"2024-09-17T05:23:26.071423+0000","last_peered":"2024-09-17T05:23:26.071423+0000","last_clean":"2024-09-17T05:23:26.071423+0000","last_became_active":"2024-09-17T05:23:24.054951+0000","last_became_peered":"2024-09-17T05:23:24.054951+0000","last_unstale":"2024-09-17T05:23:26.071423+0000","last_undegraded":"2024-09-17T05:23:26.071423+0000","last_fullsized":"2024-09-17T05:23:26.071423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:11.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052393+0000","last_change":"2024-09-17T05:23:25.056791+0000","last_active":"2024-09-17T05:23:26.052393+0000","last_peered":"2024-09-17T05:23:26.052393+0000","last_clean":"2024-09-17T05:23:26.052393+0000","last_became_active":"2024-09-17T05:23:25.056445+0000","last_became_peered":"2024-09-17T05:23:25.056445+0000","last_unstale":"2024-09-17T05:23:26.052393+0000","last_undegraded":"2024-09-17T05:23:26.052393+0000","last_fullsized":"2024-09-17T05:23:26.052393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052571+0000","last_change":"2024-09-17T05:23:24.058736+0000","last_active":"2024-09-17T05:23:26.052571+0000","last_peered":"2024-09-17T05:23:26.052571+0000","last_clean":"2024-09-17T05:23:26.052571+0000","last_became_active":"2024-09-17T05:23:24.056996+0000","last_became_peered":"2024-09-17T05:23:24.056996+0000","last_unstale":"2024-09-17T05:23:26.052571+0000","last_undegraded":"2024-09-17T05:23:26.052571+0000","last_fullsized":"2024-09-17T05:23:26.052571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:09.319432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070023+0000","last_change":"2024-09-17T05:23:25.072427+0000","last_active":"2024-09-17T05:23:26.070023+0000","last_peered":"2024-09-17T05:23:26.070023+0000","last_clean":"2024-09-17T05:23:26.070023+0000","last_became_active":"2024-09-17T05:23:25.071951+0000","last_became_peered":"2024-09-17T05:23:25.071951+0000","last_unstale":"2024-09-17T05:23:26.070023+0000","last_undegraded":"2024-09-17T05:23:26.070023+0000","last_fullsized":"2024-09-17T05:23:26.070023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:45.608054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071621+0000","last_change":"2024-09-17T05:23:24.064058+0000","last_active":"2024-09-17T05:23:26.071621+0000","last_peered":"2024-09-17T05:23:26.071621+0000","last_clean":"2024-09-17T05:23:26.071621+0000","last_became_active":"2024-09-17T05:23:24.063686+0000","last_became_peered":"2024-09-17T05:23:24.063686+0000","last_unstale":"2024-09-17T05:23:26.071621+0000","last_undegraded":"2024-09-17T05:23:26.071621+0000","last_fullsized":"2024-09-17T05:23:26.071621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:03.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070421+0000","last_change":"2024-09-17T05:23:24.061808+0000","last_active":"2024-09-17T05:23:26.070421+0000","last_peered":"2024-09-17T05:23:26.070421+0000","last_clean":"2024-09-17T05:23:26.070421+0000","last_became_active":"2024-09-17T05:23:24.061393+0000","last_became_peered":"2024-09-17T05:23:24.061393+0000","last_unstale":"2024-09-17T05:23:26.070421+0000","last_undegraded":"2024-09-17T05:23:26.070421+0000","last_fullsized":"2024-09-17T05:23:26.070421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:10.321148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070599+0000","last_change":"2024-09-17T05:23:25.065038+0000","last_active":"2024-09-17T05:23:26.070599+0000","last_peered":"2024-09-17T05:23:26.070599+0000","last_clean":"2024-09-17T05:23:26.070599+0000","last_became_active":"2024-09-17T05:23:25.063557+0000","last_became_peered":"2024-09-17T05:23:25.063557+0000","last_unstale":"2024-09-17T05:23:26.070599+0000","last_undegraded":"2024-09-17T05:23:26.070599+0000","last_fullsized":"2024-09-17T05:23:26.070599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:46.016555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071788+0000","last_change":"2024-09-17T05:23:24.061738+0000","last_active":"2024-09-17T05:23:26.071788+0000","last_peered":"2024-09-17T05:23:26.071788+0000","last_clean":"2024-09-17T05:23:26.071788+0000","last_became_active":"2024-09-17T05:23:24.061344+0000","last_became_peered":"2024-09-17T05:23:24.061344+0000","last_unstale":"2024-09-17T05:23:26.071788+0000","last_undegraded":"2024-09-17T05:23:26.071788+0000","last_fullsized":"2024-09-17T05:23:26.071788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:00.795235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070211+0000","last_change":"2024-09-17T05:23:25.059622+0000","last_active":"2024-09-17T05:23:26.070211+0000","last_peered":"2024-09-17T05:23:26.070211+0000","last_clean":"2024-09-17T05:23:26.070211+0000","last_became_active":"2024-09-17T05:23:25.059267+0000","last_became_peered":"2024-09-17T05:23:25.059267+0000","last_unstale":"2024-09-17T05:23:26.070211+0000","last_undegraded":"2024-09-17T05:23:26.070211+0000","last_fullsized":"2024-09-17T05:23:26.070211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:58.979778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.051888+0000","last_change":"2024-09-17T05:23:24.053175+0000","last_active":"2024-09-17T05:23:26.051888+0000","last_peered":"2024-09-17T05:23:26.051888+0000","last_clean":"2024-09-17T05:23:26.051888+0000","last_became_active":"2024-09-17T05:23:24.052859+0000","last_became_peered":"2024-09-17T05:23:24.052859+0000","last_unstale":"2024-09-17T05:23:26.051888+0000","last_undegraded":"2024-09-17T05:23:26.051888+0000","last_fullsized":"2024-09-17T05:23:26.051888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071095+0000","last_change":"2024-09-17T05:23:25.057826+0000","last_active":"2024-09-17T05:23:26.071095+0000","last_peered":"2024-09-17T05:23:26.071095+0000","last_clean":"2024-09-17T05:23:26.071095+0000","last_became_active":"2024-09-17T05:23:25.057507+0000","last_became_peered":"2024-09-17T05:23:25.057507+0000","last_unstale":"2024-09-17T05:23:26.071095+0000","last_undegraded":"2024-09-17T05:23:26.071095+0000","last_fullsized":"2024-09-17T05:23:26.071095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068904+0000","last_change":"2024-09-17T05:23:24.068858+0000","last_active":"2024-09-17T05:23:26.068904+0000","last_peered":"2024-09-17T05:23:26.068904+0000","last_clean":"2024-09-17T05:23:26.068904+0000","last_became_active":"2024-09-17T05:23:24.068387+0000","last_became_peered":"2024-09-17T05:23:24.068387+0000","last_unstale":"2024-09-17T05:23:26.068904+0000","last_undegraded":"2024-09-17T05:23:26.068904+0000","last_fullsized":"2024-09-17T05:23:26.068904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068658+0000","last_change":"2024-09-17T05:23:25.049296+0000","last_active":"2024-09-17T05:23:26.068658+0000","last_peered":"2024-09-17T05:23:26.068658+0000","last_clean":"2024-09-17T05:23:26.068658+0000","last_became_active":"2024-09-17T05:23:25.048948+0000","last_became_peered":"2024-09-17T05:23:25.048948+0000","last_unstale":"2024-09-17T05:23:26.068658+0000","last_undegraded":"2024-09-17T05:23:26.068658+0000","last_fullsized":"2024-09-17T05:23:26.068658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:49.995105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069337+0000","last_change":"2024-09-17T05:23:25.067211+0000","last_active":"2024-09-17T05:23:26.069337+0000","last_peered":"2024-09-17T05:23:26.069337+0000","last_clean":"2024-09-17T05:23:26.069337+0000","last_became_active":"2024-09-17T05:23:25.066883+0000","last_became_peered":"2024-09-17T05:23:25.066883+0000","last_unstale":"2024-09-17T05:23:26.069337+0000","last_undegraded":"2024-09-17T05:23:26.069337+0000","last_fullsized":"2024-09-17T05:23:26.069337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071212+0000","last_change":"2024-09-17T05:23:24.061156+0000","last_active":"2024-09-17T05:23:26.071212+0000","last_peered":"2024-09-17T05:23:26.071212+0000","last_clean":"2024-09-17T05:23:26.071212+0000","last_became_active":"2024-09-17T05:23:24.060751+0000","last_became_peered":"2024-09-17T05:23:24.060751+0000","last_unstale":"2024-09-17T05:23:26.071212+0000","last_undegraded":"2024-09-17T05:23:26.071212+0000","last_fullsized":"2024-09-17T05:23:26.071212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.506573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071131+0000","last_change":"2024-09-17T05:23:25.085054+0000","last_active":"2024-09-17T05:23:26.071131+0000","last_peered":"2024-09-17T05:23:26.071131+0000","last_clean":"2024-09-17T05:23:26.071131+0000","last_became_active":"2024-09-17T05:23:25.084307+0000","last_became_peered":"2024-09-17T05:23:25.084307+0000","last_unstale":"2024-09-17T05:23:26.071131+0000","last_undegraded":"2024-09-17T05:23:26.071131+0000","last_fullsized":"2024-09-17T05:23:26.071131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071557+0000","last_change":"2024-09-17T05:23:25.052805+0000","last_active":"2024-09-17T05:23:26.071557+0000","last_peered":"2024-09-17T05:23:26.071557+0000","last_clean":"2024-09-17T05:23:26.071557+0000","last_became_active":"2024-09-17T05:23:25.052440+0000","last_became_peered":"2024-09-17T05:23:25.052440+0000","last_unstale":"2024-09-17T05:23:26.071557+0000","last_undegraded":"2024-09-17T05:23:26.071557+0000","last_fullsized":"2024-09-17T05:23:26.071557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:51.739703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053334+0000","last_change":"2024-09-17T05:23:24.061930+0000","last_active":"2024-09-17T05:23:26.053334+0000","last_peered":"2024-09-17T05:23:26.053334+0000","last_clean":"2024-09-17T05:23:26.053334+0000","last_became_active":"2024-09-17T05:23:24.060988+0000","last_became_peered":"2024-09-17T05:23:24.060988+0000","last_unstale":"2024-09-17T05:23:26.053334+0000","last_undegraded":"2024-09-17T05:23:26.053334+0000","last_fullsized":"2024-09-17T05:23:26.053334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:47.573508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054561+0000","last_change":"2024-09-17T05:23:25.065743+0000","last_active":"2024-09-17T05:23:26.054561+0000","last_peered":"2024-09-17T05:23:26.054561+0000","last_clean":"2024-09-17T05:23:26.054561+0000","last_became_active":"2024-09-17T05:23:25.062840+0000","last_became_peered":"2024-09-17T05:23:25.062840+0000","last_unstale":"2024-09-17T05:23:26.054561+0000","last_undegraded":"2024-09-17T05:23:26.054561+0000","last_fullsized":"2024-09-17T05:23:26.054561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:01.803022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052543+0000","last_change":"2024-09-17T05:23:24.060124+0000","last_active":"2024-09-17T05:23:26.052543+0000","last_peered":"2024-09-17T05:23:26.052543+0000","last_clean":"2024-09-17T05:23:26.052543+0000","last_became_active":"2024-09-17T05:23:24.057538+0000","last_became_peered":"2024-09-17T05:23:24.057538+0000","last_unstale":"2024-09-17T05:23:26.052543+0000","last_undegraded":"2024-09-17T05:23:26.052543+0000","last_fullsized":"2024-09-17T05:23:26.052543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:41.202615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053337+0000","last_change":"2024-09-17T05:23:25.079567+0000","last_active":"2024-09-17T05:23:26.053337+0000","last_peered":"2024-09-17T05:23:26.053337+0000","last_clean":"2024-09-17T05:23:26.053337+0000","last_became_active":"2024-09-17T05:23:25.078899+0000","last_became_peered":"2024-09-17T05:23:25.078899+0000","last_unstale":"2024-09-17T05:23:26.053337+0000","last_undegraded":"2024-09-17T05:23:26.053337+0000","last_fullsized":"2024-09-17T05:23:26.053337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:39.680169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053486+0000","last_change":"2024-09-17T05:23:24.061891+0000","last_active":"2024-09-17T05:23:26.053486+0000","last_peered":"2024-09-17T05:23:26.053486+0000","last_clean":"2024-09-17T05:23:26.053486+0000","last_became_active":"2024-09-17T05:23:24.055527+0000","last_became_peered":"2024-09-17T05:23:24.055527+0000","last_unstale":"2024-09-17T05:23:26.053486+0000","last_undegraded":"2024-09-17T05:23:26.053486+0000","last_fullsized":"2024-09-17T05:23:26.053486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053220+0000","last_change":"2024-09-17T05:23:25.076838+0000","last_active":"2024-09-17T05:23:26.053220+0000","last_peered":"2024-09-17T05:23:26.053220+0000","last_clean":"2024-09-17T05:23:26.053220+0000","last_became_active":"2024-09-17T05:23:25.076547+0000","last_became_peered":"2024-09-17T05:23:25.076547+0000","last_unstale":"2024-09-17T05:23:26.053220+0000","last_undegraded":"2024-09-17T05:23:26.053220+0000","last_fullsized":"2024-09-17T05:23:26.053220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.142652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070483+0000","last_change":"2024-09-17T05:23:24.059741+0000","last_active":"2024-09-17T05:23:26.070483+0000","last_peered":"2024-09-17T05:23:26.070483+0000","last_clean":"2024-09-17T05:23:26.070483+0000","last_became_active":"2024-09-17T05:23:24.058336+0000","last_became_peered":"2024-09-17T05:23:24.058336+0000","last_unstale":"2024-09-17T05:23:26.070483+0000","last_undegraded":"2024-09-17T05:23:26.070483+0000","last_fullsized":"2024-09-17T05:23:26.070483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:01.640938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:33.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.112+0000 7fa005800640 1 -- 172.21.15.43:0/801519914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fe4003d50 msgr2=0x7f9fe40041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.112+0000 7fa005800640 1 --2- 172.21.15.43:0/801519914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fe4003d50 0x7f9fe40041a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009b40 tx=0x7f9ff001d210 comp rx=0 tx=0).stop 2024-09-17T05:23:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7f9fff400640 1 -- 172.21.15.43:0/801519914 reap_dead start 2024-09-17T05:23:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 -- 172.21.15.43:0/801519914 shutdown_connections 2024-09-17T05:23:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 -- 172.21.15.43:0/801519914 >> 172.21.15.43:0/801519914 conn(0x7fa0000b23d0 msgr2=0x7fa0000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 -- 172.21.15.43:0/801519914 shutdown_connections 2024-09-17T05:23:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 -- 172.21.15.43:0/801519914 wait complete. 2024-09-17T05:23:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 Processor -- start 2024-09-17T05:23:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 -- start start 2024-09-17T05:23:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe4003d50 0x7fa000156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa000157180 0x7fa000157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.116+0000 7fa005800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa000168fe0 con 0x7fa0001584e0 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa000168e60 con 0x7f9fe4003d50 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa000169160 con 0x7fa000157180 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41320/0 (socket says 172.21.15.43:41320) 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 -- 172.21.15.43:0/708827221 learned_addr learned my addr 172.21.15.43:0/708827221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ffea00640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe4003d50 0x7fa000156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ffe000640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa000157180 0x7fa000157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 -- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa000157180 msgr2=0x7fa000157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa000157180 0x7fa000157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 -- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe4003d50 msgr2=0x7fa000156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe4003d50 0x7fa000156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 -- 172.21.15.43:0/708827221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0001588f0 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9fff400640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0000b3900 tx=0x7f9ff80064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8011900 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff8018070 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8011d60 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0001f8a70 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001f9030 con 0x7fa0001584e0 2024-09-17T05:23:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0001fcdd0 con 0x7fa0001584e0 2024-09-17T05:23:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.120+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ff801d460 con 0x7fa0001584e0 2024-09-17T05:23:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.124+0000 7f9ff6a00640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 0x7f9fe0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.124+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9ff8094910 con 0x7fa0001584e0 2024-09-17T05:23:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.124+0000 7f9ffea00640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 0x7f9fe0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.124+0000 7f9ffea00640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 0x7f9fe0075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ad000 tx=0x7f9ff001a010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.128+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa0001588f0 con 0x7fa0001584e0 2024-09-17T05:23:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.296+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0000cc850 con 0x7f9fe0073230 2024-09-17T05:23:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.304+0000 7f9ff6a00640 1 -- 172.21.15.43:0/708827221 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7fa0000cc850 con 0x7f9fe0073230 2024-09-17T05:23:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:34.306 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:34.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 msgr2=0x7f9fe0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 0x7f9fe0075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00ad000 tx=0x7f9ff001a010 comp rx=0 tx=0).stop 2024-09-17T05:23:34.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 msgr2=0x7fa000073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0000b3900 tx=0x7f9ff80064f0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 shutdown_connections 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fe0073230 0x7f9fe0075740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0001584e0 0x7fa000073f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa000157180 0x7fa000157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 --2- 172.21.15.43:0/708827221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe4003d50 0x7fa000156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 >> 172.21.15.43:0/708827221 conn(0x7fa0000b23d0 msgr2=0x7fa0000ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 shutdown_connections 2024-09-17T05:23:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.308+0000 7fa005800640 1 -- 172.21.15.43:0/708827221 wait complete. 2024-09-17T05:23:34.326 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-17T05:23:33.841022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235860,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738860,"statfs":{"total":773094113280,"available":772852592640,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.806808"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053057+0000","last_change":"2024-09-17T05:23:25.072214+0000","last_active":"2024-09-17T05:23:26.053057+0000","last_peered":"2024-09-17T05:23:26.053057+0000","last_clean":"2024-09-17T05:23:26.053057+0000","last_became_active":"2024-09-17T05:23:25.071839+0000","last_became_peered":"2024-09-17T05:23:25.071839+0000","last_unstale":"2024-09-17T05:23:26.053057+0000","last_undegraded":"2024-09-17T05:23:26.053057+0000","last_fullsized":"2024-09-17T05:23:26.053057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052684+0000","last_change":"2024-09-17T05:23:25.068245+0000","last_active":"2024-09-17T05:23:26.052684+0000","last_peered":"2024-09-17T05:23:26.052684+0000","last_clean":"2024-09-17T05:23:26.052684+0000","last_became_active":"2024-09-17T05:23:25.063871+0000","last_became_peered":"2024-09-17T05:23:25.063871+0000","last_unstale":"2024-09-17T05:23:26.052684+0000","last_undegraded":"2024-09-17T05:23:26.052684+0000","last_fullsized":"2024-09-17T05:23:26.052684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:20.196005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052943+0000","last_change":"2024-09-17T05:23:25.077572+0000","last_active":"2024-09-17T05:23:26.052943+0000","last_peered":"2024-09-17T05:23:26.052943+0000","last_clean":"2024-09-17T05:23:26.052943+0000","last_became_active":"2024-09-17T05:23:25.077232+0000","last_became_peered":"2024-09-17T05:23:25.077232+0000","last_unstale":"2024-09-17T05:23:26.052943+0000","last_undegraded":"2024-09-17T05:23:26.052943+0000","last_fullsized":"2024-09-17T05:23:26.052943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070147+0000","last_change":"2024-09-17T05:23:25.080315+0000","last_active":"2024-09-17T05:23:26.070147+0000","last_peered":"2024-09-17T05:23:26.070147+0000","last_clean":"2024-09-17T05:23:26.070147+0000","last_became_active":"2024-09-17T05:23:25.079991+0000","last_became_peered":"2024-09-17T05:23:25.079991+0000","last_unstale":"2024-09-17T05:23:26.070147+0000","last_undegraded":"2024-09-17T05:23:26.070147+0000","last_fullsized":"2024-09-17T05:23:26.070147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:58.988827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052636+0000","last_change":"2024-09-17T05:23:25.062672+0000","last_active":"2024-09-17T05:23:26.052636+0000","last_peered":"2024-09-17T05:23:26.052636+0000","last_clean":"2024-09-17T05:23:26.052636+0000","last_became_active":"2024-09-17T05:23:25.060858+0000","last_became_peered":"2024-09-17T05:23:25.060858+0000","last_unstale":"2024-09-17T05:23:26.052636+0000","last_undegraded":"2024-09-17T05:23:26.052636+0000","last_fullsized":"2024-09-17T05:23:26.052636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.102130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069891+0000","last_change":"2024-09-17T05:23:25.065376+0000","last_active":"2024-09-17T05:23:26.069891+0000","last_peered":"2024-09-17T05:23:26.069891+0000","last_clean":"2024-09-17T05:23:26.069891+0000","last_became_active":"2024-09-17T05:23:25.064806+0000","last_became_peered":"2024-09-17T05:23:25.064806+0000","last_unstale":"2024-09-17T05:23:26.069891+0000","last_undegraded":"2024-09-17T05:23:26.069891+0000","last_fullsized":"2024-09-17T05:23:26.069891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052148+0000","last_change":"2024-09-17T05:23:25.053855+0000","last_active":"2024-09-17T05:23:26.052148+0000","last_peered":"2024-09-17T05:23:26.052148+0000","last_clean":"2024-09-17T05:23:26.052148+0000","last_became_active":"2024-09-17T05:23:25.053569+0000","last_became_peered":"2024-09-17T05:23:25.053569+0000","last_unstale":"2024-09-17T05:23:26.052148+0000","last_undegraded":"2024-09-17T05:23:26.052148+0000","last_fullsized":"2024-09-17T05:23:26.052148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052419+0000","last_change":"2024-09-17T05:23:25.056925+0000","last_active":"2024-09-17T05:23:26.052419+0000","last_peered":"2024-09-17T05:23:26.052419+0000","last_clean":"2024-09-17T05:23:26.052419+0000","last_became_active":"2024-09-17T05:23:25.056619+0000","last_became_peered":"2024-09-17T05:23:25.056619+0000","last_unstale":"2024-09-17T05:23:26.052419+0000","last_undegraded":"2024-09-17T05:23:26.052419+0000","last_fullsized":"2024-09-17T05:23:26.052419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054267+0000","last_change":"2024-09-17T05:23:25.061010+0000","last_active":"2024-09-17T05:23:26.054267+0000","last_peered":"2024-09-17T05:23:26.054267+0000","last_clean":"2024-09-17T05:23:26.054267+0000","last_became_active":"2024-09-17T05:23:25.060714+0000","last_became_peered":"2024-09-17T05:23:25.060714+0000","last_unstale":"2024-09-17T05:23:26.054267+0000","last_undegraded":"2024-09-17T05:23:26.054267+0000","last_fullsized":"2024-09-17T05:23:26.054267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.664289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052921+0000","last_change":"2024-09-17T05:23:25.079732+0000","last_active":"2024-09-17T05:23:26.052921+0000","last_peered":"2024-09-17T05:23:26.052921+0000","last_clean":"2024-09-17T05:23:26.052921+0000","last_became_active":"2024-09-17T05:23:25.079100+0000","last_became_peered":"2024-09-17T05:23:25.079100+0000","last_unstale":"2024-09-17T05:23:26.052921+0000","last_undegraded":"2024-09-17T05:23:26.052921+0000","last_fullsized":"2024-09-17T05:23:26.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:01.901521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052498+0000","last_change":"2024-09-17T05:23:25.075619+0000","last_active":"2024-09-17T05:23:26.052498+0000","last_peered":"2024-09-17T05:23:26.052498+0000","last_clean":"2024-09-17T05:23:26.052498+0000","last_became_active":"2024-09-17T05:23:25.071655+0000","last_became_peered":"2024-09-17T05:23:25.071655+0000","last_unstale":"2024-09-17T05:23:26.052498+0000","last_undegraded":"2024-09-17T05:23:26.052498+0000","last_fullsized":"2024-09-17T05:23:26.052498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052932+0000","last_change":"2024-09-17T05:23:25.077778+0000","last_active":"2024-09-17T05:23:26.052932+0000","last_peered":"2024-09-17T05:23:26.052932+0000","last_clean":"2024-09-17T05:23:26.052932+0000","last_became_active":"2024-09-17T05:23:25.077040+0000","last_became_peered":"2024-09-17T05:23:25.077040+0000","last_unstale":"2024-09-17T05:23:26.052932+0000","last_undegraded":"2024-09-17T05:23:26.052932+0000","last_fullsized":"2024-09-17T05:23:26.052932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052584+0000","last_change":"2024-09-17T05:23:25.067070+0000","last_active":"2024-09-17T05:23:26.052584+0000","last_peered":"2024-09-17T05:23:26.052584+0000","last_clean":"2024-09-17T05:23:26.052584+0000","last_became_active":"2024-09-17T05:23:25.066734+0000","last_became_peered":"2024-09-17T05:23:25.066734+0000","last_unstale":"2024-09-17T05:23:26.052584+0000","last_undegraded":"2024-09-17T05:23:26.052584+0000","last_fullsized":"2024-09-17T05:23:26.052584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:58.712013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070882+0000","last_change":"2024-09-17T05:23:25.066014+0000","last_active":"2024-09-17T05:23:26.070882+0000","last_peered":"2024-09-17T05:23:26.070882+0000","last_clean":"2024-09-17T05:23:26.070882+0000","last_became_active":"2024-09-17T05:23:25.063873+0000","last_became_peered":"2024-09-17T05:23:25.063873+0000","last_unstale":"2024-09-17T05:23:26.070882+0000","last_undegraded":"2024-09-17T05:23:26.070882+0000","last_fullsized":"2024-09-17T05:23:26.070882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071924+0000","last_change":"2024-09-17T05:23:25.055365+0000","last_active":"2024-09-17T05:23:26.071924+0000","last_peered":"2024-09-17T05:23:26.071924+0000","last_clean":"2024-09-17T05:23:26.071924+0000","last_became_active":"2024-09-17T05:23:25.055046+0000","last_became_peered":"2024-09-17T05:23:25.055046+0000","last_unstale":"2024-09-17T05:23:26.071924+0000","last_undegraded":"2024-09-17T05:23:26.071924+0000","last_fullsized":"2024-09-17T05:23:26.071924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:48.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070072+0000","last_change":"2024-09-17T05:23:25.076193+0000","last_active":"2024-09-17T05:23:26.070072+0000","last_peered":"2024-09-17T05:23:26.070072+0000","last_clean":"2024-09-17T05:23:26.070072+0000","last_became_active":"2024-09-17T05:23:25.075582+0000","last_became_peered":"2024-09-17T05:23:25.075582+0000","last_unstale":"2024-09-17T05:23:26.070072+0000","last_undegraded":"2024-09-17T05:23:26.070072+0000","last_fullsized":"2024-09-17T05:23:26.070072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071700+0000","last_change":"2024-09-17T05:23:25.053491+0000","last_active":"2024-09-17T05:23:26.071700+0000","last_peered":"2024-09-17T05:23:26.071700+0000","last_clean":"2024-09-17T05:23:26.071700+0000","last_became_active":"2024-09-17T05:23:25.053209+0000","last_became_peered":"2024-09-17T05:23:25.053209+0000","last_unstale":"2024-09-17T05:23:26.071700+0000","last_undegraded":"2024-09-17T05:23:26.071700+0000","last_fullsized":"2024-09-17T05:23:26.071700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:33.151198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053156+0000","last_change":"2024-09-17T05:23:25.068427+0000","last_active":"2024-09-17T05:23:26.053156+0000","last_peered":"2024-09-17T05:23:26.053156+0000","last_clean":"2024-09-17T05:23:26.053156+0000","last_became_active":"2024-09-17T05:23:25.068043+0000","last_became_peered":"2024-09-17T05:23:25.068043+0000","last_unstale":"2024-09-17T05:23:26.053156+0000","last_undegraded":"2024-09-17T05:23:26.053156+0000","last_fullsized":"2024-09-17T05:23:26.053156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070102+0000","last_change":"2024-09-17T05:23:25.057241+0000","last_active":"2024-09-17T05:23:26.070102+0000","last_peered":"2024-09-17T05:23:26.070102+0000","last_clean":"2024-09-17T05:23:26.070102+0000","last_became_active":"2024-09-17T05:23:25.056934+0000","last_became_peered":"2024-09-17T05:23:25.056934+0000","last_unstale":"2024-09-17T05:23:26.070102+0000","last_undegraded":"2024-09-17T05:23:26.070102+0000","last_fullsized":"2024-09-17T05:23:26.070102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054089+0000","last_change":"2024-09-17T05:23:25.064365+0000","last_active":"2024-09-17T05:23:26.054089+0000","last_peered":"2024-09-17T05:23:26.054089+0000","last_clean":"2024-09-17T05:23:26.054089+0000","last_became_active":"2024-09-17T05:23:25.064064+0000","last_became_peered":"2024-09-17T05:23:25.064064+0000","last_unstale":"2024-09-17T05:23:26.054089+0000","last_undegraded":"2024-09-17T05:23:26.054089+0000","last_fullsized":"2024-09-17T05:23:26.054089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052827+0000","last_change":"2024-09-17T05:23:25.059278+0000","last_active":"2024-09-17T05:23:26.052827+0000","last_peered":"2024-09-17T05:23:26.052827+0000","last_clean":"2024-09-17T05:23:26.052827+0000","last_became_active":"2024-09-17T05:23:25.058926+0000","last_became_peered":"2024-09-17T05:23:25.058926+0000","last_unstale":"2024-09-17T05:23:26.052827+0000","last_undegraded":"2024-09-17T05:23:26.052827+0000","last_fullsized":"2024-09-17T05:23:26.052827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070891+0000","last_change":"2024-09-17T05:23:25.064634+0000","last_active":"2024-09-17T05:23:26.070891+0000","last_peered":"2024-09-17T05:23:26.070891+0000","last_clean":"2024-09-17T05:23:26.070891+0000","last_became_active":"2024-09-17T05:23:25.064250+0000","last_became_peered":"2024-09-17T05:23:25.064250+0000","last_unstale":"2024-09-17T05:23:26.070891+0000","last_undegraded":"2024-09-17T05:23:26.070891+0000","last_fullsized":"2024-09-17T05:23:26.070891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053318+0000","last_change":"2024-09-17T05:23:25.079930+0000","last_active":"2024-09-17T05:23:26.053318+0000","last_peered":"2024-09-17T05:23:26.053318+0000","last_clean":"2024-09-17T05:23:26.053318+0000","last_became_active":"2024-09-17T05:23:25.079276+0000","last_became_peered":"2024-09-17T05:23:25.079276+0000","last_unstale":"2024-09-17T05:23:26.053318+0000","last_undegraded":"2024-09-17T05:23:26.053318+0000","last_fullsized":"2024-09-17T05:23:26.053318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070179+0000","last_change":"2024-09-17T05:23:25.083629+0000","last_active":"2024-09-17T05:23:26.070179+0000","last_peered":"2024-09-17T05:23:26.070179+0000","last_clean":"2024-09-17T05:23:26.070179+0000","last_became_active":"2024-09-17T05:23:25.083394+0000","last_became_peered":"2024-09-17T05:23:25.083394+0000","last_unstale":"2024-09-17T05:23:26.070179+0000","last_undegraded":"2024-09-17T05:23:26.070179+0000","last_fullsized":"2024-09-17T05:23:26.070179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:22.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053114+0000","last_change":"2024-09-17T05:23:25.075408+0000","last_active":"2024-09-17T05:23:26.053114+0000","last_peered":"2024-09-17T05:23:26.053114+0000","last_clean":"2024-09-17T05:23:26.053114+0000","last_became_active":"2024-09-17T05:23:25.075101+0000","last_became_peered":"2024-09-17T05:23:25.075101+0000","last_unstale":"2024-09-17T05:23:26.053114+0000","last_undegraded":"2024-09-17T05:23:26.053114+0000","last_fullsized":"2024-09-17T05:23:26.053114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071351+0000","last_change":"2024-09-17T05:23:25.068874+0000","last_active":"2024-09-17T05:23:26.071351+0000","last_peered":"2024-09-17T05:23:26.071351+0000","last_clean":"2024-09-17T05:23:26.071351+0000","last_became_active":"2024-09-17T05:23:25.064181+0000","last_became_peered":"2024-09-17T05:23:25.064181+0000","last_unstale":"2024-09-17T05:23:26.071351+0000","last_undegraded":"2024-09-17T05:23:26.071351+0000","last_fullsized":"2024-09-17T05:23:26.071351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:43.495535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068665+0000","last_change":"2024-09-17T05:23:25.064004+0000","last_active":"2024-09-17T05:23:26.068665+0000","last_peered":"2024-09-17T05:23:26.068665+0000","last_clean":"2024-09-17T05:23:26.068665+0000","last_became_active":"2024-09-17T05:23:25.063599+0000","last_became_peered":"2024-09-17T05:23:25.063599+0000","last_unstale":"2024-09-17T05:23:26.068665+0000","last_undegraded":"2024-09-17T05:23:26.068665+0000","last_fullsized":"2024-09-17T05:23:26.068665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.655466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069668+0000","last_change":"2024-09-17T05:23:25.071820+0000","last_active":"2024-09-17T05:23:26.069668+0000","last_peered":"2024-09-17T05:23:26.069668+0000","last_clean":"2024-09-17T05:23:26.069668+0000","last_became_active":"2024-09-17T05:23:25.070565+0000","last_became_peered":"2024-09-17T05:23:25.070565+0000","last_unstale":"2024-09-17T05:23:26.069668+0000","last_undegraded":"2024-09-17T05:23:26.069668+0000","last_fullsized":"2024-09-17T05:23:26.069668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:53.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054175+0000","last_change":"2024-09-17T05:23:25.064336+0000","last_active":"2024-09-17T05:23:26.054175+0000","last_peered":"2024-09-17T05:23:26.054175+0000","last_clean":"2024-09-17T05:23:26.054175+0000","last_became_active":"2024-09-17T05:23:25.063789+0000","last_became_peered":"2024-09-17T05:23:25.063789+0000","last_unstale":"2024-09-17T05:23:26.054175+0000","last_undegraded":"2024-09-17T05:23:26.054175+0000","last_fullsized":"2024-09-17T05:23:26.054175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:27.415615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052848+0000","last_change":"2024-09-17T05:23:24.057092+0000","last_active":"2024-09-17T05:23:26.052848+0000","last_peered":"2024-09-17T05:23:26.052848+0000","last_clean":"2024-09-17T05:23:26.052848+0000","last_became_active":"2024-09-17T05:23:24.056017+0000","last_became_peered":"2024-09-17T05:23:24.056017+0000","last_unstale":"2024-09-17T05:23:26.052848+0000","last_undegraded":"2024-09-17T05:23:26.052848+0000","last_fullsized":"2024-09-17T05:23:26.052848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:21.643758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070140+0000","last_change":"2024-09-17T05:23:25.050194+0000","last_active":"2024-09-17T05:23:26.070140+0000","last_peered":"2024-09-17T05:23:26.070140+0000","last_clean":"2024-09-17T05:23:26.070140+0000","last_became_active":"2024-09-17T05:23:25.049834+0000","last_became_peered":"2024-09-17T05:23:25.049834+0000","last_unstale":"2024-09-17T05:23:26.070140+0000","last_undegraded":"2024-09-17T05:23:26.070140+0000","last_fullsized":"2024-09-17T05:23:26.070140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071221+0000","last_change":"2024-09-17T05:23:24.060034+0000","last_active":"2024-09-17T05:23:26.071221+0000","last_peered":"2024-09-17T05:23:26.071221+0000","last_clean":"2024-09-17T05:23:26.071221+0000","last_became_active":"2024-09-17T05:23:24.059693+0000","last_became_peered":"2024-09-17T05:23:24.059693+0000","last_unstale":"2024-09-17T05:23:26.071221+0000","last_undegraded":"2024-09-17T05:23:26.071221+0000","last_fullsized":"2024-09-17T05:23:26.071221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:09.436373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052733+0000","last_change":"2024-09-17T05:23:25.066266+0000","last_active":"2024-09-17T05:23:26.052733+0000","last_peered":"2024-09-17T05:23:26.052733+0000","last_clean":"2024-09-17T05:23:26.052733+0000","last_became_active":"2024-09-17T05:23:25.064050+0000","last_became_peered":"2024-09-17T05:23:25.064050+0000","last_unstale":"2024-09-17T05:23:26.052733+0000","last_undegraded":"2024-09-17T05:23:26.052733+0000","last_fullsized":"2024-09-17T05:23:26.052733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070980+0000","last_change":"2024-09-17T05:23:24.052942+0000","last_active":"2024-09-17T05:23:26.070980+0000","last_peered":"2024-09-17T05:23:26.070980+0000","last_clean":"2024-09-17T05:23:26.070980+0000","last_became_active":"2024-09-17T05:23:24.052628+0000","last_became_peered":"2024-09-17T05:23:24.052628+0000","last_unstale":"2024-09-17T05:23:26.070980+0000","last_undegraded":"2024-09-17T05:23:26.070980+0000","last_fullsized":"2024-09-17T05:23:26.070980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.647679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069032+0000","last_change":"2024-09-17T05:23:25.062501+0000","last_active":"2024-09-17T05:23:26.069032+0000","last_peered":"2024-09-17T05:23:26.069032+0000","last_clean":"2024-09-17T05:23:26.069032+0000","last_became_active":"2024-09-17T05:23:25.062195+0000","last_became_peered":"2024-09-17T05:23:25.062195+0000","last_unstale":"2024-09-17T05:23:26.069032+0000","last_undegraded":"2024-09-17T05:23:26.069032+0000","last_fullsized":"2024-09-17T05:23:26.069032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:28.768705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054381+0000","last_change":"2024-09-17T05:23:24.051900+0000","last_active":"2024-09-17T05:23:26.054381+0000","last_peered":"2024-09-17T05:23:26.054381+0000","last_clean":"2024-09-17T05:23:26.054381+0000","last_became_active":"2024-09-17T05:23:24.051525+0000","last_became_peered":"2024-09-17T05:23:24.051525+0000","last_unstale":"2024-09-17T05:23:26.054381+0000","last_undegraded":"2024-09-17T05:23:26.054381+0000","last_fullsized":"2024-09-17T05:23:26.054381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070296+0000","last_change":"2024-09-17T05:23:25.074116+0000","last_active":"2024-09-17T05:23:26.070296+0000","last_peered":"2024-09-17T05:23:26.070296+0000","last_clean":"2024-09-17T05:23:26.070296+0000","last_became_active":"2024-09-17T05:23:25.073750+0000","last_became_peered":"2024-09-17T05:23:25.073750+0000","last_unstale":"2024-09-17T05:23:26.070296+0000","last_undegraded":"2024-09-17T05:23:26.070296+0000","last_fullsized":"2024-09-17T05:23:26.070296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:23.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052816+0000","last_change":"2024-09-17T05:23:24.058892+0000","last_active":"2024-09-17T05:23:26.052816+0000","last_peered":"2024-09-17T05:23:26.052816+0000","last_clean":"2024-09-17T05:23:26.052816+0000","last_became_active":"2024-09-17T05:23:24.057777+0000","last_became_peered":"2024-09-17T05:23:24.057777+0000","last_unstale":"2024-09-17T05:23:26.052816+0000","last_undegraded":"2024-09-17T05:23:26.052816+0000","last_fullsized":"2024-09-17T05:23:26.052816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:52.022043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053572+0000","last_change":"2024-09-17T05:23:25.065444+0000","last_active":"2024-09-17T05:23:26.053572+0000","last_peered":"2024-09-17T05:23:26.053572+0000","last_clean":"2024-09-17T05:23:26.053572+0000","last_became_active":"2024-09-17T05:23:25.064932+0000","last_became_peered":"2024-09-17T05:23:25.064932+0000","last_unstale":"2024-09-17T05:23:26.053572+0000","last_undegraded":"2024-09-17T05:23:26.053572+0000","last_fullsized":"2024-09-17T05:23:26.053572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069641+0000","last_change":"2024-09-17T05:23:24.058300+0000","last_active":"2024-09-17T05:23:26.069641+0000","last_peered":"2024-09-17T05:23:26.069641+0000","last_clean":"2024-09-17T05:23:26.069641+0000","last_became_active":"2024-09-17T05:23:24.057127+0000","last_became_peered":"2024-09-17T05:23:24.057127+0000","last_unstale":"2024-09-17T05:23:26.069641+0000","last_undegraded":"2024-09-17T05:23:26.069641+0000","last_fullsized":"2024-09-17T05:23:26.069641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071142+0000","last_change":"2024-09-17T05:23:25.067068+0000","last_active":"2024-09-17T05:23:26.071142+0000","last_peered":"2024-09-17T05:23:26.071142+0000","last_clean":"2024-09-17T05:23:26.071142+0000","last_became_active":"2024-09-17T05:23:25.066755+0000","last_became_peered":"2024-09-17T05:23:25.066755+0000","last_unstale":"2024-09-17T05:23:26.071142+0000","last_undegraded":"2024-09-17T05:23:26.071142+0000","last_fullsized":"2024-09-17T05:23:26.071142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:11.227030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069906+0000","last_change":"2024-09-17T05:23:24.053733+0000","last_active":"2024-09-17T05:23:26.069906+0000","last_peered":"2024-09-17T05:23:26.069906+0000","last_clean":"2024-09-17T05:23:26.069906+0000","last_became_active":"2024-09-17T05:23:24.053370+0000","last_became_peered":"2024-09-17T05:23:24.053370+0000","last_unstale":"2024-09-17T05:23:26.069906+0000","last_undegraded":"2024-09-17T05:23:26.069906+0000","last_fullsized":"2024-09-17T05:23:26.069906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.457450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052622+0000","last_change":"2024-09-17T05:23:25.069241+0000","last_active":"2024-09-17T05:23:26.052622+0000","last_peered":"2024-09-17T05:23:26.052622+0000","last_clean":"2024-09-17T05:23:26.052622+0000","last_became_active":"2024-09-17T05:23:25.067530+0000","last_became_peered":"2024-09-17T05:23:25.067530+0000","last_unstale":"2024-09-17T05:23:26.052622+0000","last_undegraded":"2024-09-17T05:23:26.052622+0000","last_fullsized":"2024-09-17T05:23:26.052622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069422+0000","last_change":"2024-09-17T05:23:24.069042+0000","last_active":"2024-09-17T05:23:26.069422+0000","last_peered":"2024-09-17T05:23:26.069422+0000","last_clean":"2024-09-17T05:23:26.069422+0000","last_became_active":"2024-09-17T05:23:24.068629+0000","last_became_peered":"2024-09-17T05:23:24.068629+0000","last_unstale":"2024-09-17T05:23:26.069422+0000","last_undegraded":"2024-09-17T05:23:26.069422+0000","last_fullsized":"2024-09-17T05:23:26.069422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:54.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071355+0000","last_change":"2024-09-17T05:23:24.059071+0000","last_active":"2024-09-17T05:23:26.071355+0000","last_peered":"2024-09-17T05:23:26.071355+0000","last_clean":"2024-09-17T05:23:26.071355+0000","last_became_active":"2024-09-17T05:23:24.058788+0000","last_became_peered":"2024-09-17T05:23:24.058788+0000","last_unstale":"2024-09-17T05:23:26.071355+0000","last_undegraded":"2024-09-17T05:23:26.071355+0000","last_fullsized":"2024-09-17T05:23:26.071355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:26.407828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071163+0000","last_change":"2024-09-17T05:23:25.070863+0000","last_active":"2024-09-17T05:23:26.071163+0000","last_peered":"2024-09-17T05:23:26.071163+0000","last_clean":"2024-09-17T05:23:26.071163+0000","last_became_active":"2024-09-17T05:23:25.070508+0000","last_became_peered":"2024-09-17T05:23:25.070508+0000","last_unstale":"2024-09-17T05:23:26.071163+0000","last_undegraded":"2024-09-17T05:23:26.071163+0000","last_fullsized":"2024-09-17T05:23:26.071163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:08.540151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052361+0000","last_change":"2024-09-17T05:23:24.054907+0000","last_active":"2024-09-17T05:23:26.052361+0000","last_peered":"2024-09-17T05:23:26.052361+0000","last_clean":"2024-09-17T05:23:26.052361+0000","last_became_active":"2024-09-17T05:23:24.054532+0000","last_became_peered":"2024-09-17T05:23:24.054532+0000","last_unstale":"2024-09-17T05:23:26.052361+0000","last_undegraded":"2024-09-17T05:23:26.052361+0000","last_fullsized":"2024-09-17T05:23:26.052361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053550+0000","last_change":"2024-09-17T05:23:25.072781+0000","last_active":"2024-09-17T05:23:26.053550+0000","last_peered":"2024-09-17T05:23:26.053550+0000","last_clean":"2024-09-17T05:23:26.053550+0000","last_became_active":"2024-09-17T05:23:25.072303+0000","last_became_peered":"2024-09-17T05:23:25.072303+0000","last_unstale":"2024-09-17T05:23:26.053550+0000","last_undegraded":"2024-09-17T05:23:26.053550+0000","last_fullsized":"2024-09-17T05:23:26.053550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068751+0000","last_change":"2024-09-17T05:23:24.050270+0000","last_active":"2024-09-17T05:23:26.068751+0000","last_peered":"2024-09-17T05:23:26.068751+0000","last_clean":"2024-09-17T05:23:26.068751+0000","last_became_active":"2024-09-17T05:23:24.049787+0000","last_became_peered":"2024-09-17T05:23:24.049787+0000","last_unstale":"2024-09-17T05:23:26.068751+0000","last_undegraded":"2024-09-17T05:23:26.068751+0000","last_fullsized":"2024-09-17T05:23:26.068751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071171+0000","last_change":"2024-09-17T05:23:25.065390+0000","last_active":"2024-09-17T05:23:26.071171+0000","last_peered":"2024-09-17T05:23:26.071171+0000","last_clean":"2024-09-17T05:23:26.071171+0000","last_became_active":"2024-09-17T05:23:25.065054+0000","last_became_peered":"2024-09-17T05:23:25.065054+0000","last_unstale":"2024-09-17T05:23:26.071171+0000","last_undegraded":"2024-09-17T05:23:26.071171+0000","last_fullsized":"2024-09-17T05:23:26.071171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:55.059902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053136+0000","last_change":"2024-09-17T05:23:24.066652+0000","last_active":"2024-09-17T05:23:26.053136+0000","last_peered":"2024-09-17T05:23:26.053136+0000","last_clean":"2024-09-17T05:23:26.053136+0000","last_became_active":"2024-09-17T05:23:24.065549+0000","last_became_peered":"2024-09-17T05:23:24.065549+0000","last_unstale":"2024-09-17T05:23:26.053136+0000","last_undegraded":"2024-09-17T05:23:26.053136+0000","last_fullsized":"2024-09-17T05:23:26.053136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:44.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070411+0000","last_change":"2024-09-17T05:23:25.074083+0000","last_active":"2024-09-17T05:23:26.070411+0000","last_peered":"2024-09-17T05:23:26.070411+0000","last_clean":"2024-09-17T05:23:26.070411+0000","last_became_active":"2024-09-17T05:23:25.073740+0000","last_became_peered":"2024-09-17T05:23:25.073740+0000","last_unstale":"2024-09-17T05:23:26.070411+0000","last_undegraded":"2024-09-17T05:23:26.070411+0000","last_fullsized":"2024-09-17T05:23:26.070411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:15.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070325+0000","last_change":"2024-09-17T05:23:24.060840+0000","last_active":"2024-09-17T05:23:26.070325+0000","last_peered":"2024-09-17T05:23:26.070325+0000","last_clean":"2024-09-17T05:23:26.070325+0000","last_became_active":"2024-09-17T05:23:24.060518+0000","last_became_peered":"2024-09-17T05:23:24.060518+0000","last_unstale":"2024-09-17T05:23:26.070325+0000","last_undegraded":"2024-09-17T05:23:26.070325+0000","last_fullsized":"2024-09-17T05:23:26.070325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:47.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053110+0000","last_change":"2024-09-17T05:23:25.067357+0000","last_active":"2024-09-17T05:23:26.053110+0000","last_peered":"2024-09-17T05:23:26.053110+0000","last_clean":"2024-09-17T05:23:26.053110+0000","last_became_active":"2024-09-17T05:23:25.067017+0000","last_became_peered":"2024-09-17T05:23:25.067017+0000","last_unstale":"2024-09-17T05:23:26.053110+0000","last_undegraded":"2024-09-17T05:23:26.053110+0000","last_fullsized":"2024-09-17T05:23:26.053110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:05.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052624+0000","last_change":"2024-09-17T05:23:24.058023+0000","last_active":"2024-09-17T05:23:26.052624+0000","last_peered":"2024-09-17T05:23:26.052624+0000","last_clean":"2024-09-17T05:23:26.052624+0000","last_became_active":"2024-09-17T05:23:24.057266+0000","last_became_peered":"2024-09-17T05:23:24.057266+0000","last_unstale":"2024-09-17T05:23:26.052624+0000","last_undegraded":"2024-09-17T05:23:26.052624+0000","last_fullsized":"2024-09-17T05:23:26.052624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053458+0000","last_change":"2024-09-17T05:23:25.051518+0000","last_active":"2024-09-17T05:23:26.053458+0000","last_peered":"2024-09-17T05:23:26.053458+0000","last_clean":"2024-09-17T05:23:26.053458+0000","last_became_active":"2024-09-17T05:23:25.051182+0000","last_became_peered":"2024-09-17T05:23:25.051182+0000","last_unstale":"2024-09-17T05:23:26.053458+0000","last_undegraded":"2024-09-17T05:23:26.053458+0000","last_fullsized":"2024-09-17T05:23:26.053458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071578+0000","last_change":"2024-09-17T05:23:25.085148+0000","last_active":"2024-09-17T05:23:26.071578+0000","last_peered":"2024-09-17T05:23:26.071578+0000","last_clean":"2024-09-17T05:23:26.071578+0000","last_became_active":"2024-09-17T05:23:25.083807+0000","last_became_peered":"2024-09-17T05:23:25.083807+0000","last_unstale":"2024-09-17T05:23:26.071578+0000","last_undegraded":"2024-09-17T05:23:26.071578+0000","last_fullsized":"2024-09-17T05:23:26.071578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:32.917183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053650+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:26.053650+0000","last_peered":"2024-09-17T05:23:26.053650+0000","last_clean":"2024-09-17T05:23:26.053650+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:26.053650+0000","last_undegraded":"2024-09-17T05:23:26.053650+0000","last_fullsized":"2024-09-17T05:23:26.053650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071284+0000","last_change":"2024-09-17T05:23:24.056449+0000","last_active":"2024-09-17T05:23:26.071284+0000","last_peered":"2024-09-17T05:23:26.071284+0000","last_clean":"2024-09-17T05:23:26.071284+0000","last_became_active":"2024-09-17T05:23:24.056055+0000","last_became_peered":"2024-09-17T05:23:24.056055+0000","last_unstale":"2024-09-17T05:23:26.071284+0000","last_undegraded":"2024-09-17T05:23:26.071284+0000","last_fullsized":"2024-09-17T05:23:26.071284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.624751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069696+0000","last_change":"2024-09-17T05:23:25.055754+0000","last_active":"2024-09-17T05:23:26.069696+0000","last_peered":"2024-09-17T05:23:26.069696+0000","last_clean":"2024-09-17T05:23:26.069696+0000","last_became_active":"2024-09-17T05:23:25.055368+0000","last_became_peered":"2024-09-17T05:23:25.055368+0000","last_unstale":"2024-09-17T05:23:26.069696+0000","last_undegraded":"2024-09-17T05:23:26.069696+0000","last_fullsized":"2024-09-17T05:23:26.069696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:35.911660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070994+0000","last_change":"2024-09-17T05:23:24.065017+0000","last_active":"2024-09-17T05:23:26.070994+0000","last_peered":"2024-09-17T05:23:26.070994+0000","last_clean":"2024-09-17T05:23:26.070994+0000","last_became_active":"2024-09-17T05:23:24.064688+0000","last_became_peered":"2024-09-17T05:23:24.064688+0000","last_unstale":"2024-09-17T05:23:26.070994+0000","last_undegraded":"2024-09-17T05:23:26.070994+0000","last_fullsized":"2024-09-17T05:23:26.070994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:38.132272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068472+0000","last_change":"2024-09-17T05:23:24.065406+0000","last_active":"2024-09-17T05:23:26.068472+0000","last_peered":"2024-09-17T05:23:26.068472+0000","last_clean":"2024-09-17T05:23:26.068472+0000","last_became_active":"2024-09-17T05:23:24.065058+0000","last_became_peered":"2024-09-17T05:23:24.065058+0000","last_unstale":"2024-09-17T05:23:26.068472+0000","last_undegraded":"2024-09-17T05:23:26.068472+0000","last_fullsized":"2024-09-17T05:23:26.068472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053208+0000","last_change":"2024-09-17T05:23:25.074915+0000","last_active":"2024-09-17T05:23:26.053208+0000","last_peered":"2024-09-17T05:23:26.053208+0000","last_clean":"2024-09-17T05:23:26.053208+0000","last_became_active":"2024-09-17T05:23:25.074603+0000","last_became_peered":"2024-09-17T05:23:25.074603+0000","last_unstale":"2024-09-17T05:23:26.053208+0000","last_undegraded":"2024-09-17T05:23:26.053208+0000","last_fullsized":"2024-09-17T05:23:26.053208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071423+0000","last_change":"2024-09-17T05:23:24.058441+0000","last_active":"2024-09-17T05:23:26.071423+0000","last_peered":"2024-09-17T05:23:26.071423+0000","last_clean":"2024-09-17T05:23:26.071423+0000","last_became_active":"2024-09-17T05:23:24.054951+0000","last_became_peered":"2024-09-17T05:23:24.054951+0000","last_unstale":"2024-09-17T05:23:26.071423+0000","last_undegraded":"2024-09-17T05:23:26.071423+0000","last_fullsized":"2024-09-17T05:23:26.071423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:11.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052393+0000","last_change":"2024-09-17T05:23:25.056791+0000","last_active":"2024-09-17T05:23:26.052393+0000","last_peered":"2024-09-17T05:23:26.052393+0000","last_clean":"2024-09-17T05:23:26.052393+0000","last_became_active":"2024-09-17T05:23:25.056445+0000","last_became_peered":"2024-09-17T05:23:25.056445+0000","last_unstale":"2024-09-17T05:23:26.052393+0000","last_undegraded":"2024-09-17T05:23:26.052393+0000","last_fullsized":"2024-09-17T05:23:26.052393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052571+0000","last_change":"2024-09-17T05:23:24.058736+0000","last_active":"2024-09-17T05:23:26.052571+0000","last_peered":"2024-09-17T05:23:26.052571+0000","last_clean":"2024-09-17T05:23:26.052571+0000","last_became_active":"2024-09-17T05:23:24.056996+0000","last_became_peered":"2024-09-17T05:23:24.056996+0000","last_unstale":"2024-09-17T05:23:26.052571+0000","last_undegraded":"2024-09-17T05:23:26.052571+0000","last_fullsized":"2024-09-17T05:23:26.052571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:09.319432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070023+0000","last_change":"2024-09-17T05:23:25.072427+0000","last_active":"2024-09-17T05:23:26.070023+0000","last_peered":"2024-09-17T05:23:26.070023+0000","last_clean":"2024-09-17T05:23:26.070023+0000","last_became_active":"2024-09-17T05:23:25.071951+0000","last_became_peered":"2024-09-17T05:23:25.071951+0000","last_unstale":"2024-09-17T05:23:26.070023+0000","last_undegraded":"2024-09-17T05:23:26.070023+0000","last_fullsized":"2024-09-17T05:23:26.070023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:45.608054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071621+0000","last_change":"2024-09-17T05:23:24.064058+0000","last_active":"2024-09-17T05:23:26.071621+0000","last_peered":"2024-09-17T05:23:26.071621+0000","last_clean":"2024-09-17T05:23:26.071621+0000","last_became_active":"2024-09-17T05:23:24.063686+0000","last_became_peered":"2024-09-17T05:23:24.063686+0000","last_unstale":"2024-09-17T05:23:26.071621+0000","last_undegraded":"2024-09-17T05:23:26.071621+0000","last_fullsized":"2024-09-17T05:23:26.071621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:03.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070421+0000","last_change":"2024-09-17T05:23:24.061808+0000","last_active":"2024-09-17T05:23:26.070421+0000","last_peered":"2024-09-17T05:23:26.070421+0000","last_clean":"2024-09-17T05:23:26.070421+0000","last_became_active":"2024-09-17T05:23:24.061393+0000","last_became_peered":"2024-09-17T05:23:24.061393+0000","last_unstale":"2024-09-17T05:23:26.070421+0000","last_undegraded":"2024-09-17T05:23:26.070421+0000","last_fullsized":"2024-09-17T05:23:26.070421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:10.321148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070599+0000","last_change":"2024-09-17T05:23:25.065038+0000","last_active":"2024-09-17T05:23:26.070599+0000","last_peered":"2024-09-17T05:23:26.070599+0000","last_clean":"2024-09-17T05:23:26.070599+0000","last_became_active":"2024-09-17T05:23:25.063557+0000","last_became_peered":"2024-09-17T05:23:25.063557+0000","last_unstale":"2024-09-17T05:23:26.070599+0000","last_undegraded":"2024-09-17T05:23:26.070599+0000","last_fullsized":"2024-09-17T05:23:26.070599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:46.016555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071788+0000","last_change":"2024-09-17T05:23:24.061738+0000","last_active":"2024-09-17T05:23:26.071788+0000","last_peered":"2024-09-17T05:23:26.071788+0000","last_clean":"2024-09-17T05:23:26.071788+0000","last_became_active":"2024-09-17T05:23:24.061344+0000","last_became_peered":"2024-09-17T05:23:24.061344+0000","last_unstale":"2024-09-17T05:23:26.071788+0000","last_undegraded":"2024-09-17T05:23:26.071788+0000","last_fullsized":"2024-09-17T05:23:26.071788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:00.795235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070211+0000","last_change":"2024-09-17T05:23:25.059622+0000","last_active":"2024-09-17T05:23:26.070211+0000","last_peered":"2024-09-17T05:23:26.070211+0000","last_clean":"2024-09-17T05:23:26.070211+0000","last_became_active":"2024-09-17T05:23:25.059267+0000","last_became_peered":"2024-09-17T05:23:25.059267+0000","last_unstale":"2024-09-17T05:23:26.070211+0000","last_undegraded":"2024-09-17T05:23:26.070211+0000","last_fullsized":"2024-09-17T05:23:26.070211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:58.979778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.051888+0000","last_change":"2024-09-17T05:23:24.053175+0000","last_active":"2024-09-17T05:23:26.051888+0000","last_peered":"2024-09-17T05:23:26.051888+0000","last_clean":"2024-09-17T05:23:26.051888+0000","last_became_active":"2024-09-17T05:23:24.052859+0000","last_became_peered":"2024-09-17T05:23:24.052859+0000","last_unstale":"2024-09-17T05:23:26.051888+0000","last_undegraded":"2024-09-17T05:23:26.051888+0000","last_fullsized":"2024-09-17T05:23:26.051888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071095+0000","last_change":"2024-09-17T05:23:25.057826+0000","last_active":"2024-09-17T05:23:26.071095+0000","last_peered":"2024-09-17T05:23:26.071095+0000","last_clean":"2024-09-17T05:23:26.071095+0000","last_became_active":"2024-09-17T05:23:25.057507+0000","last_became_peered":"2024-09-17T05:23:25.057507+0000","last_unstale":"2024-09-17T05:23:26.071095+0000","last_undegraded":"2024-09-17T05:23:26.071095+0000","last_fullsized":"2024-09-17T05:23:26.071095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068904+0000","last_change":"2024-09-17T05:23:24.068858+0000","last_active":"2024-09-17T05:23:26.068904+0000","last_peered":"2024-09-17T05:23:26.068904+0000","last_clean":"2024-09-17T05:23:26.068904+0000","last_became_active":"2024-09-17T05:23:24.068387+0000","last_became_peered":"2024-09-17T05:23:24.068387+0000","last_unstale":"2024-09-17T05:23:26.068904+0000","last_undegraded":"2024-09-17T05:23:26.068904+0000","last_fullsized":"2024-09-17T05:23:26.068904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068658+0000","last_change":"2024-09-17T05:23:25.049296+0000","last_active":"2024-09-17T05:23:26.068658+0000","last_peered":"2024-09-17T05:23:26.068658+0000","last_clean":"2024-09-17T05:23:26.068658+0000","last_became_active":"2024-09-17T05:23:25.048948+0000","last_became_peered":"2024-09-17T05:23:25.048948+0000","last_unstale":"2024-09-17T05:23:26.068658+0000","last_undegraded":"2024-09-17T05:23:26.068658+0000","last_fullsized":"2024-09-17T05:23:26.068658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:49.995105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069337+0000","last_change":"2024-09-17T05:23:25.067211+0000","last_active":"2024-09-17T05:23:26.069337+0000","last_peered":"2024-09-17T05:23:26.069337+0000","last_clean":"2024-09-17T05:23:26.069337+0000","last_became_active":"2024-09-17T05:23:25.066883+0000","last_became_peered":"2024-09-17T05:23:25.066883+0000","last_unstale":"2024-09-17T05:23:26.069337+0000","last_undegraded":"2024-09-17T05:23:26.069337+0000","last_fullsized":"2024-09-17T05:23:26.069337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071212+0000","last_change":"2024-09-17T05:23:24.061156+0000","last_active":"2024-09-17T05:23:26.071212+0000","last_peered":"2024-09-17T05:23:26.071212+0000","last_clean":"2024-09-17T05:23:26.071212+0000","last_became_active":"2024-09-17T05:23:24.060751+0000","last_became_peered":"2024-09-17T05:23:24.060751+0000","last_unstale":"2024-09-17T05:23:26.071212+0000","last_undegraded":"2024-09-17T05:23:26.071212+0000","last_fullsized":"2024-09-17T05:23:26.071212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.506573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071131+0000","last_change":"2024-09-17T05:23:25.085054+0000","last_active":"2024-09-17T05:23:26.071131+0000","last_peered":"2024-09-17T05:23:26.071131+0000","last_clean":"2024-09-17T05:23:26.071131+0000","last_became_active":"2024-09-17T05:23:25.084307+0000","last_became_peered":"2024-09-17T05:23:25.084307+0000","last_unstale":"2024-09-17T05:23:26.071131+0000","last_undegraded":"2024-09-17T05:23:26.071131+0000","last_fullsized":"2024-09-17T05:23:26.071131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071557+0000","last_change":"2024-09-17T05:23:25.052805+0000","last_active":"2024-09-17T05:23:26.071557+0000","last_peered":"2024-09-17T05:23:26.071557+0000","last_clean":"2024-09-17T05:23:26.071557+0000","last_became_active":"2024-09-17T05:23:25.052440+0000","last_became_peered":"2024-09-17T05:23:25.052440+0000","last_unstale":"2024-09-17T05:23:26.071557+0000","last_undegraded":"2024-09-17T05:23:26.071557+0000","last_fullsized":"2024-09-17T05:23:26.071557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:51.739703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053334+0000","last_change":"2024-09-17T05:23:24.061930+0000","last_active":"2024-09-17T05:23:26.053334+0000","last_peered":"2024-09-17T05:23:26.053334+0000","last_clean":"2024-09-17T05:23:26.053334+0000","last_became_active":"2024-09-17T05:23:24.060988+0000","last_became_peered":"2024-09-17T05:23:24.060988+0000","last_unstale":"2024-09-17T05:23:26.053334+0000","last_undegraded":"2024-09-17T05:23:26.053334+0000","last_fullsized":"2024-09-17T05:23:26.053334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:47.573508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054561+0000","last_change":"2024-09-17T05:23:25.065743+0000","last_active":"2024-09-17T05:23:26.054561+0000","last_peered":"2024-09-17T05:23:26.054561+0000","last_clean":"2024-09-17T05:23:26.054561+0000","last_became_active":"2024-09-17T05:23:25.062840+0000","last_became_peered":"2024-09-17T05:23:25.062840+0000","last_unstale":"2024-09-17T05:23:26.054561+0000","last_undegraded":"2024-09-17T05:23:26.054561+0000","last_fullsized":"2024-09-17T05:23:26.054561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:01.803022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052543+0000","last_change":"2024-09-17T05:23:24.060124+0000","last_active":"2024-09-17T05:23:26.052543+0000","last_peered":"2024-09-17T05:23:26.052543+0000","last_clean":"2024-09-17T05:23:26.052543+0000","last_became_active":"2024-09-17T05:23:24.057538+0000","last_became_peered":"2024-09-17T05:23:24.057538+0000","last_unstale":"2024-09-17T05:23:26.052543+0000","last_undegraded":"2024-09-17T05:23:26.052543+0000","last_fullsized":"2024-09-17T05:23:26.052543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:41.202615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053337+0000","last_change":"2024-09-17T05:23:25.079567+0000","last_active":"2024-09-17T05:23:26.053337+0000","last_peered":"2024-09-17T05:23:26.053337+0000","last_clean":"2024-09-17T05:23:26.053337+0000","last_became_active":"2024-09-17T05:23:25.078899+0000","last_became_peered":"2024-09-17T05:23:25.078899+0000","last_unstale":"2024-09-17T05:23:26.053337+0000","last_undegraded":"2024-09-17T05:23:26.053337+0000","last_fullsized":"2024-09-17T05:23:26.053337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:39.680169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053486+0000","last_change":"2024-09-17T05:23:24.061891+0000","last_active":"2024-09-17T05:23:26.053486+0000","last_peered":"2024-09-17T05:23:26.053486+0000","last_clean":"2024-09-17T05:23:26.053486+0000","last_became_active":"2024-09-17T05:23:24.055527+0000","last_became_peered":"2024-09-17T05:23:24.055527+0000","last_unstale":"2024-09-17T05:23:26.053486+0000","last_undegraded":"2024-09-17T05:23:26.053486+0000","last_fullsized":"2024-09-17T05:23:26.053486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053220+0000","last_change":"2024-09-17T05:23:25.076838+0000","last_active":"2024-09-17T05:23:26.053220+0000","last_peered":"2024-09-17T05:23:26.053220+0000","last_clean":"2024-09-17T05:23:26.053220+0000","last_became_active":"2024-09-17T05:23:25.076547+0000","last_became_peered":"2024-09-17T05:23:25.076547+0000","last_unstale":"2024-09-17T05:23:26.053220+0000","last_undegraded":"2024-09-17T05:23:26.053220+0000","last_fullsized":"2024-09-17T05:23:26.053220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.142652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070483+0000","last_change":"2024-09-17T05:23:24.059741+0000","last_active":"2024-09-17T05:23:26.070483+0000","last_peered":"2024-09-17T05:23:26.070483+0000","last_clean":"2024-09-17T05:23:26.070483+0000","last_became_active":"2024-09-17T05:23:24.058336+0000","last_became_peered":"2024-09-17T05:23:24.058336+0000","last_unstale":"2024-09-17T05:23:26.070483+0000","last_undegraded":"2024-09-17T05:23:26.070483+0000","last_fullsized":"2024-09-17T05:23:26.070483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:01.640938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:34.331 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:23:34.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T05:23:34.332 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T05:23:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 -- 172.21.15.43:0/603271330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac0b3920 msgr2=0x7fd1ac0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 --2- 172.21.15.43:0/603271330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac0b3920 0x7fd1ac0ad0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0711f0 tx=0x7fd1a401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 -- 172.21.15.43:0/603271330 shutdown_connections 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 --2- 172.21.15.43:0/603271330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac0b3920 0x7fd1ac0ad0b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 --2- 172.21.15.43:0/603271330 >> v2:172.21.15.62:3300/0 conn(0x7fd1ac06f110 0x7fd1ac0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 --2- 172.21.15.43:0/603271330 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac155d70 0x7fd1ac15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 -- 172.21.15.43:0/603271330 >> 172.21.15.43:0/603271330 conn(0x7fd1ac0b23d0 msgr2=0x7fd1ac0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 -- 172.21.15.43:0/603271330 shutdown_connections 2024-09-17T05:23:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.528+0000 7fd1b4600640 1 -- 172.21.15.43:0/603271330 wait complete. 2024-09-17T05:23:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 Processor -- start 2024-09-17T05:23:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- start start 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 0x7fd1ac1f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac0b3920 0x7fd1ac09d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac15f830 con 0x7fd1ac155d70 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac15f6b0 con 0x7fd1ac06f110 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ac15f9b0 con 0x7fd1ac0b3920 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41348/0 (socket says 172.21.15.43:41348) 2024-09-17T05:23:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 -- 172.21.15.43:0/1975837202 learned_addr learned my addr 172.21.15.43:0/1975837202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b1200640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac0b3920 0x7fd1ac09d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b1c00640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 0x7fd1ac1f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 -- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac0b3920 msgr2=0x7fd1ac09d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac0b3920 0x7fd1ac09d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 -- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 msgr2=0x7fd1ac1f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 0x7fd1ac1f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 -- 172.21.15.43:0/1975837202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac150190 con 0x7fd1ac155d70 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b1c00640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 0x7fd1ac1f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b2600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1a401b370 tx=0x7fd1a40a5b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40bb070 con 0x7fd1ac155d70 2024-09-17T05:23:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a4004e00 con 0x7fd1ac155d70 2024-09-17T05:23:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac150470 con 0x7fd1ac155d70 2024-09-17T05:23:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40b2040 con 0x7fd1ac155d70 2024-09-17T05:23:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.532+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac14fc20 con 0x7fd1ac155d70 2024-09-17T05:23:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.536+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ac14ff00 con 0x7fd1ac155d70 2024-09-17T05:23:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.536+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1a40be070 con 0x7fd1ac155d70 2024-09-17T05:23:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.536+0000 7fd1a9600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 0x7fd18c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.536+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd1a412e380 con 0x7fd1ac155d70 2024-09-17T05:23:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.536+0000 7fd1b1c00640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 0x7fd18c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.540+0000 7fd1b1c00640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 0x7fd18c075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005950 tx=0x7fd1a0005880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.540+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1a4016970 con 0x7fd1ac155d70 2024-09-17T05:23:34.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd1ac0b0fe0 con 0x7fd1ac155d70 2024-09-17T05:23:34.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1a9600640 1 -- 172.21.15.43:0/1975837202 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fd1a40f8e50 con 0x7fd1ac155d70 2024-09-17T05:23:34.715 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 msgr2=0x7fd18c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 0x7fd18c075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005950 tx=0x7fd1a0005880 comp rx=0 tx=0).stop 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 msgr2=0x7fd1ac1fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1a401b370 tx=0x7fd1a40a5b50 comp rx=0 tx=0).stop 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 shutdown_connections 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd18c073200 0x7fd18c075710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac155d70 0x7fd1ac1fc760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1ac0b3920 0x7fd1ac09d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 --2- 172.21.15.43:0/1975837202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1ac06f110 0x7fd1ac1f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 >> 172.21.15.43:0/1975837202 conn(0x7fd1ac0b23d0 msgr2=0x7fd1ac0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:34.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.716+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 shutdown_connections 2024-09-17T05:23:34.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.720+0000 7fd1b4600640 1 -- 172.21.15.43:0/1975837202 wait complete. 2024-09-17T05:23:34.738 INFO:teuthology.orchestra.run.smithi043.stdout:[{"version":31,"timestamp":"2024-09-17T05:23:12.731003+0000","name":"","changes":[{"name":"osd.7/osd_mclock_max_capacity_iops_ssd","new_value":"4223.597904"}]}] 2024-09-17T05:23:34.739 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T05:23:34.739 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T05:23:34.739 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T05:23:34.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T05:23:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- 172.21.15.43:0/3727934658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab80b3af0 msgr2=0x7fbab80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 --2- 172.21.15.43:0/3727934658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab80b3af0 0x7fbab80ad0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009c80 tx=0x7fbab001cce0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- 172.21.15.43:0/3727934658 shutdown_connections 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 --2- 172.21.15.43:0/3727934658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab80b3af0 0x7fbab80ad0b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 --2- 172.21.15.43:0/3727934658 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab8155d70 0x7fbab815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- 172.21.15.43:0/3727934658 >> 172.21.15.43:0/3727934658 conn(0x7fbab80b23d0 msgr2=0x7fbab80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- 172.21.15.43:0/3727934658 shutdown_connections 2024-09-17T05:23:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- 172.21.15.43:0/3727934658 wait complete. 2024-09-17T05:23:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 Processor -- start 2024-09-17T05:23:34.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.900+0000 7fbabf200640 1 -- start start 2024-09-17T05:23:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab8155d70 0x7fbab8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab809cbe0 0x7fbab8164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8168be0 con 0x7fbab806f110 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8168a60 con 0x7fbab8155d70 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbab8168d60 con 0x7fbab809cbe0 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41358/0 (socket says 172.21.15.43:41358) 2024-09-17T05:23:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 -- 172.21.15.43:0/1856866669 learned_addr learned my addr 172.21.15.43:0/1856866669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbaaf400640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab8155d70 0x7fbab8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabd200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab809cbe0 0x7fbab8164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 -- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab809cbe0 msgr2=0x7fbab8164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab809cbe0 0x7fbab8164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 -- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab8155d70 msgr2=0x7fbab8153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab8155d70 0x7fbab8153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 -- 172.21.15.43:0/1856866669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab81f8960 con 0x7fbab806f110 2024-09-17T05:23:34.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab7e00640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800d6d0 tx=0x7fbaa800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8017070 con 0x7fbab806f110 2024-09-17T05:23:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa8009b10 con 0x7fbab806f110 2024-09-17T05:23:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8010460 con 0x7fbab806f110 2024-09-17T05:23:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab81f8c40 con 0x7fbab806f110 2024-09-17T05:23:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81f9200 con 0x7fbab806f110 2024-09-17T05:23:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.904+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab80ad0b0 con 0x7fbab806f110 2024-09-17T05:23:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.908+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbaa801a070 con 0x7fbab806f110 2024-09-17T05:23:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.908+0000 7fbab4c00640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 0x7fba94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.908+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbaa8094600 con 0x7fbab806f110 2024-09-17T05:23:34.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.908+0000 7fbaaf400640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 0x7fba94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:34.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.908+0000 7fbaaf400640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 0x7fba94075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbab8156c50 tx=0x7fbaa0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:34.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:34.912+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbaa805ef30 con 0x7fbab806f110 2024-09-17T05:23:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.128+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbab80d0350 con 0x7fbab806f110 2024-09-17T05:23:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.132+0000 7fbab4c00640 1 -- 172.21.15.43:0/1856866669 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186455 (secure 0 0 0) 0x7fbaa8063960 con 0x7fbab806f110 2024-09-17T05:23:35.133 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 msgr2=0x7fba94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 0x7fba94075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbab8156c50 tx=0x7fbaa0009040 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 msgr2=0x7fbab81fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800d6d0 tx=0x7fbaa800db90 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 shutdown_connections 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94073230 0x7fba94075740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab8155d70 0x7fbab8153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbab809cbe0 0x7fbab8164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 --2- 172.21.15.43:0/1856866669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806f110 0x7fbab81fc750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 >> 172.21.15.43:0/1856866669 conn(0x7fbab80b23d0 msgr2=0x7fbab815dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 shutdown_connections 2024-09-17T05:23:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.136+0000 7fbabf200640 1 -- 172.21.15.43:0/1856866669 wait complete. 2024-09-17T05:23:35.154 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1563535289},{"type":"v1","addr":"172.21.15.62:6801","nonce":1563535289}]},"active_addr":"172.21.15.62:6801/1563535289","active_change":"2024-09-17T05:23:03.827432+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4120,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3334696308}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":445435249}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1484873532}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":380632384}]}]} 2024-09-17T05:23:35.157 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T05:23:35.157 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T05:23:35.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 -- 172.21.15.43:0/2952319630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c0b3af0 msgr2=0x7fa08c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 --2- 172.21.15.43:0/2952319630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c0b3af0 0x7fa08c15c7e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa074009900 tx=0x7fa07401c820 comp rx=0 tx=0).stop 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 -- 172.21.15.43:0/2952319630 shutdown_connections 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 --2- 172.21.15.43:0/2952319630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c06f110 0x7fa08c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 --2- 172.21.15.43:0/2952319630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c0b3af0 0x7fa08c15c7e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 -- 172.21.15.43:0/2952319630 >> 172.21.15.43:0/2952319630 conn(0x7fa08c0b23d0 msgr2=0x7fa08c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 -- 172.21.15.43:0/2952319630 shutdown_connections 2024-09-17T05:23:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.316+0000 7fa091200640 1 -- 172.21.15.43:0/2952319630 wait complete. 2024-09-17T05:23:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 Processor -- start 2024-09-17T05:23:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- start start 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c1f9580 0x7fa08c1f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c168c60 con 0x7fa08c06f110 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c168ae0 con 0x7fa08c155d70 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c168de0 con 0x7fa08c1f9580 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41370/0 (socket says 172.21.15.43:41370) 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 -- 172.21.15.43:0/2108947752 learned_addr learned my addr 172.21.15.43:0/2108947752 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa08a800640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c1f9580 0x7fa08c1f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa083e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 -- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c1f9580 msgr2=0x7fa08c1f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c1f9580 0x7fa08c1f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 -- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 msgr2=0x7fa08c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 -- 172.21.15.43:0/2108947752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0cd7b0 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa089e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c3c0 tx=0x7fa07c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c017070 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa07c0053c0 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c0cadd0 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c003860 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.320+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c0cb390 con 0x7fa08c06f110 2024-09-17T05:23:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.324+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c0ad0b0 con 0x7fa08c06f110 2024-09-17T05:23:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.324+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa07c01a070 con 0x7fa08c06f110 2024-09-17T05:23:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.324+0000 7fa082000640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.324+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa07c093530 con 0x7fa08c06f110 2024-09-17T05:23:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.324+0000 7fa083e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.328+0000 7fa083e00640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0740af000 tx=0x7fa07401b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:35.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.328+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa07c05def0 con 0x7fa08c06f110 2024-09-17T05:23:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.504+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa08c1534c0 con 0x7fa08c06f110 2024-09-17T05:23:35.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa082000640 1 -- 172.21.15.43:0/2108947752 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fa07c062920 con 0x7fa08c06f110 2024-09-17T05:23:35.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:35.506 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:35.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 msgr2=0x7fa068075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0740af000 tx=0x7fa07401b060 comp rx=0 tx=0).stop 2024-09-17T05:23:35.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 msgr2=0x7fa08c09d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c3c0 tx=0x7fa07c00c880 comp rx=0 tx=0).stop 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 shutdown_connections 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c1f9580 0x7fa08c1f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 --2- 172.21.15.43:0/2108947752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c09d9b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 >> 172.21.15.43:0/2108947752 conn(0x7fa08c0b23d0 msgr2=0x7fa08c0aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 shutdown_connections 2024-09-17T05:23:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.508+0000 7fa091200640 1 -- 172.21.15.43:0/2108947752 wait complete. 2024-09-17T05:23:35.529 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T05:23:35.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 -- 172.21.15.43:0/1575769970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b3af0 msgr2=0x7fb5e80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 --2- 172.21.15.43:0/1575769970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b3af0 0x7fb5e80ad0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009c80 tx=0x7fb5e001cce0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 -- 172.21.15.43:0/1575769970 shutdown_connections 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 --2- 172.21.15.43:0/1575769970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b3af0 0x7fb5e80ad0b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 --2- 172.21.15.43:0/1575769970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e8155d70 0x7fb5e815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 -- 172.21.15.43:0/1575769970 >> 172.21.15.43:0/1575769970 conn(0x7fb5e80b23d0 msgr2=0x7fb5e80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 -- 172.21.15.43:0/1575769970 shutdown_connections 2024-09-17T05:23:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 -- 172.21.15.43:0/1575769970 wait complete. 2024-09-17T05:23:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.688+0000 7fb5eda00640 1 Processor -- start 2024-09-17T05:23:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- start start 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 0x7fb5e8151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 0x7fb5e809c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8168cb0 con 0x7fb5e815a710 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8168b30 con 0x7fb5e8155d70 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e8168e30 con 0x7fb5e806f110 2024-09-17T05:23:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41386/0 (socket says 172.21.15.43:41386) 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 -- 172.21.15.43:0/3539959925 learned_addr learned my addr 172.21.15.43:0/3539959925 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 -- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 msgr2=0x7fb5e8151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5dfe00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 0x7fb5e809c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e6a00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 0x7fb5e8151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 0x7fb5e8151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 -- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 msgr2=0x7fb5e809c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 0x7fb5e809c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 -- 172.21.15.43:0/3539959925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e815b060 con 0x7fb5e815a710 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5dfe00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 0x7fb5e809c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e6a00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 0x7fb5e8151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5e7400640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009bd0 tx=0x7fb5e00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00bc070 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5e00042f0 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e809c950 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00b2040 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e80cff40 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.692+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e80ad0b0 con 0x7fb5e815a710 2024-09-17T05:23:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.696+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5e00bf070 con 0x7fb5e815a710 2024-09-17T05:23:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.696+0000 7fb5dea00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 0x7fb5c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.696+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb5e012e130 con 0x7fb5e815a710 2024-09-17T05:23:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.696+0000 7fb5e6a00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 0x7fb5c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:35.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.696+0000 7fb5e6a00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 0x7fb5c4075740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80715c0 tx=0x7fb5d8008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.700+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb5e815b060 con 0x7fb5e815a710 2024-09-17T05:23:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.876+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5e80d1150 con 0x7fb5e815a710 2024-09-17T05:23:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.876+0000 7fb5dea00640 1 -- 172.21.15.43:0/3539959925 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fb5e80d1150 con 0x7fb5e815a710 2024-09-17T05:23:35.875 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:35.875 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 msgr2=0x7fb5c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 0x7fb5c4075740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80715c0 tx=0x7fb5d8008040 comp rx=0 tx=0).stop 2024-09-17T05:23:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 msgr2=0x7fb5e815ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009bd0 tx=0x7fb5e00093d0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 shutdown_connections 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5c4073230 0x7fb5c4075740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5e8155d70 0x7fb5e809c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a710 0x7fb5e815ab20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 --2- 172.21.15.43:0/3539959925 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5e806f110 0x7fb5e8151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 >> 172.21.15.43:0/3539959925 conn(0x7fb5e80b23d0 msgr2=0x7fb5e80ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 shutdown_connections 2024-09-17T05:23:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:35.880+0000 7fb5eda00640 1 -- 172.21.15.43:0/3539959925 wait complete. 2024-09-17T05:23:35.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T05:23:35.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T05:23:35.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T05:23:35.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T05:23:35.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T05:23:35.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T05:23:35.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T05:23:35.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T05:23:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 -- 172.21.15.43:0/1055710992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e660 msgr2=0x7fe6dc15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe14be00640 1 -- 172.21.15.43:0/1036788203 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b1070 con 0x7fe14c0cc840 2024-09-17T05:23:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 -- 172.21.15.43:0/1036788203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 msgr2=0x7fe14c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 --2- 172.21.15.43:0/1036788203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c0ccc50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1400074e0 tx=0x7fe14001d3b0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 -- 172.21.15.43:0/1036788203 shutdown_connections 2024-09-17T05:23:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1055710992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e660 0x7fe6dc15ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009890 tx=0x7fe6d001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 --2- 172.21.15.43:0/1036788203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0ca850 0x7fe14c0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 --2- 172.21.15.43:0/1036788203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c0ccc50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 -- 172.21.15.43:0/1036788203 >> 172.21.15.43:0/1036788203 conn(0x7fe14c0b1dd0 msgr2=0x7fe14c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 -- 172.21.15.43:0/1036788203 shutdown_connections 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 -- 172.21.15.43:0/1055710992 shutdown_connections 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1055710992 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc0b5940 0x7fe6dc15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1055710992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e660 0x7fe6dc15ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 -- 172.21.15.43:0/1055710992 >> 172.21.15.43:0/1055710992 conn(0x7fe6dc0cee60 msgr2=0x7fe6dc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 -- 172.21.15.43:0/1055710992 shutdown_connections 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe6e3400640 1 -- 172.21.15.43:0/1055710992 wait complete. 2024-09-17T05:23:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.192+0000 7fe154000640 1 -- 172.21.15.43:0/1036788203 wait complete. 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 -- 172.21.15.43:0/2425731353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 msgr2=0x7ff7680b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 --2- 172.21.15.43:0/2425731353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 0x7ff7680b4290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff754009840 tx=0x7ff75401c820 comp rx=0 tx=0).stop 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 -- 172.21.15.43:0/2425731353 shutdown_connections 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 --2- 172.21.15.43:0/2425731353 >> v2:172.21.15.43:3300/0 conn(0x7ff7680b3bf0 0x7ff7680b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 --2- 172.21.15.43:0/2425731353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 0x7ff7680b4290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 -- 172.21.15.43:0/2425731353 >> 172.21.15.43:0/2425731353 conn(0x7ff7680b1dd0 msgr2=0x7ff7680b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 -- 172.21.15.43:0/2425731353 shutdown_connections 2024-09-17T05:23:36.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7ff76ee00640 1 -- 172.21.15.43:0/2425731353 wait complete. 2024-09-17T05:23:36.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7fe154000640 1 Processor -- start 2024-09-17T05:23:36.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7fe154000640 1 -- start start 2024-09-17T05:23:36.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7f6474c00640 1 -- 172.21.15.43:0/1507761538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6468006540 msgr2=0x7f6468035e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7f6474c00640 1 --2- 172.21.15.43:0/1507761538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6468006540 0x7f6468035e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f646800f020 tx=0x7f646401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 -- 172.21.15.43:0/1507761538 shutdown_connections 2024-09-17T05:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7f2d75000640 1 -- 172.21.15.43:0/2982177604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d70155000 msgr2=0x7f2d700cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 --2- 172.21.15.43:0/1507761538 >> v2:172.21.15.43:3300/0 conn(0x7f64680e4ac0 0x7f64680e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 --2- 172.21.15.43:0/1507761538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6468006540 0x7f6468035e40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 -- 172.21.15.43:0/1507761538 >> 172.21.15.43:0/1507761538 conn(0x7f646801db80 msgr2=0x7f6468039840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 -- 172.21.15.43:0/1507761538 shutdown_connections 2024-09-17T05:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.196+0000 7f2d75000640 1 --2- 172.21.15.43:0/2982177604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d70155000 0x7f2d700cf740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d70069d90 tx=0x7f2d5c01d520 comp rx=0 tx=0).stop 2024-09-17T05:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 -- 172.21.15.43:0/2982177604 shutdown_connections 2024-09-17T05:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 --2- 172.21.15.43:0/2982177604 >> v2:172.21.15.43:3300/0 conn(0x7f2d700ce010 0x7f2d700ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 --2- 172.21.15.43:0/2982177604 >> v2:172.21.15.62:3300/0 conn(0x7f2d70073300 0x7f2d70073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 --2- 172.21.15.43:0/2982177604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d70155000 0x7f2d700cf740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 -- 172.21.15.43:0/2982177604 >> 172.21.15.43:0/2982177604 conn(0x7f2d700d0440 msgr2=0x7f2d700d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe6e3400640 1 Processor -- start 2024-09-17T05:23:36.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 -- 172.21.15.43:0/2982177604 shutdown_connections 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0ca850 0x7fe14c0c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0cb1a0 0x7fe14c0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7ff76ee00640 1 Processor -- start 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe150c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0cb1a0 0x7fe14c0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe150c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0cb1a0 0x7fe14c0c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47808/0 (socket says 172.21.15.43:47808) 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0cd930 con 0x7fe14c0ca850 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0b1ab0 con 0x7fe14c0cc840 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe154000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c162c70 con 0x7fe14c0cb1a0 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7ff76ee00640 1 -- start start 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe150c00640 1 -- 172.21.15.43:0/1807632954 learned_addr learned my addr 172.21.15.43:0/1807632954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe151600640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0ca850 0x7fe14c0c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe152000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f6474c00640 1 -- 172.21.15.43:0/1507761538 wait complete. 2024-09-17T05:23:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7f2d75000640 1 -- 172.21.15.43:0/2982177604 wait complete. 2024-09-17T05:23:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7f6474c00640 1 Processor -- start 2024-09-17T05:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 -- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0cb1a0 msgr2=0x7fe14c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7f6474c00640 1 -- start start 2024-09-17T05:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe14c0cb1a0 0x7fe14c0c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.200+0000 7fe6e3400640 1 -- start start 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e250 0x7fe6dc0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc06e660 0x7fe6dc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 0x7fe6dc15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0d0f60 con 0x7fe6dc06e250 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc15fa70 con 0x7fe6dc0b5940 2024-09-17T05:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e3400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc0d10e0 con 0x7fe6dc06e660 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 -- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0ca850 msgr2=0x7fe14c0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0ca850 0x7fe14c0c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c099810 con 0x7fe14c0cc840 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe152000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c154ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14400be00 tx=0x7fe14400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144003980 con 0x7fe14c0cc840 2024-09-17T05:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe144003c10 con 0x7fe14c0cc840 2024-09-17T05:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144004280 con 0x7fe14c0cc840 2024-09-17T05:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 0x7fe6dc15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c099ad0 con 0x7fe14c0cc840 2024-09-17T05:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c099db0 con 0x7fe14c0cc840 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7f2d75000640 1 Processor -- start 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc06e660 0x7fe6dc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 0x7ff7680af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7680b3bf0 0x7ff7680c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7680cf390 0x7ff7680c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff768168f70 con 0x7ff7680b3bf0 2024-09-17T05:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff768168df0 con 0x7ff7680cf390 2024-09-17T05:23:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff7681690f0 con 0x7ff7680b37e0 2024-09-17T05:23:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 0x7fe6dc15e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42436/0 (socket says 172.21.15.43:42436) 2024-09-17T05:23:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 -- 172.21.15.43:0/1549122154 learned_addr learned my addr 172.21.15.43:0/1549122154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e0a00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e250 0x7fe6dc0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7f2d75000640 1 -- start start 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 -- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc06e660 msgr2=0x7fe6dc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc06e660 0x7fe6dc15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 -- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e250 msgr2=0x7fe6dc0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e250 0x7fe6dc0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe6e1400640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc200fd0 con 0x7fe6dc0b5940 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6dbe00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6dc06e660 0x7fe6dc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6e0a00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc06e250 0x7fe6dc0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6e1400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 0x7fe6dc15e1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400e970 tx=0x7fe6d400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7680cf390 0x7ff7680c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7680cf390 0x7ff7680c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42450/0 (socket says 172.21.15.43:42450) 2024-09-17T05:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7ff76ce00640 1 -- 172.21.15.43:0/494995712 learned_addr learned my addr 172.21.15.43:0/494995712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7680b3bf0 0x7ff7680c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.204+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe14c0cac60 con 0x7fe14c0cc840 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6468033ce0 0x7f64680d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680e4ac0 0x7f6468052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680d2ea0 0x7f6468053200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64680f0a80 con 0x7f6468033ce0 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f64680f0900 con 0x7f64680e4ac0 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6474c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f64680f0c00 con 0x7f64680d2ea0 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6476200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6468033ce0 0x7f64680d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6476200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6468033ce0 0x7f64680d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41486/0 (socket says 172.21.15.43:41486) 2024-09-17T05:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6476200640 1 -- 172.21.15.43:0/911674634 learned_addr learned my addr 172.21.15.43:0/911674634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70073300 0x7f2d7014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 0x7f2d7014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d70155000 0x7f2d701fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680e4ac0 0x7f6468052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680d2ea0 0x7f6468053200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4004d30 con 0x7fe6dc0b5940 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 -- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 msgr2=0x7ff7680af4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7680b37e0 0x7ff7680af4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 -- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7680cf390 msgr2=0x7ff7680c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7680cf390 0x7ff7680c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff768159960 con 0x7ff7680b3bf0 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d400cb20 con 0x7fe6dc0b5940 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4004d30 con 0x7fe6dc0b5940 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7015efe0 con 0x7f2d70073300 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7015ee60 con 0x7f2d70155000 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d75000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7015f160 con 0x7f2d700ce010 2024-09-17T05:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 -- 172.21.15.43:0/835624833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634006d570 msgr2=0x7f634006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 --2- 172.21.15.43:0/835624833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634006d570 0x7f634006f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63400b2cd0 tx=0x7f632c01c800 comp rx=0 tx=0).stop 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff767400640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7680b3bf0 0x7ff7680c7900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7680ce5a0 tx=0x7ff754005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d70155000 0x7f2d701fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 0x7f2d7014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 0x7f2d7014bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47856/0 (socket says 172.21.15.43:47856) 2024-09-17T05:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 -- 172.21.15.43:0/2102713725 learned_addr learned my addr 172.21.15.43:0/2102713725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 -- 172.21.15.43:0/835624833 shutdown_connections 2024-09-17T05:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 --2- 172.21.15.43:0/835624833 >> v2:172.21.15.62:3300/0 conn(0x7f63400cc6d0 0x7f634006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 --2- 172.21.15.43:0/835624833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634006d570 0x7f634006f1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 -- 172.21.15.43:0/835624833 >> 172.21.15.43:0/835624833 conn(0x7f63400b23d0 msgr2=0x7f63400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70073300 0x7f2d7014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 -- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680e4ac0 msgr2=0x7f6468052cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680e4ac0 0x7f6468052cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 -- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6468033ce0 msgr2=0x7f64680d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6468033ce0 0x7f64680d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64680554d0 con 0x7f64680d2ea0 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe144006d10 con 0x7fe14c0cc840 2024-09-17T05:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc200210 con 0x7fe6dc0b5940 2024-09-17T05:23:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc2007d0 con 0x7fe6dc0b5940 2024-09-17T05:23:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 -- 172.21.15.43:0/3398668808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c15adc0 msgr2=0x7fb37c06d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 --2- 172.21.15.43:0/3398668808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c15adc0 0x7fb37c06d3a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb37c072b60 tx=0x7fb37001cd80 comp rx=0 tx=0).stop 2024-09-17T05:23:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f646f400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680d2ea0 0x7f6468053200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f647002ebe0 tx=0x7f647006a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540b2040 con 0x7ff7680b3bf0 2024-09-17T05:23:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 -- 172.21.15.43:0/835624833 shutdown_connections 2024-09-17T05:23:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6346800640 1 -- 172.21.15.43:0/835624833 wait complete. 2024-09-17T05:23:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6dc0b5230 con 0x7fe6dc0b5940 2024-09-17T05:23:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7540ba070 con 0x7ff7680b3bf0 2024-09-17T05:23:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754003600 con 0x7ff7680b3bf0 2024-09-17T05:23:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe149600640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe12c073200 0x7fe12c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe1440946d0 con 0x7fe14c0cc840 2024-09-17T05:23:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647006ce60 con 0x7f64680d2ea0 2024-09-17T05:23:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6470074830 con 0x7f64680d2ea0 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 -- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d70155000 msgr2=0x7f2d701fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d70155000 0x7f2d701fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 -- 172.21.15.43:0/2102713725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70073300 msgr2=0x7f2d7014ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70073300 0x7f2d7014ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d70155bb0 con 0x7f2d700ce010 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7f2d6ea00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 0x7f2d7014bfb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ca20 tx=0x7f2d6000cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 -- 172.21.15.43:0/3398668808 shutdown_connections 2024-09-17T05:23:36.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 --2- 172.21.15.43:0/3398668808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c15adc0 0x7fb37c06d3a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 --2- 172.21.15.43:0/3398668808 >> v2:172.21.15.62:3300/0 conn(0x7fb37c0711a0 0x7fb37c06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 -- 172.21.15.43:0/3398668808 >> 172.21.15.43:0/3398668808 conn(0x7fb37c0b23d0 msgr2=0x7fb37c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 -- 172.21.15.43:0/3398668808 shutdown_connections 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.208+0000 7fb384000640 1 -- 172.21.15.43:0/3398668808 wait complete. 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 -- 172.21.15.43:0/3466296143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72340cc6d0 msgr2=0x7f723406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 --2- 172.21.15.43:0/3466296143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72340cc6d0 0x7f723406f6e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f722400b330 tx=0x7f722401d100 comp rx=0 tx=0).stop 2024-09-17T05:23:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe149600640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fe12c07b3f0 0x7fe12c07d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe12c07df10 con 0x7fe12c07b3f0 2024-09-17T05:23:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe144094a80 con 0x7fe14c0cc840 2024-09-17T05:23:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470074c60 con 0x7f64680d2ea0 2024-09-17T05:23:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6474c00640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64680d2480 con 0x7f64680d2ea0 2024-09-17T05:23:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7680afa10 con 0x7ff7680b3bf0 2024-09-17T05:23:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681593a0 con 0x7ff7680b3bf0 2024-09-17T05:23:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6474c00640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64680d2a40 con 0x7f64680d2ea0 2024-09-17T05:23:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe150c00640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fe12c07b3f0 0x7fe12c07d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe151600640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe12c073200 0x7fe12c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60007c70 con 0x7f2d700ce010 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d6000f450 con 0x7f2d700ce010 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f8a0 con 0x7f2d700ce010 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d75000640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d701fdde0 con 0x7f2d700ce010 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d75000640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7014e0f0 con 0x7f2d700ce010 2024-09-17T05:23:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6346800640 1 Processor -- start 2024-09-17T05:23:36.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 -- 172.21.15.43:0/3466296143 shutdown_connections 2024-09-17T05:23:36.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 --2- 172.21.15.43:0/3466296143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f72340cc6d0 0x7f723406f6e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 --2- 172.21.15.43:0/3466296143 >> v2:172.21.15.43:3300/0 conn(0x7f723406d570 0x7f723406f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f7238e00640 1 -- 172.21.15.43:0/3466296143 >> 172.21.15.43:0/3466296143 conn(0x7f72340b23d0 msgr2=0x7f72340d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6346800640 1 -- start start 2024-09-17T05:23:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7680b30d0 con 0x7ff7680b3bf0 2024-09-17T05:23:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe150c00640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fe12c07b3f0 0x7fe12c07d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6d4011470 con 0x7fe6dc0b5940 2024-09-17T05:23:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7fe6d8c00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6bc073430 0x7fe6bc075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe151600640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe12c073200 0x7fe12c075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1400ae000 tx=0x7fe14001bbf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64680d2c80 con 0x7f64680d2ea0 2024-09-17T05:23:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.212+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d44001040 con 0x7f2d700ce010 2024-09-17T05:23:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fb384000640 1 Processor -- start 2024-09-17T05:23:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fb384000640 1 -- start start 2024-09-17T05:23:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f7238e00640 1 -- 172.21.15.43:0/3466296143 shutdown_connections 2024-09-17T05:23:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f7238e00640 1 -- 172.21.15.43:0/3466296143 wait complete. 2024-09-17T05:23:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== osd.2 v2:172.21.15.43:6808/3128122281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe12c07df10 con 0x7fe12c07b3f0 2024-09-17T05:23:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7540bc030 con 0x7ff7680b3bf0 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400cc6d0 0x7f634015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634015adc0 0x7f634015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 0x7f63400c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6340170d80 con 0x7f634015adc0 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6340170c00 con 0x7f63400cc6d0 2024-09-17T05:23:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d4092c30 con 0x7fe6dc0b5940 2024-09-17T05:23:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6346800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6340170f00 con 0x7f634015bd90 2024-09-17T05:23:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400cc6d0 0x7f634015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 0x7f63400c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 0x7f63400c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47870/0 (socket says 172.21.15.43:47870) 2024-09-17T05:23:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 -- 172.21.15.43:0/3520807827 learned_addr learned my addr 172.21.15.43:0/3520807827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633ea00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634015adc0 0x7f634015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2d6000fa40 con 0x7f2d700ce010 2024-09-17T05:23:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff748073300 0x7ff748075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 -- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400cc6d0 msgr2=0x7f634015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff754131840 con 0x7ff7680b3bf0 2024-09-17T05:23:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe6d8c00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7fe6bc07c2f0 0x7fe6bc07e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400cc6d0 0x7f634015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6470094430 con 0x7f64680d2ea0 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6463400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6450073230 0x7f6450075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f647010fef0 con 0x7f64680d2ea0 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6463400640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f645007b420 0x7f645007d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6470072e50 con 0x7f645007b420 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7ff74807b5d0 0x7ff74807da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7540b9ec0 con 0x7ff74807b5d0 2024-09-17T05:23:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff754131c30 con 0x7ff7680b3bf0 2024-09-17T05:23:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 -- 172.21.15.43:0/3520807827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634015adc0 msgr2=0x7f634015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634015adc0 0x7f634015b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633fe00640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634015c1a0 con 0x7f634015bd90 2024-09-17T05:23:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7f633f400640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400cc6d0 0x7f634015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6bc07eeb0 con 0x7fe6bc07c2f0 2024-09-17T05:23:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.216+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe6d405b840 con 0x7fe6dc0b5940 2024-09-17T05:23:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fe6e0a00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6bc073430 0x7fe6bc075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7ff767e00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff748073300 0x7ff748075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fe6dbe00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7fe6bc07c2f0 0x7fe6bc07e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7ff76ce00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7ff74807b5d0 0x7ff74807da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d67400640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2d500732d0 0x7f2d500757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fe6e0a00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6bc073430 0x7fe6bc075940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8007fd0 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f633fe00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 0x7f63400c95a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f633800eca0 tx=0x7f633800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f7238e00640 1 Processor -- start 2024-09-17T05:23:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2d60094eb0 con 0x7f2d700ce010 2024-09-17T05:23:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f7238e00640 1 -- start start 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d67400640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f2d5007b540 0x7f2d5007d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d60018e50 con 0x7f2d5007b540 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2d600952a0 con 0x7f2d700ce010 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f64700d6e50 con 0x7f64680d2ea0 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6476200640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6450073230 0x7f6450075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f646ea00640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f645007b420 0x7f645007d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6476200640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6450073230 0x7f6450075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f645c0016a0 tx=0x7f645c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d6f400640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2d500732d0 0x7f2d500757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fe6dbe00640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7fe6bc07c2f0 0x7fe6bc07e7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7ff767e00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff748073300 0x7ff748075810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7580014e0 tx=0x7ff7580078b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d6fe00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f2d5007b540 0x7f2d5007d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 0x7fb37c09db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb37c0cc6d0 0x7fb37c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb37c0c8140 0x7fb37c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1602e0 con 0x7fb37c0711a0 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c160160 con 0x7fb37c0c8140 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb384000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c160460 con 0x7fb37c0cc6d0 2024-09-17T05:23:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338003b80 con 0x7f634015bd90 2024-09-17T05:23:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f646ea00640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f645007b420 0x7f645007d870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6338003d20 con 0x7f634015bd90 2024-09-17T05:23:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004280 con 0x7f634015bd90 2024-09-17T05:23:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 0x7fb37c09db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 0x7fb37c09db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41500/0 (socket says 172.21.15.43:41500) 2024-09-17T05:23:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 -- 172.21.15.43:0/3609121650 learned_addr learned my addr 172.21.15.43:0/3609121650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63400ad480 con 0x7f634015bd90 2024-09-17T05:23:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb380c00640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb37c0cc6d0 0x7fb37c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6340158560 con 0x7f634015bd90 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb382000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb37c0c8140 0x7fb37c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 -- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb37c0cc6d0 msgr2=0x7fb37c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb37c0cc6d0 0x7fb37c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 -- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb37c0c8140 msgr2=0x7fb37c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb37c0c8140 0x7fb37c157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb381600640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c15d6a0 con 0x7fb37c0711a0 2024-09-17T05:23:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7ff76ce00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7ff74807b5d0 0x7ff74807da20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d6fe00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f2d5007b540 0x7f2d5007d990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7f2d6f400640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2d500732d0 0x7f2d500757e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0ae000 tx=0x7f2d5c01bd60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fb382000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb37c0c8140 0x7fb37c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.220+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6bc07eeb0 con 0x7fe6bc07c2f0 2024-09-17T05:23:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6318001040 con 0x7f634015bd90 2024-09-17T05:23:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb381600640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 0x7fb37c09db60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3740058d0 tx=0x7fb37400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 0x7f72340bba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7540b9ec0 con 0x7ff74807b5d0 2024-09-17T05:23:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6470072e50 con 0x7f645007b420 2024-09-17T05:23:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== osd.4 v2:172.21.15.62:6818/1078882315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d60018e50 con 0x7f2d5007b540 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f723415adc0 0x7f72341f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72340ca220 0x7f72341f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7234168570 con 0x7f72340cc6d0 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72341683f0 con 0x7f72340ca220 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7238e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f72341686f0 con 0x7f723415adc0 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 0x7f72340bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f723415adc0 0x7f72341f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374017070 con 0x7fb37c0711a0 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb374009d20 con 0x7fb37c0711a0 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3740106c0 con 0x7fb37c0711a0 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c15d980 con 0x7fb37c0711a0 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 0x7f72340bba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41514/0 (socket says 172.21.15.43:41514) 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 -- 172.21.15.43:0/1906218385 learned_addr learned my addr 172.21.15.43:0/1906218385 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7232800640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72340ca220 0x7f72341f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c15df40 con 0x7fb37c0711a0 2024-09-17T05:23:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 -- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f723415adc0 msgr2=0x7f72341f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f723415adc0 0x7f72341f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb37c0ce100 con 0x7fb37c0711a0 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 -- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72340ca220 msgr2=0x7f72341f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72340ca220 0x7f72341f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231e00640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723415dbf0 con 0x7f72340cc6d0 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f7231400640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f723415adc0 0x7f72341f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f7232800640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72340ca220 0x7f72341f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f7231e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 0x7f72340bba20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cbf0 tx=0x7f722c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c016020 con 0x7f72340cc6d0 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72340bbf60 con 0x7f72340cc6d0 2024-09-17T05:23:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723415e180 con 0x7f72340cc6d0 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f722c00ab40 con 0x7f72340cc6d0 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c012420 con 0x7f72340cc6d0 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb37401a070 con 0x7fb37c0711a0 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.224+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6338004b50 con 0x7f634015bd90 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f6337400640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6320073230 0x7f6320075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6338095190 con 0x7f634015bd90 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f6337400640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f632007b500 0x7f632007d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f632007e020 con 0x7f632007b500 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.228+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6338095580 con 0x7f634015bd90 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7f633f400640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6320073230 0x7f6320075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb360073280 0x7fb360075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb37405aef0 con 0x7fb37c0711a0 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f720c001040 con 0x7f72340cc6d0 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb380c00640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb360073280 0x7fb360075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7fb36007c100 0x7fb36007e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb36007ecc0 con 0x7fb36007c100 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb37405b470 con 0x7fb37c0711a0 2024-09-17T05:23:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb382000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7fb36007c100 0x7fb36007e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb380c00640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb360073280 0x7fb360075790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb37c072b60 tx=0x7fb37001b5c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb382000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7fb36007c100 0x7fb36007e5f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.232+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb36007ecc0 con 0x7fb36007c100 2024-09-17T05:23:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f633ea00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f632007b500 0x7f632007d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f722c019020 con 0x7f72340cc6d0 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f722a000640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7214073230 0x7f7214075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f722c094460 con 0x7f72340cc6d0 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f722a000640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f721407b480 0x7f721407d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f722c013e50 con 0x7f721407b480 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f7231400640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7214073230 0x7f7214075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f7232800640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f721407b480 0x7f721407d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.236+0000 7f633f400640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6320073230 0x7f6320075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6330007f30 tx=0x7f6330007ec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe14c099390 con 0x7fe12c07b3f0 2024-09-17T05:23:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f722c09b050 con 0x7f72340cc6d0 2024-09-17T05:23:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f633ea00640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f632007b500 0x7f632007d950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe149600640 1 -- 172.21.15.43:0/1807632954 <== osd.2 v2:172.21.15.43:6808/3128122281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe14c099390 con 0x7fe12c07b3f0 2024-09-17T05:23:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f7232800640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f721407b480 0x7f721407d8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fe12c07b3f0 msgr2=0x7fe12c07d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fe12c07b3f0 0x7fe12c07d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f7231400640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7214073230 0x7f7214075740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7220005fd0 tx=0x7f7220005d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe12c073200 msgr2=0x7fe12c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe12c073200 0x7fe12c075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1400ae000 tx=0x7fe14001bbf0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 msgr2=0x7fe14c154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 --2- 172.21.15.43:0/1807632954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe14c0cc840 0x7fe14c154ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe14400be00 tx=0x7fe14400bef0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe152000640 1 -- 172.21.15.43:0/1807632954 reap_dead start 2024-09-17T05:23:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 shutdown_connections 2024-09-17T05:23:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 >> 172.21.15.43:0/1807632954 conn(0x7fe14c0b1dd0 msgr2=0x7fe14c070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f632007e020 con 0x7f632007b500 2024-09-17T05:23:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.240+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f722c013e50 con 0x7f721407b480 2024-09-17T05:23:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.244+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 shutdown_connections 2024-09-17T05:23:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.244+0000 7fe154000640 1 -- 172.21.15.43:0/1807632954 wait complete. 2024-09-17T05:23:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d44002de0 con 0x7f2d5007b540 2024-09-17T05:23:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d67400640 1 -- 172.21.15.43:0/2102713725 <== osd.4 v2:172.21.15.62:6818/1078882315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d44002de0 con 0x7f2d5007b540 2024-09-17T05:23:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f2d5007b540 msgr2=0x7f2d5007d990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f2d5007b540 0x7f2d5007d990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6468055280 con 0x7f645007b420 2024-09-17T05:23:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f6463400640 1 -- 172.21.15.43:0/911674634 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6468055280 con 0x7f645007b420 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2d500732d0 msgr2=0x7f2d500757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2d500732d0 0x7f2d500757e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0ae000 tx=0x7f2d5c01bd60 comp rx=0 tx=0).stop 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 msgr2=0x7f2d7014bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 --2- 172.21.15.43:0/2102713725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2d700ce010 0x7f2d7014bfb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ca20 tx=0x7f2d6000cee0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d6fe00640 1 -- 172.21.15.43:0/2102713725 reap_dead start 2024-09-17T05:23:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 shutdown_connections 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f645007b420 msgr2=0x7f645007d870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f6460c00640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f645007b420 0x7f645007d870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6450073230 msgr2=0x7f6450075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6450073230 0x7f6450075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f645c0016a0 tx=0x7f645c009040 comp rx=0 tx=0).stop 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680d2ea0 msgr2=0x7f6468053200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 --2- 172.21.15.43:0/911674634 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64680d2ea0 0x7f6468053200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f647002ebe0 tx=0x7f647006a190 comp rx=0 tx=0).stop 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f646f400640 1 -- 172.21.15.43:0/911674634 reap_dead start 2024-09-17T05:23:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 >> 172.21.15.43:0/2102713725 conn(0x7f2d700d0440 msgr2=0x7f2d700700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.252+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 shutdown_connections 2024-09-17T05:23:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f2d64c00640 1 -- 172.21.15.43:0/2102713725 wait complete. 2024-09-17T05:23:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe6dc0b3860 con 0x7fe6bc07c2f0 2024-09-17T05:23:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 shutdown_connections 2024-09-17T05:23:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 >> 172.21.15.43:0/911674634 conn(0x7f646801db80 msgr2=0x7f6468033060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 shutdown_connections 2024-09-17T05:23:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7f6460c00640 1 -- 172.21.15.43:0/911674634 wait complete. 2024-09-17T05:23:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6d8c00640 1 -- 172.21.15.43:0/1549122154 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe6dc0b3860 con 0x7fe6bc07c2f0 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7fe6bc07c2f0 msgr2=0x7fe6bc07e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7fe6bc07c2f0 0x7fe6bc07e7e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6bc073430 msgr2=0x7fe6bc075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6bc073430 0x7fe6bc075940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8007fd0 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 msgr2=0x7fe6dc15e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 --2- 172.21.15.43:0/1549122154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6dc0b5940 0x7fe6dc15e1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400e970 tx=0x7fe6d400ee30 comp rx=0 tx=0).stop 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e1400640 1 -- 172.21.15.43:0/1549122154 reap_dead start 2024-09-17T05:23:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 shutdown_connections 2024-09-17T05:23:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 >> 172.21.15.43:0/1549122154 conn(0x7fe6dc0cee60 msgr2=0x7fe6dc0b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 shutdown_connections 2024-09-17T05:23:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.256+0000 7fe6e3400640 1 -- 172.21.15.43:0/1549122154 wait complete. 2024-09-17T05:23:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.260+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb37c06d070 con 0x7fb36007c100 2024-09-17T05:23:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb379600640 1 -- 172.21.15.43:0/3609121650 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb37c06d070 con 0x7fb36007c100 2024-09-17T05:23:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7680aff40 con 0x7ff74807b5d0 2024-09-17T05:23:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7fb36007c100 msgr2=0x7fb36007e5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7fb36007c100 0x7fb36007e5f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb360073280 msgr2=0x7fb360075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb360073280 0x7fb360075790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb37c072b60 tx=0x7fb37001b5c0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 msgr2=0x7fb37c09db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 --2- 172.21.15.43:0/3609121650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0711a0 0x7fb37c09db60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3740058d0 tx=0x7fb37400dde0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb382000640 1 -- 172.21.15.43:0/3609121650 reap_dead start 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 shutdown_connections 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 >> 172.21.15.43:0/3609121650 conn(0x7fb37c0b23d0 msgr2=0x7fb37c0caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 shutdown_connections 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7fb384000640 1 -- 172.21.15.43:0/3609121650 wait complete. 2024-09-17T05:23:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.264+0000 7ff75fe00640 1 -- 172.21.15.43:0/494995712 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff7680aff40 con 0x7ff74807b5d0 2024-09-17T05:23:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7ff74807b5d0 msgr2=0x7ff74807da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7ff74807b5d0 0x7ff74807da20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff748073300 msgr2=0x7ff748075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff748073300 0x7ff748075810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7580014e0 tx=0x7ff7580078b0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7680b3bf0 msgr2=0x7ff7680c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 --2- 172.21.15.43:0/494995712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7680b3bf0 0x7ff7680c7900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7680ce5a0 tx=0x7ff754005780 comp rx=0 tx=0).stop 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ce00640 1 -- 172.21.15.43:0/494995712 reap_dead start 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 shutdown_connections 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 >> 172.21.15.43:0/494995712 conn(0x7ff7680b1dd0 msgr2=0x7ff768156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6318002de0 con 0x7f632007b500 2024-09-17T05:23:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6337400640 1 -- 172.21.15.43:0/3520807827 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6318002de0 con 0x7f632007b500 2024-09-17T05:23:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f632007b500 msgr2=0x7f632007d950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f632007b500 0x7f632007d950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 shutdown_connections 2024-09-17T05:23:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6320073230 msgr2=0x7f6320075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6320073230 0x7f6320075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6330007f30 tx=0x7f6330007ec0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7ff76ee00640 1 -- 172.21.15.43:0/494995712 wait complete. 2024-09-17T05:23:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 msgr2=0x7f63400c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 --2- 172.21.15.43:0/3520807827 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f634015bd90 0x7f63400c95a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f633800eca0 tx=0x7f633800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f633fe00640 1 -- 172.21.15.43:0/3520807827 reap_dead start 2024-09-17T05:23:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 shutdown_connections 2024-09-17T05:23:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.268+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 >> 172.21.15.43:0/3520807827 conn(0x7f63400b23d0 msgr2=0x7f6340157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.278 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:23:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 shutdown_connections 2024-09-17T05:23:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f6346800640 1 -- 172.21.15.43:0/3520807827 wait complete. 2024-09-17T05:23:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f720c002de0 con 0x7f721407b480 2024-09-17T05:23:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f722a000640 1 -- 172.21.15.43:0/1906218385 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f720c002de0 con 0x7f721407b480 2024-09-17T05:23:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f721407b480 msgr2=0x7f721407d8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.276+0000 7f7238e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f721407b480 0x7f721407d8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7214073230 msgr2=0x7f7214075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7214073230 0x7f7214075740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7220005fd0 tx=0x7f7220005d00 comp rx=0 tx=0).stop 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 msgr2=0x7f72340bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 --2- 172.21.15.43:0/1906218385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72340cc6d0 0x7f72340bba20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cbf0 tx=0x7f722c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7232800640 1 -- 172.21.15.43:0/1906218385 reap_dead start 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 shutdown_connections 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 >> 172.21.15.43:0/1906218385 conn(0x7f72340b23d0 msgr2=0x7f72340cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 shutdown_connections 2024-09-17T05:23:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.280+0000 7f7238e00640 1 -- 172.21.15.43:0/1906218385 wait complete. 2024-09-17T05:23:36.299 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:23:36.300 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:23:36.302 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T05:23:36.306 INFO:teuthology.orchestra.run.smithi043.stdout:64424509448 2024-09-17T05:23:36.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:23:36.316 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.317 INFO:teuthology.orchestra.run.smithi043.stdout:64424509448 2024-09-17T05:23:36.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:23:36.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:23:36.320 INFO:teuthology.orchestra.run.smithi043.stdout:60129542151 2024-09-17T05:23:36.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:23:36.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f37e00640 1 -- 172.21.15.43:0/4194595044 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c003860 con 0x7f1f30052cd0 2024-09-17T05:23:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 -- 172.21.15.43:0/4194595044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30052cd0 msgr2=0x7f1f30034640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/4194595044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30052cd0 0x7f1f30034640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009840 tx=0x7f1f2c01c5d0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f9626000640 1 -- 172.21.15.43:0/3948482669 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200b2070 con 0x7f962815e800 2024-09-17T05:23:36.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 -- 172.21.15.43:0/4194595044 shutdown_connections 2024-09-17T05:23:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/4194595044 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30052cd0 0x7f1f30034640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 -- 172.21.15.43:0/4194595044 >> 172.21.15.43:0/4194595044 conn(0x7f1f3001eb20 msgr2=0x7f1f3003a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 -- 172.21.15.43:0/3948482669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f962815e800 msgr2=0x7f962815ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 -- 172.21.15.43:0/4194595044 shutdown_connections 2024-09-17T05:23:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 --2- 172.21.15.43:0/3948482669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f962815e800 0x7f962815ec10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9620009bc0 tx=0x7f962001cad0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 -- 172.21.15.43:0/3948482669 shutdown_connections 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 --2- 172.21.15.43:0/3948482669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f962815e800 0x7f962815ec10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 --2- 172.21.15.43:0/3948482669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96280ce3e0 0x7f96280b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 -- 172.21.15.43:0/3948482669 >> 172.21.15.43:0/3948482669 conn(0x7f96280b1dd0 msgr2=0x7f96280b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 -- 172.21.15.43:0/3948482669 shutdown_connections 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f962e800640 1 -- 172.21.15.43:0/3948482669 wait complete. 2024-09-17T05:23:36.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.560+0000 7f1f3ee00640 1 -- 172.21.15.43:0/4194595044 wait complete. 2024-09-17T05:23:36.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.564+0000 7f962e800640 1 Processor -- start 2024-09-17T05:23:36.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.564+0000 7f962e800640 1 -- start start 2024-09-17T05:23:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f1f3ee00640 1 Processor -- start 2024-09-17T05:23:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f1f3ee00640 1 -- start start 2024-09-17T05:23:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f23d9e00640 1 -- 172.21.15.43:0/901456609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b3bf0 msgr2=0x7f23d40b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96280b5b20 0x7f96280c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815e800 0x7f962815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9628170910 con 0x7f96280b5b20 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9628170790 con 0x7f96280ce3e0 2024-09-17T05:23:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f962e800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9628170a90 con 0x7f962815e800 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9627e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815e800 0x7f962815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42526/0 (socket says 172.21.15.43:42526) 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 -- 172.21.15.43:0/581582987 learned_addr learned my addr 172.21.15.43:0/581582987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9627400640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96280b5b20 0x7f96280c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 -- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815e800 msgr2=0x7f962815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815e800 0x7f962815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 -- 172.21.15.43:0/581582987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96280b5b20 msgr2=0x7f96280c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96280b5b20 0x7f96280c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f9626a00640 1 -- 172.21.15.43:0/581582987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962815ba80 con 0x7f96280ce3e0 2024-09-17T05:23:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.568+0000 7f23d9e00640 1 --2- 172.21.15.43:0/901456609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b3bf0 0x7f23d40b49a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009bc0 tx=0x7f23cc01cba0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f23d9e00640 1 -- 172.21.15.43:0/901456609 shutdown_connections 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f23d9e00640 1 --2- 172.21.15.43:0/901456609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b3bf0 0x7f23d40b49a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f23d9e00640 1 -- 172.21.15.43:0/901456609 >> 172.21.15.43:0/901456609 conn(0x7f23d40b1dd0 msgr2=0x7f23d40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f23d9e00640 1 -- 172.21.15.43:0/901456609 shutdown_connections 2024-09-17T05:23:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f23d9e00640 1 -- 172.21.15.43:0/901456609 wait complete. 2024-09-17T05:23:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42536/0 (socket says 172.21.15.43:42536) 2024-09-17T05:23:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47918/0 (socket says 172.21.15.43:47918) 2024-09-17T05:23:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41562/0 (socket says 172.21.15.43:41562) 2024-09-17T05:23:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3d000640 1 -- 172.21.15.43:0/3028107265 learned_addr learned my addr 172.21.15.43:0/3028107265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300f1fc0 con 0x7f1f300e7460 2024-09-17T05:23:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300f1e40 con 0x7f1f30052cd0 2024-09-17T05:23:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1f300f2140 con 0x7f1f30034b80 2024-09-17T05:23:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f9626a00640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f961800d6e0 tx=0x7f961800dba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618017070 con 0x7f96280ce3e0 2024-09-17T05:23:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9618005310 con 0x7f96280ce3e0 2024-09-17T05:23:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962815ff40 con 0x7f96280ce3e0 2024-09-17T05:23:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962815bd20 con 0x7f96280ce3e0 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 -- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 msgr2=0x7f1f300de1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 -- 172.21.15.43:0/3028107265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 msgr2=0x7f1f30007e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3da00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f30006aa0 con 0x7f1f30034b80 2024-09-17T05:23:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3e400640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f1f3d000640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618009820 con 0x7f96280ce3e0 2024-09-17T05:23:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f0bdf200640 1 -- 172.21.15.43:0/2685542852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc4003a10 msgr2=0x7f0bc4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.572+0000 7f0bdf200640 1 --2- 172.21.15.43:0/2685542852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc4003a10 0x7f0bc4003e60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0004580 tx=0x7f0bd001cb00 comp rx=0 tx=0).stop 2024-09-17T05:23:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f3da00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f30055700 tx=0x7f1f2c003da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f0bdf200640 1 -- 172.21.15.43:0/2685542852 shutdown_connections 2024-09-17T05:23:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f0bdf200640 1 --2- 172.21.15.43:0/2685542852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0bc4003a10 0x7f0bc4003e60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f0bdf200640 1 -- 172.21.15.43:0/2685542852 >> 172.21.15.43:0/2685542852 conn(0x7f0be006ce60 msgr2=0x7f0be006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0b0030 con 0x7f1f30034b80 2024-09-17T05:23:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f2c0044d0 con 0x7f1f30034b80 2024-09-17T05:23:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c003fd0 con 0x7f1f30034b80 2024-09-17T05:23:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 Processor -- start 2024-09-17T05:23:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 -- start start 2024-09-17T05:23:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f30006d80 con 0x7f1f30034b80 2024-09-17T05:23:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f30007340 con 0x7f1f30034b80 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 0x7f23d40ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d40cf390 0x7f23d40ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40b01b0 con 0x7f23d40b37e0 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40b0030 con 0x7f23d40cf390 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f23d9e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40b0330 con 0x7f23d40b3bf0 2024-09-17T05:23:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.576+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96000052a0 con 0x7f96280ce3e0 2024-09-17T05:23:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41580/0 (socket says 172.21.15.43:41580) 2024-09-17T05:23:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 -- 172.21.15.43:0/780465123 learned_addr learned my addr 172.21.15.43:0/780465123 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f0bdf200640 1 -- 172.21.15.43:0/2685542852 shutdown_connections 2024-09-17T05:23:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d3400640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d40cf390 0x7f23d40ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96180099c0 con 0x7f96280ce3e0 2024-09-17T05:23:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f0bdf200640 1 -- 172.21.15.43:0/2685542852 wait complete. 2024-09-17T05:23:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 -- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 msgr2=0x7f23d40ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2000640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 0x7f23d40ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 0x7f23d40ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 -- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d40cf390 msgr2=0x7f23d40ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d40cf390 0x7f23d40ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 -- 172.21.15.43:0/780465123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40aed80 con 0x7f23d40b37e0 2024-09-17T05:23:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2000640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 0x7f23d40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f2c0bf030 con 0x7f1f30034b80 2024-09-17T05:23:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f961f400640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 0x7f96040757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f140052a0 con 0x7f1f30034b80 2024-09-17T05:23:36.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9618094060 con 0x7f96280ce3e0 2024-09-17T05:23:36.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f1f35600640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 0x7f1f18075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f0bdf200640 1 Processor -- start 2024-09-17T05:23:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2c12dd40 con 0x7f1f30034b80 2024-09-17T05:23:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 -- start start 2024-09-17T05:23:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f23d2a00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c460 tx=0x7f23c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.580+0000 7f1f3d000640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 0x7f1f18075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f9627400640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 0x7f96040757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4016020 con 0x7f23d40b37e0 2024-09-17T05:23:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c4012960 con 0x7f23d40b37e0 2024-09-17T05:23:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4012db0 con 0x7f23d40b37e0 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 -- 172.21.15.43:0/761335407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 msgr2=0x7fdc1006e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 --2- 172.21.15.43:0/761335407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 0x7fdc1006e5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009840 tx=0x7fdc0801c840 comp rx=0 tx=0).stop 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 -- 172.21.15.43:0/761335407 shutdown_connections 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 --2- 172.21.15.43:0/761335407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 0x7fdc1006e5a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 --2- 172.21.15.43:0/761335407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc100b6590 0x7fdc100b9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 0x7f0be0150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 0x7f0be01fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 -- 172.21.15.43:0/761335407 >> 172.21.15.43:0/761335407 conn(0x7fdc100d4400 msgr2=0x7fdc100d4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 0x7f0be015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f1f3d000640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 0x7f1f18075760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f24006fd0 tx=0x7f1f24008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d40c9c80 con 0x7f23d40b37e0 2024-09-17T05:23:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 0x7f0be0150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 0x7f0be01fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 -- 172.21.15.43:0/761335407 shutdown_connections 2024-09-17T05:23:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f9627400640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 0x7f96040757e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9610001660 tx=0x7f9610009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 0x7f0be01fdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47930/0 (socket says 172.21.15.43:47930) 2024-09-17T05:23:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 0x7f0be0150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42554/0 (socket says 172.21.15.43:42554) 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0be00bc630 con 0x7f0be015c370 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0be015a080 con 0x7f0bc4003a10 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0be00bc7b0 con 0x7f0be0151380 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bd5400640 1 -- 172.21.15.43:0/314613990 learned_addr learned my addr 172.21.15.43:0/314613990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d40af2d0 con 0x7f23d40b37e0 2024-09-17T05:23:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7f0bde800640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 0x7f0be015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.584+0000 7fdc0f200640 1 -- 172.21.15.43:0/761335407 wait complete. 2024-09-17T05:23:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d40b30d0 con 0x7f23d40b37e0 2024-09-17T05:23:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f0bde800640 1 -- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 msgr2=0x7f0be01fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f0bde800640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 0x7f0be01fdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f0bde800640 1 -- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 msgr2=0x7f0be0150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f0bde800640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 0x7f0be0150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f0bde800640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be015ccc0 con 0x7f0be015c370 2024-09-17T05:23:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f962815ba80 con 0x7f96280ce3e0 2024-09-17T05:23:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7fdc0f200640 1 Processor -- start 2024-09-17T05:23:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7fdc0f200640 1 -- start start 2024-09-17T05:23:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f2c0c6020 con 0x7f1f30034b80 2024-09-17T05:23:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7fdc0f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 0x7fdc10158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7f0bde800640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 0x7f0be015c780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0be0071d30 tx=0x7f0bd800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.588+0000 7fdc0f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc100b33a0 0x7fdc1015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1016c060 con 0x7fdc1006e190 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1016bee0 con 0x7fdc100b33a0 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1016c1e0 con 0x7fdc101fc7c0 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47940/0 (socket says 172.21.15.43:47940) 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 -- 172.21.15.43:0/4136584462 learned_addr learned my addr 172.21.15.43:0/4136584462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 -- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc100b33a0 msgr2=0x7fdc1015a9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:23:36.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc100b33a0 0x7fdc1015a9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 -- 172.21.15.43:0/4136584462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 msgr2=0x7fdc10158b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 0x7fdc10158b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 -- 172.21.15.43:0/4136584462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1015bb70 con 0x7fdc101fc7c0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0e800640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc0801a470 tx=0x7fdc080a55d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080b2040 con 0x7fdc101fc7c0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc080043b0 con 0x7fdc101fc7c0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1015be50 con 0x7fdc101fc7c0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc101fd110 con 0x7fdc101fc7c0 2024-09-17T05:23:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08005280 con 0x7fdc101fc7c0 2024-09-17T05:23:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23c4019020 con 0x7f23d40b37e0 2024-09-17T05:23:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7f23cb400640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 0x7f23b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f23c4094410 con 0x7f23d40b37e0 2024-09-17T05:23:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.592+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe80052a0 con 0x7fdc101fc7c0 2024-09-17T05:23:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7f23d2000640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 0x7f23b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc080be060 con 0x7fdc101fc7c0 2024-09-17T05:23:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7fdc06000640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 0x7fdbec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdc0812d9d0 con 0x7fdc101fc7c0 2024-09-17T05:23:36.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7fdc0de00640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 0x7fdbec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.596+0000 7fdc0de00640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 0x7fdbec075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001280 tx=0x7fdbf8001210 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f23d2000640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 0x7f23b4075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23d40729e0 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8010890 con 0x7f0be015c370 2024-09-17T05:23:36.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd8016070 con 0x7f0be015c370 2024-09-17T05:23:36.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8010dc0 con 0x7f0be015c370 2024-09-17T05:23:36.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01510f0 con 0x7f0be015c370 2024-09-17T05:23:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23c4020030 con 0x7f23d40b37e0 2024-09-17T05:23:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be00b97d0 con 0x7f0be015c370 2024-09-17T05:23:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.600+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc08016430 con 0x7fdc101fc7c0 2024-09-17T05:23:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.604+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0bd801b460 con 0x7f0be015c370 2024-09-17T05:23:36.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.608+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb80052a0 con 0x7f0be015c370 2024-09-17T05:23:36.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.608+0000 7f0bd6a00640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 0x7f0bbc075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.608+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd8094300 con 0x7f0be015c370 2024-09-17T05:23:36.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.612+0000 7f0bdde00640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 0x7f0bbc075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.616+0000 7f0bdde00640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 0x7f0bbc075760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00ab000 tx=0x7f0bd00198a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.620+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0be015ccc0 con 0x7f0be015c370 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.656+0000 7fdc56000640 1 -- 172.21.15.43:0/964239115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5015e280 msgr2=0x7fdc5015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.656+0000 7fdc56000640 1 --2- 172.21.15.43:0/964239115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5015e280 0x7fdc5015e690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0085a0 tx=0x7fdc3c01c4a0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.656+0000 7fdc56000640 1 -- 172.21.15.43:0/964239115 shutdown_connections 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7fdc56000640 1 --2- 172.21.15.43:0/964239115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc5015e280 0x7fdc5015e690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7fdc56000640 1 --2- 172.21.15.43:0/964239115 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7fdc56000640 1 -- 172.21.15.43:0/964239115 >> 172.21.15.43:0/964239115 conn(0x7fdc500b1c60 msgr2=0x7fdc500b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 -- 172.21.15.43:0/3251334512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 msgr2=0x7f420c1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 --2- 172.21.15.43:0/3251334512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 0x7f420c1586f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f420c15fd40 tx=0x7f420401cb50 comp rx=0 tx=0).stop 2024-09-17T05:23:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 -- 172.21.15.43:0/3251334512 shutdown_connections 2024-09-17T05:23:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 --2- 172.21.15.43:0/3251334512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 0x7f420c1586f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7fdc56000640 1 -- 172.21.15.43:0/964239115 shutdown_connections 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7fdc56000640 1 -- 172.21.15.43:0/964239115 wait complete. 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 --2- 172.21.15.43:0/3251334512 >> v2:172.21.15.62:3300/0 conn(0x7f420c072ed0 0x7f420c0732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 -- 172.21.15.43:0/3251334512 >> 172.21.15.43:0/3251334512 conn(0x7f420c0b1c60 msgr2=0x7f420c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 -- 172.21.15.43:0/3251334512 shutdown_connections 2024-09-17T05:23:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.660+0000 7f4211000640 1 -- 172.21.15.43:0/3251334512 wait complete. 2024-09-17T05:23:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.664+0000 7f4211000640 1 Processor -- start 2024-09-17T05:23:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.664+0000 7f4211000640 1 -- start start 2024-09-17T05:23:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.664+0000 7fdc56000640 1 Processor -- start 2024-09-17T05:23:36.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.664+0000 7fdc56000640 1 -- start start 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f420c0af5f0 0x7f420c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 0x7f420c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f420c070030 con 0x7f420c1582e0 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f420c06feb0 con 0x7f420c0af5f0 2024-09-17T05:23:36.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4211000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0701b0 con 0x7f420c072ed0 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc500cef70 0x7fdc501552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50152660 0x7fdc501541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500700b0 con 0x7fdc500cef70 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5006ff30 con 0x7fdc50152660 2024-09-17T05:23:36.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc56000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdc50070230 con 0x7fdc500b3790 2024-09-17T05:23:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f4203e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f420c0af5f0 0x7f420c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47954/0 (socket says 172.21.15.43:47954) 2024-09-17T05:23:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 -- 172.21.15.43:0/3132638664 learned_addr learned my addr 172.21.15.43:0/3132638664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:47956/0 (socket says 172.21.15.43:47956) 2024-09-17T05:23:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 -- 172.21.15.43:0/940104929 learned_addr learned my addr 172.21.15.43:0/940104929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc4fe00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc500cef70 0x7fdc501552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 -- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f420c0af5f0 msgr2=0x7f420c154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f420c0af5f0 0x7f420c154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 -- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50152660 msgr2=0x7fdc501541a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 -- 172.21.15.43:0/3132638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 msgr2=0x7f420c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 0x7f420c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7f420b400640 1 -- 172.21.15.43:0/3132638664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c1580b0 con 0x7f420c072ed0 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f420b400640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c490 tx=0x7f41fc00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50152660 0x7fdc501541a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 -- 172.21.15.43:0/940104929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc500cef70 msgr2=0x7fdc501552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc017070 con 0x7f420c072ed0 2024-09-17T05:23:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41fc003600 con 0x7f420c072ed0 2024-09-17T05:23:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc017070 con 0x7f420c072ed0 2024-09-17T05:23:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc500cef70 0x7fdc501552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.668+0000 7fdc54c00640 1 -- 172.21.15.43:0/940104929 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc50159b00 con 0x7fdc500b3790 2024-09-17T05:23:36.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc54c00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097d0 tx=0x7fdc40009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c150f90 con 0x7f420c072ed0 2024-09-17T05:23:36.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c1578b0 con 0x7f420c072ed0 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40004020 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc400041c0 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000e8b0 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5015d900 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5015a0c0 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5015e690 con 0x7fdc500b3790 2024-09-17T05:23:36.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.672+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c1573c0 con 0x7f420c072ed0 2024-09-17T05:23:36.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.676+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f41fc01a070 con 0x7f420c072ed0 2024-09-17T05:23:36.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.676+0000 7f4203400640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 0x7f41e8075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7f4203e00640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 0x7f41e8075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f41fc0930b0 con 0x7f420c072ed0 2024-09-17T05:23:36.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.676+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc4001b020 con 0x7fdc500b3790 2024-09-17T05:23:36.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7fdc4cc00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 0x7fdc30075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdc40093e40 con 0x7fdc500b3790 2024-09-17T05:23:36.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7fdc4fe00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 0x7fdc30075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7f4203e00640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 0x7f41e8075940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f420c09bf10 tx=0x7f41f40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.680+0000 7fdc4fe00640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 0x7fdc30075760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0ae000 tx=0x7fdc3c019260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.684+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41fc05da70 con 0x7f420c072ed0 2024-09-17T05:23:36.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.688+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc4005efd0 con 0x7fdc500b3790 2024-09-17T05:23:36.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.708+0000 7f10d5a00640 1 -- 172.21.15.43:0/4085961513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 msgr2=0x7f10b4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.708+0000 7f10d5a00640 1 --2- 172.21.15.43:0/4085961513 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10b4003e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006cb0 tx=0x7f10c00307b0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.708+0000 7f10cf400640 1 -- 172.21.15.43:0/4085961513 reap_dead start 2024-09-17T05:23:36.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.712+0000 7f10d5a00640 1 -- 172.21.15.43:0/4085961513 shutdown_connections 2024-09-17T05:23:36.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.712+0000 7f10d5a00640 1 -- 172.21.15.43:0/4085961513 >> 172.21.15.43:0/4085961513 conn(0x7f10c801fbf0 msgr2=0x7f10c8056950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.716+0000 7f10d5a00640 1 -- 172.21.15.43:0/4085961513 shutdown_connections 2024-09-17T05:23:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.716+0000 7f10d5a00640 1 -- 172.21.15.43:0/4085961513 wait complete. 2024-09-17T05:23:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 Processor -- start 2024-09-17T05:23:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 -- start start 2024-09-17T05:23:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10c80e3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c80cac90 0x7f10c80344c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8051120 con 0x7f10c80cac90 2024-09-17T05:23:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8050fa0 con 0x7f10c80d4f00 2024-09-17T05:23:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.720+0000 7f10d5a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f10c80512a0 con 0x7f10b4003a10 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42616/0 (socket says 172.21.15.43:42616) 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 -- 172.21.15.43:0/992197298 learned_addr learned my addr 172.21.15.43:0/992197298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10cf400640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c80cac90 0x7f10c80344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10cea00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10c80e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 -- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 msgr2=0x7f10c80e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10c80e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 -- 172.21.15.43:0/992197298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c80cac90 msgr2=0x7f10c80344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c80cac90 0x7f10c80344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 -- 172.21.15.43:0/992197298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c80e1f90 con 0x7f10c80d4f00 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10cea00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10c80e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10ce000640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d5f0 tx=0x7f10bc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10c80d4f00 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10bc00f8b0 con 0x7f10c80d4f00 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00fd00 con 0x7f10c80d4f00 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c80e2270 con 0x7f10c80d4f00 2024-09-17T05:23:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.724+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c80e2830 con 0x7f10c80d4f00 2024-09-17T05:23:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.728+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f10bc01a070 con 0x7f10c80d4f00 2024-09-17T05:23:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.728+0000 7f10c6a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 0x7f10b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.732+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c80397a0 con 0x7f10c80d4f00 2024-09-17T05:23:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.732+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f10bc092630 con 0x7f10c80d4f00 2024-09-17T05:23:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.732+0000 7f10cea00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 0x7f10b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.732+0000 7f10cea00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 0x7f10b0075710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10c8035ea0 tx=0x7f10c002d590 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.736+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f10bc0633f0 con 0x7f10c80d4f00 2024-09-17T05:23:36.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.960+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9600005590 con 0x7f96280ce3e0 2024-09-17T05:23:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.960+0000 7f961f400640 1 -- 172.21.15.43:0/581582987 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f961805ea10 con 0x7f96280ce3e0 2024-09-17T05:23:36.958 INFO:teuthology.orchestra.run.smithi043.stdout:64424509446 2024-09-17T05:23:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 msgr2=0x7f96040757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 0x7f96040757e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9610001660 tx=0x7f9610009040 comp rx=0 tx=0).stop 2024-09-17T05:23:36.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 msgr2=0x7f96280ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f961800d6e0 tx=0x7f961800dba0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 shutdown_connections 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96040732d0 0x7f96040757e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96280ce3e0 0x7f96280ccb90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f962815e800 0x7f962815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 --2- 172.21.15.43:0/581582987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96280b5b20 0x7f96280c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 >> 172.21.15.43:0/581582987 conn(0x7f96280b1dd0 msgr2=0x7f96280b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 shutdown_connections 2024-09-17T05:23:36.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.964+0000 7f962e800640 1 -- 172.21.15.43:0/581582987 wait complete. 2024-09-17T05:23:36.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.968+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdbe8005590 con 0x7fdc101fc7c0 2024-09-17T05:23:36.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.968+0000 7fdc06000640 1 -- 172.21.15.43:0/4136584462 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc080f89e0 con 0x7fdc101fc7c0 2024-09-17T05:23:36.966 INFO:teuthology.orchestra.run.smithi043.stdout:64424509446 2024-09-17T05:23:36.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.968+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f23d406ec90 con 0x7f23d40b37e0 2024-09-17T05:23:36.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1f14005590 con 0x7f1f30034b80 2024-09-17T05:23:36.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7f23cb400640 1 -- 172.21.15.43:0/780465123 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23c405f410 con 0x7f23d40b37e0 2024-09-17T05:23:36.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 msgr2=0x7fdbec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 0x7fdbec075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001280 tx=0x7fdbf8001210 comp rx=0 tx=0).stop 2024-09-17T05:23:36.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7f1f35600640 1 -- 172.21.15.43:0/3028107265 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f300535c0 con 0x7f1f30034b80 2024-09-17T05:23:36.971 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:36.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 msgr2=0x7fdc101fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc0801a470 tx=0x7fdc080a55d0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 shutdown_connections 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbec073200 0x7fdbec075710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc100b33a0 0x7fdc1015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc101fc7c0 0x7fdc101fcbd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 --2- 172.21.15.43:0/4136584462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1006e190 0x7fdc10158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 >> 172.21.15.43:0/4136584462 conn(0x7fdc100d4400 msgr2=0x7fdc100d4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stdout:64424509446 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 shutdown_connections 2024-09-17T05:23:36.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.972+0000 7fdc0f200640 1 -- 172.21.15.43:0/4136584462 wait complete. 2024-09-17T05:23:36.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 msgr2=0x7f1f18075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 0x7f1f18075760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f24006fd0 tx=0x7f1f24008040 comp rx=0 tx=0).stop 2024-09-17T05:23:36.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 msgr2=0x7f1f300d6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f30055700 tx=0x7f1f2c003da0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 shutdown_connections 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18073250 0x7f1f18075760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f30052cd0 0x7f1f300de1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300e7460 0x7f1f30007e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 --2- 172.21.15.43:0/3028107265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30034b80 0x7f1f300d6490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 >> 172.21.15.43:0/3028107265 conn(0x7f1f3001eb20 msgr2=0x7f1f3000c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 shutdown_connections 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 msgr2=0x7f23b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 0x7f23b4075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23d40729e0 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 msgr2=0x7f23d40990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c460 tx=0x7f23c400c920 comp rx=0 tx=0).stop 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 shutdown_connections 2024-09-17T05:23:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23b4073230 0x7f23b4075740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23d40cf390 0x7f23d40ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23d40b3bf0 0x7f23d40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 --2- 172.21.15.43:0/780465123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d40b37e0 0x7f23d40990b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 >> 172.21.15.43:0/780465123 conn(0x7f23d40b1dd0 msgr2=0x7f23d40b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f1f3ee00640 1 -- 172.21.15.43:0/3028107265 wait complete. 2024-09-17T05:23:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 shutdown_connections 2024-09-17T05:23:36.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.976+0000 7f23d9e00640 1 -- 172.21.15.43:0/780465123 wait complete. 2024-09-17T05:23:36.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.984+0000 7f0bdf200640 1 -- 172.21.15.43:0/314613990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0bb8005590 con 0x7f0be015c370 2024-09-17T05:23:36.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.984+0000 7f0bd6a00640 1 -- 172.21.15.43:0/314613990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0be015ccc0 con 0x7f0be015c370 2024-09-17T05:23:36.983 INFO:teuthology.orchestra.run.smithi043.stdout:64424509446 2024-09-17T05:23:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 msgr2=0x7f0bbc075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 0x7f0bbc075760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00ab000 tx=0x7f0bd00198a0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 msgr2=0x7f0be015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 0x7f0be015c780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0be0071d30 tx=0x7f0bd800e9a0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 shutdown_connections 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0bbc073250 0x7f0bbc075760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be015c370 0x7f0be015c780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0be0151380 0x7f0be01fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 --2- 172.21.15.43:0/314613990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4003a10 0x7f0be0150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 >> 172.21.15.43:0/314613990 conn(0x7f0be006ce60 msgr2=0x7f0be01626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 shutdown_connections 2024-09-17T05:23:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:36.988+0000 7f0bcf400640 1 -- 172.21.15.43:0/314613990 wait complete. 2024-09-17T05:23:37.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-17T05:23:37.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-17T05:23:37.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-17T05:23:37.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-17T05:23:37.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-17T05:23:37.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.036+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f420c0732e0 con 0x7f420c072ed0 2024-09-17T05:23:37.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.036+0000 7f4203400640 1 -- 172.21.15.43:0/3132638664 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41fc0624a0 con 0x7f420c072ed0 2024-09-17T05:23:37.035 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdc500b23b0 con 0x7fdc500b3790 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc4cc00640 1 -- 172.21.15.43:0/940104929 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc4005e8d0 con 0x7fdc500b3790 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 msgr2=0x7f41e8075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 0x7f41e8075940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f420c09bf10 tx=0x7f41f40091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 msgr2=0x7f420c206500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c490 tx=0x7f41fc00c950 comp rx=0 tx=0).stop 2024-09-17T05:23:37.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 shutdown_connections 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stdout:64424509446 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41e8073430 0x7f41e8075940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f420c0af5f0 0x7f420c154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f420c1582e0 0x7f420c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 --2- 172.21.15.43:0/3132638664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f420c072ed0 0x7f420c206500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 >> 172.21.15.43:0/3132638664 conn(0x7f420c0b1c60 msgr2=0x7f420c0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:37.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 shutdown_connections 2024-09-17T05:23:37.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7f4211000640 1 -- 172.21.15.43:0/3132638664 wait complete. 2024-09-17T05:23:37.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 msgr2=0x7fdc30075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 0x7fdc30075760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0ae000 tx=0x7fdc3c019260 comp rx=0 tx=0).stop 2024-09-17T05:23:37.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 msgr2=0x7fdc50147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc400097d0 tx=0x7fdc40009c90 comp rx=0 tx=0).stop 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 shutdown_connections 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc30073250 0x7fdc30075760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50152660 0x7fdc501541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc500cef70 0x7fdc501552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 --2- 172.21.15.43:0/940104929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc500b3790 0x7fdc50147b20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 >> 172.21.15.43:0/940104929 conn(0x7fdc500b1c60 msgr2=0x7fdc500cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 shutdown_connections 2024-09-17T05:23:37.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.040+0000 7fdc56000640 1 -- 172.21.15.43:0/940104929 wait complete. 2024-09-17T05:23:37.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2024-09-17T05:23:37.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f10c803bc80 con 0x7f10c80d4f00 2024-09-17T05:23:37.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-09-17T05:23:37.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10c6a00640 1 -- 172.21.15.43:0/992197298 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f10bc068330 con 0x7f10c80d4f00 2024-09-17T05:23:37.063 INFO:teuthology.orchestra.run.smithi043.stdout:60129542150 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 msgr2=0x7f10b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 0x7f10b0075710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10c8035ea0 tx=0x7f10c002d590 comp rx=0 tx=0).stop 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 msgr2=0x7f10c80e1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d5f0 tx=0x7f10bc00dab0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 shutdown_connections 2024-09-17T05:23:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f10b0073200 0x7f10b0075710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80d4f00 0x7f10c80e1620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c80cac90 0x7f10c80344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 --2- 172.21.15.43:0/992197298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f10b4003a10 0x7f10c80e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 >> 172.21.15.43:0/992197298 conn(0x7f10c801fbf0 msgr2=0x7f10c800d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 shutdown_connections 2024-09-17T05:23:37.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:37.064+0000 7f10d5a00640 1 -- 172.21.15.43:0/992197298 wait complete. 2024-09-17T05:23:37.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.6 2024-09-17T05:23:38.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:23:38.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:23:38.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T05:23:38.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:23:38.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:23:38.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:23:38.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:23:38.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:23:38.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d0a00640 1 -- 172.21.15.43:0/2164599 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc006330 con 0x7fd6b40037b0 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 -- 172.21.15.43:0/2164599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 msgr2=0x7fd6b4003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 --2- 172.21.15.43:0/2164599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 0x7fd6b4003be0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0017f0 tx=0x7fd6bc01c820 comp rx=0 tx=0).stop 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d2800640 1 -- 172.21.15.43:0/2164599 reap_dead start 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 -- 172.21.15.43:0/2164599 shutdown_connections 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 -- 172.21.15.43:0/2164599 >> 172.21.15.43:0/2164599 conn(0x7fd6cc0b3de0 msgr2=0x7fd6cc0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 -- 172.21.15.43:0/2164599 shutdown_connections 2024-09-17T05:23:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.256+0000 7fd6d4800640 1 -- 172.21.15.43:0/2164599 wait complete. 2024-09-17T05:23:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.260+0000 7fd6d4800640 1 Processor -- start 2024-09-17T05:23:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.264+0000 7fd6d4800640 1 -- start start 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 0x7fd6cc1fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 0x7fd6cc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc168b60 con 0x7fd6cc0b2c30 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1689e0 con 0x7fd6b40037b0 2024-09-17T05:23:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d4800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd6cc168ce0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 0x7fd6cc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48016/0 (socket says 172.21.15.43:48016) 2024-09-17T05:23:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d2800640 1 -- 172.21.15.43:0/1603722398 learned_addr learned my addr 172.21.15.43:0/1603722398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.268+0000 7fd6d1e00640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 0x7fd6cc1fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 -- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 msgr2=0x7fd6cc1fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 0x7fd6cc1fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 -- 172.21.15.43:0/1603722398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 msgr2=0x7fd6cc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 0x7fd6cc157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 -- 172.21.15.43:0/1603722398 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc1fabc0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d1400640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 0x7fd6cc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:38.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d2800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b7f60 tx=0x7fd6c800de50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800bbe0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6c8019070 con 0x7fd6cc15bb70 2024-09-17T05:23:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80106c0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc1faea0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1f70f0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.276+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc1fa930 con 0x7fd6cc15bb70 2024-09-17T05:23:38.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.280+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6c8010860 con 0x7fd6cc15bb70 2024-09-17T05:23:38.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.284+0000 7fd6c6000640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 0x7fd6b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.284+0000 7fd6d1e00640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 0x7fd6b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.284+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd6c805bb10 con 0x7fd6cc15bb70 2024-09-17T05:23:38.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.284+0000 7fd6d1e00640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 0x7fd6b0075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00018c0 tx=0x7fd6c0006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.284+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6c80642d0 con 0x7fd6cc15bb70 2024-09-17T05:23:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 -- 172.21.15.43:0/2512177371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015f540 msgr2=0x7f021015f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 --2- 172.21.15.43:0/2512177371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015f540 0x7f021015f950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f801c850 comp rx=0 tx=0).stop 2024-09-17T05:23:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 -- 172.21.15.43:0/2512177371 shutdown_connections 2024-09-17T05:23:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 --2- 172.21.15.43:0/2512177371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015f540 0x7f021015f950 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 --2- 172.21.15.43:0/2512177371 >> v2:172.21.15.43:3300/0 conn(0x7f021015cbd0 0x7f021015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 -- 172.21.15.43:0/2512177371 >> 172.21.15.43:0/2512177371 conn(0x7f021009aa50 msgr2=0x7f02100b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 -- 172.21.15.43:0/2512177371 shutdown_connections 2024-09-17T05:23:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7f020f200640 1 -- 172.21.15.43:0/2512177371 wait complete. 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 -- 172.21.15.43:0/980023065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 msgr2=0x7fd16c073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 --2- 172.21.15.43:0/980023065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c073930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009840 tx=0x7fd15c01c790 comp rx=0 tx=0).stop 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 -- 172.21.15.43:0/980023065 shutdown_connections 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 --2- 172.21.15.43:0/980023065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c073930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 -- 172.21.15.43:0/980023065 >> 172.21.15.43:0/980023065 conn(0x7fd16c0d0010 msgr2=0x7fd16c0d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 -- 172.21.15.43:0/980023065 shutdown_connections 2024-09-17T05:23:38.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.308+0000 7fd172400640 1 -- 172.21.15.43:0/980023065 wait complete. 2024-09-17T05:23:38.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7f020f200640 1 Processor -- start 2024-09-17T05:23:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7f020f200640 1 -- start start 2024-09-17T05:23:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7f155fe00640 1 -- 172.21.15.43:0/3800176520 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558003d90 con 0x7f15480053b0 2024-09-17T05:23:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7fd172400640 1 Processor -- start 2024-09-17T05:23:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7f1567e00640 1 -- 172.21.15.43:0/3800176520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15480053b0 msgr2=0x7f154800fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7fd172400640 1 -- start start 2024-09-17T05:23:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.312+0000 7f1567e00640 1 --2- 172.21.15.43:0/3800176520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15480053b0 0x7f154800fe60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ef0 tx=0x7f155801d250 comp rx=0 tx=0).stop 2024-09-17T05:23:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f1567e00640 1 -- 172.21.15.43:0/3800176520 shutdown_connections 2024-09-17T05:23:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f1567e00640 1 --2- 172.21.15.43:0/3800176520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15480053b0 0x7f154800fe60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f1567e00640 1 -- 172.21.15.43:0/3800176520 >> 172.21.15.43:0/3800176520 conn(0x7f15600cee60 msgr2=0x7f15600cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f1567e00640 1 -- 172.21.15.43:0/3800176520 shutdown_connections 2024-09-17T05:23:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015cbd0 0x7f021009d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 0x7f021009d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02101691c0 con 0x7f021015f540 2024-09-17T05:23:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd172400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 0x7fd16c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0210169040 con 0x7f02100b4cb0 2024-09-17T05:23:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020f200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0210169340 con 0x7f021015cbd0 2024-09-17T05:23:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd172400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 0x7fd16c1fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd172400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42654/0 (socket says 172.21.15.43:42654) 2024-09-17T05:23:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f020de00640 1 -- 172.21.15.43:0/578689507 learned_addr learned my addr 172.21.15.43:0/578689507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd172400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c169db0 con 0x7fd16c070550 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd171a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd171a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42662/0 (socket says 172.21.15.43:42662) 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd171a00640 1 -- 172.21.15.43:0/3266479448 learned_addr learned my addr 172.21.15.43:0/3266479448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7fd16be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 0x7fd16c1fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c169c30 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171000640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 0x7fd16c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c169f30 con 0x7fd16c073e70 2024-09-17T05:23:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 -- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015cbd0 msgr2=0x7f021009d1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:23:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015cbd0 0x7f021009d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 -- 172.21.15.43:0/578689507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 msgr2=0x7f021009d720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:23:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.316+0000 7f1567e00640 1 -- 172.21.15.43:0/3800176520 wait complete. 2024-09-17T05:23:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020e800640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 0x7f021009d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f1567e00640 1 Processor -- start 2024-09-17T05:23:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 0x7f021009d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f1567e00640 1 -- start start 2024-09-17T05:23:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 -- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 msgr2=0x7fd16c1fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 0x7fd16c1fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 -- 172.21.15.43:0/3266479448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 msgr2=0x7fd16c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 0x7fd16c153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c1fde40 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd16be00640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 0x7fd16c1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171000640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 0x7fd16c153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd171a00640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c730 tx=0x7fd16400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 -- 172.21.15.43:0/578689507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02100d14d0 con 0x7f02100b4cb0 2024-09-17T05:23:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164003b80 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c1fe120 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020e800640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 0x7f021009d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:38.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c1517d0 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd164003d20 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164003b80 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.320+0000 7f020de00640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f020000d750 tx=0x7f020000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1440052a0 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 0x7f15601f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 0x7f156015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15600b3670 con 0x7f15480053b0 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15600cd040 con 0x7f15600c7b40 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1567e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f15600c8750 con 0x7f156014c880 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41718/0 (socket says 172.21.15.43:41718) 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 -- 172.21.15.43:0/3276596555 learned_addr learned my addr 172.21.15.43:0/3276596555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200017070 con 0x7f02100b4cb0 2024-09-17T05:23:38.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1564a00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 0x7f15601f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 0x7f156015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0200004310 con 0x7f02100b4cb0 2024-09-17T05:23:38.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020000de60 con 0x7f02100b4cb0 2024-09-17T05:23:38.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021006d9e0 con 0x7f02100b4cb0 2024-09-17T05:23:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 -- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 msgr2=0x7f15601f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 0x7f15601f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 -- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 msgr2=0x7f156015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f1565400640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 0x7f156015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101fd2a0 con 0x7f02100b4cb0 2024-09-17T05:23:38.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.324+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd16400a850 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1565400640 1 -- 172.21.15.43:0/3276596555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600c6f30 con 0x7f15480053b0 2024-09-17T05:23:38.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1565e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 0x7f156015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:38.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1564a00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 0x7f15601f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:38.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021015f000 con 0x7f02100b4cb0 2024-09-17T05:23:38.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1565400640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1554004cf0 tx=0x7f1554001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7fd168c00640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 0x7fd148075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd1640939b0 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554011660 con 0x7f15480053b0 2024-09-17T05:23:38.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7fd171000640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 0x7fd148075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601f85f0 con 0x7f15480053b0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15600c5f10 con 0x7f15480053b0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1554016070 con 0x7f15480053b0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554011b70 con 0x7f15480053b0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.328+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c0052a0 con 0x7f15480053b0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.332+0000 7fd171000640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 0x7fd148075910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af000 tx=0x7fd15c01afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.332+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd16405ea50 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.336+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0200009c30 con 0x7f02100b4cb0 2024-09-17T05:23:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.336+0000 7f0206a00640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 0x7f01ec075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.336+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f020005b820 con 0x7f02100b4cb0 2024-09-17T05:23:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.340+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0200020020 con 0x7f02100b4cb0 2024-09-17T05:23:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.340+0000 7f0205400640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 0x7f01ec075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.340+0000 7f0205400640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 0x7f01ec075790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01f80af000 tx=0x7f01f801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.344+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1554025430 con 0x7f15480053b0 2024-09-17T05:23:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.344+0000 7f155d600640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 0x7f15400757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.344+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1554095400 con 0x7f15480053b0 2024-09-17T05:23:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.348+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1554060400 con 0x7f15480053b0 2024-09-17T05:23:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.348+0000 7f1564a00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 0x7f15400757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.348+0000 7f1564a00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 0x7f15400757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1560156d80 tx=0x7f154c006720 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.360+0000 7f3f28a00640 1 -- 172.21.15.43:0/3018493664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 msgr2=0x7f3f240ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.360+0000 7f3f28a00640 1 --2- 172.21.15.43:0/3018493664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 0x7f3f240ce720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009bc0 tx=0x7f3f1401d620 comp rx=0 tx=0).stop 2024-09-17T05:23:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.360+0000 7f3f28a00640 1 -- 172.21.15.43:0/3018493664 shutdown_connections 2024-09-17T05:23:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.360+0000 7f3f28a00640 1 --2- 172.21.15.43:0/3018493664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 0x7f3f240ce720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.360+0000 7f3f28a00640 1 -- 172.21.15.43:0/3018493664 >> 172.21.15.43:0/3018493664 conn(0x7f3f240b2f80 msgr2=0x7f3f240b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.364+0000 7f3f28a00640 1 -- 172.21.15.43:0/3018493664 shutdown_connections 2024-09-17T05:23:38.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.364+0000 7f3f28a00640 1 -- 172.21.15.43:0/3018493664 wait complete. 2024-09-17T05:23:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 Processor -- start 2024-09-17T05:23:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 -- start start 2024-09-17T05:23:38.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f240c67a0 0x7f3f2415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 0x7f3f24208eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24169ec0 con 0x7f3f240b0df0 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24169d40 con 0x7f3f240d0730 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.368+0000 7f3f28a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2416a040 con 0x7f3f240c67a0 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41722/0 (socket says 172.21.15.43:41722) 2024-09-17T05:23:38.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 -- 172.21.15.43:0/2569048732 learned_addr learned my addr 172.21.15.43:0/2569048732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 -- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f240c67a0 msgr2=0x7f3f2415f980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:23:38.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f240c67a0 0x7f3f2415f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 -- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 msgr2=0x7f3f24208eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:23:38.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 0x7f3f24208eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 -- 172.21.15.43:0/2569048732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f240c7840 con 0x7f3f240b0df0 2024-09-17T05:23:38.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f21e00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009b10 tx=0x7f3f14005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140b2040 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f140a6e50 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.372+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14005230 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.376+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241fabb0 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.376+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241f9df0 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.380+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f140053d0 con 0x7f3f240b0df0 2024-09-17T05:23:38.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.380+0000 7f3f1a000640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 0x7f3f04075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.380+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3f14002020 con 0x7f3f240b0df0 2024-09-17T05:23:38.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.384+0000 7f3f21400640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 0x7f3f04075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.384+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f240ce720 con 0x7f3f240b0df0 2024-09-17T05:23:38.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.384+0000 7f3f21400640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 0x7f3f04075760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f2409b780 tx=0x7f3f100079c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.392+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f140167b0 con 0x7f3f240b0df0 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 -- 172.21.15.43:0/1213051920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9800b7e0 msgr2=0x7fda98036870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 --2- 172.21.15.43:0/1213051920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9800b7e0 0x7fda98036870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda9001f010 comp rx=0 tx=0).stop 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 -- 172.21.15.43:0/1213051920 shutdown_connections 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 --2- 172.21.15.43:0/1213051920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9800b7e0 0x7fda98036870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 --2- 172.21.15.43:0/1213051920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda98052370 0x7fda98036330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fda9fe00640 1 -- 172.21.15.43:0/1213051920 >> 172.21.15.43:0/1213051920 conn(0x7fda980513f0 msgr2=0x7fda98051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fc6c4200640 1 -- 172.21.15.43:0/330821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 msgr2=0x7fc6bc15eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fda9fe00640 1 -- 172.21.15.43:0/1213051920 shutdown_connections 2024-09-17T05:23:38.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.408+0000 7fc6c4200640 1 --2- 172.21.15.43:0/330821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc15eb10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06b340 tx=0x7fc6a801d530 comp rx=0 tx=0).stop 2024-09-17T05:23:38.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fc6c4200640 1 -- 172.21.15.43:0/330821055 shutdown_connections 2024-09-17T05:23:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fda9fe00640 1 -- 172.21.15.43:0/1213051920 wait complete. 2024-09-17T05:23:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fc6c4200640 1 --2- 172.21.15.43:0/330821055 >> v2:172.21.15.43:3300/0 conn(0x7fc6bc15a9c0 0x7fc6bc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fc6c4200640 1 --2- 172.21.15.43:0/330821055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc15eb10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.412+0000 7fc6c4200640 1 -- 172.21.15.43:0/330821055 >> 172.21.15.43:0/330821055 conn(0x7fc6bc0b1cd0 msgr2=0x7fc6bc0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.416+0000 7fc6c4200640 1 -- 172.21.15.43:0/330821055 shutdown_connections 2024-09-17T05:23:38.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.416+0000 7fc6c4200640 1 -- 172.21.15.43:0/330821055 wait complete. 2024-09-17T05:23:38.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.416+0000 7fda9fe00640 1 Processor -- start 2024-09-17T05:23:38.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.416+0000 7fda9fe00640 1 -- start start 2024-09-17T05:23:38.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fc6c4200640 1 Processor -- start 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 0x7fda98182e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda98052370 0x7fda98047880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 0x7fda98035610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda9801c5c0 con 0x7fda9800b7e0 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda9801c740 con 0x7fda98052780 2024-09-17T05:23:38.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9fe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fda9801c8c0 con 0x7fda98052370 2024-09-17T05:23:38.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 0x7fda98182e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 0x7fda98035610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 0x7fda98035610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42708/0 (socket says 172.21.15.43:42708) 2024-09-17T05:23:38.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9f400640 1 -- 172.21.15.43:0/681335881 learned_addr learned my addr 172.21.15.43:0/681335881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fc6c4200640 1 -- start start 2024-09-17T05:23:38.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 -- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda98052370 msgr2=0x7fda98047880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:23:38.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda98052370 0x7fda98047880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 -- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 msgr2=0x7fda98035610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 0x7fda98035610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 -- 172.21.15.43:0/681335881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98020c60 con 0x7fda9800b7e0 2024-09-17T05:23:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.420+0000 7fda9ea00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 0x7fda98182e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdaa006cdc0 tx=0x7fdaa0069dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0098070 con 0x7fda9800b7e0 2024-09-17T05:23:38.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda98020f40 con 0x7fda9800b7e0 2024-09-17T05:23:38.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda98021500 con 0x7fda9800b7e0 2024-09-17T05:23:38.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa0072b20 con 0x7fda9800b7e0 2024-09-17T05:23:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00a1060 con 0x7fda9800b7e0 2024-09-17T05:23:38.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fc6c4200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda780052a0 con 0x7fda9800b7e0 2024-09-17T05:23:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.424+0000 7fc6c4200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6bc15a9c0 0x7fc6bc1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c4200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6bc15f050 0x7fc6bc155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c4200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc168180 con 0x7fc6bc15a9c0 2024-09-17T05:23:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c4200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc168000 con 0x7fc6bc0af390 2024-09-17T05:23:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c4200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc6bc168300 con 0x7fc6bc15f050 2024-09-17T05:23:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42712/0 (socket says 172.21.15.43:42712) 2024-09-17T05:23:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 -- 172.21.15.43:0/3356384990 learned_addr learned my addr 172.21.15.43:0/3356384990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c2200640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6bc15f050 0x7fc6bc155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 -- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6bc15f050 msgr2=0x7fc6bc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6bc15f050 0x7fc6bc155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 -- 172.21.15.43:0/3356384990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6bc15a9c0 msgr2=0x7fc6bc1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.428+0000 7fc6c1800640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6bc15a9c0 0x7fc6bc1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6c1800640 1 -- 172.21.15.43:0/3356384990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc14eea0 con 0x7fc6bc0af390 2024-09-17T05:23:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6c1800640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80074b0 tx=0x7fc6a8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8003600 con 0x7fc6bc0af390 2024-09-17T05:23:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6a8003890 con 0x7fc6bc0af390 2024-09-17T05:23:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6c4200640 1 -- 172.21.15.43:0/3356384990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc13c6b0 con 0x7fc6bc0af390 2024-09-17T05:23:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6c4200640 1 -- 172.21.15.43:0/3356384990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc13cc70 con 0x7fc6bc0af390 2024-09-17T05:23:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.432+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8003c80 con 0x7fc6bc0af390 2024-09-17T05:23:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.436+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdaa009b360 con 0x7fda9800b7e0 2024-09-17T05:23:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc069c20 con 0x7fc6bc0af390 2024-09-17T05:23:38.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fda96a00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 0x7fda80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa0110590 con 0x7fda9800b7e0 2024-09-17T05:23:38.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6a80aa070 con 0x7fc6bc0af390 2024-09-17T05:23:38.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdaa01109b0 con 0x7fda9800b7e0 2024-09-17T05:23:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fda9e000640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 0x7fda80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fda9e000640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 0x7fda80075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda9802b180 tx=0x7fda8c007a20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fc6b9600640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 0x7fc69c075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fc6c0e00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 0x7fc69c075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.440+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc6a812e460 con 0x7fc6bc0af390 2024-09-17T05:23:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.444+0000 7fc6c0e00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 0x7fc69c075810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f70 tx=0x7fc6b00079c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.444+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6a80c4060 con 0x7fc6bc0af390 2024-09-17T05:23:38.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 -- 172.21.15.43:0/1947899559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93806d570 msgr2=0x7fc93806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 --2- 172.21.15.43:0/1947899559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93806d570 0x7fc93806f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc930009a10 tx=0x7fc93001cb10 comp rx=0 tx=0).stop 2024-09-17T05:23:38.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 -- 172.21.15.43:0/1947899559 shutdown_connections 2024-09-17T05:23:38.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 --2- 172.21.15.43:0/1947899559 >> v2:172.21.15.62:3300/0 conn(0x7fc9380cc6d0 0x7fc93806f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 --2- 172.21.15.43:0/1947899559 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93806d570 0x7fc93806f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 -- 172.21.15.43:0/1947899559 >> 172.21.15.43:0/1947899559 conn(0x7fc9380b23d0 msgr2=0x7fc9380d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 -- 172.21.15.43:0/1947899559 shutdown_connections 2024-09-17T05:23:38.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.456+0000 7fc93fe00640 1 -- 172.21.15.43:0/1947899559 wait complete. 2024-09-17T05:23:38.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.460+0000 7fc93fe00640 1 Processor -- start 2024-09-17T05:23:38.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.460+0000 7fc93fe00640 1 -- start start 2024-09-17T05:23:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93815adc0 0x7fc93809d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 0x7fc9380adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41768/0 (socket says 172.21.15.43:41768) 2024-09-17T05:23:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93d400640 1 -- 172.21.15.43:0/3172134760 learned_addr learned my addr 172.21.15.43:0/3172134760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc938168580 con 0x7fc9380cc6d0 2024-09-17T05:23:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93de00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 0x7fc9380adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93ca00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93815adc0 0x7fc93809d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc938168400 con 0x7fc9380ad0b0 2024-09-17T05:23:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.464+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc938168700 con 0x7fc93815adc0 2024-09-17T05:23:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 -- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93815adc0 msgr2=0x7fc93809d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93815adc0 0x7fc93809d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 -- 172.21.15.43:0/3172134760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 msgr2=0x7fc938158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9381f8b90 con 0x7fc9380ad0b0 2024-09-17T05:23:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93d400640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:23:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93de00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 0x7fc9380adc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc924009600 tx=0x7fc924009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924005020 con 0x7fc9380ad0b0 2024-09-17T05:23:38.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9240051c0 con 0x7fc9380ad0b0 2024-09-17T05:23:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924005610 con 0x7fc9380ad0b0 2024-09-17T05:23:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9381f8e70 con 0x7fc9380ad0b0 2024-09-17T05:23:38.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.468+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9381f9430 con 0x7fc9380ad0b0 2024-09-17T05:23:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.472+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc92401d020 con 0x7fc9380ad0b0 2024-09-17T05:23:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.472+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9140052a0 con 0x7fc9380ad0b0 2024-09-17T05:23:38.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.476+0000 7fc935600640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 0x7fc9180757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:38.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.476+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc9240938e0 con 0x7fc9380ad0b0 2024-09-17T05:23:38.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.476+0000 7fc93d400640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 0x7fc9180757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:38.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.476+0000 7fc93d400640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 0x7fc9180757e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc92c001c60 tx=0x7fc92c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:38.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.484+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc92405e290 con 0x7fc9380ad0b0 2024-09-17T05:23:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.672+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd6cc0b5140 con 0x7fd6cc15bb70 2024-09-17T05:23:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.672+0000 7fd6c6000640 1 -- 172.21.15.43:0/1603722398 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6c8069180 con 0x7fd6cc15bb70 2024-09-17T05:23:38.670 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 msgr2=0x7fd6b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 0x7fd6b0075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00018c0 tx=0x7fd6c0006040 comp rx=0 tx=0).stop 2024-09-17T05:23:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 msgr2=0x7fd6cc1f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b7f60 tx=0x7fd6c800de50 comp rx=0 tx=0).stop 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 shutdown_connections 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6b0073200 0x7fd6b0075710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6cc15bb70 0x7fd6cc1f7610 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6cc0b2c30 0x7fd6cc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 --2- 172.21.15.43:0/1603722398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40037b0 0x7fd6cc1fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 >> 172.21.15.43:0/1603722398 conn(0x7fd6cc0b3de0 msgr2=0x7fd6cc0cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 shutdown_connections 2024-09-17T05:23:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.676+0000 7fd6d4800640 1 -- 172.21.15.43:0/1603722398 wait complete. 2024-09-17T05:23:38.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.700+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd144005590 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.704+0000 7fd168c00640 1 -- 172.21.15.43:0/3266479448 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd16405e350 con 0x7fd16c0ad2d0 2024-09-17T05:23:38.702 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:38.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 msgr2=0x7fd148075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 0x7fd148075910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af000 tx=0x7fd15c01afd0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 msgr2=0x7fd16c150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c730 tx=0x7fd16400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 shutdown_connections 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073400 0x7fd148075910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c0ad2d0 0x7fd16c150ae0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c073e70 0x7fd16c1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 --2- 172.21.15.43:0/3266479448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c070550 0x7fd16c153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 >> 172.21.15.43:0/3266479448 conn(0x7fd16c0d0010 msgr2=0x7fd16c0b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 shutdown_connections 2024-09-17T05:23:38.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.708+0000 7fd172400640 1 -- 172.21.15.43:0/3266479448 wait complete. 2024-09-17T05:23:38.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-17T05:23:38.709 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.712+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f021006ef80 con 0x7f02100b4cb0 2024-09-17T05:23:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f0206a00640 1 -- 172.21.15.43:0/578689507 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02100d14d0 con 0x7f02100b4cb0 2024-09-17T05:23:38.715 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f153c005860 con 0x7f15480053b0 2024-09-17T05:23:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f155d600640 1 -- 172.21.15.43:0/3276596555 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f155405fd00 con 0x7f15480053b0 2024-09-17T05:23:38.716 INFO:teuthology.orchestra.run.smithi043.stdout:64424509448 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 msgr2=0x7f01ec075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 0x7f01ec075790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01f80af000 tx=0x7f01f801b090 comp rx=0 tx=0).stop 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 msgr2=0x7f021014a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f020000d750 tx=0x7f020000dc10 comp rx=0 tx=0).stop 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 shutdown_connections 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01ec073280 0x7f01ec075790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f021015f540 0x7f021009d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f021015cbd0 0x7f021009d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 --2- 172.21.15.43:0/578689507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02100b4cb0 0x7f021014a080 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f020000d750 tx=0x7f020000dc10 comp rx=0 tx=0).stop 2024-09-17T05:23:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.716+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 >> 172.21.15.43:0/578689507 conn(0x7f021009aa50 msgr2=0x7f0210159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 shutdown_connections 2024-09-17T05:23:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f020f200640 1 -- 172.21.15.43:0/578689507 wait complete. 2024-09-17T05:23:38.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 msgr2=0x7f15400757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 0x7f15400757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1560156d80 tx=0x7f154c006720 comp rx=0 tx=0).stop 2024-09-17T05:23:38.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 msgr2=0x7f15600c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.720+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1554004cf0 tx=0x7f1554001640 comp rx=0 tx=0).stop 2024-09-17T05:23:38.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 shutdown_connections 2024-09-17T05:23:38.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15400732d0 0x7f15400757e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600c7b40 0x7f156015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156014c880 0x7f15601f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 --2- 172.21.15.43:0/3276596555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15480053b0 0x7f15600c69f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 >> 172.21.15.43:0/3276596555 conn(0x7f15600cee60 msgr2=0x7f15600cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 shutdown_connections 2024-09-17T05:23:38.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.728+0000 7f1567e00640 1 -- 172.21.15.43:0/3276596555 wait complete. 2024-09-17T05:23:38.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-17T05:23:38.750 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-17T05:23:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.756+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f240c6370 con 0x7f3f240b0df0 2024-09-17T05:23:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.756+0000 7f3f1a000640 1 -- 172.21.15.43:0/2569048732 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f140f9c00 con 0x7f3f240b0df0 2024-09-17T05:23:38.757 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.757 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 msgr2=0x7f3f04075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 0x7f3f04075760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f2409b780 tx=0x7f3f100079c0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 msgr2=0x7f3f241f9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009b10 tx=0x7f3f14005040 comp rx=0 tx=0).stop 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 shutdown_connections 2024-09-17T05:23:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f04073250 0x7f3f04075760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f240d0730 0x7f3f24208eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3f240c67a0 0x7f3f2415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 --2- 172.21.15.43:0/2569048732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f240b0df0 0x7f3f241f9210 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 >> 172.21.15.43:0/2569048732 conn(0x7f3f240b2f80 msgr2=0x7f3f240b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 shutdown_connections 2024-09-17T05:23:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.760+0000 7f3f28a00640 1 -- 172.21.15.43:0/2569048732 wait complete. 2024-09-17T05:23:38.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-17T05:23:38.763 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.768+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fda78005860 con 0x7fda9800b7e0 2024-09-17T05:23:38.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.768+0000 7fda96a00640 1 -- 172.21.15.43:0/681335881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdaa00daf50 con 0x7fda9800b7e0 2024-09-17T05:23:38.767 INFO:teuthology.orchestra.run.smithi043.stdout:64424509447 2024-09-17T05:23:38.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 msgr2=0x7fda80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 0x7fda80075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda9802b180 tx=0x7fda8c007a20 comp rx=0 tx=0).stop 2024-09-17T05:23:38.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 msgr2=0x7fda98182e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 0x7fda98182e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdaa006cdc0 tx=0x7fdaa0069dc0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 shutdown_connections 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fda80073200 0x7fda80075710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda98052780 0x7fda98035610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda98052370 0x7fda98047880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 --2- 172.21.15.43:0/681335881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda9800b7e0 0x7fda98182e90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 >> 172.21.15.43:0/681335881 conn(0x7fda980513f0 msgr2=0x7fda98054450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 shutdown_connections 2024-09-17T05:23:38.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.772+0000 7fda9fe00640 1 -- 172.21.15.43:0/681335881 wait complete. 2024-09-17T05:23:38.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc6bc0afbc0 con 0x7fc6bc0af390 2024-09-17T05:23:38.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6b9600640 1 -- 172.21.15.43:0/3356384990 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6bc14eea0 con 0x7fc6bc0af390 2024-09-17T05:23:38.777 INFO:teuthology.orchestra.run.smithi043.stdout:64424509448 2024-09-17T05:23:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 msgr2=0x7fc69c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 0x7fc69c075810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f70 tx=0x7fc6b00079c0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 msgr2=0x7fc6bc13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80074b0 tx=0x7fc6a8004040 comp rx=0 tx=0).stop 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 shutdown_connections 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc69c073300 0x7fc69c075810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6bc15f050 0x7fc6bc155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6bc15a9c0 0x7fc6bc1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.776+0000 7fc6aea00640 1 --2- 172.21.15.43:0/3356384990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6bc0af390 0x7fc6bc13bff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.780+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 >> 172.21.15.43:0/3356384990 conn(0x7fc6bc0b1cd0 msgr2=0x7fc6bc0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.780+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 shutdown_connections 2024-09-17T05:23:38.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.780+0000 7fc6aea00640 1 -- 172.21.15.43:0/3356384990 wait complete. 2024-09-17T05:23:38.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-17T05:23:38.792 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-17T05:23:38.802 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-17T05:23:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc914005590 con 0x7fc9380ad0b0 2024-09-17T05:23:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc935600640 1 -- 172.21.15.43:0/3172134760 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc924062cc0 con 0x7fc9380ad0b0 2024-09-17T05:23:38.804 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.805 INFO:teuthology.orchestra.run.smithi043.stdout:60129542152 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 msgr2=0x7fc9180757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 0x7fc9180757e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc92c001c60 tx=0x7fc92c009040 comp rx=0 tx=0).stop 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 msgr2=0x7fc9380adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 0x7fc9380adc90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc924009600 tx=0x7fc924009ac0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 shutdown_connections 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc9180732d0 0x7fc9180757e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9380ad0b0 0x7fc9380adc90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc93815adc0 0x7fc93809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 --2- 172.21.15.43:0/3172134760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cc6d0 0x7fc938158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 >> 172.21.15.43:0/3172134760 conn(0x7fc9380b23d0 msgr2=0x7fc938157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 shutdown_connections 2024-09-17T05:23:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:38.804+0000 7fc93fe00640 1 -- 172.21.15.43:0/3172134760 wait complete. 2024-09-17T05:23:38.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.6 2024-09-17T05:23:38.824 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:38.824 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:23:38.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:39.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 -- 172.21.15.43:0/3138401380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20240b3af0 msgr2=0x7f20240ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 --2- 172.21.15.43:0/3138401380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20240b3af0 0x7f20240ad0b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f201c009c80 tx=0x7f201c01cce0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 -- 172.21.15.43:0/3138401380 shutdown_connections 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 --2- 172.21.15.43:0/3138401380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20240b3af0 0x7f20240ad0b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 --2- 172.21.15.43:0/3138401380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2024155d70 0x7f202415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 -- 172.21.15.43:0/3138401380 >> 172.21.15.43:0/3138401380 conn(0x7f20240b23d0 msgr2=0x7f20240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 -- 172.21.15.43:0/3138401380 shutdown_connections 2024-09-17T05:23:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.040+0000 7f2028a00640 1 -- 172.21.15.43:0/3138401380 wait complete. 2024-09-17T05:23:39.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 Processor -- start 2024-09-17T05:23:39.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- start start 2024-09-17T05:23:39.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 0x7f202408bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 0x7f2024074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2024168c80 con 0x7f202406f110 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2024168b00 con 0x7f2024155d70 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2024168e00 con 0x7f20240cbcb0 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41784/0 (socket says 172.21.15.43:41784) 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 -- 172.21.15.43:0/2864232955 learned_addr learned my addr 172.21.15.43:0/2864232955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2019400640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 0x7f202408bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2022800640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 0x7f2024074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 -- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 msgr2=0x7f2024074800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 0x7f2024074800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 -- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 msgr2=0x7f202408bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 0x7f202408bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 -- 172.21.15.43:0/2864232955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20240cac10 con 0x7f202406f110 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2022800640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 0x7f2024074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2019400640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 0x7f202408bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2021e00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f201400e7c0 tx=0x7f201400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400ccc0 con 0x7f202406f110 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201400ce80 con 0x7f202406f110 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014004290 con 0x7f202406f110 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20240caef0 con 0x7f202406f110 2024-09-17T05:23:39.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.044+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20240cb4b0 con 0x7f202406f110 2024-09-17T05:23:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240ad0b0 con 0x7f202406f110 2024-09-17T05:23:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2014003600 con 0x7f202406f110 2024-09-17T05:23:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f201aa00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 0x7f2000075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2014094ca0 con 0x7f202406f110 2024-09-17T05:23:39.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f2019400640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 0x7f2000075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.048+0000 7f2019400640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 0x7f2000075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20240ca790 tx=0x7f200c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.052+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f201405fcb0 con 0x7f202406f110 2024-09-17T05:23:39.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.224+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f202415e810 con 0x7f2000073200 2024-09-17T05:23:39.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.228+0000 7f201aa00640 1 -- 172.21.15.43:0/2864232955 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f202415e810 con 0x7f2000073200 2024-09-17T05:23:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:39.232 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 msgr2=0x7f2000075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 0x7f2000075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20240ca790 tx=0x7f200c009040 comp rx=0 tx=0).stop 2024-09-17T05:23:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 msgr2=0x7f2024155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f201400e7c0 tx=0x7f201400ec80 comp rx=0 tx=0).stop 2024-09-17T05:23:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 shutdown_connections 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2000073200 0x7f2000075710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2024155d70 0x7f202408bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20240cbcb0 0x7f2024074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 --2- 172.21.15.43:0/2864232955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f202406f110 0x7f2024155240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 >> 172.21.15.43:0/2864232955 conn(0x7f20240b23d0 msgr2=0x7f20240c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 shutdown_connections 2024-09-17T05:23:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.232+0000 7f2028a00640 1 -- 172.21.15.43:0/2864232955 wait complete. 2024-09-17T05:23:39.254 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T05:23:37.842162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235860,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738860,"statfs":{"total":773094113280,"available":772852592640,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.792434"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053057+0000","last_change":"2024-09-17T05:23:25.072214+0000","last_active":"2024-09-17T05:23:26.053057+0000","last_peered":"2024-09-17T05:23:26.053057+0000","last_clean":"2024-09-17T05:23:26.053057+0000","last_became_active":"2024-09-17T05:23:25.071839+0000","last_became_peered":"2024-09-17T05:23:25.071839+0000","last_unstale":"2024-09-17T05:23:26.053057+0000","last_undegraded":"2024-09-17T05:23:26.053057+0000","last_fullsized":"2024-09-17T05:23:26.053057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052684+0000","last_change":"2024-09-17T05:23:25.068245+0000","last_active":"2024-09-17T05:23:26.052684+0000","last_peered":"2024-09-17T05:23:26.052684+0000","last_clean":"2024-09-17T05:23:26.052684+0000","last_became_active":"2024-09-17T05:23:25.063871+0000","last_became_peered":"2024-09-17T05:23:25.063871+0000","last_unstale":"2024-09-17T05:23:26.052684+0000","last_undegraded":"2024-09-17T05:23:26.052684+0000","last_fullsized":"2024-09-17T05:23:26.052684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:20.196005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052943+0000","last_change":"2024-09-17T05:23:25.077572+0000","last_active":"2024-09-17T05:23:26.052943+0000","last_peered":"2024-09-17T05:23:26.052943+0000","last_clean":"2024-09-17T05:23:26.052943+0000","last_became_active":"2024-09-17T05:23:25.077232+0000","last_became_peered":"2024-09-17T05:23:25.077232+0000","last_unstale":"2024-09-17T05:23:26.052943+0000","last_undegraded":"2024-09-17T05:23:26.052943+0000","last_fullsized":"2024-09-17T05:23:26.052943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070147+0000","last_change":"2024-09-17T05:23:25.080315+0000","last_active":"2024-09-17T05:23:26.070147+0000","last_peered":"2024-09-17T05:23:26.070147+0000","last_clean":"2024-09-17T05:23:26.070147+0000","last_became_active":"2024-09-17T05:23:25.079991+0000","last_became_peered":"2024-09-17T05:23:25.079991+0000","last_unstale":"2024-09-17T05:23:26.070147+0000","last_undegraded":"2024-09-17T05:23:26.070147+0000","last_fullsized":"2024-09-17T05:23:26.070147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:58.988827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052636+0000","last_change":"2024-09-17T05:23:25.062672+0000","last_active":"2024-09-17T05:23:26.052636+0000","last_peered":"2024-09-17T05:23:26.052636+0000","last_clean":"2024-09-17T05:23:26.052636+0000","last_became_active":"2024-09-17T05:23:25.060858+0000","last_became_peered":"2024-09-17T05:23:25.060858+0000","last_unstale":"2024-09-17T05:23:26.052636+0000","last_undegraded":"2024-09-17T05:23:26.052636+0000","last_fullsized":"2024-09-17T05:23:26.052636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.102130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069891+0000","last_change":"2024-09-17T05:23:25.065376+0000","last_active":"2024-09-17T05:23:26.069891+0000","last_peered":"2024-09-17T05:23:26.069891+0000","last_clean":"2024-09-17T05:23:26.069891+0000","last_became_active":"2024-09-17T05:23:25.064806+0000","last_became_peered":"2024-09-17T05:23:25.064806+0000","last_unstale":"2024-09-17T05:23:26.069891+0000","last_undegraded":"2024-09-17T05:23:26.069891+0000","last_fullsized":"2024-09-17T05:23:26.069891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052148+0000","last_change":"2024-09-17T05:23:25.053855+0000","last_active":"2024-09-17T05:23:26.052148+0000","last_peered":"2024-09-17T05:23:26.052148+0000","last_clean":"2024-09-17T05:23:26.052148+0000","last_became_active":"2024-09-17T05:23:25.053569+0000","last_became_peered":"2024-09-17T05:23:25.053569+0000","last_unstale":"2024-09-17T05:23:26.052148+0000","last_undegraded":"2024-09-17T05:23:26.052148+0000","last_fullsized":"2024-09-17T05:23:26.052148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052419+0000","last_change":"2024-09-17T05:23:25.056925+0000","last_active":"2024-09-17T05:23:26.052419+0000","last_peered":"2024-09-17T05:23:26.052419+0000","last_clean":"2024-09-17T05:23:26.052419+0000","last_became_active":"2024-09-17T05:23:25.056619+0000","last_became_peered":"2024-09-17T05:23:25.056619+0000","last_unstale":"2024-09-17T05:23:26.052419+0000","last_undegraded":"2024-09-17T05:23:26.052419+0000","last_fullsized":"2024-09-17T05:23:26.052419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054267+0000","last_change":"2024-09-17T05:23:25.061010+0000","last_active":"2024-09-17T05:23:26.054267+0000","last_peered":"2024-09-17T05:23:26.054267+0000","last_clean":"2024-09-17T05:23:26.054267+0000","last_became_active":"2024-09-17T05:23:25.060714+0000","last_became_peered":"2024-09-17T05:23:25.060714+0000","last_unstale":"2024-09-17T05:23:26.054267+0000","last_undegraded":"2024-09-17T05:23:26.054267+0000","last_fullsized":"2024-09-17T05:23:26.054267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.664289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052921+0000","last_change":"2024-09-17T05:23:25.079732+0000","last_active":"2024-09-17T05:23:26.052921+0000","last_peered":"2024-09-17T05:23:26.052921+0000","last_clean":"2024-09-17T05:23:26.052921+0000","last_became_active":"2024-09-17T05:23:25.079100+0000","last_became_peered":"2024-09-17T05:23:25.079100+0000","last_unstale":"2024-09-17T05:23:26.052921+0000","last_undegraded":"2024-09-17T05:23:26.052921+0000","last_fullsized":"2024-09-17T05:23:26.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:01.901521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052498+0000","last_change":"2024-09-17T05:23:25.075619+0000","last_active":"2024-09-17T05:23:26.052498+0000","last_peered":"2024-09-17T05:23:26.052498+0000","last_clean":"2024-09-17T05:23:26.052498+0000","last_became_active":"2024-09-17T05:23:25.071655+0000","last_became_peered":"2024-09-17T05:23:25.071655+0000","last_unstale":"2024-09-17T05:23:26.052498+0000","last_undegraded":"2024-09-17T05:23:26.052498+0000","last_fullsized":"2024-09-17T05:23:26.052498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052932+0000","last_change":"2024-09-17T05:23:25.077778+0000","last_active":"2024-09-17T05:23:26.052932+0000","last_peered":"2024-09-17T05:23:26.052932+0000","last_clean":"2024-09-17T05:23:26.052932+0000","last_became_active":"2024-09-17T05:23:25.077040+0000","last_became_peered":"2024-09-17T05:23:25.077040+0000","last_unstale":"2024-09-17T05:23:26.052932+0000","last_undegraded":"2024-09-17T05:23:26.052932+0000","last_fullsized":"2024-09-17T05:23:26.052932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052584+0000","last_change":"2024-09-17T05:23:25.067070+0000","last_active":"2024-09-17T05:23:26.052584+0000","last_peered":"2024-09-17T05:23:26.052584+0000","last_clean":"2024-09-17T05:23:26.052584+0000","last_became_active":"2024-09-17T05:23:25.066734+0000","last_became_peered":"2024-09-17T05:23:25.066734+0000","last_unstale":"2024-09-17T05:23:26.052584+0000","last_undegraded":"2024-09-17T05:23:26.052584+0000","last_fullsized":"2024-09-17T05:23:26.052584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:58.712013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070882+0000","last_change":"2024-09-17T05:23:25.066014+0000","last_active":"2024-09-17T05:23:26.070882+0000","last_peered":"2024-09-17T05:23:26.070882+0000","last_clean":"2024-09-17T05:23:26.070882+0000","last_became_active":"2024-09-17T05:23:25.063873+0000","last_became_peered":"2024-09-17T05:23:25.063873+0000","last_unstale":"2024-09-17T05:23:26.070882+0000","last_undegraded":"2024-09-17T05:23:26.070882+0000","last_fullsized":"2024-09-17T05:23:26.070882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071924+0000","last_change":"2024-09-17T05:23:25.055365+0000","last_active":"2024-09-17T05:23:26.071924+0000","last_peered":"2024-09-17T05:23:26.071924+0000","last_clean":"2024-09-17T05:23:26.071924+0000","last_became_active":"2024-09-17T05:23:25.055046+0000","last_became_peered":"2024-09-17T05:23:25.055046+0000","last_unstale":"2024-09-17T05:23:26.071924+0000","last_undegraded":"2024-09-17T05:23:26.071924+0000","last_fullsized":"2024-09-17T05:23:26.071924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:48.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070072+0000","last_change":"2024-09-17T05:23:25.076193+0000","last_active":"2024-09-17T05:23:26.070072+0000","last_peered":"2024-09-17T05:23:26.070072+0000","last_clean":"2024-09-17T05:23:26.070072+0000","last_became_active":"2024-09-17T05:23:25.075582+0000","last_became_peered":"2024-09-17T05:23:25.075582+0000","last_unstale":"2024-09-17T05:23:26.070072+0000","last_undegraded":"2024-09-17T05:23:26.070072+0000","last_fullsized":"2024-09-17T05:23:26.070072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071700+0000","last_change":"2024-09-17T05:23:25.053491+0000","last_active":"2024-09-17T05:23:26.071700+0000","last_peered":"2024-09-17T05:23:26.071700+0000","last_clean":"2024-09-17T05:23:26.071700+0000","last_became_active":"2024-09-17T05:23:25.053209+0000","last_became_peered":"2024-09-17T05:23:25.053209+0000","last_unstale":"2024-09-17T05:23:26.071700+0000","last_undegraded":"2024-09-17T05:23:26.071700+0000","last_fullsized":"2024-09-17T05:23:26.071700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:33.151198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053156+0000","last_change":"2024-09-17T05:23:25.068427+0000","last_active":"2024-09-17T05:23:26.053156+0000","last_peered":"2024-09-17T05:23:26.053156+0000","last_clean":"2024-09-17T05:23:26.053156+0000","last_became_active":"2024-09-17T05:23:25.068043+0000","last_became_peered":"2024-09-17T05:23:25.068043+0000","last_unstale":"2024-09-17T05:23:26.053156+0000","last_undegraded":"2024-09-17T05:23:26.053156+0000","last_fullsized":"2024-09-17T05:23:26.053156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070102+0000","last_change":"2024-09-17T05:23:25.057241+0000","last_active":"2024-09-17T05:23:26.070102+0000","last_peered":"2024-09-17T05:23:26.070102+0000","last_clean":"2024-09-17T05:23:26.070102+0000","last_became_active":"2024-09-17T05:23:25.056934+0000","last_became_peered":"2024-09-17T05:23:25.056934+0000","last_unstale":"2024-09-17T05:23:26.070102+0000","last_undegraded":"2024-09-17T05:23:26.070102+0000","last_fullsized":"2024-09-17T05:23:26.070102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054089+0000","last_change":"2024-09-17T05:23:25.064365+0000","last_active":"2024-09-17T05:23:26.054089+0000","last_peered":"2024-09-17T05:23:26.054089+0000","last_clean":"2024-09-17T05:23:26.054089+0000","last_became_active":"2024-09-17T05:23:25.064064+0000","last_became_peered":"2024-09-17T05:23:25.064064+0000","last_unstale":"2024-09-17T05:23:26.054089+0000","last_undegraded":"2024-09-17T05:23:26.054089+0000","last_fullsized":"2024-09-17T05:23:26.054089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052827+0000","last_change":"2024-09-17T05:23:25.059278+0000","last_active":"2024-09-17T05:23:26.052827+0000","last_peered":"2024-09-17T05:23:26.052827+0000","last_clean":"2024-09-17T05:23:26.052827+0000","last_became_active":"2024-09-17T05:23:25.058926+0000","last_became_peered":"2024-09-17T05:23:25.058926+0000","last_unstale":"2024-09-17T05:23:26.052827+0000","last_undegraded":"2024-09-17T05:23:26.052827+0000","last_fullsized":"2024-09-17T05:23:26.052827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070891+0000","last_change":"2024-09-17T05:23:25.064634+0000","last_active":"2024-09-17T05:23:26.070891+0000","last_peered":"2024-09-17T05:23:26.070891+0000","last_clean":"2024-09-17T05:23:26.070891+0000","last_became_active":"2024-09-17T05:23:25.064250+0000","last_became_peered":"2024-09-17T05:23:25.064250+0000","last_unstale":"2024-09-17T05:23:26.070891+0000","last_undegraded":"2024-09-17T05:23:26.070891+0000","last_fullsized":"2024-09-17T05:23:26.070891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053318+0000","last_change":"2024-09-17T05:23:25.079930+0000","last_active":"2024-09-17T05:23:26.053318+0000","last_peered":"2024-09-17T05:23:26.053318+0000","last_clean":"2024-09-17T05:23:26.053318+0000","last_became_active":"2024-09-17T05:23:25.079276+0000","last_became_peered":"2024-09-17T05:23:25.079276+0000","last_unstale":"2024-09-17T05:23:26.053318+0000","last_undegraded":"2024-09-17T05:23:26.053318+0000","last_fullsized":"2024-09-17T05:23:26.053318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070179+0000","last_change":"2024-09-17T05:23:25.083629+0000","last_active":"2024-09-17T05:23:26.070179+0000","last_peered":"2024-09-17T05:23:26.070179+0000","last_clean":"2024-09-17T05:23:26.070179+0000","last_became_active":"2024-09-17T05:23:25.083394+0000","last_became_peered":"2024-09-17T05:23:25.083394+0000","last_unstale":"2024-09-17T05:23:26.070179+0000","last_undegraded":"2024-09-17T05:23:26.070179+0000","last_fullsized":"2024-09-17T05:23:26.070179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:22.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053114+0000","last_change":"2024-09-17T05:23:25.075408+0000","last_active":"2024-09-17T05:23:26.053114+0000","last_peered":"2024-09-17T05:23:26.053114+0000","last_clean":"2024-09-17T05:23:26.053114+0000","last_became_active":"2024-09-17T05:23:25.075101+0000","last_became_peered":"2024-09-17T05:23:25.075101+0000","last_unstale":"2024-09-17T05:23:26.053114+0000","last_undegraded":"2024-09-17T05:23:26.053114+0000","last_fullsized":"2024-09-17T05:23:26.053114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071351+0000","last_change":"2024-09-17T05:23:25.068874+0000","last_active":"2024-09-17T05:23:26.071351+0000","last_peered":"2024-09-17T05:23:26.071351+0000","last_clean":"2024-09-17T05:23:26.071351+0000","last_became_active":"2024-09-17T05:23:25.064181+0000","last_became_peered":"2024-09-17T05:23:25.064181+0000","last_unstale":"2024-09-17T05:23:26.071351+0000","last_undegraded":"2024-09-17T05:23:26.071351+0000","last_fullsized":"2024-09-17T05:23:26.071351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:43.495535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068665+0000","last_change":"2024-09-17T05:23:25.064004+0000","last_active":"2024-09-17T05:23:26.068665+0000","last_peered":"2024-09-17T05:23:26.068665+0000","last_clean":"2024-09-17T05:23:26.068665+0000","last_became_active":"2024-09-17T05:23:25.063599+0000","last_became_peered":"2024-09-17T05:23:25.063599+0000","last_unstale":"2024-09-17T05:23:26.068665+0000","last_undegraded":"2024-09-17T05:23:26.068665+0000","last_fullsized":"2024-09-17T05:23:26.068665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.655466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069668+0000","last_change":"2024-09-17T05:23:25.071820+0000","last_active":"2024-09-17T05:23:26.069668+0000","last_peered":"2024-09-17T05:23:26.069668+0000","last_clean":"2024-09-17T05:23:26.069668+0000","last_became_active":"2024-09-17T05:23:25.070565+0000","last_became_peered":"2024-09-17T05:23:25.070565+0000","last_unstale":"2024-09-17T05:23:26.069668+0000","last_undegraded":"2024-09-17T05:23:26.069668+0000","last_fullsized":"2024-09-17T05:23:26.069668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:53.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054175+0000","last_change":"2024-09-17T05:23:25.064336+0000","last_active":"2024-09-17T05:23:26.054175+0000","last_peered":"2024-09-17T05:23:26.054175+0000","last_clean":"2024-09-17T05:23:26.054175+0000","last_became_active":"2024-09-17T05:23:25.063789+0000","last_became_peered":"2024-09-17T05:23:25.063789+0000","last_unstale":"2024-09-17T05:23:26.054175+0000","last_undegraded":"2024-09-17T05:23:26.054175+0000","last_fullsized":"2024-09-17T05:23:26.054175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:27.415615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052848+0000","last_change":"2024-09-17T05:23:24.057092+0000","last_active":"2024-09-17T05:23:26.052848+0000","last_peered":"2024-09-17T05:23:26.052848+0000","last_clean":"2024-09-17T05:23:26.052848+0000","last_became_active":"2024-09-17T05:23:24.056017+0000","last_became_peered":"2024-09-17T05:23:24.056017+0000","last_unstale":"2024-09-17T05:23:26.052848+0000","last_undegraded":"2024-09-17T05:23:26.052848+0000","last_fullsized":"2024-09-17T05:23:26.052848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:21.643758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070140+0000","last_change":"2024-09-17T05:23:25.050194+0000","last_active":"2024-09-17T05:23:26.070140+0000","last_peered":"2024-09-17T05:23:26.070140+0000","last_clean":"2024-09-17T05:23:26.070140+0000","last_became_active":"2024-09-17T05:23:25.049834+0000","last_became_peered":"2024-09-17T05:23:25.049834+0000","last_unstale":"2024-09-17T05:23:26.070140+0000","last_undegraded":"2024-09-17T05:23:26.070140+0000","last_fullsized":"2024-09-17T05:23:26.070140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071221+0000","last_change":"2024-09-17T05:23:24.060034+0000","last_active":"2024-09-17T05:23:26.071221+0000","last_peered":"2024-09-17T05:23:26.071221+0000","last_clean":"2024-09-17T05:23:26.071221+0000","last_became_active":"2024-09-17T05:23:24.059693+0000","last_became_peered":"2024-09-17T05:23:24.059693+0000","last_unstale":"2024-09-17T05:23:26.071221+0000","last_undegraded":"2024-09-17T05:23:26.071221+0000","last_fullsized":"2024-09-17T05:23:26.071221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:09.436373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052733+0000","last_change":"2024-09-17T05:23:25.066266+0000","last_active":"2024-09-17T05:23:26.052733+0000","last_peered":"2024-09-17T05:23:26.052733+0000","last_clean":"2024-09-17T05:23:26.052733+0000","last_became_active":"2024-09-17T05:23:25.064050+0000","last_became_peered":"2024-09-17T05:23:25.064050+0000","last_unstale":"2024-09-17T05:23:26.052733+0000","last_undegraded":"2024-09-17T05:23:26.052733+0000","last_fullsized":"2024-09-17T05:23:26.052733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070980+0000","last_change":"2024-09-17T05:23:24.052942+0000","last_active":"2024-09-17T05:23:26.070980+0000","last_peered":"2024-09-17T05:23:26.070980+0000","last_clean":"2024-09-17T05:23:26.070980+0000","last_became_active":"2024-09-17T05:23:24.052628+0000","last_became_peered":"2024-09-17T05:23:24.052628+0000","last_unstale":"2024-09-17T05:23:26.070980+0000","last_undegraded":"2024-09-17T05:23:26.070980+0000","last_fullsized":"2024-09-17T05:23:26.070980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.647679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069032+0000","last_change":"2024-09-17T05:23:25.062501+0000","last_active":"2024-09-17T05:23:26.069032+0000","last_peered":"2024-09-17T05:23:26.069032+0000","last_clean":"2024-09-17T05:23:26.069032+0000","last_became_active":"2024-09-17T05:23:25.062195+0000","last_became_peered":"2024-09-17T05:23:25.062195+0000","last_unstale":"2024-09-17T05:23:26.069032+0000","last_undegraded":"2024-09-17T05:23:26.069032+0000","last_fullsized":"2024-09-17T05:23:26.069032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:28.768705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054381+0000","last_change":"2024-09-17T05:23:24.051900+0000","last_active":"2024-09-17T05:23:26.054381+0000","last_peered":"2024-09-17T05:23:26.054381+0000","last_clean":"2024-09-17T05:23:26.054381+0000","last_became_active":"2024-09-17T05:23:24.051525+0000","last_became_peered":"2024-09-17T05:23:24.051525+0000","last_unstale":"2024-09-17T05:23:26.054381+0000","last_undegraded":"2024-09-17T05:23:26.054381+0000","last_fullsized":"2024-09-17T05:23:26.054381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070296+0000","last_change":"2024-09-17T05:23:25.074116+0000","last_active":"2024-09-17T05:23:26.070296+0000","last_peered":"2024-09-17T05:23:26.070296+0000","last_clean":"2024-09-17T05:23:26.070296+0000","last_became_active":"2024-09-17T05:23:25.073750+0000","last_became_peered":"2024-09-17T05:23:25.073750+0000","last_unstale":"2024-09-17T05:23:26.070296+0000","last_undegraded":"2024-09-17T05:23:26.070296+0000","last_fullsized":"2024-09-17T05:23:26.070296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:23.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052816+0000","last_change":"2024-09-17T05:23:24.058892+0000","last_active":"2024-09-17T05:23:26.052816+0000","last_peered":"2024-09-17T05:23:26.052816+0000","last_clean":"2024-09-17T05:23:26.052816+0000","last_became_active":"2024-09-17T05:23:24.057777+0000","last_became_peered":"2024-09-17T05:23:24.057777+0000","last_unstale":"2024-09-17T05:23:26.052816+0000","last_undegraded":"2024-09-17T05:23:26.052816+0000","last_fullsized":"2024-09-17T05:23:26.052816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:52.022043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053572+0000","last_change":"2024-09-17T05:23:25.065444+0000","last_active":"2024-09-17T05:23:26.053572+0000","last_peered":"2024-09-17T05:23:26.053572+0000","last_clean":"2024-09-17T05:23:26.053572+0000","last_became_active":"2024-09-17T05:23:25.064932+0000","last_became_peered":"2024-09-17T05:23:25.064932+0000","last_unstale":"2024-09-17T05:23:26.053572+0000","last_undegraded":"2024-09-17T05:23:26.053572+0000","last_fullsized":"2024-09-17T05:23:26.053572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069641+0000","last_change":"2024-09-17T05:23:24.058300+0000","last_active":"2024-09-17T05:23:26.069641+0000","last_peered":"2024-09-17T05:23:26.069641+0000","last_clean":"2024-09-17T05:23:26.069641+0000","last_became_active":"2024-09-17T05:23:24.057127+0000","last_became_peered":"2024-09-17T05:23:24.057127+0000","last_unstale":"2024-09-17T05:23:26.069641+0000","last_undegraded":"2024-09-17T05:23:26.069641+0000","last_fullsized":"2024-09-17T05:23:26.069641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071142+0000","last_change":"2024-09-17T05:23:25.067068+0000","last_active":"2024-09-17T05:23:26.071142+0000","last_peered":"2024-09-17T05:23:26.071142+0000","last_clean":"2024-09-17T05:23:26.071142+0000","last_became_active":"2024-09-17T05:23:25.066755+0000","last_became_peered":"2024-09-17T05:23:25.066755+0000","last_unstale":"2024-09-17T05:23:26.071142+0000","last_undegraded":"2024-09-17T05:23:26.071142+0000","last_fullsized":"2024-09-17T05:23:26.071142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:11.227030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069906+0000","last_change":"2024-09-17T05:23:24.053733+0000","last_active":"2024-09-17T05:23:26.069906+0000","last_peered":"2024-09-17T05:23:26.069906+0000","last_clean":"2024-09-17T05:23:26.069906+0000","last_became_active":"2024-09-17T05:23:24.053370+0000","last_became_peered":"2024-09-17T05:23:24.053370+0000","last_unstale":"2024-09-17T05:23:26.069906+0000","last_undegraded":"2024-09-17T05:23:26.069906+0000","last_fullsized":"2024-09-17T05:23:26.069906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.457450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052622+0000","last_change":"2024-09-17T05:23:25.069241+0000","last_active":"2024-09-17T05:23:26.052622+0000","last_peered":"2024-09-17T05:23:26.052622+0000","last_clean":"2024-09-17T05:23:26.052622+0000","last_became_active":"2024-09-17T05:23:25.067530+0000","last_became_peered":"2024-09-17T05:23:25.067530+0000","last_unstale":"2024-09-17T05:23:26.052622+0000","last_undegraded":"2024-09-17T05:23:26.052622+0000","last_fullsized":"2024-09-17T05:23:26.052622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069422+0000","last_change":"2024-09-17T05:23:24.069042+0000","last_active":"2024-09-17T05:23:26.069422+0000","last_peered":"2024-09-17T05:23:26.069422+0000","last_clean":"2024-09-17T05:23:26.069422+0000","last_became_active":"2024-09-17T05:23:24.068629+0000","last_became_peered":"2024-09-17T05:23:24.068629+0000","last_unstale":"2024-09-17T05:23:26.069422+0000","last_undegraded":"2024-09-17T05:23:26.069422+0000","last_fullsized":"2024-09-17T05:23:26.069422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:54.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071355+0000","last_change":"2024-09-17T05:23:24.059071+0000","last_active":"2024-09-17T05:23:26.071355+0000","last_peered":"2024-09-17T05:23:26.071355+0000","last_clean":"2024-09-17T05:23:26.071355+0000","last_became_active":"2024-09-17T05:23:24.058788+0000","last_became_peered":"2024-09-17T05:23:24.058788+0000","last_unstale":"2024-09-17T05:23:26.071355+0000","last_undegraded":"2024-09-17T05:23:26.071355+0000","last_fullsized":"2024-09-17T05:23:26.071355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:26.407828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071163+0000","last_change":"2024-09-17T05:23:25.070863+0000","last_active":"2024-09-17T05:23:26.071163+0000","last_peered":"2024-09-17T05:23:26.071163+0000","last_clean":"2024-09-17T05:23:26.071163+0000","last_became_active":"2024-09-17T05:23:25.070508+0000","last_became_peered":"2024-09-17T05:23:25.070508+0000","last_unstale":"2024-09-17T05:23:26.071163+0000","last_undegraded":"2024-09-17T05:23:26.071163+0000","last_fullsized":"2024-09-17T05:23:26.071163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:08.540151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052361+0000","last_change":"2024-09-17T05:23:24.054907+0000","last_active":"2024-09-17T05:23:26.052361+0000","last_peered":"2024-09-17T05:23:26.052361+0000","last_clean":"2024-09-17T05:23:26.052361+0000","last_became_active":"2024-09-17T05:23:24.054532+0000","last_became_peered":"2024-09-17T05:23:24.054532+0000","last_unstale":"2024-09-17T05:23:26.052361+0000","last_undegraded":"2024-09-17T05:23:26.052361+0000","last_fullsized":"2024-09-17T05:23:26.052361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053550+0000","last_change":"2024-09-17T05:23:25.072781+0000","last_active":"2024-09-17T05:23:26.053550+0000","last_peered":"2024-09-17T05:23:26.053550+0000","last_clean":"2024-09-17T05:23:26.053550+0000","last_became_active":"2024-09-17T05:23:25.072303+0000","last_became_peered":"2024-09-17T05:23:25.072303+0000","last_unstale":"2024-09-17T05:23:26.053550+0000","last_undegraded":"2024-09-17T05:23:26.053550+0000","last_fullsized":"2024-09-17T05:23:26.053550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068751+0000","last_change":"2024-09-17T05:23:24.050270+0000","last_active":"2024-09-17T05:23:26.068751+0000","last_peered":"2024-09-17T05:23:26.068751+0000","last_clean":"2024-09-17T05:23:26.068751+0000","last_became_active":"2024-09-17T05:23:24.049787+0000","last_became_peered":"2024-09-17T05:23:24.049787+0000","last_unstale":"2024-09-17T05:23:26.068751+0000","last_undegraded":"2024-09-17T05:23:26.068751+0000","last_fullsized":"2024-09-17T05:23:26.068751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071171+0000","last_change":"2024-09-17T05:23:25.065390+0000","last_active":"2024-09-17T05:23:26.071171+0000","last_peered":"2024-09-17T05:23:26.071171+0000","last_clean":"2024-09-17T05:23:26.071171+0000","last_became_active":"2024-09-17T05:23:25.065054+0000","last_became_peered":"2024-09-17T05:23:25.065054+0000","last_unstale":"2024-09-17T05:23:26.071171+0000","last_undegraded":"2024-09-17T05:23:26.071171+0000","last_fullsized":"2024-09-17T05:23:26.071171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:55.059902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053136+0000","last_change":"2024-09-17T05:23:24.066652+0000","last_active":"2024-09-17T05:23:26.053136+0000","last_peered":"2024-09-17T05:23:26.053136+0000","last_clean":"2024-09-17T05:23:26.053136+0000","last_became_active":"2024-09-17T05:23:24.065549+0000","last_became_peered":"2024-09-17T05:23:24.065549+0000","last_unstale":"2024-09-17T05:23:26.053136+0000","last_undegraded":"2024-09-17T05:23:26.053136+0000","last_fullsized":"2024-09-17T05:23:26.053136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:44.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070411+0000","last_change":"2024-09-17T05:23:25.074083+0000","last_active":"2024-09-17T05:23:26.070411+0000","last_peered":"2024-09-17T05:23:26.070411+0000","last_clean":"2024-09-17T05:23:26.070411+0000","last_became_active":"2024-09-17T05:23:25.073740+0000","last_became_peered":"2024-09-17T05:23:25.073740+0000","last_unstale":"2024-09-17T05:23:26.070411+0000","last_undegraded":"2024-09-17T05:23:26.070411+0000","last_fullsized":"2024-09-17T05:23:26.070411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:15.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070325+0000","last_change":"2024-09-17T05:23:24.060840+0000","last_active":"2024-09-17T05:23:26.070325+0000","last_peered":"2024-09-17T05:23:26.070325+0000","last_clean":"2024-09-17T05:23:26.070325+0000","last_became_active":"2024-09-17T05:23:24.060518+0000","last_became_peered":"2024-09-17T05:23:24.060518+0000","last_unstale":"2024-09-17T05:23:26.070325+0000","last_undegraded":"2024-09-17T05:23:26.070325+0000","last_fullsized":"2024-09-17T05:23:26.070325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:47.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053110+0000","last_change":"2024-09-17T05:23:25.067357+0000","last_active":"2024-09-17T05:23:26.053110+0000","last_peered":"2024-09-17T05:23:26.053110+0000","last_clean":"2024-09-17T05:23:26.053110+0000","last_became_active":"2024-09-17T05:23:25.067017+0000","last_became_peered":"2024-09-17T05:23:25.067017+0000","last_unstale":"2024-09-17T05:23:26.053110+0000","last_undegraded":"2024-09-17T05:23:26.053110+0000","last_fullsized":"2024-09-17T05:23:26.053110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:05.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052624+0000","last_change":"2024-09-17T05:23:24.058023+0000","last_active":"2024-09-17T05:23:26.052624+0000","last_peered":"2024-09-17T05:23:26.052624+0000","last_clean":"2024-09-17T05:23:26.052624+0000","last_became_active":"2024-09-17T05:23:24.057266+0000","last_became_peered":"2024-09-17T05:23:24.057266+0000","last_unstale":"2024-09-17T05:23:26.052624+0000","last_undegraded":"2024-09-17T05:23:26.052624+0000","last_fullsized":"2024-09-17T05:23:26.052624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053458+0000","last_change":"2024-09-17T05:23:25.051518+0000","last_active":"2024-09-17T05:23:26.053458+0000","last_peered":"2024-09-17T05:23:26.053458+0000","last_clean":"2024-09-17T05:23:26.053458+0000","last_became_active":"2024-09-17T05:23:25.051182+0000","last_became_peered":"2024-09-17T05:23:25.051182+0000","last_unstale":"2024-09-17T05:23:26.053458+0000","last_undegraded":"2024-09-17T05:23:26.053458+0000","last_fullsized":"2024-09-17T05:23:26.053458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071578+0000","last_change":"2024-09-17T05:23:25.085148+0000","last_active":"2024-09-17T05:23:26.071578+0000","last_peered":"2024-09-17T05:23:26.071578+0000","last_clean":"2024-09-17T05:23:26.071578+0000","last_became_active":"2024-09-17T05:23:25.083807+0000","last_became_peered":"2024-09-17T05:23:25.083807+0000","last_unstale":"2024-09-17T05:23:26.071578+0000","last_undegraded":"2024-09-17T05:23:26.071578+0000","last_fullsized":"2024-09-17T05:23:26.071578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:32.917183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053650+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:26.053650+0000","last_peered":"2024-09-17T05:23:26.053650+0000","last_clean":"2024-09-17T05:23:26.053650+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:26.053650+0000","last_undegraded":"2024-09-17T05:23:26.053650+0000","last_fullsized":"2024-09-17T05:23:26.053650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071284+0000","last_change":"2024-09-17T05:23:24.056449+0000","last_active":"2024-09-17T05:23:26.071284+0000","last_peered":"2024-09-17T05:23:26.071284+0000","last_clean":"2024-09-17T05:23:26.071284+0000","last_became_active":"2024-09-17T05:23:24.056055+0000","last_became_peered":"2024-09-17T05:23:24.056055+0000","last_unstale":"2024-09-17T05:23:26.071284+0000","last_undegraded":"2024-09-17T05:23:26.071284+0000","last_fullsized":"2024-09-17T05:23:26.071284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.624751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069696+0000","last_change":"2024-09-17T05:23:25.055754+0000","last_active":"2024-09-17T05:23:26.069696+0000","last_peered":"2024-09-17T05:23:26.069696+0000","last_clean":"2024-09-17T05:23:26.069696+0000","last_became_active":"2024-09-17T05:23:25.055368+0000","last_became_peered":"2024-09-17T05:23:25.055368+0000","last_unstale":"2024-09-17T05:23:26.069696+0000","last_undegraded":"2024-09-17T05:23:26.069696+0000","last_fullsized":"2024-09-17T05:23:26.069696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:35.911660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070994+0000","last_change":"2024-09-17T05:23:24.065017+0000","last_active":"2024-09-17T05:23:26.070994+0000","last_peered":"2024-09-17T05:23:26.070994+0000","last_clean":"2024-09-17T05:23:26.070994+0000","last_became_active":"2024-09-17T05:23:24.064688+0000","last_became_peered":"2024-09-17T05:23:24.064688+0000","last_unstale":"2024-09-17T05:23:26.070994+0000","last_undegraded":"2024-09-17T05:23:26.070994+0000","last_fullsized":"2024-09-17T05:23:26.070994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:38.132272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068472+0000","last_change":"2024-09-17T05:23:24.065406+0000","last_active":"2024-09-17T05:23:26.068472+0000","last_peered":"2024-09-17T05:23:26.068472+0000","last_clean":"2024-09-17T05:23:26.068472+0000","last_became_active":"2024-09-17T05:23:24.065058+0000","last_became_peered":"2024-09-17T05:23:24.065058+0000","last_unstale":"2024-09-17T05:23:26.068472+0000","last_undegraded":"2024-09-17T05:23:26.068472+0000","last_fullsized":"2024-09-17T05:23:26.068472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053208+0000","last_change":"2024-09-17T05:23:25.074915+0000","last_active":"2024-09-17T05:23:26.053208+0000","last_peered":"2024-09-17T05:23:26.053208+0000","last_clean":"2024-09-17T05:23:26.053208+0000","last_became_active":"2024-09-17T05:23:25.074603+0000","last_became_peered":"2024-09-17T05:23:25.074603+0000","last_unstale":"2024-09-17T05:23:26.053208+0000","last_undegraded":"2024-09-17T05:23:26.053208+0000","last_fullsized":"2024-09-17T05:23:26.053208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071423+0000","last_change":"2024-09-17T05:23:24.058441+0000","last_active":"2024-09-17T05:23:26.071423+0000","last_peered":"2024-09-17T05:23:26.071423+0000","last_clean":"2024-09-17T05:23:26.071423+0000","last_became_active":"2024-09-17T05:23:24.054951+0000","last_became_peered":"2024-09-17T05:23:24.054951+0000","last_unstale":"2024-09-17T05:23:26.071423+0000","last_undegraded":"2024-09-17T05:23:26.071423+0000","last_fullsized":"2024-09-17T05:23:26.071423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:11.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052393+0000","last_change":"2024-09-17T05:23:25.056791+0000","last_active":"2024-09-17T05:23:26.052393+0000","last_peered":"2024-09-17T05:23:26.052393+0000","last_clean":"2024-09-17T05:23:26.052393+0000","last_became_active":"2024-09-17T05:23:25.056445+0000","last_became_peered":"2024-09-17T05:23:25.056445+0000","last_unstale":"2024-09-17T05:23:26.052393+0000","last_undegraded":"2024-09-17T05:23:26.052393+0000","last_fullsized":"2024-09-17T05:23:26.052393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052571+0000","last_change":"2024-09-17T05:23:24.058736+0000","last_active":"2024-09-17T05:23:26.052571+0000","last_peered":"2024-09-17T05:23:26.052571+0000","last_clean":"2024-09-17T05:23:26.052571+0000","last_became_active":"2024-09-17T05:23:24.056996+0000","last_became_peered":"2024-09-17T05:23:24.056996+0000","last_unstale":"2024-09-17T05:23:26.052571+0000","last_undegraded":"2024-09-17T05:23:26.052571+0000","last_fullsized":"2024-09-17T05:23:26.052571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:09.319432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070023+0000","last_change":"2024-09-17T05:23:25.072427+0000","last_active":"2024-09-17T05:23:26.070023+0000","last_peered":"2024-09-17T05:23:26.070023+0000","last_clean":"2024-09-17T05:23:26.070023+0000","last_became_active":"2024-09-17T05:23:25.071951+0000","last_became_peered":"2024-09-17T05:23:25.071951+0000","last_unstale":"2024-09-17T05:23:26.070023+0000","last_undegraded":"2024-09-17T05:23:26.070023+0000","last_fullsized":"2024-09-17T05:23:26.070023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:45.608054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071621+0000","last_change":"2024-09-17T05:23:24.064058+0000","last_active":"2024-09-17T05:23:26.071621+0000","last_peered":"2024-09-17T05:23:26.071621+0000","last_clean":"2024-09-17T05:23:26.071621+0000","last_became_active":"2024-09-17T05:23:24.063686+0000","last_became_peered":"2024-09-17T05:23:24.063686+0000","last_unstale":"2024-09-17T05:23:26.071621+0000","last_undegraded":"2024-09-17T05:23:26.071621+0000","last_fullsized":"2024-09-17T05:23:26.071621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:03.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070421+0000","last_change":"2024-09-17T05:23:24.061808+0000","last_active":"2024-09-17T05:23:26.070421+0000","last_peered":"2024-09-17T05:23:26.070421+0000","last_clean":"2024-09-17T05:23:26.070421+0000","last_became_active":"2024-09-17T05:23:24.061393+0000","last_became_peered":"2024-09-17T05:23:24.061393+0000","last_unstale":"2024-09-17T05:23:26.070421+0000","last_undegraded":"2024-09-17T05:23:26.070421+0000","last_fullsized":"2024-09-17T05:23:26.070421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:10.321148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070599+0000","last_change":"2024-09-17T05:23:25.065038+0000","last_active":"2024-09-17T05:23:26.070599+0000","last_peered":"2024-09-17T05:23:26.070599+0000","last_clean":"2024-09-17T05:23:26.070599+0000","last_became_active":"2024-09-17T05:23:25.063557+0000","last_became_peered":"2024-09-17T05:23:25.063557+0000","last_unstale":"2024-09-17T05:23:26.070599+0000","last_undegraded":"2024-09-17T05:23:26.070599+0000","last_fullsized":"2024-09-17T05:23:26.070599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:46.016555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071788+0000","last_change":"2024-09-17T05:23:24.061738+0000","last_active":"2024-09-17T05:23:26.071788+0000","last_peered":"2024-09-17T05:23:26.071788+0000","last_clean":"2024-09-17T05:23:26.071788+0000","last_became_active":"2024-09-17T05:23:24.061344+0000","last_became_peered":"2024-09-17T05:23:24.061344+0000","last_unstale":"2024-09-17T05:23:26.071788+0000","last_undegraded":"2024-09-17T05:23:26.071788+0000","last_fullsized":"2024-09-17T05:23:26.071788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:00.795235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070211+0000","last_change":"2024-09-17T05:23:25.059622+0000","last_active":"2024-09-17T05:23:26.070211+0000","last_peered":"2024-09-17T05:23:26.070211+0000","last_clean":"2024-09-17T05:23:26.070211+0000","last_became_active":"2024-09-17T05:23:25.059267+0000","last_became_peered":"2024-09-17T05:23:25.059267+0000","last_unstale":"2024-09-17T05:23:26.070211+0000","last_undegraded":"2024-09-17T05:23:26.070211+0000","last_fullsized":"2024-09-17T05:23:26.070211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:58.979778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.051888+0000","last_change":"2024-09-17T05:23:24.053175+0000","last_active":"2024-09-17T05:23:26.051888+0000","last_peered":"2024-09-17T05:23:26.051888+0000","last_clean":"2024-09-17T05:23:26.051888+0000","last_became_active":"2024-09-17T05:23:24.052859+0000","last_became_peered":"2024-09-17T05:23:24.052859+0000","last_unstale":"2024-09-17T05:23:26.051888+0000","last_undegraded":"2024-09-17T05:23:26.051888+0000","last_fullsized":"2024-09-17T05:23:26.051888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071095+0000","last_change":"2024-09-17T05:23:25.057826+0000","last_active":"2024-09-17T05:23:26.071095+0000","last_peered":"2024-09-17T05:23:26.071095+0000","last_clean":"2024-09-17T05:23:26.071095+0000","last_became_active":"2024-09-17T05:23:25.057507+0000","last_became_peered":"2024-09-17T05:23:25.057507+0000","last_unstale":"2024-09-17T05:23:26.071095+0000","last_undegraded":"2024-09-17T05:23:26.071095+0000","last_fullsized":"2024-09-17T05:23:26.071095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068904+0000","last_change":"2024-09-17T05:23:24.068858+0000","last_active":"2024-09-17T05:23:26.068904+0000","last_peered":"2024-09-17T05:23:26.068904+0000","last_clean":"2024-09-17T05:23:26.068904+0000","last_became_active":"2024-09-17T05:23:24.068387+0000","last_became_peered":"2024-09-17T05:23:24.068387+0000","last_unstale":"2024-09-17T05:23:26.068904+0000","last_undegraded":"2024-09-17T05:23:26.068904+0000","last_fullsized":"2024-09-17T05:23:26.068904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068658+0000","last_change":"2024-09-17T05:23:25.049296+0000","last_active":"2024-09-17T05:23:26.068658+0000","last_peered":"2024-09-17T05:23:26.068658+0000","last_clean":"2024-09-17T05:23:26.068658+0000","last_became_active":"2024-09-17T05:23:25.048948+0000","last_became_peered":"2024-09-17T05:23:25.048948+0000","last_unstale":"2024-09-17T05:23:26.068658+0000","last_undegraded":"2024-09-17T05:23:26.068658+0000","last_fullsized":"2024-09-17T05:23:26.068658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:49.995105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069337+0000","last_change":"2024-09-17T05:23:25.067211+0000","last_active":"2024-09-17T05:23:26.069337+0000","last_peered":"2024-09-17T05:23:26.069337+0000","last_clean":"2024-09-17T05:23:26.069337+0000","last_became_active":"2024-09-17T05:23:25.066883+0000","last_became_peered":"2024-09-17T05:23:25.066883+0000","last_unstale":"2024-09-17T05:23:26.069337+0000","last_undegraded":"2024-09-17T05:23:26.069337+0000","last_fullsized":"2024-09-17T05:23:26.069337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071212+0000","last_change":"2024-09-17T05:23:24.061156+0000","last_active":"2024-09-17T05:23:26.071212+0000","last_peered":"2024-09-17T05:23:26.071212+0000","last_clean":"2024-09-17T05:23:26.071212+0000","last_became_active":"2024-09-17T05:23:24.060751+0000","last_became_peered":"2024-09-17T05:23:24.060751+0000","last_unstale":"2024-09-17T05:23:26.071212+0000","last_undegraded":"2024-09-17T05:23:26.071212+0000","last_fullsized":"2024-09-17T05:23:26.071212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.506573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071131+0000","last_change":"2024-09-17T05:23:25.085054+0000","last_active":"2024-09-17T05:23:26.071131+0000","last_peered":"2024-09-17T05:23:26.071131+0000","last_clean":"2024-09-17T05:23:26.071131+0000","last_became_active":"2024-09-17T05:23:25.084307+0000","last_became_peered":"2024-09-17T05:23:25.084307+0000","last_unstale":"2024-09-17T05:23:26.071131+0000","last_undegraded":"2024-09-17T05:23:26.071131+0000","last_fullsized":"2024-09-17T05:23:26.071131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071557+0000","last_change":"2024-09-17T05:23:25.052805+0000","last_active":"2024-09-17T05:23:26.071557+0000","last_peered":"2024-09-17T05:23:26.071557+0000","last_clean":"2024-09-17T05:23:26.071557+0000","last_became_active":"2024-09-17T05:23:25.052440+0000","last_became_peered":"2024-09-17T05:23:25.052440+0000","last_unstale":"2024-09-17T05:23:26.071557+0000","last_undegraded":"2024-09-17T05:23:26.071557+0000","last_fullsized":"2024-09-17T05:23:26.071557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:51.739703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053334+0000","last_change":"2024-09-17T05:23:24.061930+0000","last_active":"2024-09-17T05:23:26.053334+0000","last_peered":"2024-09-17T05:23:26.053334+0000","last_clean":"2024-09-17T05:23:26.053334+0000","last_became_active":"2024-09-17T05:23:24.060988+0000","last_became_peered":"2024-09-17T05:23:24.060988+0000","last_unstale":"2024-09-17T05:23:26.053334+0000","last_undegraded":"2024-09-17T05:23:26.053334+0000","last_fullsized":"2024-09-17T05:23:26.053334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:47.573508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054561+0000","last_change":"2024-09-17T05:23:25.065743+0000","last_active":"2024-09-17T05:23:26.054561+0000","last_peered":"2024-09-17T05:23:26.054561+0000","last_clean":"2024-09-17T05:23:26.054561+0000","last_became_active":"2024-09-17T05:23:25.062840+0000","last_became_peered":"2024-09-17T05:23:25.062840+0000","last_unstale":"2024-09-17T05:23:26.054561+0000","last_undegraded":"2024-09-17T05:23:26.054561+0000","last_fullsized":"2024-09-17T05:23:26.054561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:01.803022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052543+0000","last_change":"2024-09-17T05:23:24.060124+0000","last_active":"2024-09-17T05:23:26.052543+0000","last_peered":"2024-09-17T05:23:26.052543+0000","last_clean":"2024-09-17T05:23:26.052543+0000","last_became_active":"2024-09-17T05:23:24.057538+0000","last_became_peered":"2024-09-17T05:23:24.057538+0000","last_unstale":"2024-09-17T05:23:26.052543+0000","last_undegraded":"2024-09-17T05:23:26.052543+0000","last_fullsized":"2024-09-17T05:23:26.052543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:41.202615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053337+0000","last_change":"2024-09-17T05:23:25.079567+0000","last_active":"2024-09-17T05:23:26.053337+0000","last_peered":"2024-09-17T05:23:26.053337+0000","last_clean":"2024-09-17T05:23:26.053337+0000","last_became_active":"2024-09-17T05:23:25.078899+0000","last_became_peered":"2024-09-17T05:23:25.078899+0000","last_unstale":"2024-09-17T05:23:26.053337+0000","last_undegraded":"2024-09-17T05:23:26.053337+0000","last_fullsized":"2024-09-17T05:23:26.053337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:39.680169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053486+0000","last_change":"2024-09-17T05:23:24.061891+0000","last_active":"2024-09-17T05:23:26.053486+0000","last_peered":"2024-09-17T05:23:26.053486+0000","last_clean":"2024-09-17T05:23:26.053486+0000","last_became_active":"2024-09-17T05:23:24.055527+0000","last_became_peered":"2024-09-17T05:23:24.055527+0000","last_unstale":"2024-09-17T05:23:26.053486+0000","last_undegraded":"2024-09-17T05:23:26.053486+0000","last_fullsized":"2024-09-17T05:23:26.053486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053220+0000","last_change":"2024-09-17T05:23:25.076838+0000","last_active":"2024-09-17T05:23:26.053220+0000","last_peered":"2024-09-17T05:23:26.053220+0000","last_clean":"2024-09-17T05:23:26.053220+0000","last_became_active":"2024-09-17T05:23:25.076547+0000","last_became_peered":"2024-09-17T05:23:25.076547+0000","last_unstale":"2024-09-17T05:23:26.053220+0000","last_undegraded":"2024-09-17T05:23:26.053220+0000","last_fullsized":"2024-09-17T05:23:26.053220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.142652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070483+0000","last_change":"2024-09-17T05:23:24.059741+0000","last_active":"2024-09-17T05:23:26.070483+0000","last_peered":"2024-09-17T05:23:26.070483+0000","last_clean":"2024-09-17T05:23:26.070483+0000","last_became_active":"2024-09-17T05:23:24.058336+0000","last_became_peered":"2024-09-17T05:23:24.058336+0000","last_unstale":"2024-09-17T05:23:26.070483+0000","last_undegraded":"2024-09-17T05:23:26.070483+0000","last_fullsized":"2024-09-17T05:23:26.070483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:01.640938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:39.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:23:39.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- 172.21.15.43:0/1355493578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 msgr2=0x7f35180cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 --2- 172.21.15.43:0/1355493578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180cce70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3500009900 tx=0x7f350001c620 comp rx=0 tx=0).stop 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- 172.21.15.43:0/1355493578 shutdown_connections 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 --2- 172.21.15.43:0/1355493578 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 0x7f35181609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 --2- 172.21.15.43:0/1355493578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180cce70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- 172.21.15.43:0/1355493578 >> 172.21.15.43:0/1355493578 conn(0x7f35180cee80 msgr2=0x7f35180cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- 172.21.15.43:0/1355493578 shutdown_connections 2024-09-17T05:23:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- 172.21.15.43:0/1355493578 wait complete. 2024-09-17T05:23:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 Processor -- start 2024-09-17T05:23:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.456+0000 7f351d200640 1 -- start start 2024-09-17T05:23:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 0x7f3518157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 0x7f3518155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f351816a020 con 0x7f35180d0f50 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3518169ea0 con 0x7f35180b7ce0 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f351816a1a0 con 0x7f35180cd3b0 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41802/0 (socket says 172.21.15.43:41802) 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 -- 172.21.15.43:0/2061502253 learned_addr learned my addr 172.21.15.43:0/2061502253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f350d400640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 0x7f3518155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3515e00640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 0x7f3518157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 -- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 msgr2=0x7f3518155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 0x7f3518155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 -- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 msgr2=0x7f3518157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 0x7f3518157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 -- 172.21.15.43:0/2061502253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35181fcea0 con 0x7f35180d0f50 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3515e00640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 0x7f3518157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f3516800640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f351000da40 tx=0x7f351000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510017070 con 0x7f35180d0f50 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3510004b30 con 0x7f35180d0f50 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510005020 con 0x7f35180d0f50 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35181fd150 con 0x7f35180d0f50 2024-09-17T05:23:39.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3518157450 con 0x7f35180d0f50 2024-09-17T05:23:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.460+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35181609c0 con 0x7f35180d0f50 2024-09-17T05:23:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.464+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f35100051c0 con 0x7f35180d0f50 2024-09-17T05:23:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.464+0000 7f350ea00640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 0x7f34f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.464+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3510094640 con 0x7f35180d0f50 2024-09-17T05:23:39.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.464+0000 7f3515e00640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 0x7f34f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.464+0000 7f3515e00640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 0x7f34f4075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3518071220 tx=0x7f3508009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.468+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3510020070 con 0x7f35180d0f50 2024-09-17T05:23:39.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.636+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3518157970 con 0x7f34f4073200 2024-09-17T05:23:39.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.644+0000 7f350ea00640 1 -- 172.21.15.43:0/2061502253 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f3508001cb0 con 0x7f34f4073200 2024-09-17T05:23:39.644 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:23:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 msgr2=0x7f34f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 0x7f34f4075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3518071220 tx=0x7f3508009040 comp rx=0 tx=0).stop 2024-09-17T05:23:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 msgr2=0x7f35180b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f351000da40 tx=0x7f351000df00 comp rx=0 tx=0).stop 2024-09-17T05:23:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 shutdown_connections 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34f4073200 0x7f34f4075710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180d0f50 0x7f35180b1b10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35180cd3b0 0x7f3518155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 --2- 172.21.15.43:0/2061502253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35180b7ce0 0x7f3518157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 >> 172.21.15.43:0/2061502253 conn(0x7f35180cee80 msgr2=0x7f351815a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 shutdown_connections 2024-09-17T05:23:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.648+0000 7f351d200640 1 -- 172.21.15.43:0/2061502253 wait complete. 2024-09-17T05:23:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-17T05:23:37.842162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235860,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738860,"statfs":{"total":773094113280,"available":772852592640,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.792434"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053057+0000","last_change":"2024-09-17T05:23:25.072214+0000","last_active":"2024-09-17T05:23:26.053057+0000","last_peered":"2024-09-17T05:23:26.053057+0000","last_clean":"2024-09-17T05:23:26.053057+0000","last_became_active":"2024-09-17T05:23:25.071839+0000","last_became_peered":"2024-09-17T05:23:25.071839+0000","last_unstale":"2024-09-17T05:23:26.053057+0000","last_undegraded":"2024-09-17T05:23:26.053057+0000","last_fullsized":"2024-09-17T05:23:26.053057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052684+0000","last_change":"2024-09-17T05:23:25.068245+0000","last_active":"2024-09-17T05:23:26.052684+0000","last_peered":"2024-09-17T05:23:26.052684+0000","last_clean":"2024-09-17T05:23:26.052684+0000","last_became_active":"2024-09-17T05:23:25.063871+0000","last_became_peered":"2024-09-17T05:23:25.063871+0000","last_unstale":"2024-09-17T05:23:26.052684+0000","last_undegraded":"2024-09-17T05:23:26.052684+0000","last_fullsized":"2024-09-17T05:23:26.052684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:20.196005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052943+0000","last_change":"2024-09-17T05:23:25.077572+0000","last_active":"2024-09-17T05:23:26.052943+0000","last_peered":"2024-09-17T05:23:26.052943+0000","last_clean":"2024-09-17T05:23:26.052943+0000","last_became_active":"2024-09-17T05:23:25.077232+0000","last_became_peered":"2024-09-17T05:23:25.077232+0000","last_unstale":"2024-09-17T05:23:26.052943+0000","last_undegraded":"2024-09-17T05:23:26.052943+0000","last_fullsized":"2024-09-17T05:23:26.052943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070147+0000","last_change":"2024-09-17T05:23:25.080315+0000","last_active":"2024-09-17T05:23:26.070147+0000","last_peered":"2024-09-17T05:23:26.070147+0000","last_clean":"2024-09-17T05:23:26.070147+0000","last_became_active":"2024-09-17T05:23:25.079991+0000","last_became_peered":"2024-09-17T05:23:25.079991+0000","last_unstale":"2024-09-17T05:23:26.070147+0000","last_undegraded":"2024-09-17T05:23:26.070147+0000","last_fullsized":"2024-09-17T05:23:26.070147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:58.988827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052636+0000","last_change":"2024-09-17T05:23:25.062672+0000","last_active":"2024-09-17T05:23:26.052636+0000","last_peered":"2024-09-17T05:23:26.052636+0000","last_clean":"2024-09-17T05:23:26.052636+0000","last_became_active":"2024-09-17T05:23:25.060858+0000","last_became_peered":"2024-09-17T05:23:25.060858+0000","last_unstale":"2024-09-17T05:23:26.052636+0000","last_undegraded":"2024-09-17T05:23:26.052636+0000","last_fullsized":"2024-09-17T05:23:26.052636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.102130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069891+0000","last_change":"2024-09-17T05:23:25.065376+0000","last_active":"2024-09-17T05:23:26.069891+0000","last_peered":"2024-09-17T05:23:26.069891+0000","last_clean":"2024-09-17T05:23:26.069891+0000","last_became_active":"2024-09-17T05:23:25.064806+0000","last_became_peered":"2024-09-17T05:23:25.064806+0000","last_unstale":"2024-09-17T05:23:26.069891+0000","last_undegraded":"2024-09-17T05:23:26.069891+0000","last_fullsized":"2024-09-17T05:23:26.069891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:42.210402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052148+0000","last_change":"2024-09-17T05:23:25.053855+0000","last_active":"2024-09-17T05:23:26.052148+0000","last_peered":"2024-09-17T05:23:26.052148+0000","last_clean":"2024-09-17T05:23:26.052148+0000","last_became_active":"2024-09-17T05:23:25.053569+0000","last_became_peered":"2024-09-17T05:23:25.053569+0000","last_unstale":"2024-09-17T05:23:26.052148+0000","last_undegraded":"2024-09-17T05:23:26.052148+0000","last_fullsized":"2024-09-17T05:23:26.052148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052419+0000","last_change":"2024-09-17T05:23:25.056925+0000","last_active":"2024-09-17T05:23:26.052419+0000","last_peered":"2024-09-17T05:23:26.052419+0000","last_clean":"2024-09-17T05:23:26.052419+0000","last_became_active":"2024-09-17T05:23:25.056619+0000","last_became_peered":"2024-09-17T05:23:25.056619+0000","last_unstale":"2024-09-17T05:23:26.052419+0000","last_undegraded":"2024-09-17T05:23:26.052419+0000","last_fullsized":"2024-09-17T05:23:26.052419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054267+0000","last_change":"2024-09-17T05:23:25.061010+0000","last_active":"2024-09-17T05:23:26.054267+0000","last_peered":"2024-09-17T05:23:26.054267+0000","last_clean":"2024-09-17T05:23:26.054267+0000","last_became_active":"2024-09-17T05:23:25.060714+0000","last_became_peered":"2024-09-17T05:23:25.060714+0000","last_unstale":"2024-09-17T05:23:26.054267+0000","last_undegraded":"2024-09-17T05:23:26.054267+0000","last_fullsized":"2024-09-17T05:23:26.054267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:12.664289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052921+0000","last_change":"2024-09-17T05:23:25.079732+0000","last_active":"2024-09-17T05:23:26.052921+0000","last_peered":"2024-09-17T05:23:26.052921+0000","last_clean":"2024-09-17T05:23:26.052921+0000","last_became_active":"2024-09-17T05:23:25.079100+0000","last_became_peered":"2024-09-17T05:23:25.079100+0000","last_unstale":"2024-09-17T05:23:26.052921+0000","last_undegraded":"2024-09-17T05:23:26.052921+0000","last_fullsized":"2024-09-17T05:23:26.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:01.901521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052498+0000","last_change":"2024-09-17T05:23:25.075619+0000","last_active":"2024-09-17T05:23:26.052498+0000","last_peered":"2024-09-17T05:23:26.052498+0000","last_clean":"2024-09-17T05:23:26.052498+0000","last_became_active":"2024-09-17T05:23:25.071655+0000","last_became_peered":"2024-09-17T05:23:25.071655+0000","last_unstale":"2024-09-17T05:23:26.052498+0000","last_undegraded":"2024-09-17T05:23:26.052498+0000","last_fullsized":"2024-09-17T05:23:26.052498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:17.006366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052932+0000","last_change":"2024-09-17T05:23:25.077778+0000","last_active":"2024-09-17T05:23:26.052932+0000","last_peered":"2024-09-17T05:23:26.052932+0000","last_clean":"2024-09-17T05:23:26.052932+0000","last_became_active":"2024-09-17T05:23:25.077040+0000","last_became_peered":"2024-09-17T05:23:25.077040+0000","last_unstale":"2024-09-17T05:23:26.052932+0000","last_undegraded":"2024-09-17T05:23:26.052932+0000","last_fullsized":"2024-09-17T05:23:26.052932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070125+0000","last_change":"2024-09-17T05:23:25.065666+0000","last_active":"2024-09-17T05:23:26.070125+0000","last_peered":"2024-09-17T05:23:26.070125+0000","last_clean":"2024-09-17T05:23:26.070125+0000","last_became_active":"2024-09-17T05:23:25.065336+0000","last_became_peered":"2024-09-17T05:23:25.065336+0000","last_unstale":"2024-09-17T05:23:26.070125+0000","last_undegraded":"2024-09-17T05:23:26.070125+0000","last_fullsized":"2024-09-17T05:23:26.070125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:05.046572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069242+0000","last_change":"2024-09-17T05:23:25.073988+0000","last_active":"2024-09-17T05:23:26.069242+0000","last_peered":"2024-09-17T05:23:26.069242+0000","last_clean":"2024-09-17T05:23:26.069242+0000","last_became_active":"2024-09-17T05:23:25.073703+0000","last_became_peered":"2024-09-17T05:23:25.073703+0000","last_unstale":"2024-09-17T05:23:26.069242+0000","last_undegraded":"2024-09-17T05:23:26.069242+0000","last_fullsized":"2024-09-17T05:23:26.069242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:07.586082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052584+0000","last_change":"2024-09-17T05:23:25.067070+0000","last_active":"2024-09-17T05:23:26.052584+0000","last_peered":"2024-09-17T05:23:26.052584+0000","last_clean":"2024-09-17T05:23:26.052584+0000","last_became_active":"2024-09-17T05:23:25.066734+0000","last_became_peered":"2024-09-17T05:23:25.066734+0000","last_unstale":"2024-09-17T05:23:26.052584+0000","last_undegraded":"2024-09-17T05:23:26.052584+0000","last_fullsized":"2024-09-17T05:23:26.052584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:58.712013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070882+0000","last_change":"2024-09-17T05:23:25.066014+0000","last_active":"2024-09-17T05:23:26.070882+0000","last_peered":"2024-09-17T05:23:26.070882+0000","last_clean":"2024-09-17T05:23:26.070882+0000","last_became_active":"2024-09-17T05:23:25.063873+0000","last_became_peered":"2024-09-17T05:23:25.063873+0000","last_unstale":"2024-09-17T05:23:26.070882+0000","last_undegraded":"2024-09-17T05:23:26.070882+0000","last_fullsized":"2024-09-17T05:23:26.070882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:45.132886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071924+0000","last_change":"2024-09-17T05:23:25.055365+0000","last_active":"2024-09-17T05:23:26.071924+0000","last_peered":"2024-09-17T05:23:26.071924+0000","last_clean":"2024-09-17T05:23:26.071924+0000","last_became_active":"2024-09-17T05:23:25.055046+0000","last_became_peered":"2024-09-17T05:23:25.055046+0000","last_unstale":"2024-09-17T05:23:26.071924+0000","last_undegraded":"2024-09-17T05:23:26.071924+0000","last_fullsized":"2024-09-17T05:23:26.071924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:48.850645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070072+0000","last_change":"2024-09-17T05:23:25.076193+0000","last_active":"2024-09-17T05:23:26.070072+0000","last_peered":"2024-09-17T05:23:26.070072+0000","last_clean":"2024-09-17T05:23:26.070072+0000","last_became_active":"2024-09-17T05:23:25.075582+0000","last_became_peered":"2024-09-17T05:23:25.075582+0000","last_unstale":"2024-09-17T05:23:26.070072+0000","last_undegraded":"2024-09-17T05:23:26.070072+0000","last_fullsized":"2024-09-17T05:23:26.070072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071700+0000","last_change":"2024-09-17T05:23:25.053491+0000","last_active":"2024-09-17T05:23:26.071700+0000","last_peered":"2024-09-17T05:23:26.071700+0000","last_clean":"2024-09-17T05:23:26.071700+0000","last_became_active":"2024-09-17T05:23:25.053209+0000","last_became_peered":"2024-09-17T05:23:25.053209+0000","last_unstale":"2024-09-17T05:23:26.071700+0000","last_undegraded":"2024-09-17T05:23:26.071700+0000","last_fullsized":"2024-09-17T05:23:26.071700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:33.151198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053156+0000","last_change":"2024-09-17T05:23:25.068427+0000","last_active":"2024-09-17T05:23:26.053156+0000","last_peered":"2024-09-17T05:23:26.053156+0000","last_clean":"2024-09-17T05:23:26.053156+0000","last_became_active":"2024-09-17T05:23:25.068043+0000","last_became_peered":"2024-09-17T05:23:25.068043+0000","last_unstale":"2024-09-17T05:23:26.053156+0000","last_undegraded":"2024-09-17T05:23:26.053156+0000","last_fullsized":"2024-09-17T05:23:26.053156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070102+0000","last_change":"2024-09-17T05:23:25.057241+0000","last_active":"2024-09-17T05:23:26.070102+0000","last_peered":"2024-09-17T05:23:26.070102+0000","last_clean":"2024-09-17T05:23:26.070102+0000","last_became_active":"2024-09-17T05:23:25.056934+0000","last_became_peered":"2024-09-17T05:23:25.056934+0000","last_unstale":"2024-09-17T05:23:26.070102+0000","last_undegraded":"2024-09-17T05:23:26.070102+0000","last_fullsized":"2024-09-17T05:23:26.070102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:07.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054089+0000","last_change":"2024-09-17T05:23:25.064365+0000","last_active":"2024-09-17T05:23:26.054089+0000","last_peered":"2024-09-17T05:23:26.054089+0000","last_clean":"2024-09-17T05:23:26.054089+0000","last_became_active":"2024-09-17T05:23:25.064064+0000","last_became_peered":"2024-09-17T05:23:25.064064+0000","last_unstale":"2024-09-17T05:23:26.054089+0000","last_undegraded":"2024-09-17T05:23:26.054089+0000","last_fullsized":"2024-09-17T05:23:26.054089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:10.444160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052827+0000","last_change":"2024-09-17T05:23:25.059278+0000","last_active":"2024-09-17T05:23:26.052827+0000","last_peered":"2024-09-17T05:23:26.052827+0000","last_clean":"2024-09-17T05:23:26.052827+0000","last_became_active":"2024-09-17T05:23:25.058926+0000","last_became_peered":"2024-09-17T05:23:25.058926+0000","last_unstale":"2024-09-17T05:23:26.052827+0000","last_undegraded":"2024-09-17T05:23:26.052827+0000","last_fullsized":"2024-09-17T05:23:26.052827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070891+0000","last_change":"2024-09-17T05:23:25.064634+0000","last_active":"2024-09-17T05:23:26.070891+0000","last_peered":"2024-09-17T05:23:26.070891+0000","last_clean":"2024-09-17T05:23:26.070891+0000","last_became_active":"2024-09-17T05:23:25.064250+0000","last_became_peered":"2024-09-17T05:23:25.064250+0000","last_unstale":"2024-09-17T05:23:26.070891+0000","last_undegraded":"2024-09-17T05:23:26.070891+0000","last_fullsized":"2024-09-17T05:23:26.070891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053318+0000","last_change":"2024-09-17T05:23:25.079930+0000","last_active":"2024-09-17T05:23:26.053318+0000","last_peered":"2024-09-17T05:23:26.053318+0000","last_clean":"2024-09-17T05:23:26.053318+0000","last_became_active":"2024-09-17T05:23:25.079276+0000","last_became_peered":"2024-09-17T05:23:25.079276+0000","last_unstale":"2024-09-17T05:23:26.053318+0000","last_undegraded":"2024-09-17T05:23:26.053318+0000","last_fullsized":"2024-09-17T05:23:26.053318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:53.742046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070179+0000","last_change":"2024-09-17T05:23:25.083629+0000","last_active":"2024-09-17T05:23:26.070179+0000","last_peered":"2024-09-17T05:23:26.070179+0000","last_clean":"2024-09-17T05:23:26.070179+0000","last_became_active":"2024-09-17T05:23:25.083394+0000","last_became_peered":"2024-09-17T05:23:25.083394+0000","last_unstale":"2024-09-17T05:23:26.070179+0000","last_undegraded":"2024-09-17T05:23:26.070179+0000","last_fullsized":"2024-09-17T05:23:26.070179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:22.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:25.072561+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:25.070646+0000","last_became_peered":"2024-09-17T05:23:25.070646+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:07.163083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069443+0000","last_change":"2024-09-17T05:23:25.073264+0000","last_active":"2024-09-17T05:23:26.069443+0000","last_peered":"2024-09-17T05:23:26.069443+0000","last_clean":"2024-09-17T05:23:26.069443+0000","last_became_active":"2024-09-17T05:23:25.072874+0000","last_became_peered":"2024-09-17T05:23:25.072874+0000","last_unstale":"2024-09-17T05:23:26.069443+0000","last_undegraded":"2024-09-17T05:23:26.069443+0000","last_fullsized":"2024-09-17T05:23:26.069443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:07.432592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053114+0000","last_change":"2024-09-17T05:23:25.075408+0000","last_active":"2024-09-17T05:23:26.053114+0000","last_peered":"2024-09-17T05:23:26.053114+0000","last_clean":"2024-09-17T05:23:26.053114+0000","last_became_active":"2024-09-17T05:23:25.075101+0000","last_became_peered":"2024-09-17T05:23:25.075101+0000","last_unstale":"2024-09-17T05:23:26.053114+0000","last_undegraded":"2024-09-17T05:23:26.053114+0000","last_fullsized":"2024-09-17T05:23:26.053114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071351+0000","last_change":"2024-09-17T05:23:25.068874+0000","last_active":"2024-09-17T05:23:26.071351+0000","last_peered":"2024-09-17T05:23:26.071351+0000","last_clean":"2024-09-17T05:23:26.071351+0000","last_became_active":"2024-09-17T05:23:25.064181+0000","last_became_peered":"2024-09-17T05:23:25.064181+0000","last_unstale":"2024-09-17T05:23:26.071351+0000","last_undegraded":"2024-09-17T05:23:26.071351+0000","last_fullsized":"2024-09-17T05:23:26.071351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:43.495535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068665+0000","last_change":"2024-09-17T05:23:25.064004+0000","last_active":"2024-09-17T05:23:26.068665+0000","last_peered":"2024-09-17T05:23:26.068665+0000","last_clean":"2024-09-17T05:23:26.068665+0000","last_became_active":"2024-09-17T05:23:25.063599+0000","last_became_peered":"2024-09-17T05:23:25.063599+0000","last_unstale":"2024-09-17T05:23:26.068665+0000","last_undegraded":"2024-09-17T05:23:26.068665+0000","last_fullsized":"2024-09-17T05:23:26.068665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.655466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069668+0000","last_change":"2024-09-17T05:23:25.071820+0000","last_active":"2024-09-17T05:23:26.069668+0000","last_peered":"2024-09-17T05:23:26.069668+0000","last_clean":"2024-09-17T05:23:26.069668+0000","last_became_active":"2024-09-17T05:23:25.070565+0000","last_became_peered":"2024-09-17T05:23:25.070565+0000","last_unstale":"2024-09-17T05:23:26.069668+0000","last_undegraded":"2024-09-17T05:23:26.069668+0000","last_fullsized":"2024-09-17T05:23:26.069668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:53.029830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054175+0000","last_change":"2024-09-17T05:23:25.064336+0000","last_active":"2024-09-17T05:23:26.054175+0000","last_peered":"2024-09-17T05:23:26.054175+0000","last_clean":"2024-09-17T05:23:26.054175+0000","last_became_active":"2024-09-17T05:23:25.063789+0000","last_became_peered":"2024-09-17T05:23:25.063789+0000","last_unstale":"2024-09-17T05:23:26.054175+0000","last_undegraded":"2024-09-17T05:23:26.054175+0000","last_fullsized":"2024-09-17T05:23:26.054175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:27.415615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052848+0000","last_change":"2024-09-17T05:23:24.057092+0000","last_active":"2024-09-17T05:23:26.052848+0000","last_peered":"2024-09-17T05:23:26.052848+0000","last_clean":"2024-09-17T05:23:26.052848+0000","last_became_active":"2024-09-17T05:23:24.056017+0000","last_became_peered":"2024-09-17T05:23:24.056017+0000","last_unstale":"2024-09-17T05:23:26.052848+0000","last_undegraded":"2024-09-17T05:23:26.052848+0000","last_fullsized":"2024-09-17T05:23:26.052848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:21.643758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070140+0000","last_change":"2024-09-17T05:23:25.050194+0000","last_active":"2024-09-17T05:23:26.070140+0000","last_peered":"2024-09-17T05:23:26.070140+0000","last_clean":"2024-09-17T05:23:26.070140+0000","last_became_active":"2024-09-17T05:23:25.049834+0000","last_became_peered":"2024-09-17T05:23:25.049834+0000","last_unstale":"2024-09-17T05:23:26.070140+0000","last_undegraded":"2024-09-17T05:23:26.070140+0000","last_fullsized":"2024-09-17T05:23:26.070140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071221+0000","last_change":"2024-09-17T05:23:24.060034+0000","last_active":"2024-09-17T05:23:26.071221+0000","last_peered":"2024-09-17T05:23:26.071221+0000","last_clean":"2024-09-17T05:23:26.071221+0000","last_became_active":"2024-09-17T05:23:24.059693+0000","last_became_peered":"2024-09-17T05:23:24.059693+0000","last_unstale":"2024-09-17T05:23:26.071221+0000","last_undegraded":"2024-09-17T05:23:26.071221+0000","last_fullsized":"2024-09-17T05:23:26.071221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:09.436373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069091+0000","last_change":"2024-09-17T05:23:24.061263+0000","last_active":"2024-09-17T05:23:26.069091+0000","last_peered":"2024-09-17T05:23:26.069091+0000","last_clean":"2024-09-17T05:23:26.069091+0000","last_became_active":"2024-09-17T05:23:24.060910+0000","last_became_peered":"2024-09-17T05:23:24.060910+0000","last_unstale":"2024-09-17T05:23:26.069091+0000","last_undegraded":"2024-09-17T05:23:26.069091+0000","last_fullsized":"2024-09-17T05:23:26.069091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:54.990975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052733+0000","last_change":"2024-09-17T05:23:25.066266+0000","last_active":"2024-09-17T05:23:26.052733+0000","last_peered":"2024-09-17T05:23:26.052733+0000","last_clean":"2024-09-17T05:23:26.052733+0000","last_became_active":"2024-09-17T05:23:25.064050+0000","last_became_peered":"2024-09-17T05:23:25.064050+0000","last_unstale":"2024-09-17T05:23:26.052733+0000","last_undegraded":"2024-09-17T05:23:26.052733+0000","last_fullsized":"2024-09-17T05:23:26.052733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:23.536679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070980+0000","last_change":"2024-09-17T05:23:24.052942+0000","last_active":"2024-09-17T05:23:26.070980+0000","last_peered":"2024-09-17T05:23:26.070980+0000","last_clean":"2024-09-17T05:23:26.070980+0000","last_became_active":"2024-09-17T05:23:24.052628+0000","last_became_peered":"2024-09-17T05:23:24.052628+0000","last_unstale":"2024-09-17T05:23:26.070980+0000","last_undegraded":"2024-09-17T05:23:26.070980+0000","last_fullsized":"2024-09-17T05:23:26.070980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:13.647679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069032+0000","last_change":"2024-09-17T05:23:25.062501+0000","last_active":"2024-09-17T05:23:26.069032+0000","last_peered":"2024-09-17T05:23:26.069032+0000","last_clean":"2024-09-17T05:23:26.069032+0000","last_became_active":"2024-09-17T05:23:25.062195+0000","last_became_peered":"2024-09-17T05:23:25.062195+0000","last_unstale":"2024-09-17T05:23:26.069032+0000","last_undegraded":"2024-09-17T05:23:26.069032+0000","last_fullsized":"2024-09-17T05:23:26.069032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:28.768705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054381+0000","last_change":"2024-09-17T05:23:24.051900+0000","last_active":"2024-09-17T05:23:26.054381+0000","last_peered":"2024-09-17T05:23:26.054381+0000","last_clean":"2024-09-17T05:23:26.054381+0000","last_became_active":"2024-09-17T05:23:24.051525+0000","last_became_peered":"2024-09-17T05:23:24.051525+0000","last_unstale":"2024-09-17T05:23:26.054381+0000","last_undegraded":"2024-09-17T05:23:26.054381+0000","last_fullsized":"2024-09-17T05:23:26.054381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070296+0000","last_change":"2024-09-17T05:23:25.074116+0000","last_active":"2024-09-17T05:23:26.070296+0000","last_peered":"2024-09-17T05:23:26.070296+0000","last_clean":"2024-09-17T05:23:26.070296+0000","last_became_active":"2024-09-17T05:23:25.073750+0000","last_became_peered":"2024-09-17T05:23:25.073750+0000","last_unstale":"2024-09-17T05:23:26.070296+0000","last_undegraded":"2024-09-17T05:23:26.070296+0000","last_fullsized":"2024-09-17T05:23:26.070296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:23.398558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052816+0000","last_change":"2024-09-17T05:23:24.058892+0000","last_active":"2024-09-17T05:23:26.052816+0000","last_peered":"2024-09-17T05:23:26.052816+0000","last_clean":"2024-09-17T05:23:26.052816+0000","last_became_active":"2024-09-17T05:23:24.057777+0000","last_became_peered":"2024-09-17T05:23:24.057777+0000","last_unstale":"2024-09-17T05:23:26.052816+0000","last_undegraded":"2024-09-17T05:23:26.052816+0000","last_fullsized":"2024-09-17T05:23:26.052816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:52.022043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053572+0000","last_change":"2024-09-17T05:23:25.065444+0000","last_active":"2024-09-17T05:23:26.053572+0000","last_peered":"2024-09-17T05:23:26.053572+0000","last_clean":"2024-09-17T05:23:26.053572+0000","last_became_active":"2024-09-17T05:23:25.064932+0000","last_became_peered":"2024-09-17T05:23:25.064932+0000","last_unstale":"2024-09-17T05:23:26.053572+0000","last_undegraded":"2024-09-17T05:23:26.053572+0000","last_fullsized":"2024-09-17T05:23:26.053572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:38.125021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069641+0000","last_change":"2024-09-17T05:23:24.058300+0000","last_active":"2024-09-17T05:23:26.069641+0000","last_peered":"2024-09-17T05:23:26.069641+0000","last_clean":"2024-09-17T05:23:26.069641+0000","last_became_active":"2024-09-17T05:23:24.057127+0000","last_became_peered":"2024-09-17T05:23:24.057127+0000","last_unstale":"2024-09-17T05:23:26.069641+0000","last_undegraded":"2024-09-17T05:23:26.069641+0000","last_fullsized":"2024-09-17T05:23:26.069641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071142+0000","last_change":"2024-09-17T05:23:25.067068+0000","last_active":"2024-09-17T05:23:26.071142+0000","last_peered":"2024-09-17T05:23:26.071142+0000","last_clean":"2024-09-17T05:23:26.071142+0000","last_became_active":"2024-09-17T05:23:25.066755+0000","last_became_peered":"2024-09-17T05:23:25.066755+0000","last_unstale":"2024-09-17T05:23:26.071142+0000","last_undegraded":"2024-09-17T05:23:26.071142+0000","last_fullsized":"2024-09-17T05:23:26.071142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:11.227030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069906+0000","last_change":"2024-09-17T05:23:24.053733+0000","last_active":"2024-09-17T05:23:26.069906+0000","last_peered":"2024-09-17T05:23:26.069906+0000","last_clean":"2024-09-17T05:23:26.069906+0000","last_became_active":"2024-09-17T05:23:24.053370+0000","last_became_peered":"2024-09-17T05:23:24.053370+0000","last_unstale":"2024-09-17T05:23:26.069906+0000","last_undegraded":"2024-09-17T05:23:26.069906+0000","last_fullsized":"2024-09-17T05:23:26.069906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.457450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052622+0000","last_change":"2024-09-17T05:23:25.069241+0000","last_active":"2024-09-17T05:23:26.052622+0000","last_peered":"2024-09-17T05:23:26.052622+0000","last_clean":"2024-09-17T05:23:26.052622+0000","last_became_active":"2024-09-17T05:23:25.067530+0000","last_became_peered":"2024-09-17T05:23:25.067530+0000","last_unstale":"2024-09-17T05:23:26.052622+0000","last_undegraded":"2024-09-17T05:23:26.052622+0000","last_fullsized":"2024-09-17T05:23:26.052622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:35.751431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069422+0000","last_change":"2024-09-17T05:23:24.069042+0000","last_active":"2024-09-17T05:23:26.069422+0000","last_peered":"2024-09-17T05:23:26.069422+0000","last_clean":"2024-09-17T05:23:26.069422+0000","last_became_active":"2024-09-17T05:23:24.068629+0000","last_became_peered":"2024-09-17T05:23:24.068629+0000","last_unstale":"2024-09-17T05:23:26.069422+0000","last_undegraded":"2024-09-17T05:23:26.069422+0000","last_fullsized":"2024-09-17T05:23:26.069422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:54.665127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069223+0000","last_change":"2024-09-17T05:23:25.072397+0000","last_active":"2024-09-17T05:23:26.069223+0000","last_peered":"2024-09-17T05:23:26.069223+0000","last_clean":"2024-09-17T05:23:26.069223+0000","last_became_active":"2024-09-17T05:23:25.072038+0000","last_became_peered":"2024-09-17T05:23:25.072038+0000","last_unstale":"2024-09-17T05:23:26.069223+0000","last_undegraded":"2024-09-17T05:23:26.069223+0000","last_fullsized":"2024-09-17T05:23:26.069223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:25.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071355+0000","last_change":"2024-09-17T05:23:24.059071+0000","last_active":"2024-09-17T05:23:26.071355+0000","last_peered":"2024-09-17T05:23:26.071355+0000","last_clean":"2024-09-17T05:23:26.071355+0000","last_became_active":"2024-09-17T05:23:24.058788+0000","last_became_peered":"2024-09-17T05:23:24.058788+0000","last_unstale":"2024-09-17T05:23:26.071355+0000","last_undegraded":"2024-09-17T05:23:26.071355+0000","last_fullsized":"2024-09-17T05:23:26.071355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:26.407828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071163+0000","last_change":"2024-09-17T05:23:25.070863+0000","last_active":"2024-09-17T05:23:26.071163+0000","last_peered":"2024-09-17T05:23:26.071163+0000","last_clean":"2024-09-17T05:23:26.071163+0000","last_became_active":"2024-09-17T05:23:25.070508+0000","last_became_peered":"2024-09-17T05:23:25.070508+0000","last_unstale":"2024-09-17T05:23:26.071163+0000","last_undegraded":"2024-09-17T05:23:26.071163+0000","last_fullsized":"2024-09-17T05:23:26.071163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:08.540151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052361+0000","last_change":"2024-09-17T05:23:24.054907+0000","last_active":"2024-09-17T05:23:26.052361+0000","last_peered":"2024-09-17T05:23:26.052361+0000","last_clean":"2024-09-17T05:23:26.052361+0000","last_became_active":"2024-09-17T05:23:24.054532+0000","last_became_peered":"2024-09-17T05:23:24.054532+0000","last_unstale":"2024-09-17T05:23:26.052361+0000","last_undegraded":"2024-09-17T05:23:26.052361+0000","last_fullsized":"2024-09-17T05:23:26.052361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053550+0000","last_change":"2024-09-17T05:23:25.072781+0000","last_active":"2024-09-17T05:23:26.053550+0000","last_peered":"2024-09-17T05:23:26.053550+0000","last_clean":"2024-09-17T05:23:26.053550+0000","last_became_active":"2024-09-17T05:23:25.072303+0000","last_became_peered":"2024-09-17T05:23:25.072303+0000","last_unstale":"2024-09-17T05:23:26.053550+0000","last_undegraded":"2024-09-17T05:23:26.053550+0000","last_fullsized":"2024-09-17T05:23:26.053550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068751+0000","last_change":"2024-09-17T05:23:24.050270+0000","last_active":"2024-09-17T05:23:26.068751+0000","last_peered":"2024-09-17T05:23:26.068751+0000","last_clean":"2024-09-17T05:23:26.068751+0000","last_became_active":"2024-09-17T05:23:24.049787+0000","last_became_peered":"2024-09-17T05:23:24.049787+0000","last_unstale":"2024-09-17T05:23:26.068751+0000","last_undegraded":"2024-09-17T05:23:26.068751+0000","last_fullsized":"2024-09-17T05:23:26.068751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071171+0000","last_change":"2024-09-17T05:23:25.065390+0000","last_active":"2024-09-17T05:23:26.071171+0000","last_peered":"2024-09-17T05:23:26.071171+0000","last_clean":"2024-09-17T05:23:26.071171+0000","last_became_active":"2024-09-17T05:23:25.065054+0000","last_became_peered":"2024-09-17T05:23:25.065054+0000","last_unstale":"2024-09-17T05:23:26.071171+0000","last_undegraded":"2024-09-17T05:23:26.071171+0000","last_fullsized":"2024-09-17T05:23:26.071171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:55.059902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053136+0000","last_change":"2024-09-17T05:23:24.066652+0000","last_active":"2024-09-17T05:23:26.053136+0000","last_peered":"2024-09-17T05:23:26.053136+0000","last_clean":"2024-09-17T05:23:26.053136+0000","last_became_active":"2024-09-17T05:23:24.065549+0000","last_became_peered":"2024-09-17T05:23:24.065549+0000","last_unstale":"2024-09-17T05:23:26.053136+0000","last_undegraded":"2024-09-17T05:23:26.053136+0000","last_fullsized":"2024-09-17T05:23:26.053136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:44.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070411+0000","last_change":"2024-09-17T05:23:25.074083+0000","last_active":"2024-09-17T05:23:26.070411+0000","last_peered":"2024-09-17T05:23:26.070411+0000","last_clean":"2024-09-17T05:23:26.070411+0000","last_became_active":"2024-09-17T05:23:25.073740+0000","last_became_peered":"2024-09-17T05:23:25.073740+0000","last_unstale":"2024-09-17T05:23:26.070411+0000","last_undegraded":"2024-09-17T05:23:26.070411+0000","last_fullsized":"2024-09-17T05:23:26.070411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:15.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070325+0000","last_change":"2024-09-17T05:23:24.060840+0000","last_active":"2024-09-17T05:23:26.070325+0000","last_peered":"2024-09-17T05:23:26.070325+0000","last_clean":"2024-09-17T05:23:26.070325+0000","last_became_active":"2024-09-17T05:23:24.060518+0000","last_became_peered":"2024-09-17T05:23:24.060518+0000","last_unstale":"2024-09-17T05:23:26.070325+0000","last_undegraded":"2024-09-17T05:23:26.070325+0000","last_fullsized":"2024-09-17T05:23:26.070325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:47.457533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053110+0000","last_change":"2024-09-17T05:23:25.067357+0000","last_active":"2024-09-17T05:23:26.053110+0000","last_peered":"2024-09-17T05:23:26.053110+0000","last_clean":"2024-09-17T05:23:26.053110+0000","last_became_active":"2024-09-17T05:23:25.067017+0000","last_became_peered":"2024-09-17T05:23:25.067017+0000","last_unstale":"2024-09-17T05:23:26.053110+0000","last_undegraded":"2024-09-17T05:23:26.053110+0000","last_fullsized":"2024-09-17T05:23:26.053110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:05.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052624+0000","last_change":"2024-09-17T05:23:24.058023+0000","last_active":"2024-09-17T05:23:26.052624+0000","last_peered":"2024-09-17T05:23:26.052624+0000","last_clean":"2024-09-17T05:23:26.052624+0000","last_became_active":"2024-09-17T05:23:24.057266+0000","last_became_peered":"2024-09-17T05:23:24.057266+0000","last_unstale":"2024-09-17T05:23:26.052624+0000","last_undegraded":"2024-09-17T05:23:26.052624+0000","last_fullsized":"2024-09-17T05:23:26.052624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053458+0000","last_change":"2024-09-17T05:23:25.051518+0000","last_active":"2024-09-17T05:23:26.053458+0000","last_peered":"2024-09-17T05:23:26.053458+0000","last_clean":"2024-09-17T05:23:26.053458+0000","last_became_active":"2024-09-17T05:23:25.051182+0000","last_became_peered":"2024-09-17T05:23:25.051182+0000","last_unstale":"2024-09-17T05:23:26.053458+0000","last_undegraded":"2024-09-17T05:23:26.053458+0000","last_fullsized":"2024-09-17T05:23:26.053458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069856+0000","last_change":"2024-09-17T05:23:24.060486+0000","last_active":"2024-09-17T05:23:26.069856+0000","last_peered":"2024-09-17T05:23:26.069856+0000","last_clean":"2024-09-17T05:23:26.069856+0000","last_became_active":"2024-09-17T05:23:24.060106+0000","last_became_peered":"2024-09-17T05:23:24.060106+0000","last_unstale":"2024-09-17T05:23:26.069856+0000","last_undegraded":"2024-09-17T05:23:26.069856+0000","last_fullsized":"2024-09-17T05:23:26.069856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:13.295786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071578+0000","last_change":"2024-09-17T05:23:25.085148+0000","last_active":"2024-09-17T05:23:26.071578+0000","last_peered":"2024-09-17T05:23:26.071578+0000","last_clean":"2024-09-17T05:23:26.071578+0000","last_became_active":"2024-09-17T05:23:25.083807+0000","last_became_peered":"2024-09-17T05:23:25.083807+0000","last_unstale":"2024-09-17T05:23:26.071578+0000","last_undegraded":"2024-09-17T05:23:26.071578+0000","last_fullsized":"2024-09-17T05:23:26.071578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:32.917183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053650+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:23:26.053650+0000","last_peered":"2024-09-17T05:23:26.053650+0000","last_clean":"2024-09-17T05:23:26.053650+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:23:26.053650+0000","last_undegraded":"2024-09-17T05:23:26.053650+0000","last_fullsized":"2024-09-17T05:23:26.053650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071284+0000","last_change":"2024-09-17T05:23:24.056449+0000","last_active":"2024-09-17T05:23:26.071284+0000","last_peered":"2024-09-17T05:23:26.071284+0000","last_clean":"2024-09-17T05:23:26.071284+0000","last_became_active":"2024-09-17T05:23:24.056055+0000","last_became_peered":"2024-09-17T05:23:24.056055+0000","last_unstale":"2024-09-17T05:23:26.071284+0000","last_undegraded":"2024-09-17T05:23:26.071284+0000","last_fullsized":"2024-09-17T05:23:26.071284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.624751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069696+0000","last_change":"2024-09-17T05:23:25.055754+0000","last_active":"2024-09-17T05:23:26.069696+0000","last_peered":"2024-09-17T05:23:26.069696+0000","last_clean":"2024-09-17T05:23:26.069696+0000","last_became_active":"2024-09-17T05:23:25.055368+0000","last_became_peered":"2024-09-17T05:23:25.055368+0000","last_unstale":"2024-09-17T05:23:26.069696+0000","last_undegraded":"2024-09-17T05:23:26.069696+0000","last_fullsized":"2024-09-17T05:23:26.069696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:35.911660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070994+0000","last_change":"2024-09-17T05:23:24.065017+0000","last_active":"2024-09-17T05:23:26.070994+0000","last_peered":"2024-09-17T05:23:26.070994+0000","last_clean":"2024-09-17T05:23:26.070994+0000","last_became_active":"2024-09-17T05:23:24.064688+0000","last_became_peered":"2024-09-17T05:23:24.064688+0000","last_unstale":"2024-09-17T05:23:26.070994+0000","last_undegraded":"2024-09-17T05:23:26.070994+0000","last_fullsized":"2024-09-17T05:23:26.070994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:38.132272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068980+0000","last_change":"2024-09-17T05:23:25.078079+0000","last_active":"2024-09-17T05:23:26.068980+0000","last_peered":"2024-09-17T05:23:26.068980+0000","last_clean":"2024-09-17T05:23:26.068980+0000","last_became_active":"2024-09-17T05:23:25.077756+0000","last_became_peered":"2024-09-17T05:23:25.077756+0000","last_unstale":"2024-09-17T05:23:26.068980+0000","last_undegraded":"2024-09-17T05:23:26.068980+0000","last_fullsized":"2024-09-17T05:23:26.068980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:01.476383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068472+0000","last_change":"2024-09-17T05:23:24.065406+0000","last_active":"2024-09-17T05:23:26.068472+0000","last_peered":"2024-09-17T05:23:26.068472+0000","last_clean":"2024-09-17T05:23:26.068472+0000","last_became_active":"2024-09-17T05:23:24.065058+0000","last_became_peered":"2024-09-17T05:23:24.065058+0000","last_unstale":"2024-09-17T05:23:26.068472+0000","last_undegraded":"2024-09-17T05:23:26.068472+0000","last_fullsized":"2024-09-17T05:23:26.068472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:59.718163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053208+0000","last_change":"2024-09-17T05:23:25.074915+0000","last_active":"2024-09-17T05:23:26.053208+0000","last_peered":"2024-09-17T05:23:26.053208+0000","last_clean":"2024-09-17T05:23:26.053208+0000","last_became_active":"2024-09-17T05:23:25.074603+0000","last_became_peered":"2024-09-17T05:23:25.074603+0000","last_unstale":"2024-09-17T05:23:26.053208+0000","last_undegraded":"2024-09-17T05:23:26.053208+0000","last_fullsized":"2024-09-17T05:23:26.053208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:36.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071423+0000","last_change":"2024-09-17T05:23:24.058441+0000","last_active":"2024-09-17T05:23:26.071423+0000","last_peered":"2024-09-17T05:23:26.071423+0000","last_clean":"2024-09-17T05:23:26.071423+0000","last_became_active":"2024-09-17T05:23:24.054951+0000","last_became_peered":"2024-09-17T05:23:24.054951+0000","last_unstale":"2024-09-17T05:23:26.071423+0000","last_undegraded":"2024-09-17T05:23:26.071423+0000","last_fullsized":"2024-09-17T05:23:26.071423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:11.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052393+0000","last_change":"2024-09-17T05:23:25.056791+0000","last_active":"2024-09-17T05:23:26.052393+0000","last_peered":"2024-09-17T05:23:26.052393+0000","last_clean":"2024-09-17T05:23:26.052393+0000","last_became_active":"2024-09-17T05:23:25.056445+0000","last_became_peered":"2024-09-17T05:23:25.056445+0000","last_unstale":"2024-09-17T05:23:26.052393+0000","last_undegraded":"2024-09-17T05:23:26.052393+0000","last_fullsized":"2024-09-17T05:23:26.052393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.022012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052571+0000","last_change":"2024-09-17T05:23:24.058736+0000","last_active":"2024-09-17T05:23:26.052571+0000","last_peered":"2024-09-17T05:23:26.052571+0000","last_clean":"2024-09-17T05:23:26.052571+0000","last_became_active":"2024-09-17T05:23:24.056996+0000","last_became_peered":"2024-09-17T05:23:24.056996+0000","last_unstale":"2024-09-17T05:23:26.052571+0000","last_undegraded":"2024-09-17T05:23:26.052571+0000","last_fullsized":"2024-09-17T05:23:26.052571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:09.319432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070023+0000","last_change":"2024-09-17T05:23:25.072427+0000","last_active":"2024-09-17T05:23:26.070023+0000","last_peered":"2024-09-17T05:23:26.070023+0000","last_clean":"2024-09-17T05:23:26.070023+0000","last_became_active":"2024-09-17T05:23:25.071951+0000","last_became_peered":"2024-09-17T05:23:25.071951+0000","last_unstale":"2024-09-17T05:23:26.070023+0000","last_undegraded":"2024-09-17T05:23:26.070023+0000","last_fullsized":"2024-09-17T05:23:26.070023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:45.608054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071621+0000","last_change":"2024-09-17T05:23:24.064058+0000","last_active":"2024-09-17T05:23:26.071621+0000","last_peered":"2024-09-17T05:23:26.071621+0000","last_clean":"2024-09-17T05:23:26.071621+0000","last_became_active":"2024-09-17T05:23:24.063686+0000","last_became_peered":"2024-09-17T05:23:24.063686+0000","last_unstale":"2024-09-17T05:23:26.071621+0000","last_undegraded":"2024-09-17T05:23:26.071621+0000","last_fullsized":"2024-09-17T05:23:26.071621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:03.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069725+0000","last_change":"2024-09-17T05:23:25.070103+0000","last_active":"2024-09-17T05:23:26.069725+0000","last_peered":"2024-09-17T05:23:26.069725+0000","last_clean":"2024-09-17T05:23:26.069725+0000","last_became_active":"2024-09-17T05:23:25.069686+0000","last_became_peered":"2024-09-17T05:23:25.069686+0000","last_unstale":"2024-09-17T05:23:26.069725+0000","last_undegraded":"2024-09-17T05:23:26.069725+0000","last_fullsized":"2024-09-17T05:23:26.069725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:56.760702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070421+0000","last_change":"2024-09-17T05:23:24.061808+0000","last_active":"2024-09-17T05:23:26.070421+0000","last_peered":"2024-09-17T05:23:26.070421+0000","last_clean":"2024-09-17T05:23:26.070421+0000","last_became_active":"2024-09-17T05:23:24.061393+0000","last_became_peered":"2024-09-17T05:23:24.061393+0000","last_unstale":"2024-09-17T05:23:26.070421+0000","last_undegraded":"2024-09-17T05:23:26.070421+0000","last_fullsized":"2024-09-17T05:23:26.070421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:10.321148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070599+0000","last_change":"2024-09-17T05:23:25.065038+0000","last_active":"2024-09-17T05:23:26.070599+0000","last_peered":"2024-09-17T05:23:26.070599+0000","last_clean":"2024-09-17T05:23:26.070599+0000","last_became_active":"2024-09-17T05:23:25.063557+0000","last_became_peered":"2024-09-17T05:23:25.063557+0000","last_unstale":"2024-09-17T05:23:26.070599+0000","last_undegraded":"2024-09-17T05:23:26.070599+0000","last_fullsized":"2024-09-17T05:23:26.070599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:46.016555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071788+0000","last_change":"2024-09-17T05:23:24.061738+0000","last_active":"2024-09-17T05:23:26.071788+0000","last_peered":"2024-09-17T05:23:26.071788+0000","last_clean":"2024-09-17T05:23:26.071788+0000","last_became_active":"2024-09-17T05:23:24.061344+0000","last_became_peered":"2024-09-17T05:23:24.061344+0000","last_unstale":"2024-09-17T05:23:26.071788+0000","last_undegraded":"2024-09-17T05:23:26.071788+0000","last_fullsized":"2024-09-17T05:23:26.071788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:00.795235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070211+0000","last_change":"2024-09-17T05:23:25.059622+0000","last_active":"2024-09-17T05:23:26.070211+0000","last_peered":"2024-09-17T05:23:26.070211+0000","last_clean":"2024-09-17T05:23:26.070211+0000","last_became_active":"2024-09-17T05:23:25.059267+0000","last_became_peered":"2024-09-17T05:23:25.059267+0000","last_unstale":"2024-09-17T05:23:26.070211+0000","last_undegraded":"2024-09-17T05:23:26.070211+0000","last_fullsized":"2024-09-17T05:23:26.070211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:58.979778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.051888+0000","last_change":"2024-09-17T05:23:24.053175+0000","last_active":"2024-09-17T05:23:26.051888+0000","last_peered":"2024-09-17T05:23:26.051888+0000","last_clean":"2024-09-17T05:23:26.051888+0000","last_became_active":"2024-09-17T05:23:24.052859+0000","last_became_peered":"2024-09-17T05:23:24.052859+0000","last_unstale":"2024-09-17T05:23:26.051888+0000","last_undegraded":"2024-09-17T05:23:26.051888+0000","last_fullsized":"2024-09-17T05:23:26.051888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:13.871334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071095+0000","last_change":"2024-09-17T05:23:25.057826+0000","last_active":"2024-09-17T05:23:26.071095+0000","last_peered":"2024-09-17T05:23:26.071095+0000","last_clean":"2024-09-17T05:23:26.071095+0000","last_became_active":"2024-09-17T05:23:25.057507+0000","last_became_peered":"2024-09-17T05:23:25.057507+0000","last_unstale":"2024-09-17T05:23:26.071095+0000","last_undegraded":"2024-09-17T05:23:26.071095+0000","last_fullsized":"2024-09-17T05:23:26.071095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.095733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068904+0000","last_change":"2024-09-17T05:23:24.068858+0000","last_active":"2024-09-17T05:23:26.068904+0000","last_peered":"2024-09-17T05:23:26.068904+0000","last_clean":"2024-09-17T05:23:26.068904+0000","last_became_active":"2024-09-17T05:23:24.068387+0000","last_became_peered":"2024-09-17T05:23:24.068387+0000","last_unstale":"2024-09-17T05:23:26.068904+0000","last_undegraded":"2024-09-17T05:23:26.068904+0000","last_fullsized":"2024-09-17T05:23:26.068904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:44.504657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.068658+0000","last_change":"2024-09-17T05:23:25.049296+0000","last_active":"2024-09-17T05:23:26.068658+0000","last_peered":"2024-09-17T05:23:26.068658+0000","last_clean":"2024-09-17T05:23:26.068658+0000","last_became_active":"2024-09-17T05:23:25.048948+0000","last_became_peered":"2024-09-17T05:23:25.048948+0000","last_unstale":"2024-09-17T05:23:26.068658+0000","last_undegraded":"2024-09-17T05:23:26.068658+0000","last_fullsized":"2024-09-17T05:23:26.068658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:49.995105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069373+0000","last_change":"2024-09-17T05:23:24.061471+0000","last_active":"2024-09-17T05:23:26.069373+0000","last_peered":"2024-09-17T05:23:26.069373+0000","last_clean":"2024-09-17T05:23:26.069373+0000","last_became_active":"2024-09-17T05:23:24.061050+0000","last_became_peered":"2024-09-17T05:23:24.061050+0000","last_unstale":"2024-09-17T05:23:26.069373+0000","last_undegraded":"2024-09-17T05:23:26.069373+0000","last_fullsized":"2024-09-17T05:23:26.069373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:44.926336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.069337+0000","last_change":"2024-09-17T05:23:25.067211+0000","last_active":"2024-09-17T05:23:26.069337+0000","last_peered":"2024-09-17T05:23:26.069337+0000","last_clean":"2024-09-17T05:23:26.069337+0000","last_became_active":"2024-09-17T05:23:25.066883+0000","last_became_peered":"2024-09-17T05:23:25.066883+0000","last_unstale":"2024-09-17T05:23:26.069337+0000","last_undegraded":"2024-09-17T05:23:26.069337+0000","last_fullsized":"2024-09-17T05:23:26.069337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:39.140059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071212+0000","last_change":"2024-09-17T05:23:24.061156+0000","last_active":"2024-09-17T05:23:26.071212+0000","last_peered":"2024-09-17T05:23:26.071212+0000","last_clean":"2024-09-17T05:23:26.071212+0000","last_became_active":"2024-09-17T05:23:24.060751+0000","last_became_peered":"2024-09-17T05:23:24.060751+0000","last_unstale":"2024-09-17T05:23:26.071212+0000","last_undegraded":"2024-09-17T05:23:26.071212+0000","last_fullsized":"2024-09-17T05:23:26.071212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.506573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071131+0000","last_change":"2024-09-17T05:23:25.085054+0000","last_active":"2024-09-17T05:23:26.071131+0000","last_peered":"2024-09-17T05:23:26.071131+0000","last_clean":"2024-09-17T05:23:26.071131+0000","last_became_active":"2024-09-17T05:23:25.084307+0000","last_became_peered":"2024-09-17T05:23:25.084307+0000","last_unstale":"2024-09-17T05:23:26.071131+0000","last_undegraded":"2024-09-17T05:23:26.071131+0000","last_fullsized":"2024-09-17T05:23:26.071131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:58.367183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.071557+0000","last_change":"2024-09-17T05:23:25.052805+0000","last_active":"2024-09-17T05:23:26.071557+0000","last_peered":"2024-09-17T05:23:26.071557+0000","last_clean":"2024-09-17T05:23:26.071557+0000","last_became_active":"2024-09-17T05:23:25.052440+0000","last_became_peered":"2024-09-17T05:23:25.052440+0000","last_unstale":"2024-09-17T05:23:26.071557+0000","last_undegraded":"2024-09-17T05:23:26.071557+0000","last_fullsized":"2024-09-17T05:23:26.071557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:51.739703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053334+0000","last_change":"2024-09-17T05:23:24.061930+0000","last_active":"2024-09-17T05:23:26.053334+0000","last_peered":"2024-09-17T05:23:26.053334+0000","last_clean":"2024-09-17T05:23:26.053334+0000","last_became_active":"2024-09-17T05:23:24.060988+0000","last_became_peered":"2024-09-17T05:23:24.060988+0000","last_unstale":"2024-09-17T05:23:26.053334+0000","last_undegraded":"2024-09-17T05:23:26.053334+0000","last_fullsized":"2024-09-17T05:23:26.053334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:47.573508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.054561+0000","last_change":"2024-09-17T05:23:25.065743+0000","last_active":"2024-09-17T05:23:26.054561+0000","last_peered":"2024-09-17T05:23:26.054561+0000","last_clean":"2024-09-17T05:23:26.054561+0000","last_became_active":"2024-09-17T05:23:25.062840+0000","last_became_peered":"2024-09-17T05:23:25.062840+0000","last_unstale":"2024-09-17T05:23:26.054561+0000","last_undegraded":"2024-09-17T05:23:26.054561+0000","last_fullsized":"2024-09-17T05:23:26.054561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:01.803022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.052543+0000","last_change":"2024-09-17T05:23:24.060124+0000","last_active":"2024-09-17T05:23:26.052543+0000","last_peered":"2024-09-17T05:23:26.052543+0000","last_clean":"2024-09-17T05:23:26.052543+0000","last_became_active":"2024-09-17T05:23:24.057538+0000","last_became_peered":"2024-09-17T05:23:24.057538+0000","last_unstale":"2024-09-17T05:23:26.052543+0000","last_undegraded":"2024-09-17T05:23:26.052543+0000","last_fullsized":"2024-09-17T05:23:26.052543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:41.202615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053337+0000","last_change":"2024-09-17T05:23:25.079567+0000","last_active":"2024-09-17T05:23:26.053337+0000","last_peered":"2024-09-17T05:23:26.053337+0000","last_clean":"2024-09-17T05:23:26.053337+0000","last_became_active":"2024-09-17T05:23:25.078899+0000","last_became_peered":"2024-09-17T05:23:25.078899+0000","last_unstale":"2024-09-17T05:23:26.053337+0000","last_undegraded":"2024-09-17T05:23:26.053337+0000","last_fullsized":"2024-09-17T05:23:26.053337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:39.680169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053486+0000","last_change":"2024-09-17T05:23:24.061891+0000","last_active":"2024-09-17T05:23:26.053486+0000","last_peered":"2024-09-17T05:23:26.053486+0000","last_clean":"2024-09-17T05:23:26.053486+0000","last_became_active":"2024-09-17T05:23:24.055527+0000","last_became_peered":"2024-09-17T05:23:24.055527+0000","last_unstale":"2024-09-17T05:23:26.053486+0000","last_undegraded":"2024-09-17T05:23:26.053486+0000","last_fullsized":"2024-09-17T05:23:26.053486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:06.652971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.053220+0000","last_change":"2024-09-17T05:23:25.076838+0000","last_active":"2024-09-17T05:23:26.053220+0000","last_peered":"2024-09-17T05:23:26.053220+0000","last_clean":"2024-09-17T05:23:26.053220+0000","last_became_active":"2024-09-17T05:23:25.076547+0000","last_became_peered":"2024-09-17T05:23:25.076547+0000","last_unstale":"2024-09-17T05:23:26.053220+0000","last_undegraded":"2024-09-17T05:23:26.053220+0000","last_fullsized":"2024-09-17T05:23:26.053220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:24.024374+0000","last_clean_scrub_stamp":"2024-09-17T05:23:24.024374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.142652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:23:26.070483+0000","last_change":"2024-09-17T05:23:24.059741+0000","last_active":"2024-09-17T05:23:26.070483+0000","last_peered":"2024-09-17T05:23:26.070483+0000","last_clean":"2024-09-17T05:23:26.070483+0000","last_became_active":"2024-09-17T05:23:24.058336+0000","last_became_peered":"2024-09-17T05:23:24.058336+0000","last_unstale":"2024-09-17T05:23:26.070483+0000","last_undegraded":"2024-09-17T05:23:26.070483+0000","last_fullsized":"2024-09-17T05:23:26.070483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:23.016587+0000","last_clean_scrub_stamp":"2024-09-17T05:23:23.016587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:01.640938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:23:39.670 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:23:39.670 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T05:23:39.670 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T05:23:39.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:23:39.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.864+0000 7fce51a00640 1 -- 172.21.15.43:0/2496451816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c09ba60 msgr2=0x7fce4c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.864+0000 7fce51a00640 1 --2- 172.21.15.43:0/2496451816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c09ba60 0x7fce4c0b3130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009900 tx=0x7fce3c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- 172.21.15.43:0/2496451816 shutdown_connections 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 --2- 172.21.15.43:0/2496451816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c09ba60 0x7fce4c0b3130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- 172.21.15.43:0/2496451816 >> 172.21.15.43:0/2496451816 conn(0x7fce4c0cee60 msgr2=0x7fce4c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- 172.21.15.43:0/2496451816 shutdown_connections 2024-09-17T05:23:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- 172.21.15.43:0/2496451816 wait complete. 2024-09-17T05:23:39.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 Processor -- start 2024-09-17T05:23:39.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- start start 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 0x7fce4c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c0d0650 0x7fce4c1f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c16b280 con 0x7fce4c0d0240 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c16b100 con 0x7fce4c0d0650 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.868+0000 7fce51a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c16b400 con 0x7fce4c09ba60 2024-09-17T05:23:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41810/0 (socket says 172.21.15.43:41810) 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 -- 172.21.15.43:0/2709791852 learned_addr learned my addr 172.21.15.43:0/2709791852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce4aa00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 0x7fce4c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce4b400640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c0d0650 0x7fce4c1f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 -- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 msgr2=0x7fce4c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 0x7fce4c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 -- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c0d0650 msgr2=0x7fce4c1f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c0d0650 0x7fce4c1f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 -- 172.21.15.43:0/2709791852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c1fa0d0 con 0x7fce4c0d0240 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce4aa00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 0x7fce4c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce43e00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c390 tx=0x7fce3400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34017070 con 0x7fce4c0d0240 2024-09-17T05:23:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce340053c0 con 0x7fce4c0d0240 2024-09-17T05:23:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34003840 con 0x7fce4c0d0240 2024-09-17T05:23:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c154af0 con 0x7fce4c0d0240 2024-09-17T05:23:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c14fe80 con 0x7fce4c0d0240 2024-09-17T05:23:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c0b3130 con 0x7fce4c0d0240 2024-09-17T05:23:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.872+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce3401a070 con 0x7fce4c0d0240 2024-09-17T05:23:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.876+0000 7fce42a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.876+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fce340934f0 con 0x7fce4c0d0240 2024-09-17T05:23:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.876+0000 7fce4aa00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:39.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.876+0000 7fce4aa00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 0x7fce28075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0af000 tx=0x7fce3c01b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:39.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:39.880+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce3405e590 con 0x7fce4c0d0240 2024-09-17T05:23:40.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fce4c1f9e60 con 0x7fce4c0d0240 2024-09-17T05:23:40.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce42a00640 1 -- 172.21.15.43:0/2709791852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fce3405de90 con 0x7fce4c0d0240 2024-09-17T05:23:40.094 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:40.094 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T05:23:40.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 msgr2=0x7fce28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 0x7fce28075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0af000 tx=0x7fce3c01b120 comp rx=0 tx=0).stop 2024-09-17T05:23:40.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 msgr2=0x7fce4c1f9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c390 tx=0x7fce3400c850 comp rx=0 tx=0).stop 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 shutdown_connections 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c0d0650 0x7fce4c1f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4c0d0240 0x7fce4c1f9380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 --2- 172.21.15.43:0/2709791852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce4c09ba60 0x7fce4c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 >> 172.21.15.43:0/2709791852 conn(0x7fce4c0cee60 msgr2=0x7fce4c155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 shutdown_connections 2024-09-17T05:23:40.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.096+0000 7fce51a00640 1 -- 172.21.15.43:0/2709791852 wait complete. 2024-09-17T05:23:40.117 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T05:23:40.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 -- 172.21.15.43:0/4259292108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf480b1a70 msgr2=0x7fbf480b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 --2- 172.21.15.43:0/4259292108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf480b1a70 0x7fbf480b1e80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009c80 tx=0x7fbf4001cce0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 -- 172.21.15.43:0/4259292108 shutdown_connections 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 --2- 172.21.15.43:0/4259292108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf480b1a70 0x7fbf480b1e80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 --2- 172.21.15.43:0/4259292108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf48070d60 0x7fbf480b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 -- 172.21.15.43:0/4259292108 >> 172.21.15.43:0/4259292108 conn(0x7fbf480b24c0 msgr2=0x7fbf480b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 -- 172.21.15.43:0/4259292108 shutdown_connections 2024-09-17T05:23:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.276+0000 7fbf4d800640 1 -- 172.21.15.43:0/4259292108 wait complete. 2024-09-17T05:23:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 Processor -- start 2024-09-17T05:23:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- start start 2024-09-17T05:23:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf480b1a70 0x7fbf481fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf480b44c0 0x7fbf48138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4813faf0 con 0x7fbf4815fbb0 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4813f970 con 0x7fbf480b1a70 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4813fc70 con 0x7fbf480b44c0 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41828/0 (socket says 172.21.15.43:41828) 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 -- 172.21.15.43:0/3110580268 learned_addr learned my addr 172.21.15.43:0/3110580268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf46a00640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf480b1a70 0x7fbf481fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf47400640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf480b44c0 0x7fbf48138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 -- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf480b44c0 msgr2=0x7fbf48138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf480b44c0 0x7fbf48138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 -- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf480b1a70 msgr2=0x7fbf481fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf480b1a70 0x7fbf481fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 -- 172.21.15.43:0/3110580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481fc680 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3e000640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d750 tx=0x7fbf3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf30009d20 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf481fc960 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30005280 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.280+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4813d420 con 0x7fbf4815fbb0 2024-09-17T05:23:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.284+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf48140180 con 0x7fbf4815fbb0 2024-09-17T05:23:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.284+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf30005420 con 0x7fbf4815fbb0 2024-09-17T05:23:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.284+0000 7fbf3f400640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 0x7fbf24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.284+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf300946f0 con 0x7fbf4815fbb0 2024-09-17T05:23:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.284+0000 7fbf46a00640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 0x7fbf24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.288+0000 7fbf46a00640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 0x7fbf24075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf380098e0 tx=0x7fbf38009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:40.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.288+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf3000de60 con 0x7fbf4815fbb0 2024-09-17T05:23:40.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.500+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf4806f540 con 0x7fbf4815fbb0 2024-09-17T05:23:40.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.500+0000 7fbf3f400640 1 -- 172.21.15.43:0/3110580268 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fbf3005f130 con 0x7fbf4815fbb0 2024-09-17T05:23:40.499 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:40.499 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 msgr2=0x7fbf24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 0x7fbf24075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf380098e0 tx=0x7fbf38009060 comp rx=0 tx=0).stop 2024-09-17T05:23:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 msgr2=0x7fbf481524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d750 tx=0x7fbf3000dc10 comp rx=0 tx=0).stop 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 shutdown_connections 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf24073200 0x7fbf24075710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf480b44c0 0x7fbf48138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf4815fbb0 0x7fbf481524f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 --2- 172.21.15.43:0/3110580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf480b1a70 0x7fbf481fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 >> 172.21.15.43:0/3110580268 conn(0x7fbf480b24c0 msgr2=0x7fbf480cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 shutdown_connections 2024-09-17T05:23:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.504+0000 7fbf4d800640 1 -- 172.21.15.43:0/3110580268 wait complete. 2024-09-17T05:23:40.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 -- 172.21.15.43:0/3410625333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec15c7e0 msgr2=0x7f60ec06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 --2- 172.21.15.43:0/3410625333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec15c7e0 0x7f60ec06f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009c80 tx=0x7f60e401cd10 comp rx=0 tx=0).stop 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 -- 172.21.15.43:0/3410625333 shutdown_connections 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 --2- 172.21.15.43:0/3410625333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec15c7e0 0x7f60ec06f110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 -- 172.21.15.43:0/3410625333 >> 172.21.15.43:0/3410625333 conn(0x7f60ec0b23d0 msgr2=0x7f60ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 -- 172.21.15.43:0/3410625333 shutdown_connections 2024-09-17T05:23:40.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.680+0000 7f60f4200640 1 -- 172.21.15.43:0/3410625333 wait complete. 2024-09-17T05:23:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 Processor -- start 2024-09-17T05:23:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 -- start start 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ec15c7e0 0x7f60ec0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 0x7f60ec156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec169e60 con 0x7f60ec155d70 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec169ce0 con 0x7f60ec15c7e0 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f4200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec169fe0 con 0x7f60ec0b0d20 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41854/0 (socket says 172.21.15.43:41854) 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 -- 172.21.15.43:0/775652899 learned_addr learned my addr 172.21.15.43:0/775652899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f0e00640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ec15c7e0 0x7f60ec0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f2200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 0x7f60ec156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 -- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 msgr2=0x7f60ec156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 0x7f60ec156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 -- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ec15c7e0 msgr2=0x7f60ec0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ec15c7e0 0x7f60ec0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f1800640 1 -- 172.21.15.43:0/775652899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0b1130 con 0x7f60ec155d70 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.684+0000 7f60f2200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 0x7f60ec156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60f1800640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60e000c3f0 tx=0x7f60e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0017070 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60e00053c0 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec0cf1f0 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0003860 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec0cf4d0 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec06f110 con 0x7f60ec155d70 2024-09-17T05:23:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.688+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60e001a070 con 0x7f60ec155d70 2024-09-17T05:23:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.692+0000 7f60e9600640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 0x7f60cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.692+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f60e0093590 con 0x7f60ec155d70 2024-09-17T05:23:40.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.692+0000 7f60f0e00640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 0x7f60cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.692+0000 7f60f0e00640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 0x7f60cc075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ad4f0 tx=0x7f60d8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:40.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.696+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f60ec0b1130 con 0x7f60ec155d70 2024-09-17T05:23:40.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.904+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60ec0af690 con 0x7f60ec155d70 2024-09-17T05:23:40.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.904+0000 7f60e9600640 1 -- 172.21.15.43:0/775652899 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f60e005df50 con 0x7f60ec155d70 2024-09-17T05:23:40.905 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:40.905 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 msgr2=0x7f60cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 0x7f60cc075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ad4f0 tx=0x7f60d8009040 comp rx=0 tx=0).stop 2024-09-17T05:23:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 msgr2=0x7f60ec155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60e000c3f0 tx=0x7f60e000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 shutdown_connections 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f60cc073200 0x7f60cc075710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60ec15c7e0 0x7f60ec0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f60ec0b0d20 0x7f60ec156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 --2- 172.21.15.43:0/775652899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ec155d70 0x7f60ec155340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 >> 172.21.15.43:0/775652899 conn(0x7f60ec0b23d0 msgr2=0x7f60ec15e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 shutdown_connections 2024-09-17T05:23:40.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:40.908+0000 7f60f4200640 1 -- 172.21.15.43:0/775652899 wait complete. 2024-09-17T05:23:40.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:41.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 -- 172.21.15.43:0/246655865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b4b00 msgr2=0x7f41041603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 --2- 172.21.15.43:0/246655865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b4b00 0x7f41041603e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009c80 tx=0x7f40fc01cd10 comp rx=0 tx=0).stop 2024-09-17T05:23:41.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 -- 172.21.15.43:0/246655865 shutdown_connections 2024-09-17T05:23:41.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 --2- 172.21.15.43:0/246655865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b4b00 0x7f41041603e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 --2- 172.21.15.43:0/246655865 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f410415cea0 0x7f41040d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.088+0000 7f410ae00640 1 -- 172.21.15.43:0/246655865 >> 172.21.15.43:0/246655865 conn(0x7f41040691b0 msgr2=0x7f41040695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- 172.21.15.43:0/246655865 shutdown_connections 2024-09-17T05:23:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- 172.21.15.43:0/246655865 wait complete. 2024-09-17T05:23:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 Processor -- start 2024-09-17T05:23:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- start start 2024-09-17T05:23:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41040cccb0 0x7f41040b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f410415cea0 0x7f41040b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f410416a390 con 0x7f41040b7aa0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f410416a210 con 0x7f410415cea0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.092+0000 7f410ae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f410416a510 con 0x7f41040cccb0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41866/0 (socket says 172.21.15.43:41866) 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 -- 172.21.15.43:0/1850239442 learned_addr learned my addr 172.21.15.43:0/1850239442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4109a00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41040cccb0 0x7f41040b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4109000640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f410415cea0 0x7f41040b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 -- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41040cccb0 msgr2=0x7f41040b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41040cccb0 0x7f41040b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 -- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f410415cea0 msgr2=0x7f41040b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f410415cea0 0x7f41040b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 -- 172.21.15.43:0/1850239442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040b7eb0 con 0x7f41040b7aa0 2024-09-17T05:23:41.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410a400640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40fc01b3c0 tx=0x7f40fc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0bb070 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40fc004b60 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0b2040 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4104154760 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4104154d20 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41040b5f20 con 0x7f41040b7aa0 2024-09-17T05:23:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.096+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f40fc004000 con 0x7f41040b7aa0 2024-09-17T05:23:41.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.100+0000 7f4101600640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 0x7f40e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.100+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f40fc12de40 con 0x7f41040b7aa0 2024-09-17T05:23:41.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.100+0000 7f4109a00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 0x7f40e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.100+0000 7f4109a00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 0x7f40e4075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009ad0 tx=0x7f40f80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.104+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f40fc0f1db0 con 0x7f41040b7aa0 2024-09-17T05:23:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.280+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41041603e0 con 0x7f41040b7aa0 2024-09-17T05:23:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.280+0000 7f4101600640 1 -- 172.21.15.43:0/1850239442 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f41040b7eb0 con 0x7f41040b7aa0 2024-09-17T05:23:41.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:41.280 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 msgr2=0x7f40e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 0x7f40e4075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009ad0 tx=0x7f40f80091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 msgr2=0x7f410415e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40fc01b3c0 tx=0x7f40fc005b00 comp rx=0 tx=0).stop 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 shutdown_connections 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f40e4073230 0x7f40e4075740 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f410415cea0 0x7f41040b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41040b7aa0 0x7f410415e060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 --2- 172.21.15.43:0/1850239442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41040cccb0 0x7f41040b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 >> 172.21.15.43:0/1850239442 conn(0x7f41040691b0 msgr2=0x7f4104151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 shutdown_connections 2024-09-17T05:23:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.284+0000 7f410ae00640 1 -- 172.21.15.43:0/1850239442 wait complete. 2024-09-17T05:23:41.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 -- 172.21.15.43:0/3235817318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 msgr2=0x7f5bcc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 --2- 172.21.15.43:0/3235817318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5bcc005160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ccc50 tx=0x7f5be001d5d0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5be7e00640 1 -- 172.21.15.43:0/3235817318 reap_dead start 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 -- 172.21.15.43:0/3235817318 shutdown_connections 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 -- 172.21.15.43:0/3235817318 >> 172.21.15.43:0/3235817318 conn(0x7f5be80b23d0 msgr2=0x7f5be80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 -- 172.21.15.43:0/3235817318 shutdown_connections 2024-09-17T05:23:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.460+0000 7f5bee600640 1 -- 172.21.15.43:0/3235817318 wait complete. 2024-09-17T05:23:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 Processor -- start 2024-09-17T05:23:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 -- start start 2024-09-17T05:23:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5be80c9190 0x7f5be80ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5be809d260 0x7f5be80cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80b3250 con 0x7f5bcc004d50 2024-09-17T05:23:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80b30d0 con 0x7f5be80c9190 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5bee600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80b33d0 con 0x7f5be809d260 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5be7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5be7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41886/0 (socket says 172.21.15.43:41886) 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.464+0000 7f5be7400640 1 -- 172.21.15.43:0/3860270353 learned_addr learned my addr 172.21.15.43:0/3860270353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7e00640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5be809d260 0x7f5be80cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be6a00640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5be80c9190 0x7f5be80ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 -- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5be809d260 msgr2=0x7f5be80cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5be809d260 0x7f5be80cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 -- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5be80c9190 msgr2=0x7f5be80ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5be80c9190 0x7f5be80ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 -- 172.21.15.43:0/3860270353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80af620 con 0x7f5bcc004d50 2024-09-17T05:23:41.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5be7400640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011ae0 tx=0x7f5bd8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:41.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801e070 con 0x7f5bcc004d50 2024-09-17T05:23:41.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd800fcd0 con 0x7f5bcc004d50 2024-09-17T05:23:41.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8019670 con 0x7f5bcc004d50 2024-09-17T05:23:41.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be80aebf0 con 0x7f5bcc004d50 2024-09-17T05:23:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be80c4700 con 0x7f5bcc004d50 2024-09-17T05:23:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be80c4960 con 0x7f5bcc004d50 2024-09-17T05:23:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.468+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5bd8021070 con 0x7f5bcc004d50 2024-09-17T05:23:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.472+0000 7f5bdf400640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 0x7f5bc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.472+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd8094aa0 con 0x7f5bcc004d50 2024-09-17T05:23:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.472+0000 7f5be6a00640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 0x7f5bc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.472+0000 7f5be6a00640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 0x7f5bc8075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ae330 tx=0x7f5bd4000f00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.476+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5bd805f4f0 con 0x7f5bcc004d50 2024-09-17T05:23:41.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be80cc530 con 0x7f5bcc004d50 2024-09-17T05:23:41.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bdf400640 1 -- 172.21.15.43:0/3860270353 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f5bd8063f20 con 0x7f5bcc004d50 2024-09-17T05:23:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:41.682 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:41.682 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 msgr2=0x7f5bc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 0x7f5bc8075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ae330 tx=0x7f5bd4000f00 comp rx=0 tx=0).stop 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 msgr2=0x7f5be8157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8011ae0 tx=0x7f5bd8009ee0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 shutdown_connections 2024-09-17T05:23:41.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5bc8073200 0x7f5bc8075710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5be80c9190 0x7f5be80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5be809d260 0x7f5be80cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 --2- 172.21.15.43:0/3860270353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bcc004d50 0x7f5be8157320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 >> 172.21.15.43:0/3860270353 conn(0x7f5be80b23d0 msgr2=0x7f5be80d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 shutdown_connections 2024-09-17T05:23:41.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:41.683+0000 7f5bee600640 1 -- 172.21.15.43:0/3860270353 wait complete. 2024-09-17T05:23:42.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:23:24.470623+0000', 'modified': '2024-09-17T05:23:27.471789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4384}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4384': {'gid': 4384, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6836/401236362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 401236362}, {'type': 'v1', 'addr': '172.21.15.62:6836', 'nonce': 401236362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4384, 'qdb_cluster': [4384]} 2024-09-17T05:23:42.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:23:42.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 -- 172.21.15.43:0/4028567096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740b43c0 msgr2=0x7f43740b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 --2- 172.21.15.43:0/4028567096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740b43c0 0x7f43740b47d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43740b6c60 tx=0x7f436c01cd50 comp rx=0 tx=0).stop 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 -- 172.21.15.43:0/4028567096 shutdown_connections 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 --2- 172.21.15.43:0/4028567096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740b43c0 0x7f43740b47d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 --2- 172.21.15.43:0/4028567096 >> v2:172.21.15.62:3300/0 conn(0x7f437415fc40 0x7f4374160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 -- 172.21.15.43:0/4028567096 >> 172.21.15.43:0/4028567096 conn(0x7f43740b2e80 msgr2=0x7f43740b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 -- 172.21.15.43:0/4028567096 shutdown_connections 2024-09-17T05:23:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.271+0000 7f437c600640 1 -- 172.21.15.43:0/4028567096 wait complete. 2024-09-17T05:23:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 Processor -- start 2024-09-17T05:23:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 -- start start 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 0x7f43741f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43740b5fe0 0x7f4374157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43741690b0 con 0x7f437415fc40 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4374168f30 con 0x7f43740b43c0 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4374169230 con 0x7f43740b5fe0 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41910/0 (socket says 172.21.15.43:41910) 2024-09-17T05:23:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 -- 172.21.15.43:0/1052536479 learned_addr learned my addr 172.21.15.43:0/1052536479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 -- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43740b5fe0 msgr2=0x7f4374157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43740b5fe0 0x7f4374157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 -- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 msgr2=0x7f43741f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f4379c00640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 0x7f43741f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 0x7f43741f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 -- 172.21.15.43:0/1052536479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741f7bd0 con 0x7f437415fc40 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f4379c00640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 0x7f43741f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437a600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f436c019960 tx=0x7f436c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f4371600640 1 -- 172.21.15.43:0/1052536479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0bb070 con 0x7f437415fc40 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f4371600640 1 -- 172.21.15.43:0/1052536479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436c004d40 con 0x7f437415fc40 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f4371600640 1 -- 172.21.15.43:0/1052536479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0b2040 con 0x7f437415fc40 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.275+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43740c75f0 con 0x7f437415fc40 2024-09-17T05:23:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.279+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43740c7bb0 con 0x7f437415fc40 2024-09-17T05:23:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.279+0000 7f4371600640 1 -- 172.21.15.43:0/1052536479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f436c0ad850 con 0x7f437415fc40 2024-09-17T05:23:42.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.279+0000 7f4371600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 0x7f4354075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4371600640 1 -- 172.21.15.43:0/1052536479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f436c12e250 con 0x7f437415fc40 2024-09-17T05:23:42.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 Processor -- start 2024-09-17T05:23:42.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 0x7f4354075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 -- start start 2024-09-17T05:23:42.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 0x7f4354075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4374073990 tx=0x7f4368009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4152d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4152d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ca413fd410 con 0x55ca40fe7460 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca4133ab40 con 0x55ca413c0ca0 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ca41329f70 con 0x55ca413d7950 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41922/0 (socket says 172.21.15.43:41922) 2024-09-17T05:23:42.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 -- 172.21.15.43:0/983884209 learned_addr learned my addr 172.21.15.43:0/983884209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379200640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4152d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f437a600640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4152d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 -- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 msgr2=0x55ca4152d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4152d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 -- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 msgr2=0x55ca4152d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4152d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 -- 172.21.15.43:0/983884209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca412f15b0 con 0x55ca40fe7460 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379200640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4152d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4379c00640 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f436800d860 tx=0x7f4368010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18006265a276fdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4366000640 1 -- 172.21.15.43:0/983884209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368024070 con 0x55ca40fe7460 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4366000640 1 -- 172.21.15.43:0/983884209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4368014ba0 con 0x55ca40fe7460 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.283+0000 7f4366000640 1 -- 172.21.15.43:0/983884209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436801f400 con 0x55ca40fe7460 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- 172.21.15.43:0/983884209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 msgr2=0x55ca4152cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f436800d860 tx=0x7f4368010000 comp rx=0 tx=0).stop 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- 172.21.15.43:0/983884209 shutdown_connections 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4152d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4152d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- 172.21.15.43:0/983884209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4152cb00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- 172.21.15.43:0/983884209 >> 172.21.15.43:0/983884209 conn(0x55ca414add90 msgr2=0x55ca414dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- 172.21.15.43:0/983884209 shutdown_connections 2024-09-17T05:23:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- 172.21.15.43:0/983884209 wait complete. 2024-09-17T05:23:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- start start 2024-09-17T05:23:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 Processor -- start 2024-09-17T05:23:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4151b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4151bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ca412c45f0 con 0x55ca40fe7460 2024-09-17T05:23:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca4132dd80 con 0x55ca413c0ca0 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ca412d0520 con 0x55ca413d7950 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4383bc6000 10 client.0 _ll_register_callbacks cb 0x55ca412d4830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41934/0 (socket says 172.21.15.43:41934) 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 -- 172.21.15.43:0/2507148913 learned_addr learned my addr 172.21.15.43:0/2507148913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379200640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4151b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f437a600640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4151bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 -- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 msgr2=0x55ca4151bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4151bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 -- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 msgr2=0x55ca4151b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4151b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.287+0000 7f4379c00640 1 -- 172.21.15.43:0/2507148913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca412c8190 con 0x55ca40fe7460 2024-09-17T05:23:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4379c00640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4368009010 tx=0x7f4368014c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc087a21f10c33d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436802c070 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca412c2110 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4368028610 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4368037050 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 10 client.4674 fetch_fsmap learned FSMap version 8 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca413aef10 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ca412c2110 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f436803a020 con 0x55ca40fe7460 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 10 client.4674 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 20 client.4674 populate_metadata read hostname 'smithi043' 2024-09-17T05:23:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fcc7d0 0x55ca4151c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 4 client.4674 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:23:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 --> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4133ca90 con 0x55ca40fcc7d0 2024-09-17T05:23:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4379200640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fcc7d0 0x55ca4151c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4379200640 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fcc7d0 0x55ca4151c570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc41feba30ad29b server_cookie=8078e73181437822 in_seq=0 out_seq=0 2024-09-17T05:23:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 10 client.4674 ms_handle_connect on v2:172.21.15.62:6834/401236362 2024-09-17T05:23:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 1 -- 172.21.15.43:0/2507148913 <== mds.0 v2:172.21.15.62:6834/401236362 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca4133ca90 con 0x55ca40fcc7d0 2024-09-17T05:23:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f435aa00640 10 client.4674 handle_command_reply: tid=0 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 client.4674 shutdown 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 msgr2=0x55ca4151b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.291+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4368009010 tx=0x7f4368014c50 comp rx=0 tx=0).stop 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 shutdown_connections 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fcc7d0 0x55ca4151c570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413c0ca0 0x55ca4151b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca4151bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f435aa00640 0 client.4674 ms_handle_reset on v2:172.21.15.62:6834/401236362 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 --2- 172.21.15.43:0/2507148913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca40fe7460 0x55ca4151b0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 >> 172.21.15.43:0/2507148913 conn(0x55ca414add90 msgr2=0x55ca414e5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 shutdown_connections 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 1 -- 172.21.15.43:0/2507148913 wait complete. 2024-09-17T05:23:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.295+0000 7f4383bc6000 20 client.4674 trim_cache size 0 max 16384 2024-09-17T05:23:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.311+0000 7f4383bc6000 1 Processor -- start 2024-09-17T05:23:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.311+0000 7f4383bc6000 1 -- start start 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 0x55ca414ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 0x55ca41543110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43741fb100 con 0x55ca413c0ca0 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43740b00e0 con 0x55ca413d7950 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43740af440 con 0x55ca40fe7460 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41938/0 (socket says 172.21.15.43:41938) 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 -- 172.21.15.43:0/3648535684 learned_addr learned my addr 172.21.15.43:0/3648535684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379c00640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 0x55ca414ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f437a600640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 0x55ca41543110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 -- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 msgr2=0x55ca414ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 0x55ca414ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 -- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 msgr2=0x55ca41543110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 0x55ca41543110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 -- 172.21.15.43:0/3648535684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca41385db0 con 0x55ca413c0ca0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f437a600640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 0x55ca41543110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379c00640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 0x55ca414ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4379200640 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ca413e0bd0 tx=0x7f4360004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5535fe2b42ad1914 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4365600640 1 -- 172.21.15.43:0/3648535684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360003d30 con 0x55ca413c0ca0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4365600640 1 -- 172.21.15.43:0/3648535684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4360012450 con 0x55ca413c0ca0 2024-09-17T05:23:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4365600640 1 -- 172.21.15.43:0/3648535684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360012820 con 0x55ca413c0ca0 2024-09-17T05:23:42.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- 172.21.15.43:0/3648535684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 msgr2=0x55ca414ea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55ca413e0bd0 tx=0x7f4360004410 comp rx=0 tx=0).stop 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- 172.21.15.43:0/3648535684 shutdown_connections 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca413d7950 0x55ca41543110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca414ea6d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 --2- 172.21.15.43:0/3648535684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca40fe7460 0x55ca414ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- 172.21.15.43:0/3648535684 >> 172.21.15.43:0/3648535684 conn(0x55ca414add90 msgr2=0x55ca414e9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- 172.21.15.43:0/3648535684 shutdown_connections 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- 172.21.15.43:0/3648535684 wait complete. 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.315+0000 7f4383bc6000 1 -- start start 2024-09-17T05:23:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 Processor -- start 2024-09-17T05:23:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca40fe7460 0x55ca415a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca415a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ca412f5ee0 con 0x55ca413c0ca0 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43740afa90 con 0x55ca40fe7460 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ca412c45f0 con 0x55ca413d7950 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 10 client.0 _ll_register_callbacks cb 0x7f4368022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41954/0 (socket says 172.21.15.43:41954) 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 -- 172.21.15.43:0/56853884 learned_addr learned my addr 172.21.15.43:0/56853884 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379c00640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca40fe7460 0x55ca415a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f437a600640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca415a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 -- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 msgr2=0x55ca415a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca415a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 -- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca40fe7460 msgr2=0x55ca415a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca40fe7460 0x55ca415a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 -- 172.21.15.43:0/56853884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca412f0ab0 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4379200640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4360004710 tx=0x7f4360009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ab2528f7813b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360034060 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4360012600 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4360012450 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f436003a050 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 10 client.4680 fetch_fsmap learned FSMap version 8 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca412cf070 con 0x55ca413c0ca0 2024-09-17T05:23:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4360012450 con 0x55ca413c0ca0 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f436003d020 con 0x55ca413c0ca0 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 10 client.4680 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 20 client.4680 populate_metadata read hostname 'smithi043' 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fba830 0x55ca4153ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 4 client.4680 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:23:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.319+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 --> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] -- command(tid 0: {"prefix": "status"}) -- 0x55ca41338540 con 0x55ca40fba830 2024-09-17T05:23:42.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4379c00640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fba830 0x55ca4153ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4379c00640 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fba830 0x55ca4153ff60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724e30718d65d2cd server_cookie=fe56b99ee611dbfc in_seq=0 out_seq=0 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f435aa00640 10 client.4680 ms_handle_connect on v2:172.21.15.62:6834/401236362 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f435aa00640 1 -- 172.21.15.43:0/56853884 <== mds.0 v2:172.21.15.62:6834/401236362 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55ca41338540 con 0x55ca40fba830 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f435aa00640 10 client.4680 handle_command_reply: tid=0 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 client.4680 shutdown 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 msgr2=0x55ca415a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4360004710 tx=0x7f4360009710 comp rx=0 tx=0).stop 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 shutdown_connections 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:6834/401236362,v1:172.21.15.62:6836/401236362] conn(0x55ca40fba830 0x55ca4153ff60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ca413d7950 0x55ca415a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ca413c0ca0 0x55ca415a2c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 --2- 172.21.15.43:0/56853884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca40fe7460 0x55ca415a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 >> 172.21.15.43:0/56853884 conn(0x55ca414add90 msgr2=0x55ca414ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f435aa00640 0 client.4680 ms_handle_reset on v2:172.21.15.62:6834/401236362 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 shutdown_connections 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 1 -- 172.21.15.43:0/56853884 wait complete. 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f4383bc6000 20 client.4680 trim_cache size 0 max 16384 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4384, 2024-09-17T05:23:42.326 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 17.289626496, 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 8, 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 22, 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 22, 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 23.362799438 2024-09-17T05:23:42.327 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 msgr2=0x7f4354075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 0x7f4354075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4374073990 tx=0x7f4368009040 comp rx=0 tx=0).stop 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 msgr2=0x7f437415afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f436c019960 tx=0x7f436c0092c0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 shutdown_connections 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4354073200 0x7f4354075710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fc40 0x7f437415afd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43740b5fe0 0x7f4374157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 --2- 172.21.15.43:0/1052536479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43740b43c0 0x7f43741f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 >> 172.21.15.43:0/1052536479 conn(0x7f43740b2e80 msgr2=0x7f437415a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 shutdown_connections 2024-09-17T05:23:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.323+0000 7f437c600640 1 -- 172.21.15.43:0/1052536479 wait complete. 2024-09-17T05:23:42.343 INFO:teuthology.run_tasks:Running task kclient... 2024-09-17T05:23:42.373 INFO:tasks.kclient:Mounting kernel clients... 2024-09-17T05:23:42.373 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T05:23:42.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.373 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:23:42.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-17T05:23:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-17T05:23:42.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:23:42.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.449 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:23:42.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-17T05:23:42.567 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-17T05:23:42.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:23:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.570 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:23:42.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-17T05:23:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-09-17T05:23:42.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:23:42.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.652 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:23:42.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:42.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-17T05:23:42.772 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-09-17T05:23:42.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:23:42.774 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-17T05:23:42.774 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:23:42.775 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:23:42.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 -- 172.21.15.43:0/1309054759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f899c004e70 msgr2=0x7f899c0052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 --2- 172.21.15.43:0/1309054759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f899c004e70 0x7f899c0052e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009ec0 tx=0x7f89b001d5f0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89b6800640 1 -- 172.21.15.43:0/1309054759 reap_dead start 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 -- 172.21.15.43:0/1309054759 shutdown_connections 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 -- 172.21.15.43:0/1309054759 >> 172.21.15.43:0/1309054759 conn(0x7f89b80cee80 msgr2=0x7f89b80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 -- 172.21.15.43:0/1309054759 shutdown_connections 2024-09-17T05:23:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.935+0000 7f89bd200640 1 -- 172.21.15.43:0/1309054759 wait complete. 2024-09-17T05:23:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 Processor -- start 2024-09-17T05:23:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 -- start start 2024-09-17T05:23:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 0x7f89b814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b8159940 0x7f89b80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8165df0 con 0x7f89b81f7570 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8165c70 con 0x7f89b8159940 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89bd200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89b8165f70 con 0x7f899c004e70 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41966/0 (socket says 172.21.15.43:41966) 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 -- 172.21.15.43:0/835550133 learned_addr learned my addr 172.21.15.43:0/835550133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89b6800640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b8159940 0x7f89b80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89b5e00640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 0x7f89b814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 -- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 msgr2=0x7f89b814b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 0x7f89b814b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 -- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b8159940 msgr2=0x7f89b80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b8159940 0x7f89b80c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89ad400640 1 -- 172.21.15.43:0/835550133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80a02b0 con 0x7f89b81f7570 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.939+0000 7f89b5e00640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 0x7f89b814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89ad400640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89a0003eb0 tx=0x7f89a0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0015070 con 0x7f89b81f7570 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a0011840 con 0x7f89b81f7570 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0011c90 con 0x7f89b81f7570 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b81f6920 con 0x7f89b81f7570 2024-09-17T05:23:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b81f4ae0 con 0x7f89b81f7570 2024-09-17T05:23:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b81f7a80 con 0x7f89b81f7570 2024-09-17T05:23:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89a0018070 con 0x7f89b81f7570 2024-09-17T05:23:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 0x7f8994075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.943+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89a0094ce0 con 0x7f89b81f7570 2024-09-17T05:23:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.947+0000 7f89b5e00640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 0x7f8994075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.947+0000 7f89b5e00640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 0x7f8994075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89a8001b60 tx=0x7f89a8000fa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:42.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:42.947+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89b80a02b0 con 0x7f89b81f7570 2024-09-17T05:23:43.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89b80d0840 con 0x7f89b81f7570 2024-09-17T05:23:43.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89aea00640 1 -- 172.21.15.43:0/835550133 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f89a005f740 con 0x7f89b81f7570 2024-09-17T05:23:43.166 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:23:43.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 msgr2=0x7f8994075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:43.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 0x7f8994075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89a8001b60 tx=0x7f89a8000fa0 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 msgr2=0x7f89b80c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89a0003eb0 tx=0x7f89a0001640 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 shutdown_connections 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8994073200 0x7f8994075710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b8159940 0x7f89b80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89b81f7570 0x7f89b80c5f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 --2- 172.21.15.43:0/835550133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f899c004e70 0x7f89b814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 >> 172.21.15.43:0/835550133 conn(0x7f89b80cee80 msgr2=0x7f89b80ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 shutdown_connections 2024-09-17T05:23:43.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43.167+0000 7f89bd200640 1 -- 172.21.15.43:0/835550133 wait complete. 2024-09-17T05:23:43.188 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:23:43.188 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:23:43.189 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:23:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:43.190 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:23:43.199 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42421sec preferred_lft 42421sec 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:43.200 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:23:43.201 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:23:43.201 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:43.201 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:23:43.201 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:43.202 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:23:43.202 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:43.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:43.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:43.363 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:23:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:23:43.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:43.384 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:23:43.454 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:23:43.455 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:23:43.455 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:23:43.455 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:23:43.455 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:23:43.455 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:23:43.455 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:43.455 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:43.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:23:43.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:23:43.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:23:43.456 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:43.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:43.593 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:23:43.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:43.603 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-09-17T05:23:43.658 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:43.658 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:43.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:23:43.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T05:23:43.658 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:43.787 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T05:23:43.787 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:43.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:43.955 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:43.955 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:43.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:23:43.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:23:43.955 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:44.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:44.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:44.041 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:23:44.050 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:23:44.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:44.050 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:23:44.105 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:23:44.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:44.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:23:44.211 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:23:44.211 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:23:44.211 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:44.888 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:23:44.888 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:23:44.888 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:44.888 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:44.888 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:44.903 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:23:44.903 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:23:44.904 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:23:44.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:23:44.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:44.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:23:44.929 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:45.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:45.063 INFO:teuthology.orchestra.run.smithi043.stdout:client4601 2024-09-17T05:23:45.063 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:45.064 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:45.073 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:45.186 INFO:teuthology.orchestra.run.smithi043.stdout:client4601 2024-09-17T05:23:45.187 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:45.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:45.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:45.261 INFO:teuthology.orchestra.run.smithi043.stdout:client4601 2024-09-17T05:23:45.262 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1cac0027-7a1b-472a-9fc8-e0027668aa6d.client4601/status') 2024-09-17T05:23:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:45.335 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.4601 (3)172.21.15.43:0/2706174310 2024-09-17T05:23:45.335 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-09-17T05:23:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-09-17T05:23:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-09-17T05:23:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:70 bytes copied, 5.2679e-05 s, 1.3 MB/s 2024-09-17T05:23:45.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:45.343 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-17T05:23:45.343 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:23:45.343 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:23:45.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- 172.21.15.43:0/3210091306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 msgr2=0x7f6f780734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 --2- 172.21.15.43:0/3210091306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f780734c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f780ccb60 tx=0x7f6f7001d6c0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- 172.21.15.43:0/3210091306 shutdown_connections 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 --2- 172.21.15.43:0/3210091306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f780734c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 --2- 172.21.15.43:0/3210091306 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7806fc90 0x7f6f780700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 --2- 172.21.15.43:0/3210091306 >> v2:172.21.15.62:3300/0 conn(0x7f6f7815fe10 0x7f6f780b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- 172.21.15.43:0/3210091306 >> 172.21.15.43:0/3210091306 conn(0x7f6f780b23d0 msgr2=0x7f6f780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- 172.21.15.43:0/3210091306 shutdown_connections 2024-09-17T05:23:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- 172.21.15.43:0/3210091306 wait complete. 2024-09-17T05:23:45.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 Processor -- start 2024-09-17T05:23:45.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.503+0000 7f6f7d600640 1 -- start start 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7806fc90 0x7f6f78161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7815fe10 0x7f6f7815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780ceab0 con 0x7f6f780730b0 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780ce930 con 0x7f6f7806fc90 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780cec30 con 0x7f6f7815fe10 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54244/0 (socket says 172.21.15.43:54244) 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 -- 172.21.15.43:0/1027229775 learned_addr learned my addr 172.21.15.43:0/1027229775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f76a00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7806fc90 0x7f6f78161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f77400640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7815fe10 0x7f6f7815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 -- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7815fe10 msgr2=0x7f6f7815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7815fe10 0x7f6f7815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 -- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7806fc90 msgr2=0x7f6f78161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7806fc90 0x7f6f78161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 -- 172.21.15.43:0/1027229775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f78156a80 con 0x7f6f780730b0 2024-09-17T05:23:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6fe00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d6d0 tx=0x7f6f6000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60017070 con 0x7f6f780730b0 2024-09-17T05:23:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f60009b10 con 0x7f6f780730b0 2024-09-17T05:23:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60010460 con 0x7f6f780730b0 2024-09-17T05:23:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f78156d60 con 0x7f6f780730b0 2024-09-17T05:23:45.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.507+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f78157320 con 0x7f6f780730b0 2024-09-17T05:23:45.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f781615b0 con 0x7f6f780730b0 2024-09-17T05:23:45.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6f6001a070 con 0x7f6f780730b0 2024-09-17T05:23:45.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f6ea00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 0x7f6f54075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:45.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6f600942d0 con 0x7f6f780730b0 2024-09-17T05:23:45.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f76a00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 0x7f6f54075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:45.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.511+0000 7f6f76a00640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 0x7f6f54075770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009b40 tx=0x7f6f680091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.515+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f6005ec00 con 0x7f6f780730b0 2024-09-17T05:23:45.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.727+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6f780734c0 con 0x7f6f780730b0 2024-09-17T05:23:45.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.727+0000 7f6f6ea00640 1 -- 172.21.15.43:0/1027229775 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f6f60063630 con 0x7f6f780730b0 2024-09-17T05:23:45.728 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 msgr2=0x7f6f54075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 0x7f6f54075770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009b40 tx=0x7f6f680091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 msgr2=0x7f6f7815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000d6d0 tx=0x7f6f6000db90 comp rx=0 tx=0).stop 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 shutdown_connections 2024-09-17T05:23:45.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f54073260 0x7f6f54075770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6f7815fe10 0x7f6f7815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f780730b0 0x7f6f7815e8e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 --2- 172.21.15.43:0/1027229775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7806fc90 0x7f6f78161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 >> 172.21.15.43:0/1027229775 conn(0x7f6f780b23d0 msgr2=0x7f6f781592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 shutdown_connections 2024-09-17T05:23:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45.731+0000 7f6f7d600640 1 -- 172.21.15.43:0/1027229775 wait complete. 2024-09-17T05:23:45.750 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:23:45.751 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:23:45.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:45.751 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:23:45.760 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:23:45.761 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42418sec preferred_lft 42418sec 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.762 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:23:45.763 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:45.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:45.765 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:23:45.826 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:23:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:45.829 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:23:45.883 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:23:45.883 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:45.883 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:45.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:23:45.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:23:45.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:23:45.884 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:46.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.021 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:23:46.030 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:23:46.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.031 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-09-17T05:23:46.081 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T05:23:46.081 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:46.082 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:46.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:23:46.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T05:23:46.082 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:46.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:46.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether a6:0d:b8:d2:fb:18 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:23:46.232 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:23:46.233 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.233 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::a40d:b8ff:fed2:fb18/64 scope link 2024-09-17T05:23:46.233 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:46.301 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether a6:0d:b8:d2:fb:18 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::a40d:b8ff:fed2:fb18/64 scope link 2024-09-17T05:23:46.302 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:46.303 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T05:23:46.303 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:46.303 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T05:23:46.304 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:46.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:46.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:46.483 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:46.483 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:23:46.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2024-09-17T05:23:46.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:23:46.483 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:23:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:46.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:46.609 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T05:23:46.665 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:23:46.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:46.666 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:23:46.720 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:23:46.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:23:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:23:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:23:46.822 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:46.840 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:23:46.840 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:23:46.841 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:46.841 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:46.841 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:46.852 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:23:46.852 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:23:46.852 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:23:46.853 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:46.853 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:46.853 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:23:46.853 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:23:46.853 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:23:46.856 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:23:46.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:46.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:23:46.876 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:46.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:46.975 INFO:teuthology.orchestra.run.smithi043.stdout:client4610 2024-09-17T05:23:46.976 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:46.976 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:46.985 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:47.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:47.087 INFO:teuthology.orchestra.run.smithi043.stdout:client4610 2024-09-17T05:23:47.087 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:47.187 INFO:teuthology.orchestra.run.smithi043.stdout:client4610 2024-09-17T05:23:47.188 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1cac0027-7a1b-472a-9fc8-e0027668aa6d.client4610/status') 2024-09-17T05:23:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:47.248 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.4610 (3)172.21.15.43:0/442684862 2024-09-17T05:23:47.248 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-09-17T05:23:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-09-17T05:23:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-09-17T05:23:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:69 bytes copied, 5.5171e-05 s, 1.3 MB/s 2024-09-17T05:23:47.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:47.256 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-17T05:23:47.257 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:23:47.257 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:23:47.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- 172.21.15.43:0/3937814409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed00b3af0 msgr2=0x7f8ed00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 --2- 172.21.15.43:0/3937814409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed00b3af0 0x7f8ed00ad0b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009c80 tx=0x7f8ec801cce0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- 172.21.15.43:0/3937814409 shutdown_connections 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 --2- 172.21.15.43:0/3937814409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed00b3af0 0x7f8ed00ad0b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 --2- 172.21.15.43:0/3937814409 >> v2:172.21.15.62:3300/0 conn(0x7f8ed0155d70 0x7f8ed015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- 172.21.15.43:0/3937814409 >> 172.21.15.43:0/3937814409 conn(0x7f8ed00b23d0 msgr2=0x7f8ed00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- 172.21.15.43:0/3937814409 shutdown_connections 2024-09-17T05:23:47.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- 172.21.15.43:0/3937814409 wait complete. 2024-09-17T05:23:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 Processor -- start 2024-09-17T05:23:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.459+0000 7f8ed7200640 1 -- start start 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed0155d70 0x7f8ed00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed0165560 0x7f8ed0165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0168cb0 con 0x7f8ed006f110 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0168b30 con 0x7f8ed0155d70 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed0168e30 con 0x7f8ed0165560 2024-09-17T05:23:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54252/0 (socket says 172.21.15.43:54252) 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 -- 172.21.15.43:0/454937487 learned_addr learned my addr 172.21.15.43:0/454937487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed5200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed0165560 0x7f8ed0165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ec7e00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed0155d70 0x7f8ed00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 -- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed0165560 msgr2=0x7f8ed0165970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed0165560 0x7f8ed0165970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 -- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed0155d70 msgr2=0x7f8ed00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed0155d70 0x7f8ed00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 -- 172.21.15.43:0/454937487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed0165eb0 con 0x7f8ed006f110 2024-09-17T05:23:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ecfe00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000e790 tx=0x7f8ec000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000eea0 con 0x7f8ed006f110 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec000cce0 con 0x7f8ed006f110 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0004290 con 0x7f8ed006f110 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed0155070 con 0x7f8ed006f110 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed00752b0 con 0x7f8ed006f110 2024-09-17T05:23:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.463+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed00ad0b0 con 0x7f8ed006f110 2024-09-17T05:23:47.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.467+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ec0003600 con 0x7f8ed006f110 2024-09-17T05:23:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.467+0000 7f8eccc00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 0x7f8eac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.467+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec0094c20 con 0x7f8ed006f110 2024-09-17T05:23:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.467+0000 7f8ec7e00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 0x7f8eac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.467+0000 7f8ec7e00640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 0x7f8eac075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ed009da30 tx=0x7f8eb8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:47.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.471+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ed0165eb0 con 0x7f8ed006f110 2024-09-17T05:23:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.683+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ed015cf80 con 0x7f8ed006f110 2024-09-17T05:23:47.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.683+0000 7f8eccc00640 1 -- 172.21.15.43:0/454937487 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8ec005f4d0 con 0x7f8ed006f110 2024-09-17T05:23:47.685 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:23:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 msgr2=0x7f8eac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 0x7f8eac075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ed009da30 tx=0x7f8eb8009040 comp rx=0 tx=0).stop 2024-09-17T05:23:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 msgr2=0x7f8ed0154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000e790 tx=0x7f8ec000ec50 comp rx=0 tx=0).stop 2024-09-17T05:23:47.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 shutdown_connections 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8eac073200 0x7f8eac075710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed0155d70 0x7f8ed00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ed0165560 0x7f8ed0165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 --2- 172.21.15.43:0/454937487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed006f110 0x7f8ed0154b30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 >> 172.21.15.43:0/454937487 conn(0x7f8ed00b23d0 msgr2=0x7f8ed015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 shutdown_connections 2024-09-17T05:23:47.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:47.687+0000 7f8ed7200640 1 -- 172.21.15.43:0/454937487 wait complete. 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:23:47.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:23:47.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:23:47.708 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:23:47.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:47.708 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:47.718 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42421sec preferred_lft 42421sec 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:23:47.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 02:42:f8:d3:e0:2a brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:47.720 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:23:47.720 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:47.720 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:23:47.721 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:47.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:47.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:47.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:47.894 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:23:47.913 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-09-17T05:23:47.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:47.915 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:47.990 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:23:47.991 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:48.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:48.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:48.129 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:23:48.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:48.138 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-09-17T05:23:48.189 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:48.190 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:48.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:23:48.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T05:23:48.190 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:48.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:48.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:48.318 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T05:23:48.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T05:23:48.319 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:48.491 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:48.491 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:48.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-09-17T05:23:48.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:23:48.491 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:48.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:48.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:48.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:48.578 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:23:48.587 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:23:48.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:48.588 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:23:48.635 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:23:48.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:48.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:23:48.737 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:23:48.737 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:23:48.737 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:49.388 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:23:49.388 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:23:49.388 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:49.388 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:49.388 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:23:49.402 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:23:49.405 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:23:49.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:49.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:23:49.428 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:49.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:49.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:49.543 INFO:teuthology.orchestra.run.smithi062.stdout:client4619 2024-09-17T05:23:49.543 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:49.543 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:49.553 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:49.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:49.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:49.655 INFO:teuthology.orchestra.run.smithi062.stdout:client4619 2024-09-17T05:23:49.656 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:49.713 INFO:teuthology.orchestra.run.smithi062.stdout:client4619 2024-09-17T05:23:49.714 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1cac0027-7a1b-472a-9fc8-e0027668aa6d.client4619/status') 2024-09-17T05:23:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:49.824 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4619 (3)192.168.0.1:0/3571748500 2024-09-17T05:23:49.824 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-09-17T05:23:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-09-17T05:23:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-09-17T05:23:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:69 bytes copied, 7.0421e-05 s, 980 kB/s 2024-09-17T05:23:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:49.829 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-17T05:23:49.829 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:23:49.829 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:23:49.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 -- 172.21.15.43:0/722437113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc0737c0 msgr2=0x7fa1cc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/722437113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc0737c0 0x7fa1cc0700a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009c20 tx=0x7fa1c401caa0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 -- 172.21.15.43:0/722437113 shutdown_connections 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/722437113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc0737c0 0x7fa1cc0700a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 -- 172.21.15.43:0/722437113 >> 172.21.15.43:0/722437113 conn(0x7fa1cc0b23d0 msgr2=0x7fa1cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 -- 172.21.15.43:0/722437113 shutdown_connections 2024-09-17T05:23:50.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.003+0000 7fa1d0e00640 1 -- 172.21.15.43:0/722437113 wait complete. 2024-09-17T05:23:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 Processor -- start 2024-09-17T05:23:50.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- start start 2024-09-17T05:23:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:50.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 0x7fa1cc15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 0x7fa1cc15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc16b6c0 con 0x7fa1cc06fc90 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc16b540 con 0x7fa1cc15fe10 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc16b840 con 0x7fa1cc075050 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54264/0 (socket says 172.21.15.43:54264) 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 -- 172.21.15.43:0/2196454839 learned_addr learned my addr 172.21.15.43:0/2196454839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1ca800640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 0x7fa1cc15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c3e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 0x7fa1cc15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 -- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 msgr2=0x7fa1cc15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 0x7fa1cc15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 -- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 msgr2=0x7fa1cc15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 0x7fa1cc15b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 -- 172.21.15.43:0/2196454839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc15a140 con 0x7fa1cc06fc90 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1ca800640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 0x7fa1cc15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c3e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 0x7fa1cc15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:50.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c9e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c460 tx=0x7fa1bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc016020 con 0x7fa1cc06fc90 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1bc012a00 con 0x7fa1cc06fc90 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc012e50 con 0x7fa1cc06fc90 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc15a420 con 0x7fa1cc06fc90 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.007+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc15a9e0 con 0x7fa1cc06fc90 2024-09-17T05:23:50.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc0700a0 con 0x7fa1cc06fc90 2024-09-17T05:23:50.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1bc019020 con 0x7fa1cc06fc90 2024-09-17T05:23:50.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1c2000640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 0x7fa1a80758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:50.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa1bc094150 con 0x7fa1cc06fc90 2024-09-17T05:23:50.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1c3e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 0x7fa1a80758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:50.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.011+0000 7fa1c3e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 0x7fa1a80758e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc157d40 tx=0x7fa1b4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:50.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.015+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa1bc020020 con 0x7fa1cc06fc90 2024-09-17T05:23:50.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1cc159e30 con 0x7fa1cc06fc90 2024-09-17T05:23:50.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1c2000640 1 -- 172.21.15.43:0/2196454839 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa1bc05f160 con 0x7fa1cc06fc90 2024-09-17T05:23:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 msgr2=0x7fa1a80758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 0x7fa1a80758e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc157d40 tx=0x7fa1b4009040 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 msgr2=0x7fa1cc074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c460 tx=0x7fa1bc00c920 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 shutdown_connections 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a80733d0 0x7fa1a80758e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1cc15fe10 0x7fa1cc15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1cc075050 0x7fa1cc15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 --2- 172.21.15.43:0/2196454839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1cc06fc90 0x7fa1cc074b10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 >> 172.21.15.43:0/2196454839 conn(0x7fa1cc0b23d0 msgr2=0x7fa1cc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 shutdown_connections 2024-09-17T05:23:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:50.231+0000 7fa1d0e00640 1 -- 172.21.15.43:0/2196454839 wait complete. 2024-09-17T05:23:50.252 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:23:50.252 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:23:50.253 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:23:50.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.253 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:50.263 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42418sec preferred_lft 42418sec 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 02:42:f8:d3:e0:2a brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:23:50.264 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:23:50.265 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.266 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:23:50.326 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-09-17T05:23:50.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.328 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:23:50.388 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:23:50.389 INFO:teuthology.orchestra.run.smithi062.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:50.389 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:50.389 INFO:teuthology.orchestra.run.smithi062.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:23:50.389 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:23:50.389 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:50.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:23:50.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:23:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:50.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.529 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:23:50.580 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:23:50.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.581 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-09-17T05:23:50.631 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T05:23:50.633 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:50.633 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:50.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:23:50.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T05:23:50.633 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:50.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether be:18:e3:27:43:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.822 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::bc18:e3ff:fe27:4348/64 scope link 2024-09-17T05:23:50.823 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:50.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:23:50.885 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:23:50.885 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:23:50.885 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:23:50.885 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.885 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether be:18:e3:27:43:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::bc18:e3ff:fe27:4348/64 scope link 2024-09-17T05:23:50.886 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:23:50.887 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T05:23:50.887 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T05:23:50.888 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:50.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:51.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.069 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:23:51.069 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-09-17T05:23:51.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2024-09-17T05:23:51.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:23:51.070 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-09-17T05:23:51.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:51.155 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T05:23:51.164 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:23:51.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:23:51.164 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:23:51.216 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:23:51.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:51.216 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:23:51.322 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:23:51.322 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:23:51.322 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:51.340 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:23:51.340 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:23:51.340 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:51.340 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:51.341 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:51.351 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:23:51.351 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:23:51.352 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:23:51.355 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:23:51.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:51.356 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:23:51.375 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:51.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.473 INFO:teuthology.orchestra.run.smithi062.stdout:client4714 2024-09-17T05:23:51.474 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:51.474 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:51.529 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:51.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:51.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.632 INFO:teuthology.orchestra.run.smithi062.stdout:client4714 2024-09-17T05:23:51.633 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:23:51.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:23:51.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.689 INFO:teuthology.orchestra.run.smithi062.stdout:client4714 2024-09-17T05:23:51.690 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1cac0027-7a1b-472a-9fc8-e0027668aa6d.client4714/status') 2024-09-17T05:23:51.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:23:51.795 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4714 (3)192.168.0.2:0/3262212858 2024-09-17T05:23:51.796 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-09-17T05:23:51.796 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-09-17T05:23:51.796 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-09-17T05:23:51.796 INFO:teuthology.orchestra.run.smithi062.stderr:69 bytes copied, 6.4435e-05 s, 1.1 MB/s 2024-09-17T05:23:51.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-09-17T05:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:23:51.804 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T05:23:51.860 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:23:51.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- 172.21.15.43:0/3028406261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0155d70 msgr2=0x7faec015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 --2- 172.21.15.43:0/3028406261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0155d70 0x7faec015c7e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009900 tx=0x7faeb001c820 comp rx=0 tx=0).stop 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- 172.21.15.43:0/3028406261 shutdown_connections 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 --2- 172.21.15.43:0/3028406261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec00b3af0 0x7faec00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 --2- 172.21.15.43:0/3028406261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec0155d70 0x7faec015c7e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- 172.21.15.43:0/3028406261 >> 172.21.15.43:0/3028406261 conn(0x7faec00b23d0 msgr2=0x7faec00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- 172.21.15.43:0/3028406261 shutdown_connections 2024-09-17T05:23:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- 172.21.15.43:0/3028406261 wait complete. 2024-09-17T05:23:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 Processor -- start 2024-09-17T05:23:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.031+0000 7faec5600640 1 -- start start 2024-09-17T05:23:52.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0155d70 0x7faec01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec01fa530 0x7faec00c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec0168c60 con 0x7faec01fa530 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faec0168ae0 con 0x7faec006f110 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faec0168de0 con 0x7faec0155d70 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:53366/0 (socket says 172.21.15.43:53366) 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 -- 172.21.15.43:0/528884546 learned_addr learned my addr 172.21.15.43:0/528884546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faeb7e00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0155d70 0x7faec01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebf400640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec01fa530 0x7faec00c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 -- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0155d70 msgr2=0x7faec01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0155d70 0x7faec01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 -- 172.21.15.43:0/528884546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec01fa530 msgr2=0x7faec00c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec01fa530 0x7faec00c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 -- 172.21.15.43:0/528884546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec009def0 con 0x7faec006f110 2024-09-17T05:23:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faebea00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009850 tx=0x7faeb00a57f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00b2040 con 0x7faec006f110 2024-09-17T05:23:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeb0004340 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb00044f0 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec00c78b0 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec00c7e70 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.035+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec00ad0b0 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.039+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faeb0005290 con 0x7faec006f110 2024-09-17T05:23:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.039+0000 7faeb6a00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 0x7fae9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.039+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faeb012dd00 con 0x7faec006f110 2024-09-17T05:23:52.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.039+0000 7faeb7e00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 0x7fae9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.039+0000 7faeb7e00640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 0x7fae9c075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faec015fd40 tx=0x7faea8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.043+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faeb00164d0 con 0x7faec006f110 2024-09-17T05:23:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.219+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faec015b7a0 con 0x7faec006f110 2024-09-17T05:23:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.219+0000 7faeb6a00640 1 -- 172.21.15.43:0/528884546 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7faeb0005580 con 0x7faec006f110 2024-09-17T05:23:52.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:52.221 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 msgr2=0x7fae9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 0x7fae9c075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faec015fd40 tx=0x7faea8009040 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 msgr2=0x7faec00cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009850 tx=0x7faeb00a57f0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 shutdown_connections 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae9c073200 0x7fae9c075710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec01fa530 0x7faec00c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec0155d70 0x7faec01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 --2- 172.21.15.43:0/528884546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec006f110 0x7faec00cba60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 >> 172.21.15.43:0/528884546 conn(0x7faec00b23d0 msgr2=0x7faec015f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 shutdown_connections 2024-09-17T05:23:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.223+0000 7faec5600640 1 -- 172.21.15.43:0/528884546 wait complete. 2024-09-17T05:23:52.243 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T05:23:22.405461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T05:23:23.420902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:23:52.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- 172.21.15.43:0/271122386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 msgr2=0x7f4504004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 --2- 172.21.15.43:0/271122386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 0x7f4504004810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4508009b90 tx=0x7f450801d280 comp rx=0 tx=0).stop 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f451e800640 1 -- 172.21.15.43:0/271122386 reap_dead start 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- 172.21.15.43:0/271122386 shutdown_connections 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- 172.21.15.43:0/271122386 >> 172.21.15.43:0/271122386 conn(0x7f45200b23d0 msgr2=0x7f45200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- 172.21.15.43:0/271122386 shutdown_connections 2024-09-17T05:23:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- 172.21.15.43:0/271122386 wait complete. 2024-09-17T05:23:52.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 Processor -- start 2024-09-17T05:23:52.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.407+0000 7f4524c00640 1 -- start start 2024-09-17T05:23:52.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 0x7f452009da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45200c72b0 0x7f45201fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45201667f0 con 0x7f45200c76c0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4520166670 con 0x7f45200c72b0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4520166970 con 0x7f4520155d70 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54304/0 (socket says 172.21.15.43:54304) 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 -- 172.21.15.43:0/2254405909 learned_addr learned my addr 172.21.15.43:0/2254405909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451de00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 0x7f452009da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4517e00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45200c72b0 0x7f45201fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 -- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 msgr2=0x7f452009da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 0x7f452009da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 -- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45200c72b0 msgr2=0x7f45201fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45200c72b0 0x7f45201fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 -- 172.21.15.43:0/2254405909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452009df80 con 0x7f45200c76c0 2024-09-17T05:23:52.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f451e800640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510009710 con 0x7f45200c76c0 2024-09-17T05:23:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f451800f7a0 con 0x7f45200c76c0 2024-09-17T05:23:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451800fbf0 con 0x7f45200c76c0 2024-09-17T05:23:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45200c7bc0 con 0x7f45200c76c0 2024-09-17T05:23:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45200c8160 con 0x7f45200c76c0 2024-09-17T05:23:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.411+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45200b33e0 con 0x7f45200c76c0 2024-09-17T05:23:52.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.415+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f451801c020 con 0x7f45200c76c0 2024-09-17T05:23:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.415+0000 7f4516000640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 0x7f44fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.415+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4518094af0 con 0x7f45200c76c0 2024-09-17T05:23:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.415+0000 7f451de00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 0x7f44fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.419+0000 7f451de00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 0x7f44fc075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45200715c0 tx=0x7f45100091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.419+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f45200b3d30 con 0x7f45200c76c0 2024-09-17T05:23:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45200af170 con 0x7f45200c76c0 2024-09-17T05:23:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4516000640 1 -- 172.21.15.43:0/2254405909 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f451805f4a0 con 0x7f45200c76c0 2024-09-17T05:23:52.594 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 msgr2=0x7f44fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 0x7f44fc075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45200715c0 tx=0x7f45100091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 msgr2=0x7f45201fc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518006660 comp rx=0 tx=0).stop 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 shutdown_connections 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44fc073200 0x7f44fc075710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45200c72b0 0x7f45201fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200c76c0 0x7f45201fc860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 --2- 172.21.15.43:0/2254405909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4520155d70 0x7f452009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 >> 172.21.15.43:0/2254405909 conn(0x7f45200b23d0 msgr2=0x7f45200d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 shutdown_connections 2024-09-17T05:23:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.595+0000 7f4524c00640 1 -- 172.21.15.43:0/2254405909 wait complete. 2024-09-17T05:23:52.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T05:23:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- 172.21.15.43:0/3449654729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 msgr2=0x7fb078003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 --2- 172.21.15.43:0/3449654729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb078003fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb084009d00 tx=0x7fb08401d3d0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- 172.21.15.43:0/3449654729 shutdown_connections 2024-09-17T05:23:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 --2- 172.21.15.43:0/3449654729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb078003fb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- 172.21.15.43:0/3449654729 >> 172.21.15.43:0/3449654729 conn(0x7fb0940b23d0 msgr2=0x7fb0940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- 172.21.15.43:0/3449654729 shutdown_connections 2024-09-17T05:23:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- 172.21.15.43:0/3449654729 wait complete. 2024-09-17T05:23:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 Processor -- start 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.779+0000 7fb099800640 1 -- start start 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09415ae00 0x7fb0941f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0940c8300 0x7fb0940b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0940ad5d0 con 0x7fb078003b40 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0940ad450 con 0x7fb09415ae00 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb0940ad750 con 0x7fb0940c8300 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54334/0 (socket says 172.21.15.43:54334) 2024-09-17T05:23:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 -- 172.21.15.43:0/2999720191 learned_addr learned my addr 172.21.15.43:0/2999720191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb093400640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0940c8300 0x7fb0940b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092000640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09415ae00 0x7fb0941f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 -- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0940c8300 msgr2=0x7fb0940b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0940c8300 0x7fb0940b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 -- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09415ae00 msgr2=0x7fb0941f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09415ae00 0x7fb0941f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 -- 172.21.15.43:0/2999720191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0940cb020 con 0x7fb078003b40 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb092a00640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c50 tx=0x7fb084009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840bb030 con 0x7fb078003b40 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb084004980 con 0x7fb078003b40 2024-09-17T05:23:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0941f8ae0 con 0x7fb078003b40 2024-09-17T05:23:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084004d90 con 0x7fb078003b40 2024-09-17T05:23:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0940cad40 con 0x7fb078003b40 2024-09-17T05:23:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.783+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0941f88f0 con 0x7fb078003b40 2024-09-17T05:23:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.787+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0840be030 con 0x7fb078003b40 2024-09-17T05:23:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.787+0000 7fb08aa00640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 0x7fb074075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.787+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb08412fb50 con 0x7fb078003b40 2024-09-17T05:23:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.787+0000 7fb092000640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 0x7fb074075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.787+0000 7fb092000640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 0x7fb074075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0941f6530 tx=0x7fb0800067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.791+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb0840170e0 con 0x7fb078003b40 2024-09-17T05:23:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.967+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb0940adcb0 con 0x7fb078003b40 2024-09-17T05:23:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.967+0000 7fb08aa00640 1 -- 172.21.15.43:0/2999720191 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fb0840fa510 con 0x7fb078003b40 2024-09-17T05:23:52.968 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-09-17T05:23:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 msgr2=0x7fb074075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 0x7fb074075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb0941f6530 tx=0x7fb0800067b0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 msgr2=0x7fb09409dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb084009c50 tx=0x7fb084009980 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 shutdown_connections 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb074073230 0x7fb074075740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb09415ae00 0x7fb0941f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0940c8300 0x7fb0940b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 --2- 172.21.15.43:0/2999720191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb078003b40 0x7fb09409dc90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 >> 172.21.15.43:0/2999720191 conn(0x7fb0940b23d0 msgr2=0x7fb09415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 shutdown_connections 2024-09-17T05:23:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:52.971+0000 7fb099800640 1 -- 172.21.15.43:0/2999720191 wait complete. 2024-09-17T05:23:52.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T05:23:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 -- 172.21.15.43:0/2977218387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb006e230 msgr2=0x7f3fb00b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/2977218387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb006e230 0x7f3fb00b5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0074000 tx=0x7f3fa001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 -- 172.21.15.43:0/2977218387 shutdown_connections 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/2977218387 >> v2:172.21.15.62:3300/0 conn(0x7f3fb006e640 0x7f3fb00b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/2977218387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb006e230 0x7f3fb00b5230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 -- 172.21.15.43:0/2977218387 >> 172.21.15.43:0/2977218387 conn(0x7f3fb00cee60 msgr2=0x7f3fb00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 -- 172.21.15.43:0/2977218387 shutdown_connections 2024-09-17T05:23:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.151+0000 7f3fb5c00640 1 -- 172.21.15.43:0/2977218387 wait complete. 2024-09-17T05:23:53.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 Processor -- start 2024-09-17T05:23:53.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 -- start start 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fb006e230 0x7f3fb014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fb006e640 0x7f3fb00ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb016c0f0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb00d2180 con 0x7f3fb006e640 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fb5c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb016c270 con 0x7f3fb006e230 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54344/0 (socket says 172.21.15.43:54344) 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 -- 172.21.15.43:0/1045498683 learned_addr learned my addr 172.21.15.43:0/1045498683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faea00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fb006e230 0x7f3fb014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3fa7e00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fb006e640 0x7f3fb00ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 -- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fb006e230 msgr2=0x7f3fb014f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fb006e230 0x7f3fb014f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 -- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fb006e640 msgr2=0x7f3fb00ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fb006e640 0x7f3fb00ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.155+0000 7f3faf400640 1 -- 172.21.15.43:0/1045498683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb014fbc0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3faf400640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800c770 tx=0x7f3fa800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8003b80 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fa8003d20 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800aab0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb00ce5d0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb0154c30 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0163400 con 0x7f3fb0162ff0 2024-09-17T05:23:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3fa800ac50 con 0x7f3fb0162ff0 2024-09-17T05:23:53.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 0x7f3f8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.159+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3fa8093fe0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.163+0000 7f3faea00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 0x7f3f8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.163+0000 7f3faea00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 0x7f3f8c075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00af000 tx=0x7f3fa00a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:53.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.167+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3fa805e8b0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.339+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fb015cc50 con 0x7f3fb0162ff0 2024-09-17T05:23:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.339+0000 7f3fa6a00640 1 -- 172.21.15.43:0/1045498683 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f3fa80632e0 con 0x7f3fb0162ff0 2024-09-17T05:23:53.340 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 msgr2=0x7f3f8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 0x7f3f8c075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00af000 tx=0x7f3fa00a7040 comp rx=0 tx=0).stop 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 msgr2=0x7f3fb01fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800c770 tx=0x7f3fa800cc30 comp rx=0 tx=0).stop 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 shutdown_connections 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f8c073200 0x7f3f8c075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fb006e640 0x7f3fb00ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb0162ff0 0x7f3fb01fc390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 --2- 172.21.15.43:0/1045498683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fb006e230 0x7f3fb014f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 >> 172.21.15.43:0/1045498683 conn(0x7f3fb00cee60 msgr2=0x7f3fb00aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 shutdown_connections 2024-09-17T05:23:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.343+0000 7f3fb5c00640 1 -- 172.21.15.43:0/1045498683 wait complete. 2024-09-17T05:23:53.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 -- 172.21.15.43:0/3393855669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e640 msgr2=0x7f33840d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 --2- 172.21.15.43:0/3393855669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e640 0x7f33840d1760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3384074000 tx=0x7f337401c880 comp rx=0 tx=0).stop 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 -- 172.21.15.43:0/3393855669 shutdown_connections 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 --2- 172.21.15.43:0/3393855669 >> v2:172.21.15.62:3300/0 conn(0x7f33840b5940 0x7f33840d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 --2- 172.21.15.43:0/3393855669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e640 0x7f33840d1760 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 -- 172.21.15.43:0/3393855669 >> 172.21.15.43:0/3393855669 conn(0x7f33840cee60 msgr2=0x7f33840cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:53.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 -- 172.21.15.43:0/3393855669 shutdown_connections 2024-09-17T05:23:53.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.519+0000 7f338c800640 1 -- 172.21.15.43:0/3393855669 wait complete. 2024-09-17T05:23:53.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 Processor -- start 2024-09-17T05:23:53.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 -- start start 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f338406e640 0x7f33840b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 0x7f33840b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f338416c0f0 con 0x7f338406e230 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3384163b80 con 0x7f338406e640 2024-09-17T05:23:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f338c800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f338416c270 con 0x7f33840b5940 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f3389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f3389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54366/0 (socket says 172.21.15.43:54366) 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.523+0000 7f3389e00640 1 -- 172.21.15.43:0/1555965954 learned_addr learned my addr 172.21.15.43:0/1555965954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389400640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f338406e640 0x7f33840b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f338a800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 0x7f33840b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 -- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 msgr2=0x7f33840b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 0x7f33840b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 -- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f338406e640 msgr2=0x7f33840b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f338406e640 0x7f33840b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 -- 172.21.15.43:0/1555965954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33840c98b0 con 0x7f338406e230 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f338a800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 0x7f33840b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f3389e00640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f337800b440 tx=0x7f337800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:53.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378015020 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3378011980 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33840c92f0 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378011dd0 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841561b0 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33840b5230 con 0x7f338406e230 2024-09-17T05:23:53.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.527+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3378018020 con 0x7f338406e230 2024-09-17T05:23:53.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.531+0000 7f337e000640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 0x7f3368075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:53.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.531+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3378094070 con 0x7f338406e230 2024-09-17T05:23:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.531+0000 7f3389400640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 0x7f3368075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.531+0000 7f3389400640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 0x7f3368075740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33740af000 tx=0x7f33740a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.535+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f337801f030 con 0x7f338406e230 2024-09-17T05:23:53.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:53.771+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f3384070230 con 0x7f338406e230 2024-09-17T05:23:54.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.179+0000 7f337e000640 1 -- 172.21.15.43:0/1555965954 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v52) ==== 137+0+0 (secure 0 0 0) 0x7f337800d040 con 0x7f338406e230 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 msgr2=0x7f3368075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 0x7f3368075740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33740af000 tx=0x7f33740a7040 comp rx=0 tx=0).stop 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 msgr2=0x7f33840b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f337800b440 tx=0x7f337800b900 comp rx=0 tx=0).stop 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 shutdown_connections 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3368073230 0x7f3368075740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f338406e640 0x7f33840b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33840b5940 0x7f33840b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 --2- 172.21.15.43:0/1555965954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f338406e230 0x7f33840b3020 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 >> 172.21.15.43:0/1555965954 conn(0x7f33840cee60 msgr2=0x7f33840cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 shutdown_connections 2024-09-17T05:23:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.183+0000 7f338c800640 1 -- 172.21.15.43:0/1555965954 wait complete. 2024-09-17T05:23:54.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:23:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 -- 172.21.15.43:0/3155699637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395015c7e0 msgr2=0x7f395006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 --2- 172.21.15.43:0/3155699637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395015c7e0 0x7f395006f110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3948009bc0 tx=0x7f394801cb10 comp rx=0 tx=0).stop 2024-09-17T05:23:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 -- 172.21.15.43:0/3155699637 shutdown_connections 2024-09-17T05:23:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 --2- 172.21.15.43:0/3155699637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395015c7e0 0x7f395006f110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 -- 172.21.15.43:0/3155699637 >> 172.21.15.43:0/3155699637 conn(0x7f39500b23d0 msgr2=0x7f39500d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 -- 172.21.15.43:0/3155699637 shutdown_connections 2024-09-17T05:23:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.363+0000 7f3956800640 1 -- 172.21.15.43:0/3155699637 wait complete. 2024-09-17T05:23:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 Processor -- start 2024-09-17T05:23:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- start start 2024-09-17T05:23:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 0x7f39500cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395015c7e0 0x7f3950156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3950169e60 con 0x7f395009d1f0 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3950169ce0 con 0x7f395015c7e0 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3950169fe0 con 0x7f3950155d70 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40678/0 (socket says 172.21.15.43:40678) 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 -- 172.21.15.43:0/1453625555 learned_addr learned my addr 172.21.15.43:0/1453625555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3947e00640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395015c7e0 0x7f3950156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394f400640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 0x7f39500cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 -- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 msgr2=0x7f39500cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 0x7f39500cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 -- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395015c7e0 msgr2=0x7f3950156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395015c7e0 0x7f3950156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 -- 172.21.15.43:0/1453625555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3950050f70 con 0x7f395009d1f0 2024-09-17T05:23:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394f400640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 0x7f39500cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f394fe00640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3948019720 tx=0x7f39480a5940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480bb070 con 0x7f395009d1f0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3948004d40 con 0x7f395009d1f0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480b2040 con 0x7f395009d1f0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395009df20 con 0x7f395009d1f0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.367+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3950153ed0 con 0x7f395009d1f0 2024-09-17T05:23:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395006f110 con 0x7f395009d1f0 2024-09-17T05:23:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3948004320 con 0x7f395009d1f0 2024-09-17T05:23:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f3947400640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 0x7f392c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f394812e270 con 0x7f395009d1f0 2024-09-17T05:23:54.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f394f400640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 0x7f392c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.371+0000 7f394f400640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 0x7f392c075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39400046c0 tx=0x7f3940009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.375+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3948016790 con 0x7f395009d1f0 2024-09-17T05:23:54.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.551+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f395015ec10 con 0x7f395009d1f0 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.551+0000 7f3947400640 1 -- 172.21.15.43:0/1453625555 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f39480f8c30 con 0x7f395009d1f0 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:23:54.551 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:23:54.552 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:23:54.553 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:23:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 msgr2=0x7f392c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 0x7f392c075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39400046c0 tx=0x7f3940009040 comp rx=0 tx=0).stop 2024-09-17T05:23:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 msgr2=0x7f3950156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3948019720 tx=0x7f39480a5940 comp rx=0 tx=0).stop 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 shutdown_connections 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f392c073200 0x7f392c075710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395015c7e0 0x7f3950156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f395009d1f0 0x7f3950156840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 --2- 172.21.15.43:0/1453625555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3950155d70 0x7f39500cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 >> 172.21.15.43:0/1453625555 conn(0x7f39500b23d0 msgr2=0x7f395006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 shutdown_connections 2024-09-17T05:23:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.555+0000 7f3956800640 1 -- 172.21.15.43:0/1453625555 wait complete. 2024-09-17T05:23:54.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:23:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 -- 172.21.15.43:0/167555462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00b43c0 msgr2=0x7f82c00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 --2- 172.21.15.43:0/167555462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00b43c0 0x7f82c00b47d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82c00b6c60 tx=0x7f82b801cd50 comp rx=0 tx=0).stop 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 -- 172.21.15.43:0/167555462 shutdown_connections 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 --2- 172.21.15.43:0/167555462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00b43c0 0x7f82c00b47d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 --2- 172.21.15.43:0/167555462 >> v2:172.21.15.62:3300/0 conn(0x7f82c015fc40 0x7f82c0160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 -- 172.21.15.43:0/167555462 >> 172.21.15.43:0/167555462 conn(0x7f82c00b2e80 msgr2=0x7f82c00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 -- 172.21.15.43:0/167555462 shutdown_connections 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.735+0000 7f82c4e00640 1 -- 172.21.15.43:0/167555462 wait complete. 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 Processor -- start 2024-09-17T05:23:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- start start 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c00b43c0 0x7f82c0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c00b5fe0 0x7f82c00aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82c01690e0 con 0x7f82c015fc40 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0168f60 con 0x7f82c00b5fe0 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f82c0169260 con 0x7f82c00b43c0 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40700/0 (socket says 172.21.15.43:40700) 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 -- 172.21.15.43:0/2350297000 learned_addr learned my addr 172.21.15.43:0/2350297000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82b7e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c00b5fe0 0x7f82c00aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82bde00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c00b43c0 0x7f82c0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 -- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c00b43c0 msgr2=0x7f82c0155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c00b43c0 0x7f82c0155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 -- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c00b5fe0 msgr2=0x7f82c00aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c00b5fe0 0x7f82c00aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 -- 172.21.15.43:0/2350297000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c015d220 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82be800640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82b801b400 tx=0x7f82b8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80bb070 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b8004d40 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c015d500 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80b2040 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.739+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c015dac0 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c00b47d0 con 0x7f82c015fc40 2024-09-17T05:23:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82b80be070 con 0x7f82c015fc40 2024-09-17T05:23:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82b6000640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 0x7f829c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f82b812df70 con 0x7f82c015fc40 2024-09-17T05:23:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82bde00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 0x7f829c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.743+0000 7f82bde00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 0x7f829c075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82c0073990 tx=0x7f82b0009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:54.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.747+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82b8016970 con 0x7f82c015fc40 2024-09-17T05:23:54.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.923+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f82c0069350 con 0x7f82c015fc40 2024-09-17T05:23:54.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82b6000640 1 -- 172.21.15.43:0/2350297000 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f82b80f8a40 con 0x7f82c015fc40 2024-09-17T05:23:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 msgr2=0x7f829c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 0x7f829c075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82c0073990 tx=0x7f82b0009060 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 msgr2=0x7f82c0156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82b801b400 tx=0x7f82b8005b00 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 shutdown_connections 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f829c073200 0x7f829c075710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c00b5fe0 0x7f82c00aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c015fc40 0x7f82c0156730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 --2- 172.21.15.43:0/2350297000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c00b43c0 0x7f82c0155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 >> 172.21.15.43:0/2350297000 conn(0x7f82c00b2e80 msgr2=0x7f82c0159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 shutdown_connections 2024-09-17T05:23:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:54.927+0000 7f82c4e00640 1 -- 172.21.15.43:0/2350297000 wait complete. 2024-09-17T05:23:54.948 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:23:54.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:23:55.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:23:55.123 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:23:55.123 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:55.123 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.132 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:55.133 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.185 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:55.185 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:23:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.439+0000 7f893ac00640 1 -- 172.21.15.43:0/2959750966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89340d0240 msgr2=0x7f89340cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.439+0000 7f893ac00640 1 --2- 172.21.15.43:0/2959750966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89340d0240 0x7f89340cd040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f89340751a0 tx=0x7f891c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- 172.21.15.43:0/2959750966 shutdown_connections 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- 172.21.15.43:0/2959750966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89340d0240 0x7f89340cd040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- 172.21.15.43:0/2959750966 >> v2:172.21.15.62:3300/0 conn(0x7f89340d0650 0x7f89340cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- 172.21.15.43:0/2959750966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f893409ba60 0x7f89340b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- 172.21.15.43:0/2959750966 >> 172.21.15.43:0/2959750966 conn(0x7f89340cee60 msgr2=0x7f89340cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- 172.21.15.43:0/2959750966 shutdown_connections 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- 172.21.15.43:0/2959750966 wait complete. 2024-09-17T05:23:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 Processor -- start 2024-09-17T05:23:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- start start 2024-09-17T05:23:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89340d0240 0x7f89340af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89340d0650 0x7f89340af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8934161950 con 0x7f893409ba60 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89341617d0 con 0x7f89340d0650 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.443+0000 7f893ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89340b1d40 con 0x7f89340d0240 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40720/0 (socket says 172.21.15.43:40720) 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 -- 172.21.15.43:0/2979023188 learned_addr learned my addr 172.21.15.43:0/2979023188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f892be00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89340d0240 0x7f89340af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8938c00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89340d0650 0x7f89340af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 -- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89340d0240 msgr2=0x7f89340af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89340d0240 0x7f89340af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 -- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89340d0650 msgr2=0x7f89340af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89340d0650 0x7f89340af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 -- 172.21.15.43:0/2979023188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893406f250 con 0x7f893409ba60 2024-09-17T05:23:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8933e00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f892400e770 tx=0x7f892400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400ccc0 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f892400ce60 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8934157b80 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924005280 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8934158140 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89340b3130 con 0x7f893409ba60 2024-09-17T05:23:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.447+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8924005420 con 0x7f893409ba60 2024-09-17T05:23:55.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.451+0000 7f8930c00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:55.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.451+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8924094c90 con 0x7f893409ba60 2024-09-17T05:23:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.451+0000 7f892be00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.451+0000 7f892be00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f891c0af000 tx=0x7f891c01b100 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:55.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.455+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f892405fc20 con 0x7f893409ba60 2024-09-17T05:23:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.639+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8934166130 con 0x7f893409ba60 2024-09-17T05:23:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.639+0000 7f8930c00640 1 -- 172.21.15.43:0/2979023188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f892405f520 con 0x7f893409ba60 2024-09-17T05:23:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:23:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 msgr2=0x7f8910075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f891c0af000 tx=0x7f891c01b100 comp rx=0 tx=0).stop 2024-09-17T05:23:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 msgr2=0x7f893415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f892400e770 tx=0x7f892400ec30 comp rx=0 tx=0).stop 2024-09-17T05:23:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 shutdown_connections 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89340d0650 0x7f89340af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89340d0240 0x7f89340af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 --2- 172.21.15.43:0/2979023188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893409ba60 0x7f893415b7f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 >> 172.21.15.43:0/2979023188 conn(0x7f89340cee60 msgr2=0x7f893415c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 shutdown_connections 2024-09-17T05:23:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:55.643+0000 7f893ac00640 1 -- 172.21.15.43:0/2979023188 wait complete. 2024-09-17T05:23:55.664 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:23:55.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:55.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:23:55.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:23:55.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:55.697 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:23:55.755 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:55.755 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.766 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:55.766 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.817 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:55.817 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:55.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- 172.21.15.43:0/2254549174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 msgr2=0x7fa68415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 --2- 172.21.15.43:0/2254549174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa68415ce90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009840 tx=0x7fa66c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- 172.21.15.43:0/2254549174 shutdown_connections 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 --2- 172.21.15.43:0/2254549174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa68415ce90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 --2- 172.21.15.43:0/2254549174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6840d1cb0 0x7fa6840ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- 172.21.15.43:0/2254549174 >> 172.21.15.43:0/2254549174 conn(0x7fa684068fe0 msgr2=0x7fa684069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:56.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- 172.21.15.43:0/2254549174 shutdown_connections 2024-09-17T05:23:56.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- 172.21.15.43:0/2254549174 wait complete. 2024-09-17T05:23:56.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 Processor -- start 2024-09-17T05:23:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.031+0000 7fa689800640 1 -- start start 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6840d1cb0 0x7fa6841f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6841605e0 0x7fa684167380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6841535e0 con 0x7fa68415ca80 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa684153460 con 0x7fa6840d1cb0 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa684153760 con 0x7fa6841605e0 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40728/0 (socket says 172.21.15.43:40728) 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 -- 172.21.15.43:0/958511118 learned_addr learned my addr 172.21.15.43:0/958511118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa683e00640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6840d1cb0 0x7fa6841f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa688e00640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6841605e0 0x7fa684167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 -- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6841605e0 msgr2=0x7fa684167380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6841605e0 0x7fa684167380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 -- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6840d1cb0 msgr2=0x7fa6841f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6840d1cb0 0x7fa6841f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 -- 172.21.15.43:0/958511118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68406df70 con 0x7fa68415ca80 2024-09-17T05:23:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa67b400640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009790 tx=0x7fa66c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0b2040 con 0x7fa68415ca80 2024-09-17T05:23:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa66c004330 con 0x7fa68415ca80 2024-09-17T05:23:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6841f9480 con 0x7fa68415ca80 2024-09-17T05:23:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003800 con 0x7fa68415ca80 2024-09-17T05:23:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6841f9a40 con 0x7fa68415ca80 2024-09-17T05:23:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.035+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6841f91f0 con 0x7fa68415ca80 2024-09-17T05:23:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.039+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa66c0bd060 con 0x7fa68415ca80 2024-09-17T05:23:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.039+0000 7fa680c00640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 0x7fa660075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.039+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa66c12da60 con 0x7fa68415ca80 2024-09-17T05:23:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.039+0000 7fa683e00640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 0x7fa660075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.039+0000 7fa683e00640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 0x7fa660075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ec0 tx=0x7fa674009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.043+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa66c016410 con 0x7fa68415ca80 2024-09-17T05:23:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.231+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa6840ccc40 con 0x7fa68415ca80 2024-09-17T05:23:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.231+0000 7fa680c00640 1 -- 172.21.15.43:0/958511118 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa66c0f8390 con 0x7fa68415ca80 2024-09-17T05:23:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 msgr2=0x7fa660075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 0x7fa660075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ec0 tx=0x7fa674009500 comp rx=0 tx=0).stop 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 msgr2=0x7fa6840b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009790 tx=0x7fa66c005d20 comp rx=0 tx=0).stop 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 shutdown_connections 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa660073200 0x7fa660075710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6841605e0 0x7fa684167380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa68415ca80 0x7fa6840b79c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 --2- 172.21.15.43:0/958511118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6840d1cb0 0x7fa6841f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 >> 172.21.15.43:0/958511118 conn(0x7fa684068fe0 msgr2=0x7fa68415f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 shutdown_connections 2024-09-17T05:23:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.235+0000 7fa689800640 1 -- 172.21.15.43:0/958511118 wait complete. 2024-09-17T05:23:56.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:23:56.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:56.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-17T05:23:56.294 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:23:56.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:56.294 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:23:56.351 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:56.352 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:56.361 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:56.361 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:56.413 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:23:56.413 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:56.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 -- 172.21.15.43:0/2858761980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27080b3af0 msgr2=0x7f270815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 --2- 172.21.15.43:0/2858761980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27080b3af0 0x7f270815c7e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009900 tx=0x7f26f001c850 comp rx=0 tx=0).stop 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 -- 172.21.15.43:0/2858761980 shutdown_connections 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 --2- 172.21.15.43:0/2858761980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27080b3af0 0x7f270815c7e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 -- 172.21.15.43:0/2858761980 >> 172.21.15.43:0/2858761980 conn(0x7f27080b23d0 msgr2=0x7f27080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 -- 172.21.15.43:0/2858761980 shutdown_connections 2024-09-17T05:23:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.667+0000 7f270f600640 1 -- 172.21.15.43:0/2858761980 wait complete. 2024-09-17T05:23:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 Processor -- start 2024-09-17T05:23:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 -- start start 2024-09-17T05:23:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 0x7f270809dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 0x7f27081f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2708169e10 con 0x7f2708155d70 2024-09-17T05:23:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2708169c90 con 0x7f27080c4d80 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2708169f90 con 0x7f270806f110 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40744/0 (socket says 172.21.15.43:40744) 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 -- 172.21.15.43:0/3002905048 learned_addr learned my addr 172.21.15.43:0/3002905048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270cc00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 0x7f270809dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270d600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 0x7f27081f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 -- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 msgr2=0x7f270809dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 0x7f270809dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 -- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 msgr2=0x7f27081f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 0x7f27081f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 -- 172.21.15.43:0/3002905048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270809e180 con 0x7f2708155d70 2024-09-17T05:23:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270cc00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 0x7f270809dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270d600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 0x7f27081f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f26ffe00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26f00b0040 tx=0x7f26f00a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00ba020 con 0x7f2708155d70 2024-09-17T05:23:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26f00043b0 con 0x7f2708155d70 2024-09-17T05:23:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27080cb640 con 0x7f2708155d70 2024-09-17T05:23:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.671+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00b2040 con 0x7f2708155d70 2024-09-17T05:23:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270809ad90 con 0x7f2708155d70 2024-09-17T05:23:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270815c7e0 con 0x7f2708155d70 2024-09-17T05:23:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26f0005290 con 0x7f2708155d70 2024-09-17T05:23:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f2705600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 0x7f26e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f26f012e2b0 con 0x7f2708155d70 2024-09-17T05:23:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.675+0000 7f270cc00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 0x7f26e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.679+0000 7f270cc00640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 0x7f26e4075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27080715c0 tx=0x7f26f8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:56.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.679+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26f00c4060 con 0x7f2708155d70 2024-09-17T05:23:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.867+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f270809aa00 con 0x7f2708155d70 2024-09-17T05:23:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.867+0000 7f2705600640 1 -- 172.21.15.43:0/3002905048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f26f00164d0 con 0x7f2708155d70 2024-09-17T05:23:56.869 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 msgr2=0x7f26e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 0x7f26e4075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27080715c0 tx=0x7f26f8009040 comp rx=0 tx=0).stop 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 msgr2=0x7f270809a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26f00b0040 tx=0x7f26f00a5660 comp rx=0 tx=0).stop 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 shutdown_connections 2024-09-17T05:23:56.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26e4073200 0x7f26e4075710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27080c4d80 0x7f27081f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2708155d70 0x7f270809a4c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 --2- 172.21.15.43:0/3002905048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f270806f110 0x7f270809dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 >> 172.21.15.43:0/3002905048 conn(0x7f27080b23d0 msgr2=0x7f270815d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 shutdown_connections 2024-09-17T05:23:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:56.871+0000 7f270f600640 1 -- 172.21.15.43:0/3002905048 wait complete. 2024-09-17T05:23:56.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-17T05:23:56.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:56.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-17T05:23:56.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:23:56.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:56.915 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T05:23:56.965 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:56.966 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:56.975 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:56.975 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:57.029 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:23:57.029 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:23:57.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- 172.21.15.43:0/4017305667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c003b20 msgr2=0x7f525c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 --2- 172.21.15.43:0/4017305667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c003b20 0x7f525c003f90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f52680016c0 tx=0x7f526801c790 comp rx=0 tx=0).stop 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f5277e00640 1 -- 172.21.15.43:0/4017305667 reap_dead start 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- 172.21.15.43:0/4017305667 shutdown_connections 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- 172.21.15.43:0/4017305667 >> 172.21.15.43:0/4017305667 conn(0x7f52780b23d0 msgr2=0x7f52780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- 172.21.15.43:0/4017305667 shutdown_connections 2024-09-17T05:23:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- 172.21.15.43:0/4017305667 wait complete. 2024-09-17T05:23:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 Processor -- start 2024-09-17T05:23:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.247+0000 7f527e600640 1 -- start start 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525c003b20 0x7f52781ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527809c670 0x7f52781fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5278167b80 con 0x7f52781fc4a0 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5278167a00 con 0x7f527809c670 2024-09-17T05:23:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5278167d00 con 0x7f525c003b20 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40764/0 (socket says 172.21.15.43:40764) 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 -- 172.21.15.43:0/1770869321 learned_addr learned my addr 172.21.15.43:0/1770869321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f526fe00640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527809c670 0x7f52781fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277400640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525c003b20 0x7f52781ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 -- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525c003b20 msgr2=0x7f52781ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525c003b20 0x7f52781ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 -- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527809c670 msgr2=0x7f52781fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527809c670 0x7f52781fbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 -- 172.21.15.43:0/1770869321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52781fc8b0 con 0x7f52781fc4a0 2024-09-17T05:23:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f5277e00640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f527815f170 tx=0x7f527000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270004b30 con 0x7f52781fc4a0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5270004cd0 con 0x7f52781fc4a0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52780ce310 con 0x7f52781fc4a0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270003be0 con 0x7f52781fc4a0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52780731c0 con 0x7f52781fc4a0 2024-09-17T05:23:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.251+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52780cea60 con 0x7f52781fc4a0 2024-09-17T05:23:57.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.255+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5270004030 con 0x7f52781fc4a0 2024-09-17T05:23:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.255+0000 7f526f400640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 0x7f5254075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.255+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5270094b70 con 0x7f52781fc4a0 2024-09-17T05:23:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.255+0000 7f5277400640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 0x7f5254075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.255+0000 7f5277400640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 0x7f5254075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52780715c0 tx=0x7f52680a8040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.259+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52780cea60 con 0x7f52781fc4a0 2024-09-17T05:23:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.443+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52781edbd0 con 0x7f52781fc4a0 2024-09-17T05:23:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.443+0000 7f526f400640 1 -- 172.21.15.43:0/1770869321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f52781fc8b0 con 0x7f52781fc4a0 2024-09-17T05:23:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:23:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 msgr2=0x7f5254075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 0x7f5254075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52780715c0 tx=0x7f52680a8040 comp rx=0 tx=0).stop 2024-09-17T05:23:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 msgr2=0x7f5278073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f527815f170 tx=0x7f527000f3c0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 shutdown_connections 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5254073200 0x7f5254075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527809c670 0x7f52781fbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52781fc4a0 0x7f5278073ca0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 --2- 172.21.15.43:0/1770869321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f525c003b20 0x7f52781ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 >> 172.21.15.43:0/1770869321 conn(0x7f52780b23d0 msgr2=0x7f52780b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 shutdown_connections 2024-09-17T05:23:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.447+0000 7f527e600640 1 -- 172.21.15.43:0/1770869321 wait complete. 2024-09-17T05:23:57.466 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-17T05:23:57.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:57.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-17T05:23:57.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:23:57.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:23:57.494 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T05:23:57.545 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:23:57.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 -- 172.21.15.43:0/3019967081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc48004810 msgr2=0x7efc48004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 --2- 172.21.15.43:0/3019967081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc48004810 0x7efc48004c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00c310 tx=0x7efc4c01cf60 comp rx=0 tx=0).stop 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc66400640 1 -- 172.21.15.43:0/3019967081 reap_dead start 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 -- 172.21.15.43:0/3019967081 shutdown_connections 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 -- 172.21.15.43:0/3019967081 >> 172.21.15.43:0/3019967081 conn(0x7efc600b23d0 msgr2=0x7efc600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 -- 172.21.15.43:0/3019967081 shutdown_connections 2024-09-17T05:23:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.703+0000 7efc68400640 1 -- 172.21.15.43:0/3019967081 wait complete. 2024-09-17T05:23:57.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 Processor -- start 2024-09-17T05:23:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- start start 2024-09-17T05:23:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc48004810 0x7efc60166af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc6009d830 0x7efc600c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc60167ad0 con 0x7efc601fa5b0 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc60167950 con 0x7efc6009d830 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efc60167c50 con 0x7efc48004810 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40788/0 (socket says 172.21.15.43:40788) 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 -- 172.21.15.43:0/4130296491 learned_addr learned my addr 172.21.15.43:0/4130296491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65a00640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc48004810 0x7efc60166af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc66400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc6009d830 0x7efc600c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 -- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc48004810 msgr2=0x7efc60166af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc48004810 0x7efc60166af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 -- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc6009d830 msgr2=0x7efc600c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc6009d830 0x7efc600c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 -- 172.21.15.43:0/4130296491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc600cd8e0 con 0x7efc601fa5b0 2024-09-17T05:23:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc65000640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc4c01b660 tx=0x7efc4c001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c001580 con 0x7efc601fa5b0 2024-09-17T05:23:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc4c006280 con 0x7efc601fa5b0 2024-09-17T05:23:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c003600 con 0x7efc601fa5b0 2024-09-17T05:23:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc601f9ad0 con 0x7efc601fa5b0 2024-09-17T05:23:57.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.707+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc601fa090 con 0x7efc601fa5b0 2024-09-17T05:23:57.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.711+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc600ad0b0 con 0x7efc601fa5b0 2024-09-17T05:23:57.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.711+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc4c0037a0 con 0x7efc601fa5b0 2024-09-17T05:23:57.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.715+0000 7efc5d600640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 0x7efc40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:57.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.715+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efc4c12df00 con 0x7efc601fa5b0 2024-09-17T05:23:57.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.715+0000 7efc65a00640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 0x7efc40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:57.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.715+0000 7efc65a00640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 0x7efc40075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efc600715c0 tx=0x7efc540091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:57.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.715+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc4c0c0030 con 0x7efc601fa5b0 2024-09-17T05:23:57.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.923+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc6006e070 con 0x7efc601fa5b0 2024-09-17T05:23:57.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc5d600640 1 -- 172.21.15.43:0/4130296491 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7efc4c016d10 con 0x7efc601fa5b0 2024-09-17T05:23:57.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:57.926 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:57.926 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 msgr2=0x7efc40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 0x7efc40075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efc600715c0 tx=0x7efc540091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 msgr2=0x7efc600c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc4c01b660 tx=0x7efc4c001d00 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 shutdown_connections 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc40073200 0x7efc40075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc6009d830 0x7efc600c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc601fa5b0 0x7efc600c94c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 --2- 172.21.15.43:0/4130296491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc48004810 0x7efc60166af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 >> 172.21.15.43:0/4130296491 conn(0x7efc600b23d0 msgr2=0x7efc600ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:57.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 shutdown_connections 2024-09-17T05:23:57.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:57.927+0000 7efc68400640 1 -- 172.21.15.43:0/4130296491 wait complete. 2024-09-17T05:23:57.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 -- 172.21.15.43:0/1994944133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 msgr2=0x7f00540b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 --2- 172.21.15.43:0/1994944133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f00540b4b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00540d0190 tx=0x7f004801c7a0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 -- 172.21.15.43:0/1994944133 shutdown_connections 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 --2- 172.21.15.43:0/1994944133 >> v2:172.21.15.62:3300/0 conn(0x7f005406fc90 0x7f00540700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 --2- 172.21.15.43:0/1994944133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f00540b4b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 -- 172.21.15.43:0/1994944133 >> 172.21.15.43:0/1994944133 conn(0x7f00540b23d0 msgr2=0x7f00540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 -- 172.21.15.43:0/1994944133 shutdown_connections 2024-09-17T05:23:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.107+0000 7f005ba00640 1 -- 172.21.15.43:0/1994944133 wait complete. 2024-09-17T05:23:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 Processor -- start 2024-09-17T05:23:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 -- start start 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 0x7f005415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 0x7f005415e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f005416a000 con 0x7f005415fe10 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0054169e80 con 0x7f00540730b0 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f005ba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f005416a180 con 0x7f005406fc90 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40800/0 (socket says 172.21.15.43:40800) 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 -- 172.21.15.43:0/2238700997 learned_addr learned my addr 172.21.15.43:0/2238700997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059000640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 0x7f005415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 -- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 msgr2=0x7f005415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 0x7f005415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 -- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 msgr2=0x7f005415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0053e00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 0x7f005415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 0x7f005415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 -- 172.21.15.43:0/2238700997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005415fa40 con 0x7f005415fe10 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059000640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 0x7f005415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0053e00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 0x7f005415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.111+0000 7f0059a00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b990 tx=0x7f004c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c019070 con 0x7f005415fe10 2024-09-17T05:23:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f004c00cc90 con 0x7f005415fe10 2024-09-17T05:23:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005415e970 con 0x7f005415fe10 2024-09-17T05:23:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c0146a0 con 0x7f005415fe10 2024-09-17T05:23:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00540cd4c0 con 0x7f005415fe10 2024-09-17T05:23:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00540700a0 con 0x7f005415fe10 2024-09-17T05:23:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f004c01c070 con 0x7f005415fe10 2024-09-17T05:23:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 0x7f0034075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f004c093ff0 con 0x7f005415fe10 2024-09-17T05:23:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.115+0000 7f0059000640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 0x7f0034075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.119+0000 7f0059000640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 0x7f0034075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00480af000 tx=0x7f004801afe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.119+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f005415fa40 con 0x7f005415fe10 2024-09-17T05:23:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.331+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00541565d0 con 0x7f005415fe10 2024-09-17T05:23:58.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.331+0000 7f0050c00640 1 -- 172.21.15.43:0/2238700997 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f004c05f060 con 0x7f005415fe10 2024-09-17T05:23:58.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:58.331 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:58.332 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:58.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 msgr2=0x7f0034075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 0x7f0034075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00480af000 tx=0x7f004801afe0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 msgr2=0x7f0054153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b990 tx=0x7f004c00be50 comp rx=0 tx=0).stop 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 shutdown_connections 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0034073230 0x7f0034075740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540730b0 0x7f005415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415fe10 0x7f0054153830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 --2- 172.21.15.43:0/2238700997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005406fc90 0x7f005415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 >> 172.21.15.43:0/2238700997 conn(0x7f00540b23d0 msgr2=0x7f00540ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 shutdown_connections 2024-09-17T05:23:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.335+0000 7f005ba00640 1 -- 172.21.15.43:0/2238700997 wait complete. 2024-09-17T05:23:58.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:58.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- 172.21.15.43:0/1874069662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 msgr2=0x7f36a00cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 --2- 172.21.15.43:0/1874069662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a00cd580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3698009c80 tx=0x7f369801cda0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- 172.21.15.43:0/1874069662 shutdown_connections 2024-09-17T05:23:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 --2- 172.21.15.43:0/1874069662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a00cd580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 --2- 172.21.15.43:0/1874069662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 0x7f36a00cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- 172.21.15.43:0/1874069662 >> 172.21.15.43:0/1874069662 conn(0x7f36a00cee60 msgr2=0x7f36a00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- 172.21.15.43:0/1874069662 shutdown_connections 2024-09-17T05:23:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- 172.21.15.43:0/1874069662 wait complete. 2024-09-17T05:23:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 Processor -- start 2024-09-17T05:23:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.515+0000 7f36a7000640 1 -- start start 2024-09-17T05:23:58.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a009ba60 0x7f36a00b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 0x7f36a0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b2b80 con 0x7f36a00d0650 2024-09-17T05:23:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b2a00 con 0x7f36a009ba60 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0169e30 con 0x7f36a00d0240 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40820/0 (socket says 172.21.15.43:40820) 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 -- 172.21.15.43:0/3170036446 learned_addr learned my addr 172.21.15.43:0/3170036446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f3697e00640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 0x7f36a0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f369fe00640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a009ba60 0x7f36a00b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 -- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 msgr2=0x7f36a0158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 0x7f36a0158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 -- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a009ba60 msgr2=0x7f36a00b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a009ba60 0x7f36a00b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 -- 172.21.15.43:0/3170036446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a01fced0 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a5000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f369801b450 tx=0x7f3698004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980bb070 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3698004a50 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980b2040 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a01f9940 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01fc110 con 0x7f36a00d0650 2024-09-17T05:23:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a00b70b0 con 0x7f36a00d0650 2024-09-17T05:23:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.519+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36980be070 con 0x7f36a00d0650 2024-09-17T05:23:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.523+0000 7f369cc00640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.523+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f369812e0f0 con 0x7f36a00d0650 2024-09-17T05:23:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.523+0000 7f369fe00640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.523+0000 7f369fe00640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3690001650 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.527+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3698016990 con 0x7f36a00d0650 2024-09-17T05:23:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36a0165430 con 0x7f36a00d0650 2024-09-17T05:23:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f369cc00640 1 -- 172.21.15.43:0/3170036446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f36980f8a20 con 0x7f36a00d0650 2024-09-17T05:23:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:58.707 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 msgr2=0x7f367c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3690001650 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 msgr2=0x7f36a0165950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f369801b450 tx=0x7f3698004270 comp rx=0 tx=0).stop 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 shutdown_connections 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a00d0650 0x7f36a0165950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36a00d0240 0x7f36a0158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 --2- 172.21.15.43:0/3170036446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a009ba60 0x7f36a00b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 >> 172.21.15.43:0/3170036446 conn(0x7f36a00cee60 msgr2=0x7f36a00712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.707+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 shutdown_connections 2024-09-17T05:23:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.711+0000 7f36a7000640 1 -- 172.21.15.43:0/3170036446 wait complete. 2024-09-17T05:23:58.730 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:23:58.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- 172.21.15.43:0/4208762834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f48003c40 msgr2=0x7f7f48004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 --2- 172.21.15.43:0/4208762834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f48003c40 0x7f7f48004090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400ac20 tx=0x7f7f5401e050 comp rx=0 tx=0).stop 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- 172.21.15.43:0/4208762834 shutdown_connections 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 --2- 172.21.15.43:0/4208762834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f48003c40 0x7f7f48004090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- 172.21.15.43:0/4208762834 >> 172.21.15.43:0/4208762834 conn(0x7f7f600d0010 msgr2=0x7f7f600d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- 172.21.15.43:0/4208762834 shutdown_connections 2024-09-17T05:23:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- 172.21.15.43:0/4208762834 wait complete. 2024-09-17T05:23:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 Processor -- start 2024-09-17T05:23:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 -- start start 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.891+0000 7f7f66400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f48003c40 0x7f7f6014f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 0x7f7f6009d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6006fe40 con 0x7f7f601f9050 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600b4520 con 0x7f7f48003c40 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6006ffc0 con 0x7f7f6009b160 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40840/0 (socket says 172.21.15.43:40840) 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 -- 172.21.15.43:0/2013939996 learned_addr learned my addr 172.21.15.43:0/2013939996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f65a00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 0x7f7f6009d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f65000640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f48003c40 0x7f7f6014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 -- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 msgr2=0x7f7f6009d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 0x7f7f6009d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 -- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f48003c40 msgr2=0x7f7f6014f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f48003c40 0x7f7f6014f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 -- 172.21.15.43:0/2013939996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6014fbf0 con 0x7f7f601f9050 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f65a00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 0x7f7f6009d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:23:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5fe00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006480 tx=0x7f7f4c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c015070 con 0x7f7f601f9050 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f4c011860 con 0x7f7f601f9050 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6014fed0 con 0x7f7f601f9050 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011cb0 con 0x7f7f601f9050 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6009cf20 con 0x7f7f601f9050 2024-09-17T05:23:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6014f960 con 0x7f7f601f9050 2024-09-17T05:23:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.895+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7f4c018070 con 0x7f7f601f9050 2024-09-17T05:23:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.899+0000 7f7f5cc00640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 0x7f7f40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.899+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7f4c094f30 con 0x7f7f601f9050 2024-09-17T05:23:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.899+0000 7f7f65000640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 0x7f7f40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:58.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.899+0000 7f7f65000640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 0x7f7f40075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7f540af000 tx=0x7f7f5401adf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:58.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:58.903+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f4c05feb0 con 0x7f7f601f9050 2024-09-17T05:23:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f6015e2f0 con 0x7f7f601f9050 2024-09-17T05:23:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f5cc00640 1 -- 172.21.15.43:0/2013939996 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f7f4c05f7b0 con 0x7f7f601f9050 2024-09-17T05:23:59.110 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:59.110 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:59.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 msgr2=0x7f7f40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 0x7f7f40075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7f540af000 tx=0x7f7f5401adf0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 msgr2=0x7f7f6009ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c006480 tx=0x7f7f4c0039d0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 shutdown_connections 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f40073230 0x7f7f40075740 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6009b160 0x7f7f6009d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f601f9050 0x7f7f6009ac20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 --2- 172.21.15.43:0/2013939996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f48003c40 0x7f7f6014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 >> 172.21.15.43:0/2013939996 conn(0x7f7f600d0010 msgr2=0x7f7f600761f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 shutdown_connections 2024-09-17T05:23:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.111+0000 7f7f66400640 1 -- 172.21.15.43:0/2013939996 wait complete. 2024-09-17T05:23:59.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- 172.21.15.43:0/570564789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf80b45a0 msgr2=0x7fbcf80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/570564789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf80b45a0 0x7fbcf80b6130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009840 tx=0x7fbce801c790 comp rx=0 tx=0).stop 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- 172.21.15.43:0/570564789 shutdown_connections 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/570564789 >> v2:172.21.15.62:3300/0 conn(0x7fbcf80b6670 0x7fbcf815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/570564789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf80b45a0 0x7fbcf80b6130 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- 172.21.15.43:0/570564789 >> 172.21.15.43:0/570564789 conn(0x7fbcf80b2e80 msgr2=0x7fbcf80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- 172.21.15.43:0/570564789 shutdown_connections 2024-09-17T05:23:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- 172.21.15.43:0/570564789 wait complete. 2024-09-17T05:23:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 Processor -- start 2024-09-17T05:23:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.295+0000 7fbcfdc00640 1 -- start start 2024-09-17T05:23:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcf80b6670 0x7fbcf8164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf815fa70 0x7fbcf8164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8168cb0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8168b30 con 0x7fbcf80b6670 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf8168e30 con 0x7fbcf815fa70 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40850/0 (socket says 172.21.15.43:40850) 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 -- 172.21.15.43:0/1009061400 learned_addr learned my addr 172.21.15.43:0/1009061400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcefe00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcf80b6670 0x7fbcf8164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf7400640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf815fa70 0x7fbcf8164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 -- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf815fa70 msgr2=0x7fbcf8164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf815fa70 0x7fbcf8164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 -- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcf80b6670 msgr2=0x7fbcf8164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcf80b6670 0x7fbcf8164a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 -- 172.21.15.43:0/1009061400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf80c6cb0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcf6a00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009790 tx=0x7fbce8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80b2040 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce80043e0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf80c5520 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80052d0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf80c7210 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.299+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80b6130 con 0x7fbcf80b45a0 2024-09-17T05:23:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.303+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbce8005470 con 0x7fbcf80b45a0 2024-09-17T05:23:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.303+0000 7fbceea00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 0x7fbcd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.303+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbce812df10 con 0x7fbcf80b45a0 2024-09-17T05:23:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.303+0000 7fbcefe00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 0x7fbcd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.303+0000 7fbcefe00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 0x7fbcd4075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8166060 tx=0x7fbce00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.307+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbce80bd060 con 0x7fbcf80b45a0 2024-09-17T05:23:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcf81600a0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbceea00640 1 -- 172.21.15.43:0/1009061400 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fbcf80c6cb0 con 0x7fbcf80b45a0 2024-09-17T05:23:59.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:59.519 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-17T05:23:28:056823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1146787273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1146787273},{"type":"v1","addr":"172.21.15.62:6837","nonce":1146787273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3305868115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3305868115},{"type":"v1","addr":"172.21.15.43:6835","nonce":3305868115}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1764685203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1764685203},{"type":"v1","addr":"172.21.15.43:6834","nonce":1764685203}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:23:24.470623+0000","modified":"2024-09-17T05:23:27.471789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6836/401236362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":401236362},{"type":"v1","addr":"172.21.15.62:6836","nonce":401236362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T05:23:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 msgr2=0x7fbcd4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 0x7fbcd4075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8166060 tx=0x7fbce00091e0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 msgr2=0x7fbcf80c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009790 tx=0x7fbce8005d90 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 shutdown_connections 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbcd4073200 0x7fbcd4075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcf80b6670 0x7fbcf8164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbcf815fa70 0x7fbcf8164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 --2- 172.21.15.43:0/1009061400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcf80b45a0 0x7fbcf80c7a10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 >> 172.21.15.43:0/1009061400 conn(0x7fbcf80b2e80 msgr2=0x7fbcf80d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 shutdown_connections 2024-09-17T05:23:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.519+0000 7fbcfdc00640 1 -- 172.21.15.43:0/1009061400 wait complete. 2024-09-17T05:23:59.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:23:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.699+0000 7f20c7200640 1 -- 172.21.15.43:0/1588360597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a8004810 msgr2=0x7f20a8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.699+0000 7f20c7200640 1 --2- 172.21.15.43:0/1588360597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a8004810 0x7f20a8004c20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20c0069610 tx=0x7f20b001c6f0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.699+0000 7f20c7200640 1 -- 172.21.15.43:0/1588360597 shutdown_connections 2024-09-17T05:23:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.699+0000 7f20c7200640 1 --2- 172.21.15.43:0/1588360597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a8004810 0x7f20a8004c20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.699+0000 7f20c7200640 1 -- 172.21.15.43:0/1588360597 >> 172.21.15.43:0/1588360597 conn(0x7f20c00b26c0 msgr2=0x7f20c00b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- 172.21.15.43:0/1588360597 shutdown_connections 2024-09-17T05:23:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- 172.21.15.43:0/1588360597 wait complete. 2024-09-17T05:23:59.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 Processor -- start 2024-09-17T05:23:59.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- start start 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 0x7f20c0155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20c01f5820 0x7f20c0149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20c015f2b0 con 0x7f20c0157f30 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20c015f130 con 0x7f20a8004810 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.703+0000 7f20c7200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f20c0073b80 con 0x7f20c01f5820 2024-09-17T05:23:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40876/0 (socket says 172.21.15.43:40876) 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 -- 172.21.15.43:0/2366585193 learned_addr learned my addr 172.21.15.43:0/2366585193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 -- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20c01f5820 msgr2=0x7f20c0149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c5400640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20c01f5820 0x7f20c0149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c5e00640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 0x7f20c0155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20c01f5820 0x7f20c0149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 -- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 msgr2=0x7f20c0155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 0x7f20c0155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 -- 172.21.15.43:0/2366585193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c0158880 con 0x7f20c0157f30 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c5e00640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 0x7f20c0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c6800640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20c015c930 tx=0x7f20bc004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc004b30 con 0x7f20c0157f30 2024-09-17T05:23:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20bc004cd0 con 0x7f20c0157f30 2024-09-17T05:23:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc010c90 con 0x7f20c0157f30 2024-09-17T05:23:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c01f62e0 con 0x7f20c0157f30 2024-09-17T05:23:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c0159670 con 0x7f20c0157f30 2024-09-17T05:23:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c01f6050 con 0x7f20c0157f30 2024-09-17T05:23:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.707+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20bc01c020 con 0x7f20c0157f30 2024-09-17T05:23:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.711+0000 7f20ba000640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 0x7f20a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:23:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.711+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f20bc0947e0 con 0x7f20c0157f30 2024-09-17T05:23:59.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.711+0000 7f20c5e00640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 0x7f20a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:23:59.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.711+0000 7f20c5e00640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 0x7f20a4075740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006780 tx=0x7f20b40067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:23:59.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.715+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20c0158880 con 0x7f20c0157f30 2024-09-17T05:23:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20c015f430 con 0x7f20c0157f30 2024-09-17T05:23:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20ba000640 1 -- 172.21.15.43:0/2366585193 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f20bc05f880 con 0x7f20c0157f30 2024-09-17T05:23:59.891 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:23:59.891 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:23:26.043723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:23:22.405461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:23:23.420902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 msgr2=0x7f20a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 0x7f20a4075740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006780 tx=0x7f20b40067b0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 msgr2=0x7f20c0158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20c015c930 tx=0x7f20bc004640 comp rx=0 tx=0).stop 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 shutdown_connections 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20a4073230 0x7f20a4075740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c0157f30 0x7f20c0158340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20c01f5820 0x7f20c0149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 --2- 172.21.15.43:0/2366585193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a8004810 0x7f20c0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:23:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 >> 172.21.15.43:0/2366585193 conn(0x7f20c00b26c0 msgr2=0x7f20c015df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:23:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.891+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 shutdown_connections 2024-09-17T05:23:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:23:59.895+0000 7f20c7200640 1 -- 172.21.15.43:0/2366585193 wait complete. 2024-09-17T05:23:59.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:24:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- 172.21.15.43:0/387821405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 msgr2=0x7ffaf80ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 --2- 172.21.15.43:0/387821405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf80ccc40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009870 tx=0x7ffaec01c830 comp rx=0 tx=0).stop 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- 172.21.15.43:0/387821405 shutdown_connections 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 --2- 172.21.15.43:0/387821405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf80ccc40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- 172.21.15.43:0/387821405 >> 172.21.15.43:0/387821405 conn(0x7ffaf8068fe0 msgr2=0x7ffaf8069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- 172.21.15.43:0/387821405 shutdown_connections 2024-09-17T05:24:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- 172.21.15.43:0/387821405 wait complete. 2024-09-17T05:24:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 Processor -- start 2024-09-17T05:24:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.075+0000 7ffaff200640 1 -- start start 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 0x7ffaf80b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf8160cf0 0x7ffaf813e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf815ce90 con 0x7ffaf815ca80 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf81539c0 con 0x7ffaf8160cf0 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf815d010 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 0x7ffaf80b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:49934/0 (socket says 172.21.15.43:49934) 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 0x7ffaf80b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40882/0 (socket says 172.21.15.43:40882) 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 -- 172.21.15.43:0/761395935 learned_addr learned my addr 172.21.15.43:0/761395935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafe800640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf8160cf0 0x7ffaf813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 -- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf8160cf0 msgr2=0x7ffaf813e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf8160cf0 0x7ffaf813e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 -- 172.21.15.43:0/761395935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 msgr2=0x7ffaf80b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 0x7ffaf80b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 -- 172.21.15.43:0/761395935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf8141350 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffafde00640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c10 tx=0x7ffaec0a5750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0bb070 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaec004470 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf81fc8b0 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0b2040 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf80b67c0 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.079+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf80ccc40 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.083+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffaec0be070 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.083+0000 7ffaf2000640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 0x7ffadc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.083+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffaec12de30 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.083+0000 7ffafd400640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 0x7ffadc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.083+0000 7ffafd400640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 0x7ffadc075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000c10 tx=0x7ffae8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.087+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffaf8141350 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.295+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffaf806a1e0 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.298 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:24:00.299+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:24:00.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.307+0000 7ffaf2000640 1 -- 172.21.15.43:0/761395935 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ffaec016450 con 0x7ffaf80d1cb0 2024-09-17T05:24:00.308 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:24:00.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 msgr2=0x7ffadc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 0x7ffadc075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000c10 tx=0x7ffae8009040 comp rx=0 tx=0).stop 2024-09-17T05:24:00.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 msgr2=0x7ffaf8140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c10 tx=0x7ffaec0a5750 comp rx=0 tx=0).stop 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 shutdown_connections 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffadc073200 0x7ffadc075710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffaf8160cf0 0x7ffaf813e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffaf815ca80 0x7ffaf80b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 --2- 172.21.15.43:0/761395935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffaf80d1cb0 0x7ffaf8140e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 >> 172.21.15.43:0/761395935 conn(0x7ffaf8068fe0 msgr2=0x7ffaf80cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 shutdown_connections 2024-09-17T05:24:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.311+0000 7ffaff200640 1 -- 172.21.15.43:0/761395935 wait complete. 2024-09-17T05:24:00.317 INFO:tasks.ceph.mds.d.smithi062.stderr: -7120> 2024-09-17T05:23:29.000+0000 7f2e8ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:24:00.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:24:00.441 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- 172.21.15.43:0/1286829311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00cdfb0 msgr2=0x7f3fc00cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/1286829311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00cd370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00afb40 tx=0x7f3fb001c830 comp rx=0 tx=0).stop 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- 172.21.15.43:0/1286829311 shutdown_connections 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/1286829311 >> v2:172.21.15.62:3300/0 conn(0x7f3fc00ce3c0 0x7f3fc00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/1286829311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00cd370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- 172.21.15.43:0/1286829311 >> 172.21.15.43:0/1286829311 conn(0x7f3fc00b2810 msgr2=0x7f3fc00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- 172.21.15.43:0/1286829311 shutdown_connections 2024-09-17T05:24:00.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- 172.21.15.43:0/1286829311 wait complete. 2024-09-17T05:24:00.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 Processor -- start 2024-09-17T05:24:00.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.495+0000 7f3fc5e00640 1 -- start start 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fc01603e0 0x7f3fc0155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc016a150 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0169fd0 con 0x7f3fc00cdfb0 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc016a2d0 con 0x7f3fc01603e0 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40896/0 (socket says 172.21.15.43:40896) 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 -- 172.21.15.43:0/3419039631 learned_addr learned my addr 172.21.15.43:0/3419039631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fbea00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fbf400640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fc01603e0 0x7f3fc0155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 -- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fc01603e0 msgr2=0x7f3fc0155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fc01603e0 0x7f3fc0155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 -- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc00cdfb0 msgr2=0x7f3fc00b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 -- 172.21.15.43:0/3419039631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc01f8ec0 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb6000640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e7a0 tx=0x7f3fa800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800ccc0 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fa8010430 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc00b5880 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8010860 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01f9430 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc00b55f0 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.499+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3fa8010a00 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.503+0000 7f3fb7e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 0x7f3f9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.503+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f3fa8094bb0 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.503+0000 7f3fbea00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 0x7f3f9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.503+0000 7f3fbea00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 0x7f3f9c075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00af000 tx=0x7f3fb001b070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.507+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3fa805f5b0 con 0x7f3fc00ce3c0 2024-09-17T05:24:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:00.719+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3fc01607f0 con 0x7f3fc00ce3c0 2024-09-17T05:24:01.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fb7e00640 1 -- 172.21.15.43:0/3419039631 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f3fa8063fe0 con 0x7f3fc00ce3c0 2024-09-17T05:24:01.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 msgr2=0x7f3f9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:01.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 0x7f3f9c075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00af000 tx=0x7f3fb001b070 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 msgr2=0x7f3fc01483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e7a0 tx=0x7f3fa800ec60 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 shutdown_connections 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3f9c073200 0x7f3f9c075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3fc01603e0 0x7f3fc0155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc00ce3c0 0x7f3fc01483b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 --2- 172.21.15.43:0/3419039631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fc00cdfb0 0x7f3fc00b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 >> 172.21.15.43:0/3419039631 conn(0x7f3fc00b2810 msgr2=0x7f3fc015bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 shutdown_connections 2024-09-17T05:24:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.303+0000 7f3fc5e00640 1 -- 172.21.15.43:0/3419039631 wait complete. 2024-09-17T05:24:01.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:24:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 -- 172.21.15.43:0/2962360956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc4004810 msgr2=0x7fcbc4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 --2- 172.21.15.43:0/2962360956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc4004810 0x7fcbc4004c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0b4340 tx=0x7fcbc801dfd0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe1400640 1 -- 172.21.15.43:0/2962360956 reap_dead start 2024-09-17T05:24:01.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 -- 172.21.15.43:0/2962360956 shutdown_connections 2024-09-17T05:24:01.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 -- 172.21.15.43:0/2962360956 >> 172.21.15.43:0/2962360956 conn(0x7fcbdc06ce60 msgr2=0x7fcbdc06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:01.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 -- 172.21.15.43:0/2962360956 shutdown_connections 2024-09-17T05:24:01.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.491+0000 7fcbe3400640 1 -- 172.21.15.43:0/2962360956 wait complete. 2024-09-17T05:24:01.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 Processor -- start 2024-09-17T05:24:01.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 -- start start 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbc4004810 0x7fcbdc0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 0x7fcbdc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc070750 con 0x7fcbdc0cbb30 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc0705d0 con 0x7fcbc4004810 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbe3400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcbdc0708d0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.495+0000 7fcbdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 0x7fcbdc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40912/0 (socket says 172.21.15.43:40912) 2024-09-17T05:24:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbdbe00640 1 -- 172.21.15.43:0/3767296247 learned_addr learned my addr 172.21.15.43:0/3767296247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe0a00640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbc4004810 0x7fcbdc0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 -- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbc4004810 msgr2=0x7fcbdc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbc4004810 0x7fcbdc0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 -- 172.21.15.43:0/3767296247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 msgr2=0x7fcbdc0ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 -- 172.21.15.43:0/3767296247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc0af9e0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbdbe00640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe1400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 0x7fcbdc0ae6e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc06df60 tx=0x7fcbd400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004b30 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbd4004cd0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc1f6530 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4003be0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc1f6af0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbdc1f62a0 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.499+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcbd4004030 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.503+0000 7fcbd8c00640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 0x7fcbbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.503+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7fcbd4094d80 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.503+0000 7fcbe0a00640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 0x7fcbbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:01.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.503+0000 7fcbe0a00640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 0x7fcbbc075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0b4ee0 tx=0x7fcbd0001480 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:01.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.507+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcbd401f060 con 0x7fcbdc0cbf40 2024-09-17T05:24:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:01.679+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbdc15c0c0 con 0x7fcbdc0cbf40 2024-09-17T05:24:02.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.315+0000 7fcbd8c00640 1 -- 172.21.15.43:0/3767296247 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fcbd405f6e0 con 0x7fcbdc0cbf40 2024-09-17T05:24:02.316 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 msgr2=0x7fcbbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 0x7fcbbc075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0b4ee0 tx=0x7fcbd0001480 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 msgr2=0x7fcbdc0ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 0x7fcbdc0ae6e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc06df60 tx=0x7fcbd400f3c0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 shutdown_connections 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbbc073200 0x7fcbbc075710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbdc0cbf40 0x7fcbdc0ae6e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc0cbb30 0x7fcbdc0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 --2- 172.21.15.43:0/3767296247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbc4004810 0x7fcbdc0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 >> 172.21.15.43:0/3767296247 conn(0x7fcbdc06ce60 msgr2=0x7fcbdc15a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:02.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 shutdown_connections 2024-09-17T05:24:02.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.319+0000 7fcbe3400640 1 -- 172.21.15.43:0/3767296247 wait complete. 2024-09-17T05:24:02.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:24:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- 172.21.15.43:0/1523154376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800d05f0 msgr2=0x7f3780161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 --2- 172.21.15.43:0/1523154376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800d05f0 0x7f3780161010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3768009870 tx=0x7f376801c610 comp rx=0 tx=0).stop 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- 172.21.15.43:0/1523154376 shutdown_connections 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 --2- 172.21.15.43:0/1523154376 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800d05f0 0x7f3780161010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- 172.21.15.43:0/1523154376 >> 172.21.15.43:0/1523154376 conn(0x7f37800ceec0 msgr2=0x7f37800cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- 172.21.15.43:0/1523154376 shutdown_connections 2024-09-17T05:24:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- 172.21.15.43:0/1523154376 wait complete. 2024-09-17T05:24:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 Processor -- start 2024-09-17T05:24:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 -- start start 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800cddf0 0x7f37801521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.515+0000 7f3784a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 0x7f37800b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3780070d30 con 0x7f37800d01e0 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3780161760 con 0x7f37800d05f0 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f378016b800 con 0x7f37800cddf0 2024-09-17T05:24:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40916/0 (socket says 172.21.15.43:40916) 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 -- 172.21.15.43:0/4269717162 learned_addr learned my addr 172.21.15.43:0/4269717162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f377de00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800cddf0 0x7f37801521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f377e800640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 0x7f37800b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 -- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800cddf0 msgr2=0x7f37801521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800cddf0 0x7f37801521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 -- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 msgr2=0x7f37800b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 0x7f37800b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 -- 172.21.15.43:0/4269717162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378006e2e0 con 0x7f37800d01e0 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f377e800640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 0x7f37800b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3775400640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3768000c10 tx=0x7f3768005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680047d0 con 0x7f37800d01e0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3768004970 con 0x7f37800d01e0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768004da0 con 0x7f37800d01e0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378006e5c0 con 0x7f37800d01e0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378006eb80 con 0x7f37800d01e0 2024-09-17T05:24:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37800ce200 con 0x7f37800d01e0 2024-09-17T05:24:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.519+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37680b04f0 con 0x7f37800d01e0 2024-09-17T05:24:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.523+0000 7f3776a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 0x7f375c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.523+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7f376812e420 con 0x7f37800d01e0 2024-09-17T05:24:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.523+0000 7f377de00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 0x7f375c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.523+0000 7f377de00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 0x7f375c075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3770007a10 tx=0x7f3770009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.527+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f37680ac030 con 0x7f37800d01e0 2024-09-17T05:24:02.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:02.703+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f378006e030 con 0x7f37800d01e0 2024-09-17T05:24:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.307+0000 7f3776a00640 1 -- 172.21.15.43:0/4269717162 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f3768016350 con 0x7f37800d01e0 2024-09-17T05:24:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:24:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 msgr2=0x7f375c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 0x7f375c075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3770007a10 tx=0x7f3770009040 comp rx=0 tx=0).stop 2024-09-17T05:24:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 msgr2=0x7f37800aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3768000c10 tx=0x7f3768005780 comp rx=0 tx=0).stop 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 shutdown_connections 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f375c073200 0x7f375c075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37800d05f0 0x7f37800b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37800d01e0 0x7f37800aff30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 --2- 172.21.15.43:0/4269717162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37800cddf0 0x7f37801521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 >> 172.21.15.43:0/4269717162 conn(0x7f37800ceec0 msgr2=0x7f37800b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:03.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 shutdown_connections 2024-09-17T05:24:03.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.315+0000 7f3784a00640 1 -- 172.21.15.43:0/4269717162 wait complete. 2024-09-17T05:24:03.348 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:24:03.348 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:24:03.348 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:24:03.348 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:24:03.348 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:24:03.348 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:24:03.349 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:24:03.349 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:24:03.353 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:24:03.353 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:24:03.353 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:24:03.353 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:24:03.354 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:24:03.354 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:24:03.354 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:24:03.354 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:24:03.362 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:24:03.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:24:03.363 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:24:03.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:24:03.368 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:24:03.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:24:03.376 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:24:03.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:24:03.407 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:24:03.408 DEBUG:teuthology.parallel:result is None 2024-09-17T05:24:03.410 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:24:03.410 DEBUG:teuthology.parallel:result is None 2024-09-17T05:24:03.418 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:24:03.419 DEBUG:teuthology.parallel:result is None 2024-09-17T05:24:03.421 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:24:03.421 DEBUG:teuthology.parallel:result is None 2024-09-17T05:24:03.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:24:03.508 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:24:03.513 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:24:03.522 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:24:03.525 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:24:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 -- 172.21.15.43:0/722893823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc0737c0 msgr2=0x7f5ddc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 --2- 172.21.15.43:0/722893823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc0737c0 0x7f5ddc0700a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009bf0 tx=0x7f5dd401ca80 comp rx=0 tx=0).stop 2024-09-17T05:24:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 -- 172.21.15.43:0/722893823 shutdown_connections 2024-09-17T05:24:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 --2- 172.21.15.43:0/722893823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc0737c0 0x7f5ddc0700a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 --2- 172.21.15.43:0/722893823 >> v2:172.21.15.62:3300/0 conn(0x7f5ddc15fe10 0x7f5ddc0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 -- 172.21.15.43:0/722893823 >> 172.21.15.43:0/722893823 conn(0x7f5ddc0b23d0 msgr2=0x7f5ddc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 -- 172.21.15.43:0/722893823 shutdown_connections 2024-09-17T05:24:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.587+0000 7f5de3e00640 1 -- 172.21.15.43:0/722893823 wait complete. 2024-09-17T05:24:03.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 Processor -- start 2024-09-17T05:24:03.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 -- start start 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 0x7f5ddc1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 0x7f5ddc157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc16a0c0 con 0x7f5ddc06fc90 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc169f40 con 0x7f5ddc153a30 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de3e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ddc16a240 con 0x7f5ddc15fe10 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40952/0 (socket says 172.21.15.43:40952) 2024-09-17T05:24:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de1400640 1 -- 172.21.15.43:0/3809417716 learned_addr learned my addr 172.21.15.43:0/3809417716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de0a00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 0x7f5ddc1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.591+0000 7f5de1e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 0x7f5ddc157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 -- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 msgr2=0x7f5ddc1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 0x7f5ddc1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 -- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 msgr2=0x7f5ddc157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 0x7f5ddc157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 -- 172.21.15.43:0/3809417716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc153e40 con 0x7f5ddc06fc90 2024-09-17T05:24:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de0a00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 0x7f5ddc1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 0x7f5ddc157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de1400640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e9a0 tx=0x7f5dd000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000cd20 con 0x7f5ddc06fc90 2024-09-17T05:24:03.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dd0004020 con 0x7f5ddc06fc90 2024-09-17T05:24:03.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004450 con 0x7f5ddc06fc90 2024-09-17T05:24:03.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1f9b00 con 0x7f5ddc06fc90 2024-09-17T05:24:03.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc158a60 con 0x7f5ddc06fc90 2024-09-17T05:24:03.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc0700a0 con 0x7f5ddc06fc90 2024-09-17T05:24:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5dd0003600 con 0x7f5ddc06fc90 2024-09-17T05:24:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.595+0000 7f5dd9600640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 0x7f5dbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.599+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f5dd00949d0 con 0x7f5ddc06fc90 2024-09-17T05:24:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.599+0000 7f5de0a00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 0x7f5dbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.599+0000 7f5de0a00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 0x7f5dbc075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ad7f0 tx=0x7f5dc8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:03.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.603+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ddc153e40 con 0x7f5ddc06fc90 2024-09-17T05:24:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:03.787+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5ddc0751f0 con 0x7f5ddc06fc90 2024-09-17T05:24:04.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.327+0000 7f5dd9600640 1 -- 172.21.15.43:0/3809417716 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f5dd005fc60 con 0x7f5ddc06fc90 2024-09-17T05:24:04.329 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 msgr2=0x7f5dbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 0x7f5dbc075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ad7f0 tx=0x7f5dc8009040 comp rx=0 tx=0).stop 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 msgr2=0x7f5ddc0996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e9a0 tx=0x7f5dd000ee60 comp rx=0 tx=0).stop 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 shutdown_connections 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dbc073230 0x7f5dbc075740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ddc153a30 0x7f5ddc157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ddc15fe10 0x7f5ddc1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 --2- 172.21.15.43:0/3809417716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ddc06fc90 0x7f5ddc0996d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 >> 172.21.15.43:0/3809417716 conn(0x7f5ddc0b23d0 msgr2=0x7f5ddc0ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 shutdown_connections 2024-09-17T05:24:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.331+0000 7f5de3e00640 1 -- 172.21.15.43:0/3809417716 wait complete. 2024-09-17T05:24:04.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- 172.21.15.43:0/3815228517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c0b1a70 msgr2=0x7f0a3c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 --2- 172.21.15.43:0/3815228517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c0b1e80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009bc0 tx=0x7f0a3401cae0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- 172.21.15.43:0/3815228517 shutdown_connections 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 --2- 172.21.15.43:0/3815228517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c0b1e80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 --2- 172.21.15.43:0/3815228517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c070d60 0x7f0a3c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- 172.21.15.43:0/3815228517 >> 172.21.15.43:0/3815228517 conn(0x7f0a3c0b24c0 msgr2=0x7f0a3c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- 172.21.15.43:0/3815228517 shutdown_connections 2024-09-17T05:24:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- 172.21.15.43:0/3815228517 wait complete. 2024-09-17T05:24:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 Processor -- start 2024-09-17T05:24:04.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.515+0000 7f0a43600640 1 -- start start 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c1524b0 0x7f0a3c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c13faf0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c13f970 con 0x7f0a3c0b1a70 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3c13fc70 con 0x7f0a3c1524b0 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56156/0 (socket says 172.21.15.43:56156) 2024-09-17T05:24:04.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 -- 172.21.15.43:0/3320496234 learned_addr learned my addr 172.21.15.43:0/3320496234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a41600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c1524b0 0x7f0a3c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a40c00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 -- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c1524b0 msgr2=0x7f0a3c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c1524b0 0x7f0a3c0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 -- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 msgr2=0x7f0a3c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 -- 172.21.15.43:0/3320496234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c136cb0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a40c00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a3be00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800e7c0 tx=0x7f0a2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800ccc0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a2800ce80 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1521b0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28005280 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1f9e80 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.519+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c1400b0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.523+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a28005420 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.523+0000 7f0a38c00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 0x7f0a1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:04.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.523+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0a28094870 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.523+0000 7f0a40c00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 0x7f0a1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.523+0000 7f0a40c00640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 0x7f0a1c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a30000f70 tx=0x7f0a30009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.527+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0a2805fbd0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.759+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0a3c06a720 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.759+0000 7f0a38c00640 1 -- 172.21.15.43:0/3320496234 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+6645 (secure 0 0 0) 0x7f0a2805f4d0 con 0x7f0a3c15fbb0 2024-09-17T05:24:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:04.761 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:04.761 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:24:04.763 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:24:04.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:24:04.766 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:24:04.768 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.769 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.770 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.771 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:24:04.772 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulmSoNTFhAA+BqK2UEDNl6m0rEjjXFghQ==", 2024-09-17T05:24:04.774 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulmx5RTFhAAErIkoxIFka5CKIoQHJnRSg==", 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.775 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulmHKVTFhAA1yJ9PDVBQJXVR5rig9gtEA==", 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulmhbVTFhAALQvCX36fxiynoG59LUZlxA==", 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulmcsVTFhAA8v36JL6stjdwfZBoRisVcg==", 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAyEulm1tVTFhAAg/KN2qY9xMq5SG7vRGz/dA==", 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T05:24:04.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:24:04.778 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:24:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:04.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 msgr2=0x7f0a1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 0x7f0a1c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a30000f70 tx=0x7f0a30009040 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 msgr2=0x7f0a3c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800e7c0 tx=0x7f0a2800ec80 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 shutdown_connections 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073200 0x7f0a1c075710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c1524b0 0x7f0a3c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c15fbb0 0x7f0a3c0cec70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 --2- 172.21.15.43:0/3320496234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c0b1a70 0x7f0a3c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 >> 172.21.15.43:0/3320496234 conn(0x7f0a3c0b24c0 msgr2=0x7f0a3c0b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 shutdown_connections 2024-09-17T05:24:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.763+0000 7f0a43600640 1 -- 172.21.15.43:0/3320496234 wait complete. 2024-09-17T05:24:04.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T05:24:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- 172.21.15.43:0/420148705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed4003d00 msgr2=0x7fbed4004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 --2- 172.21.15.43:0/420148705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed4003d00 0x7fbed4004150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c340 tx=0x7fbee001ce60 comp rx=0 tx=0).stop 2024-09-17T05:24:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef5800640 1 -- 172.21.15.43:0/420148705 reap_dead start 2024-09-17T05:24:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- 172.21.15.43:0/420148705 shutdown_connections 2024-09-17T05:24:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- 172.21.15.43:0/420148705 >> 172.21.15.43:0/420148705 conn(0x7fbef00b2810 msgr2=0x7fbef00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- 172.21.15.43:0/420148705 shutdown_connections 2024-09-17T05:24:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- 172.21.15.43:0/420148705 wait complete. 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 Processor -- start 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.947+0000 7fbef7800640 1 -- start start 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed4003d00 0x7fbef0164140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef0164ce0 0x7fbef013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbef016b020 con 0x7fbef00b1900 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbef016aea0 con 0x7fbed4003d00 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbef016b1a0 con 0x7fbef0164ce0 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56174/0 (socket says 172.21.15.43:56174) 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 -- 172.21.15.43:0/2362533369 learned_addr learned my addr 172.21.15.43:0/2362533369 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef4e00640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed4003d00 0x7fbef0164140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbee7e00640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef0164ce0 0x7fbef013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 -- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef0164ce0 msgr2=0x7fbef013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef0164ce0 0x7fbef013fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 -- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed4003d00 msgr2=0x7fbef0164140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed4003d00 0x7fbef0164140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 -- 172.21.15.43:0/2362533369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef00b2250 con 0x7fbef00b1900 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef5800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbee8006fd0 tx=0x7fbee8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8011900 con 0x7fbef00b1900 2024-09-17T05:24:05.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbee8011aa0 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8010420 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef01408b0 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef0140e70 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef0165230 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.951+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbee80105c0 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.955+0000 7fbeed600640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 0x7fbecc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.955+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fbee80946d0 con 0x7fbef00b1900 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.955+0000 7fbef4e00640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 0x7fbecc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.955+0000 7fbef4e00640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 0x7fbecc075740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ab000 tx=0x7fbee0019ca0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:04.959+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbef00b2250 con 0x7fbef00b1900 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.195+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fbef0071150 con 0x7fbef00b1900 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.199+0000 7fbeed600640 1 -- 172.21.15.43:0/2362533369 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v15) ==== 90+0+0 (secure 0 0 0) 0x7fbee805f9a0 con 0x7fbef00b1900 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 msgr2=0x7fbecc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 0x7fbecc075740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ab000 tx=0x7fbee0019ca0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 msgr2=0x7fbef00b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbee8006fd0 tx=0x7fbee8003f90 comp rx=0 tx=0).stop 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 shutdown_connections 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbecc073230 0x7fbecc075740 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbef00b1900 0x7fbef00b1d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbef0164ce0 0x7fbef013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 --2- 172.21.15.43:0/2362533369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed4003d00 0x7fbef0164140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 >> 172.21.15.43:0/2362533369 conn(0x7fbef00b2810 msgr2=0x7fbef00ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:05.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 shutdown_connections 2024-09-17T05:24:05.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.203+0000 7fbef7800640 1 -- 172.21.15.43:0/2362533369 wait complete. 2024-09-17T05:24:05.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T05:24:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- 172.21.15.43:0/3041728180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d540b3af0 msgr2=0x7f8d540ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/3041728180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d540b3af0 0x7f8d540ad0b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8d54072b60 tx=0x7f8d4c01cac0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- 172.21.15.43:0/3041728180 shutdown_connections 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/3041728180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d540b3af0 0x7f8d540ad0b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/3041728180 >> v2:172.21.15.62:3300/0 conn(0x7f8d54155d70 0x7f8d5415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- 172.21.15.43:0/3041728180 >> 172.21.15.43:0/3041728180 conn(0x7f8d540b23d0 msgr2=0x7f8d540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- 172.21.15.43:0/3041728180 shutdown_connections 2024-09-17T05:24:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- 172.21.15.43:0/3041728180 wait complete. 2024-09-17T05:24:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 Processor -- start 2024-09-17T05:24:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.407+0000 7f8d5bc00640 1 -- start start 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d5406f110 0x7f8d5409be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 0x7f8d540cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54168c60 con 0x7f8d5409c490 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54168ae0 con 0x7f8d54155d70 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8d54168de0 con 0x7f8d5406f110 2024-09-17T05:24:05.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56182/0 (socket says 172.21.15.43:56182) 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 -- 172.21.15.43:0/2602312221 learned_addr learned my addr 172.21.15.43:0/2602312221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59200640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d5406f110 0x7f8d5409be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d53e00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 0x7f8d540cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 -- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d5406f110 msgr2=0x7f8d5409be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d5406f110 0x7f8d5409be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 -- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 msgr2=0x7f8d540cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 0x7f8d540cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 -- 172.21.15.43:0/2602312221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d540c8010 con 0x7f8d5409c490 2024-09-17T05:24:05.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d53e00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 0x7f8d540cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d59c00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c01a6f0 tx=0x7f8d4c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0bc070 con 0x7f8d5409c490 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d4c0042f0 con 0x7f8d5409c490 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d540c82f0 con 0x7f8d5409c490 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0b2040 con 0x7f8d5409c490 2024-09-17T05:24:05.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d540cccc0 con 0x7f8d5409c490 2024-09-17T05:24:05.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.411+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d540ad0b0 con 0x7f8d5409c490 2024-09-17T05:24:05.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.415+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8d4c0bf070 con 0x7f8d5409c490 2024-09-17T05:24:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.415+0000 7f8d50c00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 0x7f8d34075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.415+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8d4c12dd00 con 0x7f8d5409c490 2024-09-17T05:24:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.415+0000 7f8d59200640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 0x7f8d34075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.415+0000 7f8d59200640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 0x7f8d34075740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d480058d0 tx=0x7f8d48005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.419+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d4c016770 con 0x7f8d5409c490 2024-09-17T05:24:05.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.651+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f8d5415fea0 con 0x7f8d5409c490 2024-09-17T05:24:05.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.655+0000 7f8d50c00640 1 -- 172.21.15.43:0/2602312221 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v16) ==== 90+0+0 (secure 0 0 0) 0x7f8d4c0f8fd0 con 0x7f8d5409c490 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 msgr2=0x7f8d34075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 0x7f8d34075740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d480058d0 tx=0x7f8d48005860 comp rx=0 tx=0).stop 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 msgr2=0x7f8d540c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c01a6f0 tx=0x7f8d4c009310 comp rx=0 tx=0).stop 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 shutdown_connections 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073230 0x7f8d34075740 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d54155d70 0x7f8d540cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5409c490 0x7f8d540c46d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 --2- 172.21.15.43:0/2602312221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d5406f110 0x7f8d5409be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 >> 172.21.15.43:0/2602312221 conn(0x7f8d540b23d0 msgr2=0x7f8d5406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:05.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 shutdown_connections 2024-09-17T05:24:05.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.659+0000 7f8d5bc00640 1 -- 172.21.15.43:0/2602312221 wait complete. 2024-09-17T05:24:05.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T05:24:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 -- 172.21.15.43:0/905469550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 msgr2=0x7faeec004170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 --2- 172.21.15.43:0/905469550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faeec004170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faef8009d00 tx=0x7faef801d3f0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 -- 172.21.15.43:0/905469550 shutdown_connections 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 --2- 172.21.15.43:0/905469550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faeec004170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 -- 172.21.15.43:0/905469550 >> 172.21.15.43:0/905469550 conn(0x7faf040b23d0 msgr2=0x7faf040d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 -- 172.21.15.43:0/905469550 shutdown_connections 2024-09-17T05:24:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.839+0000 7faf0b600640 1 -- 172.21.15.43:0/905469550 wait complete. 2024-09-17T05:24:05.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 Processor -- start 2024-09-17T05:24:05.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 -- start start 2024-09-17T05:24:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf040c41d0 0x7faf040c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf040c4b20 0x7faf041fa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf040ad5d0 con 0x7faeec003d20 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf040ad450 con 0x7faf040c4b20 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faf040ad750 con 0x7faf040c41d0 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56204/0 (socket says 172.21.15.43:56204) 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 -- 172.21.15.43:0/2813067065 learned_addr learned my addr 172.21.15.43:0/2813067065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf09600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf040c4b20 0x7faf041fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf03e00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf040c41d0 0x7faf040c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 -- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf040c41d0 msgr2=0x7faf040c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf040c41d0 0x7faf040c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 -- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf040c4b20 msgr2=0x7faf041fa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf040c4b20 0x7faf041fa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 -- 172.21.15.43:0/2813067065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf040c6140 con 0x7faeec003d20 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf08c00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faef8009d00 tx=0x7faef80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80048c0 con 0x7faeec003d20 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef80bc070 con 0x7faeec003d20 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8004dd0 con 0x7faeec003d20 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.843+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf040c9110 con 0x7faeec003d20 2024-09-17T05:24:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf040c8e30 con 0x7faeec003d20 2024-09-17T05:24:05.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf040ae470 con 0x7faeec003d20 2024-09-17T05:24:05.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faef80b5040 con 0x7faeec003d20 2024-09-17T05:24:05.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf00c00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:05.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7faef812f1a0 con 0x7faeec003d20 2024-09-17T05:24:05.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.847+0000 7faf03e00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.851+0000 7faf03e00640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 0x7faee4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf0415d230 tx=0x7faef0006c00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:05.851+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faef8017100 con 0x7faeec003d20 2024-09-17T05:24:06.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.087+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7faf040b1b60 con 0x7faeec003d20 2024-09-17T05:24:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf00c00640 1 -- 172.21.15.43:0/2813067065 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v17) ==== 90+0+0 (secure 0 0 0) 0x7faef80f9e20 con 0x7faeec003d20 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 msgr2=0x7faee4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 0x7faee4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf0415d230 tx=0x7faef0006c00 comp rx=0 tx=0).stop 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 msgr2=0x7faf041f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faef8009d00 tx=0x7faef80046d0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 shutdown_connections 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf040c4b20 0x7faf041fa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf040c41d0 0x7faf040c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 --2- 172.21.15.43:0/2813067065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeec003d20 0x7faf041f6590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 >> 172.21.15.43:0/2813067065 conn(0x7faf040b23d0 msgr2=0x7faf0415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 shutdown_connections 2024-09-17T05:24:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.091+0000 7faf0b600640 1 -- 172.21.15.43:0/2813067065 wait complete. 2024-09-17T05:24:06.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T05:24:06.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 -- 172.21.15.43:0/950591103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5900155d70 msgr2=0x7f590015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 --2- 172.21.15.43:0/950591103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5900155d70 0x7f590015c7e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009840 tx=0x7f58f001c790 comp rx=0 tx=0).stop 2024-09-17T05:24:06.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 -- 172.21.15.43:0/950591103 shutdown_connections 2024-09-17T05:24:06.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 --2- 172.21.15.43:0/950591103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5900155d70 0x7f590015c7e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 -- 172.21.15.43:0/950591103 >> 172.21.15.43:0/950591103 conn(0x7f59000b23d0 msgr2=0x7f59000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:06.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 -- 172.21.15.43:0/950591103 shutdown_connections 2024-09-17T05:24:06.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.271+0000 7f5905a00640 1 -- 172.21.15.43:0/950591103 wait complete. 2024-09-17T05:24:06.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 Processor -- start 2024-09-17T05:24:06.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- start start 2024-09-17T05:24:06.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5900155d70 0x7f59001545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59000c6b20 0x7f5900154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5900169e10 con 0x7f590006f110 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5900169c90 con 0x7f5900155d70 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5900169f90 con 0x7f59000c6b20 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56216/0 (socket says 172.21.15.43:56216) 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 -- 172.21.15.43:0/2366513667 learned_addr learned my addr 172.21.15.43:0/2366513667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58f7e00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5900155d70 0x7f59001545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58ff400640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59000c6b20 0x7f5900154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 -- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59000c6b20 msgr2=0x7f5900154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59000c6b20 0x7f5900154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 -- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5900155d70 msgr2=0x7f59001545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5900155d70 0x7f59001545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 -- 172.21.15.43:0/2366513667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5900154290 con 0x7f590006f110 2024-09-17T05:24:06.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58fea00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58f001a3c0 tx=0x7f58f00a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:06.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f00bb070 con 0x7f590006f110 2024-09-17T05:24:06.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58f0004470 con 0x7f590006f110 2024-09-17T05:24:06.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59000c3ed0 con 0x7f590006f110 2024-09-17T05:24:06.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f00b2040 con 0x7f590006f110 2024-09-17T05:24:06.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.275+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59000c6600 con 0x7f590006f110 2024-09-17T05:24:06.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59000c43f0 con 0x7f590006f110 2024-09-17T05:24:06.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58f00be070 con 0x7f590006f110 2024-09-17T05:24:06.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f58f6a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 0x7f58dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f58f012d740 con 0x7f590006f110 2024-09-17T05:24:06.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f58f7e00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 0x7f58dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.279+0000 7f58f7e00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 0x7f58dc075740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f590015e460 tx=0x7f58e8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:06.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.283+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f59000c43f0 con 0x7f590006f110 2024-09-17T05:24:06.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.523+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f590015c7e0 con 0x7f590006f110 2024-09-17T05:24:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f58f6a00640 1 -- 172.21.15.43:0/2366513667 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v18) ==== 90+0+0 (secure 0 0 0) 0x7f58f0016410 con 0x7f590006f110 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 msgr2=0x7f58dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 0x7f58dc075740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f590015e460 tx=0x7f58e8009040 comp rx=0 tx=0).stop 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 msgr2=0x7f5900153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58f001a3c0 tx=0x7f58f00a55e0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 shutdown_connections 2024-09-17T05:24:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f58dc073230 0x7f58dc075740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5900155d70 0x7f59001545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f59000c6b20 0x7f5900154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 --2- 172.21.15.43:0/2366513667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f590006f110 0x7f5900153d50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 >> 172.21.15.43:0/2366513667 conn(0x7f59000b23d0 msgr2=0x7f59000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 shutdown_connections 2024-09-17T05:24:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.527+0000 7f5905a00640 1 -- 172.21.15.43:0/2366513667 wait complete. 2024-09-17T05:24:06.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T05:24:06.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 -- 172.21.15.43:0/929859997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 msgr2=0x7f20481609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 --2- 172.21.15.43:0/929859997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f20481609c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f204809bbe0 tx=0x7f204001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 -- 172.21.15.43:0/929859997 shutdown_connections 2024-09-17T05:24:06.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 --2- 172.21.15.43:0/929859997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f20481609c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 --2- 172.21.15.43:0/929859997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480b7ce0 0x7f20480d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 -- 172.21.15.43:0/929859997 >> 172.21.15.43:0/929859997 conn(0x7f20480cee80 msgr2=0x7f20480cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:06.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 -- 172.21.15.43:0/929859997 shutdown_connections 2024-09-17T05:24:06.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.707+0000 7f204e000640 1 -- 172.21.15.43:0/929859997 wait complete. 2024-09-17T05:24:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 Processor -- start 2024-09-17T05:24:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 -- start start 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20480b7ce0 0x7f2048157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480d0d80 0x7f20480afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2048169ea0 con 0x7f20480cd580 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20480b37a0 con 0x7f20480b7ce0 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f204e000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f204816a020 con 0x7f20480d0d80 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56238/0 (socket says 172.21.15.43:56238) 2024-09-17T05:24:06.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 -- 172.21.15.43:0/634319003 learned_addr learned my addr 172.21.15.43:0/634319003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f2047400640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20480b7ce0 0x7f2048157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f2047e00640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480d0d80 0x7f20480afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 -- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480d0d80 msgr2=0x7f20480afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480d0d80 0x7f20480afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 -- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20480b7ce0 msgr2=0x7f2048157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20480b7ce0 0x7f2048157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.711+0000 7f203ea00640 1 -- 172.21.15.43:0/634319003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048157ea0 con 0x7f20480cd580 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203ea00640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f203000c430 tx=0x7f203000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030011790 con 0x7f20480cd580 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2030017070 con 0x7f20480cd580 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20481f8ce0 con 0x7f20480cd580 2024-09-17T05:24:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030011ce0 con 0x7f20480cd580 2024-09-17T05:24:06.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481f8fa0 con 0x7f20480cd580 2024-09-17T05:24:06.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20480d0840 con 0x7f20480cd580 2024-09-17T05:24:06.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f203001c460 con 0x7f20480cd580 2024-09-17T05:24:06.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 0x7f2024075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:06.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.715+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f2030093c80 con 0x7f20480cd580 2024-09-17T05:24:06.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.719+0000 7f2047400640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 0x7f2024075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:06.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.719+0000 7f2047400640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 0x7f2024075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2048071220 tx=0x7f203800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:06.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.723+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f203005e9c0 con 0x7f20480cd580 2024-09-17T05:24:06.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.959+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f20481580d0 con 0x7f20480cd580 2024-09-17T05:24:06.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f203fe00640 1 -- 172.21.15.43:0/634319003 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v19) ==== 97+0+0 (secure 0 0 0) 0x7f20300633f0 con 0x7f20480cd580 2024-09-17T05:24:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 msgr2=0x7f2024075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 0x7f2024075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2048071220 tx=0x7f203800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 msgr2=0x7f2048155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f203000c430 tx=0x7f203000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 shutdown_connections 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2024073200 0x7f2024075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20480d0d80 0x7f20480afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20480cd580 0x7f2048155590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.963+0000 7f204e000640 1 --2- 172.21.15.43:0/634319003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20480b7ce0 0x7f2048157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.967+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 >> 172.21.15.43:0/634319003 conn(0x7f20480cee80 msgr2=0x7f20480cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.967+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 shutdown_connections 2024-09-17T05:24:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:06.967+0000 7f204e000640 1 -- 172.21.15.43:0/634319003 wait complete. 2024-09-17T05:24:06.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T05:24:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- 172.21.15.43:0/4003994409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc06f110 msgr2=0x7f35cc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 --2- 172.21.15.43:0/4003994409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc06f110 0x7f35cc0ad0b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009c80 tx=0x7f35c401cce0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- 172.21.15.43:0/4003994409 shutdown_connections 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 --2- 172.21.15.43:0/4003994409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc06f110 0x7f35cc0ad0b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 --2- 172.21.15.43:0/4003994409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc0b3af0 0x7f35cc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- 172.21.15.43:0/4003994409 >> 172.21.15.43:0/4003994409 conn(0x7f35cc0b23d0 msgr2=0x7f35cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- 172.21.15.43:0/4003994409 shutdown_connections 2024-09-17T05:24:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- 172.21.15.43:0/4003994409 wait complete. 2024-09-17T05:24:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 Processor -- start 2024-09-17T05:24:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.147+0000 7f35d3a00640 1 -- start start 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 0x7f35cc155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 0x7f35cc1624f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc168be0 con 0x7f35cc155d70 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc168a60 con 0x7f35cc0cd5d0 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc168d60 con 0x7f35cc06f110 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56254/0 (socket says 172.21.15.43:56254) 2024-09-17T05:24:07.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 -- 172.21.15.43:0/456857143 learned_addr learned my addr 172.21.15.43:0/456857143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d1a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 0x7f35cc1624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 -- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 msgr2=0x7f35cc155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d1000640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 0x7f35cc155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 0x7f35cc155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 -- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 msgr2=0x7f35cc1624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 0x7f35cc1624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 -- 172.21.15.43:0/456857143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc161200 con 0x7f35cc155d70 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d1a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 0x7f35cc1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d1000640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 0x7f35cc155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:07.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35cbe00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35b800d980 tx=0x7f35b800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8017070 con 0x7f35cc155d70 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35b8009d20 con 0x7f35cc155d70 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1614e0 con 0x7f35cc155d70 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004280 con 0x7f35cc155d70 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc0cd070 con 0x7f35cc155d70 2024-09-17T05:24:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.151+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc0ad0b0 con 0x7f35cc155d70 2024-09-17T05:24:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.155+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f35b8005020 con 0x7f35cc155d70 2024-09-17T05:24:07.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.155+0000 7f35c8c00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 0x7f35ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.155+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f35b8094410 con 0x7f35cc155d70 2024-09-17T05:24:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.155+0000 7f35d1000640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 0x7f35ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.155+0000 7f35d1000640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 0x7f35ac075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0715c0 tx=0x7f35c0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.159+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f35b8020020 con 0x7f35cc155d70 2024-09-17T05:24:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.395+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f35cc160f90 con 0x7f35cc155d70 2024-09-17T05:24:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35c8c00640 1 -- 172.21.15.43:0/456857143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v20) ==== 90+0+0 (secure 0 0 0) 0x7f35b805f6e0 con 0x7f35cc155d70 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 msgr2=0x7f35ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 0x7f35ac075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0715c0 tx=0x7f35c0009040 comp rx=0 tx=0).stop 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 msgr2=0x7f35cc161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35b800d980 tx=0x7f35b800de40 comp rx=0 tx=0).stop 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 shutdown_connections 2024-09-17T05:24:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35ac073200 0x7f35ac075710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35cc0cd5d0 0x7f35cc1624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35cc155d70 0x7f35cc161fb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 --2- 172.21.15.43:0/456857143 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35cc06f110 0x7f35cc155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 >> 172.21.15.43:0/456857143 conn(0x7f35cc0b23d0 msgr2=0x7f35cc15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 shutdown_connections 2024-09-17T05:24:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.399+0000 7f35d3a00640 1 -- 172.21.15.43:0/456857143 wait complete. 2024-09-17T05:24:07.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:24:07.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 -- 172.21.15.43:0/3842512105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0240 msgr2=0x7f3cac0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3842512105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0240 0x7f3cac0cd040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009840 tx=0x7f3c9401c820 comp rx=0 tx=0).stop 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 -- 172.21.15.43:0/3842512105 shutdown_connections 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3842512105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0240 0x7f3cac0cd040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 -- 172.21.15.43:0/3842512105 >> 172.21.15.43:0/3842512105 conn(0x7f3cac0cee60 msgr2=0x7f3cac0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 -- 172.21.15.43:0/3842512105 shutdown_connections 2024-09-17T05:24:07.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.579+0000 7f3cb1800640 1 -- 172.21.15.43:0/3842512105 wait complete. 2024-09-17T05:24:07.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 Processor -- start 2024-09-17T05:24:07.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 -- start start 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3cac09ba60 0x7f3cac0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac0d0240 0x7f3cac14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac16aae0 con 0x7f3cac0d0650 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac16a960 con 0x7f3cac0d0240 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cb1800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3cac16ac60 con 0x7f3cac09ba60 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56268/0 (socket says 172.21.15.43:56268) 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3cab400640 1 -- 172.21.15.43:0/3871398732 learned_addr learned my addr 172.21.15.43:0/3871398732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3caaa00640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3cac09ba60 0x7f3cac0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.583+0000 7f3ca3e00640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac0d0240 0x7f3cac14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 -- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3cac09ba60 msgr2=0x7f3cac0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3cac09ba60 0x7f3cac0c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 -- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac0d0240 msgr2=0x7f3cac14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac0d0240 0x7f3cac14c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 -- 172.21.15.43:0/3871398732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac1fa040 con 0x7f3cac0d0650 2024-09-17T05:24:07.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cab400640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400cae0 tx=0x7f3ca400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4016020 con 0x7f3cac0d0650 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ca400ab40 con 0x7f3cac0d0650 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac1fa320 con 0x7f3cac0d0650 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400ce20 con 0x7f3cac0d0650 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1fa8e0 con 0x7f3cac0d0650 2024-09-17T05:24:07.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac0b3130 con 0x7f3cac0d0650 2024-09-17T05:24:07.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.587+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ca4019020 con 0x7f3cac0d0650 2024-09-17T05:24:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.591+0000 7f3ca2a00640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 0x7f3c88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.591+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3ca4094210 con 0x7f3cac0d0650 2024-09-17T05:24:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.591+0000 7f3caaa00640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 0x7f3c88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.591+0000 7f3caaa00640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 0x7f3c88075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c001650 tx=0x7f3c9c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.595+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ca4020020 con 0x7f3cac0d0650 2024-09-17T05:24:07.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.775+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3cac0d1400 con 0x7f3cac0d0650 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.775+0000 7f3ca2a00640 1 -- 172.21.15.43:0/3871398732 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f3ca405f4e0 con 0x7f3cac0d0650 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:24:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:07.777 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:24:07.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 msgr2=0x7f3c88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 0x7f3c88075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c001650 tx=0x7f3c9c009040 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 msgr2=0x7f3cac0ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400cae0 tx=0x7f3ca400cbd0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 shutdown_connections 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c88073200 0x7f3c88075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac0d0240 0x7f3cac14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cac0d0650 0x7f3cac0ca070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 --2- 172.21.15.43:0/3871398732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3cac09ba60 0x7f3cac0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 >> 172.21.15.43:0/3871398732 conn(0x7f3cac0cee60 msgr2=0x7f3cac0b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 shutdown_connections 2024-09-17T05:24:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.779+0000 7f3cb1800640 1 -- 172.21.15.43:0/3871398732 wait complete. 2024-09-17T05:24:07.798 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:24:07.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 -- 172.21.15.43:0/230210927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940074c60 msgr2=0x7f89400b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 --2- 172.21.15.43:0/230210927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940074c60 0x7f89400b3b80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89400d0140 tx=0x7f893401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 -- 172.21.15.43:0/230210927 shutdown_connections 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 --2- 172.21.15.43:0/230210927 >> v2:172.21.15.62:3300/0 conn(0x7f89400cf980 0x7f89400731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 --2- 172.21.15.43:0/230210927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940074c60 0x7f89400b3b80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 -- 172.21.15.43:0/230210927 >> 172.21.15.43:0/230210927 conn(0x7f894006ce60 msgr2=0x7f894006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 -- 172.21.15.43:0/230210927 shutdown_connections 2024-09-17T05:24:07.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.959+0000 7f8947e00640 1 -- 172.21.15.43:0/230210927 wait complete. 2024-09-17T05:24:07.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 Processor -- start 2024-09-17T05:24:07.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 -- start start 2024-09-17T05:24:07.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8940074c60 0x7f894015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894015b7e0 0x7f894014a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89400cecb0 con 0x7f8940137df0 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89400ceb30 con 0x7f894015b7e0 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89400cee30 con 0x7f8940074c60 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56280/0 (socket says 172.21.15.43:56280) 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 -- 172.21.15.43:0/2389288276 learned_addr learned my addr 172.21.15.43:0/2389288276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945400640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8940074c60 0x7f894015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8944a00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894015b7e0 0x7f894014a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 -- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8940074c60 msgr2=0x7f894015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8940074c60 0x7f894015e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 -- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894015b7e0 msgr2=0x7f894014a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894015b7e0 0x7f894014a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 -- 172.21.15.43:0/2389288276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8940138740 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8945e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f893800e910 tx=0x7f893800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938004d30 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f893800cb20 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89401f54d0 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.963+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938005020 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.967+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89401538a0 con 0x7f8940137df0 2024-09-17T05:24:07.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.967+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89400b3b80 con 0x7f8940137df0 2024-09-17T05:24:07.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.967+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89380051c0 con 0x7f8940137df0 2024-09-17T05:24:07.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.967+0000 7f893d600640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 0x7f8920075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:07.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.967+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8938094730 con 0x7f8940137df0 2024-09-17T05:24:07.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.971+0000 7f8945400640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 0x7f8920075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:07.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.971+0000 7f8945400640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 0x7f8920075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f89340af000 tx=0x7f893401b100 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:07.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:07.975+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8940138740 con 0x7f8940137df0 2024-09-17T05:24:08.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.151+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f89400b0bd0 con 0x7f8940137df0 2024-09-17T05:24:08.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f893d600640 1 -- 172.21.15.43:0/2389288276 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f8938016030 con 0x7f8940137df0 2024-09-17T05:24:08.406 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 msgr2=0x7f8920075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 0x7f8920075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f89340af000 tx=0x7f893401b100 comp rx=0 tx=0).stop 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 msgr2=0x7f8940138200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f893800e910 tx=0x7f893800edd0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 shutdown_connections 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8920073200 0x7f8920075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894015b7e0 0x7f894014a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8940137df0 0x7f8940138200 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 --2- 172.21.15.43:0/2389288276 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8940074c60 0x7f894015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 >> 172.21.15.43:0/2389288276 conn(0x7f894006ce60 msgr2=0x7f894015fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:08.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 shutdown_connections 2024-09-17T05:24:08.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.407+0000 7f8947e00640 1 -- 172.21.15.43:0/2389288276 wait complete. 2024-09-17T05:24:08.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:24:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.599+0000 7fd8bda00640 1 -- 172.21.15.43:0/897139747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 msgr2=0x7fd8b80cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 --2- 172.21.15.43:0/897139747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80cdbe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009840 tx=0x7fd8a801c820 comp rx=0 tx=0).stop 2024-09-17T05:24:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- 172.21.15.43:0/897139747 shutdown_connections 2024-09-17T05:24:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 --2- 172.21.15.43:0/897139747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80cdbe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- 172.21.15.43:0/897139747 >> 172.21.15.43:0/897139747 conn(0x7fd8b80b2de0 msgr2=0x7fd8b80b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- 172.21.15.43:0/897139747 shutdown_connections 2024-09-17T05:24:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- 172.21.15.43:0/897139747 wait complete. 2024-09-17T05:24:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 Processor -- start 2024-09-17T05:24:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- start start 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b80cc900 0x7fd8b814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b80ce120 0x7fd8b80af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8169fe0 con 0x7fd8b80d0830 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8169e60 con 0x7fd8b80cc900 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.603+0000 7fd8bda00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b816a160 con 0x7fd8b80ce120 2024-09-17T05:24:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56298/0 (socket says 172.21.15.43:56298) 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 -- 172.21.15.43:0/112256841 learned_addr learned my addr 172.21.15.43:0/112256841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8b7e00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b80cc900 0x7fd8b814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8b7400640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b80ce120 0x7fd8b80af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 -- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b80ce120 msgr2=0x7fd8b80af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b80ce120 0x7fd8b80af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 -- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b80cc900 msgr2=0x7fd8b814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b80cc900 0x7fd8b814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 -- 172.21.15.43:0/112256841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b814e960 con 0x7fd8b80d0830 2024-09-17T05:24:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bd000640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c710 tx=0x7fd8b000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0003b80 con 0x7fd8b80d0830 2024-09-17T05:24:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b0003d20 con 0x7fd8b80d0830 2024-09-17T05:24:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b814ec40 con 0x7fd8b80d0830 2024-09-17T05:24:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000aab0 con 0x7fd8b80d0830 2024-09-17T05:24:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b814f640 con 0x7fd8b80d0830 2024-09-17T05:24:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b814e6d0 con 0x7fd8b80d0830 2024-09-17T05:24:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.607+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8b000ac50 con 0x7fd8b80d0830 2024-09-17T05:24:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.611+0000 7fd8afe00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 0x7fd898075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.611+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fd8b0093d60 con 0x7fd8b80d0830 2024-09-17T05:24:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.611+0000 7fd8b7e00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 0x7fd898075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.611+0000 7fd8b7e00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 0x7fd898075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af000 tx=0x7fd8a801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:08.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.615+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8b000ce20 con 0x7fd8b80d0830 2024-09-17T05:24:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:08.791+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd8b814f160 con 0x7fd8b80d0830 2024-09-17T05:24:09.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.407+0000 7fd8afe00640 1 -- 172.21.15.43:0/112256841 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fd8b814f160 con 0x7fd8b80d0830 2024-09-17T05:24:09.410 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 msgr2=0x7fd898075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 0x7fd898075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80af000 tx=0x7fd8a801b060 comp rx=0 tx=0).stop 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 msgr2=0x7fd8b80af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c710 tx=0x7fd8b000cbd0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 shutdown_connections 2024-09-17T05:24:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd898073200 0x7fd898075710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b80d0830 0x7fd8b80af9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8b80ce120 0x7fd8b80af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 --2- 172.21.15.43:0/112256841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b80cc900 0x7fd8b814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.411+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 >> 172.21.15.43:0/112256841 conn(0x7fd8b80b2de0 msgr2=0x7fd8b8158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:09.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.415+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 shutdown_connections 2024-09-17T05:24:09.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.415+0000 7fd8bda00640 1 -- 172.21.15.43:0/112256841 wait complete. 2024-09-17T05:24:09.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:24:09.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- 172.21.15.43:0/3992929116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300737c0 msgr2=0x7fcb300700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 --2- 172.21.15.43:0/3992929116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300737c0 0x7fcb300700a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009bc0 tx=0x7fcb2801ca80 comp rx=0 tx=0).stop 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- 172.21.15.43:0/3992929116 shutdown_connections 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 --2- 172.21.15.43:0/3992929116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300737c0 0x7fcb300700a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 --2- 172.21.15.43:0/3992929116 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 0x7fcb300b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- 172.21.15.43:0/3992929116 >> 172.21.15.43:0/3992929116 conn(0x7fcb300b23d0 msgr2=0x7fcb300d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- 172.21.15.43:0/3992929116 shutdown_connections 2024-09-17T05:24:09.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- 172.21.15.43:0/3992929116 wait complete. 2024-09-17T05:24:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 Processor -- start 2024-09-17T05:24:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.614+0000 7fcb36600640 1 -- start start 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3006fc90 0x7fcb3014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 0x7fcb3015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30169f30 con 0x7fcb300c8140 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30169db0 con 0x7fcb3006fc90 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3016a0b0 con 0x7fcb3015fe10 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56318/0 (socket says 172.21.15.43:56318) 2024-09-17T05:24:09.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 -- 172.21.15.43:0/2523814546 learned_addr learned my addr 172.21.15.43:0/2523814546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2f400640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3006fc90 0x7fcb3014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb27e00640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 0x7fcb3015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 -- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 msgr2=0x7fcb3015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 0x7fcb3015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 -- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3006fc90 msgr2=0x7fcb3014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3006fc90 0x7fcb3014af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 -- 172.21.15.43:0/2523814546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb300cc890 con 0x7fcb300c8140 2024-09-17T05:24:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb2fe00640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb2801c4f0 tx=0x7fcb28003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb280b2040 con 0x7fcb300c8140 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb280bb070 con 0x7fcb300c8140 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb300ca240 con 0x7fcb300c8140 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004370 con 0x7fcb300c8140 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb300cc2d0 con 0x7fcb300c8140 2024-09-17T05:24:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.618+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30075480 con 0x7fcb300c8140 2024-09-17T05:24:09.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.622+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb28004510 con 0x7fcb300c8140 2024-09-17T05:24:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.622+0000 7fcb27400640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 0x7fcb0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.622+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fcb2812f540 con 0x7fcb300c8140 2024-09-17T05:24:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.622+0000 7fcb2f400640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 0x7fcb0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.622+0000 7fcb2f400640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 0x7fcb0c075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009b40 tx=0x7fcb200091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:09.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.626+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb28016790 con 0x7fcb300c8140 2024-09-17T05:24:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:09.838+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcb30158f00 con 0x7fcb300c8140 2024-09-17T05:24:10.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.434+0000 7fcb27400640 1 -- 172.21.15.43:0/2523814546 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fcb280f9f50 con 0x7fcb300c8140 2024-09-17T05:24:10.438 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 msgr2=0x7fcb0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 0x7fcb0c075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009b40 tx=0x7fcb200091e0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 msgr2=0x7fcb3015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcb2801c4f0 tx=0x7fcb28003f40 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 shutdown_connections 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb0c073230 0x7fcb0c075740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb300c8140 0x7fcb3015e6e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3015fe10 0x7fcb3015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 --2- 172.21.15.43:0/2523814546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3006fc90 0x7fcb3014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 >> 172.21.15.43:0/2523814546 conn(0x7fcb300b23d0 msgr2=0x7fcb300adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 shutdown_connections 2024-09-17T05:24:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.446+0000 7fcb36600640 1 -- 172.21.15.43:0/2523814546 wait complete. 2024-09-17T05:24:10.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- 172.21.15.43:0/514282597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044406d570 msgr2=0x7f044406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 --2- 172.21.15.43:0/514282597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044406d570 0x7f044406f1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f042c00a950 tx=0x7f042c01db70 comp rx=0 tx=0).stop 2024-09-17T05:24:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- 172.21.15.43:0/514282597 shutdown_connections 2024-09-17T05:24:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 --2- 172.21.15.43:0/514282597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044406d570 0x7f044406f1a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- 172.21.15.43:0/514282597 >> 172.21.15.43:0/514282597 conn(0x7f04440b23d0 msgr2=0x7f04440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- 172.21.15.43:0/514282597 shutdown_connections 2024-09-17T05:24:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- 172.21.15.43:0/514282597 wait complete. 2024-09-17T05:24:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 Processor -- start 2024-09-17T05:24:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.650+0000 7f0449600640 1 -- start start 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04440cc6d0 0x7f04440ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0444156650 0x7f04441594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0444169720 con 0x7f044415adc0 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04441695a0 con 0x7f04440cc6d0 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f04441698a0 con 0x7f0444156650 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56336/0 (socket says 172.21.15.43:56336) 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 -- 172.21.15.43:0/1872286839 learned_addr learned my addr 172.21.15.43:0/1872286839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0442a00640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04440cc6d0 0x7f04440ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0443400640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0444156650 0x7f04441594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 -- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0444156650 msgr2=0x7f04441594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0444156650 0x7f04441594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 -- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04440cc6d0 msgr2=0x7f04440ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04440cc6d0 0x7f04440ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 -- 172.21.15.43:0/1872286839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04441510d0 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043a000640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f042c00a8a0 tx=0x7f042c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0bd070 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f042c005320 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0444156cd0 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0b3040 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044415b570 con 0x7f044415adc0 2024-09-17T05:24:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.654+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044406ce60 con 0x7f044415adc0 2024-09-17T05:24:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.658+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f042c0046d0 con 0x7f044415adc0 2024-09-17T05:24:10.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.658+0000 7f043b400640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:10.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.658+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f042c12e120 con 0x7f044415adc0 2024-09-17T05:24:10.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.658+0000 7f0442a00640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:10.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.658+0000 7f0442a00640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 0x7f0420075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0434007fd0 tx=0x7f0434009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.662+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0444070000 con 0x7f044415adc0 2024-09-17T05:24:10.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.838+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0444156a60 con 0x7f044415adc0 2024-09-17T05:24:10.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f043b400640 1 -- 172.21.15.43:0/1872286839 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f042c017760 con 0x7f044415adc0 2024-09-17T05:24:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:10.843 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:24:09.841768+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:24:08.162466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:24:08.803372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 msgr2=0x7f0420075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 0x7f0420075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0434007fd0 tx=0x7f0434009040 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 msgr2=0x7f0444156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f042c00a8a0 tx=0x7f042c004620 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 shutdown_connections 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0444156650 0x7f04441594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044415adc0 0x7f0444156110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 --2- 172.21.15.43:0/1872286839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04440cc6d0 0x7f04440ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 >> 172.21.15.43:0/1872286839 conn(0x7f04440b23d0 msgr2=0x7f04440ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 shutdown_connections 2024-09-17T05:24:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:10.842+0000 7f0449600640 1 -- 172.21.15.43:0/1872286839 wait complete. 2024-09-17T05:24:10.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 -- 172.21.15.43:0/2953014409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 msgr2=0x7f1f3006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 --2- 172.21.15.43:0/2953014409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3006f110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009bc0 tx=0x7f1f2801cb10 comp rx=0 tx=0).stop 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 -- 172.21.15.43:0/2953014409 shutdown_connections 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 --2- 172.21.15.43:0/2953014409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3006f110 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 -- 172.21.15.43:0/2953014409 >> 172.21.15.43:0/2953014409 conn(0x7f1f300b23d0 msgr2=0x7f1f300d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 -- 172.21.15.43:0/2953014409 shutdown_connections 2024-09-17T05:24:11.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.022+0000 7f1f38600640 1 -- 172.21.15.43:0/2953014409 wait complete. 2024-09-17T05:24:11.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 Processor -- start 2024-09-17T05:24:11.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 -- start start 2024-09-17T05:24:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30155d70 0x7f1f300cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3009b7b0 0x7f1f3015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30169dc0 con 0x7f1f3015c7e0 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30169c40 con 0x7f1f3009b7b0 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1f30169f40 con 0x7f1f30155d70 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56356/0 (socket says 172.21.15.43:56356) 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 -- 172.21.15.43:0/985956317 learned_addr learned my addr 172.21.15.43:0/985956317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35c00640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30155d70 0x7f1f300cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f36600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3009b7b0 0x7f1f3015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 -- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30155d70 msgr2=0x7f1f300cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30155d70 0x7f1f300cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 -- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3009b7b0 msgr2=0x7f1f3015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3009b7b0 0x7f1f3015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 -- 172.21.15.43:0/985956317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f30074140 con 0x7f1f3015c7e0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f35200640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c390 tx=0x7f1f1c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017070 con 0x7f1f3015c7e0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f1c003600 con 0x7f1f3015c7e0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3006d930 con 0x7f1f3015c7e0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.026+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c003a50 con 0x7f1f3015c7e0 2024-09-17T05:24:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f300746c0 con 0x7f1f3015c7e0 2024-09-17T05:24:11.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3006f110 con 0x7f1f3015c7e0 2024-09-17T05:24:11.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f1c01a070 con 0x7f1f3015c7e0 2024-09-17T05:24:11.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f2d600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 0x7f1f10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c093340 con 0x7f1f3015c7e0 2024-09-17T05:24:11.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.030+0000 7f1f35c00640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 0x7f1f10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.034+0000 7f1f35c00640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 0x7f1f10075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1f24007fd0 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.034+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f1c05dd10 con 0x7f1f3015c7e0 2024-09-17T05:24:11.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.210+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f3015b9d0 con 0x7f1f3015c7e0 2024-09-17T05:24:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.210+0000 7f1f2d600640 1 -- 172.21.15.43:0/985956317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16622 (secure 0 0 0) 0x7f1f1c062740 con 0x7f1f3015c7e0 2024-09-17T05:24:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:11.212 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:24:09.841768+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:24:08.162466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:24:08.803372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:11.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 msgr2=0x7f1f10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 0x7f1f10075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1f24007fd0 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2024-09-17T05:24:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 msgr2=0x7f1f3009b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c390 tx=0x7f1f1c00c850 comp rx=0 tx=0).stop 2024-09-17T05:24:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 shutdown_connections 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f10073230 0x7f1f10075740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f3009b7b0 0x7f1f3015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3015c7e0 0x7f1f3009b0f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 --2- 172.21.15.43:0/985956317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f30155d70 0x7f1f300cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 >> 172.21.15.43:0/985956317 conn(0x7f1f300b23d0 msgr2=0x7f1f3015f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 shutdown_connections 2024-09-17T05:24:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.214+0000 7f1f38600640 1 -- 172.21.15.43:0/985956317 wait complete. 2024-09-17T05:24:11.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3057839321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c141614e0 msgr2=0x7f9c140b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3057839321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c141614e0 0x7f9c140b1a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c14074180 tx=0x7f9c0c01f370 comp rx=0 tx=0).stop 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3057839321 shutdown_connections 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3057839321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c141614e0 0x7f9c140b1a30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3057839321 >> v2:172.21.15.62:3300/0 conn(0x7f9c1409ba60 0x7f9c140d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3057839321 >> 172.21.15.43:0/3057839321 conn(0x7f9c140cee10 msgr2=0x7f9c140cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3057839321 shutdown_connections 2024-09-17T05:24:11.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3057839321 wait complete. 2024-09-17T05:24:11.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 Processor -- start 2024-09-17T05:24:11.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 -- start start 2024-09-17T05:24:11.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.398+0000 7f9c1bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c141609c0 0x7f9c141e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c141614e0 0x7f9c141e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c140ceab0 con 0x7f9c1409ba60 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c140ce930 con 0x7f9c141614e0 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140b1f70 con 0x7f9c141609c0 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56382/0 (socket says 172.21.15.43:56382) 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 -- 172.21.15.43:0/3801266530 learned_addr learned my addr 172.21.15.43:0/3801266530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c0be00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c141609c0 0x7f9c141e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19c00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c141614e0 0x7f9c141e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 -- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c141609c0 msgr2=0x7f9c141e8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c141609c0 0x7f9c141e8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 -- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c141614e0 msgr2=0x7f9c141e92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c141614e0 0x7f9c141e92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 -- 172.21.15.43:0/3801266530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c141e77d0 con 0x7f9c1409ba60 2024-09-17T05:24:11.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c19200640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d770 tx=0x7f9c0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:11.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04009c30 con 0x7f9c1409ba60 2024-09-17T05:24:11.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c0400de80 con 0x7f9c1409ba60 2024-09-17T05:24:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004280 con 0x7f9c1409ba60 2024-09-17T05:24:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c141e7cd0 con 0x7f9c1409ba60 2024-09-17T05:24:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c141e8290 con 0x7f9c1409ba60 2024-09-17T05:24:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c140d01f0 con 0x7f9c1409ba60 2024-09-17T05:24:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.402+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c04003600 con 0x7f9c1409ba60 2024-09-17T05:24:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.406+0000 7f9c11600640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 0x7f9bf0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.406+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9c04094760 con 0x7f9c1409ba60 2024-09-17T05:24:11.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.406+0000 7f9c0be00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 0x7f9bf0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:11.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.406+0000 7f9c0be00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 0x7f9bf0075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c14154c10 tx=0x7f9bfc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.410+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c0405f800 con 0x7f9c1409ba60 2024-09-17T05:24:11.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:11.626+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9c140b3830 con 0x7f9c1409ba60 2024-09-17T05:24:12.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.454+0000 7f9c11600640 1 -- 172.21.15.43:0/3801266530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f9c04066100 con 0x7f9c1409ba60 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 msgr2=0x7f9bf0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 0x7f9bf0075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9c14154c10 tx=0x7f9bfc009040 comp rx=0 tx=0).stop 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 msgr2=0x7f9c1414e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d770 tx=0x7f9c0400dc30 comp rx=0 tx=0).stop 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 shutdown_connections 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9bf0073200 0x7f9bf0075710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c141614e0 0x7f9c141e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c141609c0 0x7f9c141e8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 --2- 172.21.15.43:0/3801266530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1409ba60 0x7f9c1414e500 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 >> 172.21.15.43:0/3801266530 conn(0x7f9c140cee10 msgr2=0x7f9c140cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 shutdown_connections 2024-09-17T05:24:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.458+0000 7f9c1bc00640 1 -- 172.21.15.43:0/3801266530 wait complete. 2024-09-17T05:24:12.483 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:24:12.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 -- 172.21.15.43:0/1322724859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f8003af0 msgr2=0x7ff3f8003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 --2- 172.21.15.43:0/1322724859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f8003af0 0x7ff3f8003f60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c470 tx=0x7ff40401cfa0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff416200640 1 -- 172.21.15.43:0/1322724859 reap_dead start 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 -- 172.21.15.43:0/1322724859 shutdown_connections 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 -- 172.21.15.43:0/1322724859 >> 172.21.15.43:0/1322724859 conn(0x7ff4100b23d0 msgr2=0x7ff4100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 -- 172.21.15.43:0/1322724859 shutdown_connections 2024-09-17T05:24:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 -- 172.21.15.43:0/1322724859 wait complete. 2024-09-17T05:24:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.650+0000 7ff418200640 1 Processor -- start 2024-09-17T05:24:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- start start 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3f8003af0 0x7ff41015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 0x7ff4101fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff410167ec0 con 0x7ff4101f9db0 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff410167d40 con 0x7ff4100ca700 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff410168040 con 0x7ff3f8003af0 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56396/0 (socket says 172.21.15.43:56396) 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 -- 172.21.15.43:0/1131916585 learned_addr learned my addr 172.21.15.43:0/1131916585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff415800640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3f8003af0 0x7ff41015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff414e00640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 0x7ff4101fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 -- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3f8003af0 msgr2=0x7ff41015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3f8003af0 0x7ff41015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 -- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 msgr2=0x7ff4101fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 0x7ff4101fd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 -- 172.21.15.43:0/1131916585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4100b0d20 con 0x7ff4101f9db0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff414e00640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 0x7ff4101fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff416200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4100b3c20 tx=0x7ff40800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408004b30 con 0x7ff4101f9db0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff408004cd0 con 0x7ff4101f9db0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408003be0 con 0x7ff4101f9db0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41015d040 con 0x7ff4101f9db0 2024-09-17T05:24:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101fd620 con 0x7ff4101f9db0 2024-09-17T05:24:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.654+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4101fd7f0 con 0x7ff4101f9db0 2024-09-17T05:24:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.658+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff408004030 con 0x7ff4101f9db0 2024-09-17T05:24:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.658+0000 7ff40d600640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 0x7ff3f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.658+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff408094c20 con 0x7ff4101f9db0 2024-09-17T05:24:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.658+0000 7ff415800640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 0x7ff3f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:12.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.658+0000 7ff415800640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 0x7ff3f0075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4040ac000 tx=0x7ff4040b0040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.662+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff40805fcc0 con 0x7ff4101f9db0 2024-09-17T05:24:12.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:12.870+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff4100700a0 con 0x7ff4101f9db0 2024-09-17T05:24:13.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.454+0000 7ff40d600640 1 -- 172.21.15.43:0/1131916585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7ff40805f5c0 con 0x7ff4101f9db0 2024-09-17T05:24:13.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 msgr2=0x7ff3f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 0x7ff3f0075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4040ac000 tx=0x7ff4040b0040 comp rx=0 tx=0).stop 2024-09-17T05:24:13.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 msgr2=0x7ff410153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4100b3c20 tx=0x7ff40800f3c0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 shutdown_connections 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3f0073200 0x7ff3f0075710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ca700 0x7ff4101fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4101f9db0 0x7ff410153530 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 --2- 172.21.15.43:0/1131916585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3f8003af0 0x7ff41015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 >> 172.21.15.43:0/1131916585 conn(0x7ff4100b23d0 msgr2=0x7ff410155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 shutdown_connections 2024-09-17T05:24:13.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.458+0000 7ff418200640 1 -- 172.21.15.43:0/1131916585 wait complete. 2024-09-17T05:24:13.486 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:24:13.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:13.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.646+0000 7f9237600640 1 -- 172.21.15.43:0/1361223861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300cd580 msgr2=0x7f92301609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.646+0000 7f9237600640 1 --2- 172.21.15.43:0/1361223861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300cd580 0x7f92301609c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f923009bbe0 tx=0x7f922801caa0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- 172.21.15.43:0/1361223861 shutdown_connections 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 --2- 172.21.15.43:0/1361223861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300cd580 0x7f92301609c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 --2- 172.21.15.43:0/1361223861 >> v2:172.21.15.62:3300/0 conn(0x7f92300b7ce0 0x7f92300d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- 172.21.15.43:0/1361223861 >> 172.21.15.43:0/1361223861 conn(0x7f92300cee80 msgr2=0x7f92300cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- 172.21.15.43:0/1361223861 shutdown_connections 2024-09-17T05:24:13.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- 172.21.15.43:0/1361223861 wait complete. 2024-09-17T05:24:13.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 Processor -- start 2024-09-17T05:24:13.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- start start 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300b7ce0 0x7f92301fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 0x7f9230156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f923016a020 con 0x7f92300d0d80 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9230169ea0 con 0x7f92300cd580 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.650+0000 7f9237600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f923016a1a0 con 0x7f92300b7ce0 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56416/0 (socket says 172.21.15.43:56416) 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 -- 172.21.15.43:0/3728874063 learned_addr learned my addr 172.21.15.43:0/3728874063 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9234c00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300b7ce0 0x7f92301fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922fe00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 0x7f9230156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 -- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300b7ce0 msgr2=0x7f92301fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300b7ce0 0x7f92301fc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 -- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 msgr2=0x7f9230156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 0x7f9230156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 -- 172.21.15.43:0/3728874063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301f8d90 con 0x7f92300d0d80 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922fe00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 0x7f9230156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9235600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9228009bd0 tx=0x7f9228005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92280bb070 con 0x7f92300d0d80 2024-09-17T05:24:13.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9228004d40 con 0x7f92300d0d80 2024-09-17T05:24:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92280b2040 con 0x7f92300d0d80 2024-09-17T05:24:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301f9070 con 0x7f92300d0d80 2024-09-17T05:24:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301f9630 con 0x7f92300d0d80 2024-09-17T05:24:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9230157800 con 0x7f92300d0d80 2024-09-17T05:24:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.654+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92280be070 con 0x7f92300d0d80 2024-09-17T05:24:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.658+0000 7f922cc00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 0x7f9210075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.658+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f922812dfa0 con 0x7f92300d0d80 2024-09-17T05:24:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.658+0000 7f9234c00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 0x7f9210075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.658+0000 7f9234c00640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 0x7f9210075740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9230071220 tx=0x7f9224005dd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:13.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.662+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9228016950 con 0x7f92300d0d80 2024-09-17T05:24:13.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.870+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92300d0840 con 0x7f92300d0d80 2024-09-17T05:24:13.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.870+0000 7f922cc00640 1 -- 172.21.15.43:0/3728874063 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f92280f8f20 con 0x7f92300d0d80 2024-09-17T05:24:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:13.872 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"btime":"2024-09-17T05:24:13:457749+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:12.876397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4860":{"gid":4860,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1404655827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1404655827},{"type":"v1","addr":"172.21.15.43:6835","nonce":1404655827}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4860,"qdb_cluster":[4860]},"id":2}]} 2024-09-17T05:24:13.872 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2024-09-17T05:24:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 msgr2=0x7f9210075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 0x7f9210075740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9230071220 tx=0x7f9224005dd0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 msgr2=0x7f92300b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9228009bd0 tx=0x7f9228005b00 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 shutdown_connections 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9210073230 0x7f9210075740 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92300cd580 0x7f9230156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92300d0d80 0x7f92300b6360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 --2- 172.21.15.43:0/3728874063 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92300b7ce0 0x7f92301fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 >> 172.21.15.43:0/3728874063 conn(0x7f92300cee80 msgr2=0x7f92300cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 shutdown_connections 2024-09-17T05:24:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:13.874+0000 7f9237600640 1 -- 172.21.15.43:0/3728874063 wait complete. 2024-09-17T05:24:13.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:14.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 -- 172.21.15.43:0/548158586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f8155d70 msgr2=0x7f78f815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 --2- 172.21.15.43:0/548158586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f8155d70 0x7f78f815c7e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009870 tx=0x7f78e801c6f0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 -- 172.21.15.43:0/548158586 shutdown_connections 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 --2- 172.21.15.43:0/548158586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f8155d70 0x7f78f815c7e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 -- 172.21.15.43:0/548158586 >> 172.21.15.43:0/548158586 conn(0x7f78f80b23d0 msgr2=0x7f78f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 -- 172.21.15.43:0/548158586 shutdown_connections 2024-09-17T05:24:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.050+0000 7f78fe800640 1 -- 172.21.15.43:0/548158586 wait complete. 2024-09-17T05:24:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 Processor -- start 2024-09-17T05:24:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 -- start start 2024-09-17T05:24:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f806f110 0x7f78f809d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8155d70 0x7f78f8155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8169ea0 con 0x7f78f80c69b0 2024-09-17T05:24:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8169d20 con 0x7f78f8155d70 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78fe800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f78f816a020 con 0x7f78f806f110 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56234/0 (socket says 172.21.15.43:56234) 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 -- 172.21.15.43:0/3028014920 learned_addr learned my addr 172.21.15.43:0/3028014920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7400640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f806f110 0x7f78f809d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f6a00640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8155d70 0x7f78f8155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 -- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f806f110 msgr2=0x7f78f809d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f806f110 0x7f78f809d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 -- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8155d70 msgr2=0x7f78f8155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8155d70 0x7f78f8155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.054+0000 7f78f7e00640 1 -- 172.21.15.43:0/3028014920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80c3fd0 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78f7e00640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c760 tx=0x7f78f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003b80 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78f0003d20 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f809cc10 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f000aa50 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f809d1d0 con 0x7f78f80c69b0 2024-09-17T05:24:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f815c7e0 con 0x7f78f80c69b0 2024-09-17T05:24:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f78f000abf0 con 0x7f78f80c69b0 2024-09-17T05:24:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 0x7f78d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.058+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f78f0093f80 con 0x7f78f80c69b0 2024-09-17T05:24:14.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.062+0000 7f78f7400640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 0x7f78d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.062+0000 7f78f7400640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 0x7f78d8075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f78e80ae000 tx=0x7f78e801af30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.062+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f78f005e8c0 con 0x7f78f80c69b0 2024-09-17T05:24:14.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.238+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78f815e3b0 con 0x7f78f80c69b0 2024-09-17T05:24:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.238+0000 7f78ef400640 1 -- 172.21.15.43:0/3028014920 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16609 (secure 0 0 0) 0x7f78f00632f0 con 0x7f78f80c69b0 2024-09-17T05:24:14.242 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:14.242 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":33,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:24:11.438981+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:24:08.162466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:24:08.803372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 msgr2=0x7f78d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 0x7f78d8075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f78e80ae000 tx=0x7f78e801af30 comp rx=0 tx=0).stop 2024-09-17T05:24:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 msgr2=0x7f78f808bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c760 tx=0x7f78f000cc20 comp rx=0 tx=0).stop 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 shutdown_connections 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78d8073200 0x7f78d8075710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78f8155d70 0x7f78f8155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f80c69b0 0x7f78f808bfe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 --2- 172.21.15.43:0/3028014920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78f806f110 0x7f78f809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 >> 172.21.15.43:0/3028014920 conn(0x7f78f80b23d0 msgr2=0x7f78f80ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 shutdown_connections 2024-09-17T05:24:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.242+0000 7f78fe800640 1 -- 172.21.15.43:0/3028014920 wait complete. 2024-09-17T05:24:14.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:24:14.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:24:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- 172.21.15.43:0/874127557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48300b18e0 msgr2=0x7f48300ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 --2- 172.21.15.43:0/874127557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48300b18e0 0x7f48300ae160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4830073cd0 tx=0x7f481c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- 172.21.15.43:0/874127557 shutdown_connections 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 --2- 172.21.15.43:0/874127557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48300b18e0 0x7f48300ae160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 --2- 172.21.15.43:0/874127557 >> v2:172.21.15.62:3300/0 conn(0x7f483015ee30 0x7f48300b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- 172.21.15.43:0/874127557 >> 172.21.15.43:0/874127557 conn(0x7f48300b2e80 msgr2=0x7f48300b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- 172.21.15.43:0/874127557 shutdown_connections 2024-09-17T05:24:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- 172.21.15.43:0/874127557 wait complete. 2024-09-17T05:24:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 Processor -- start 2024-09-17T05:24:14.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.422+0000 7f4835600640 1 -- start start 2024-09-17T05:24:14.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48300ae6a0 0x7f48300c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48300b18e0 0x7f48301fd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f483016c870 con 0x7f483015ee30 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f483016c6f0 con 0x7f48300b18e0 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f483016c9f0 con 0x7f48300ae6a0 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56252/0 (socket says 172.21.15.43:56252) 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 -- 172.21.15.43:0/746212157 learned_addr learned my addr 172.21.15.43:0/746212157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482ea00640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48300ae6a0 0x7f48300c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482e000640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48300b18e0 0x7f48301fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 -- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48300ae6a0 msgr2=0x7f48300c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48300ae6a0 0x7f48300c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 -- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48300b18e0 msgr2=0x7f48301fd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48300b18e0 0x7f48301fd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 -- 172.21.15.43:0/746212157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300cce30 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f482f400640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f482800c710 tx=0x7f482800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828003b80 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4828003d20 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48300cd110 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800aa50 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48300cb000 con 0x7f483015ee30 2024-09-17T05:24:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.426+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48300cad50 con 0x7f483015ee30 2024-09-17T05:24:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.430+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f482800abf0 con 0x7f483015ee30 2024-09-17T05:24:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.430+0000 7f4826a00640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 0x7f4810075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.430+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4828093fb0 con 0x7f483015ee30 2024-09-17T05:24:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.430+0000 7f482ea00640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 0x7f4810075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.430+0000 7f482ea00640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 0x7f4810075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f482000bb50 tx=0x7f482000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.434+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f482800ce20 con 0x7f483015ee30 2024-09-17T05:24:14.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.606+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48300c86e0 con 0x7f4810073200 2024-09-17T05:24:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.614+0000 7f4826a00640 1 -- 172.21.15.43:0/746212157 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f48300c86e0 con 0x7f4810073200 2024-09-17T05:24:14.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:14.619 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 msgr2=0x7f4810075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 0x7f4810075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f482000bb50 tx=0x7f482000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 msgr2=0x7f4830155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f482800c710 tx=0x7f482800cbd0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 shutdown_connections 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4810073200 0x7f4810075710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48300b18e0 0x7f48301fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f483015ee30 0x7f4830155540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 --2- 172.21.15.43:0/746212157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48300ae6a0 0x7f48300c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 >> 172.21.15.43:0/746212157 conn(0x7f48300b2e80 msgr2=0x7f48300b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:14.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 shutdown_connections 2024-09-17T05:24:14.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.618+0000 7f4835600640 1 -- 172.21.15.43:0/746212157 wait complete. 2024-09-17T05:24:14.641 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-17T05:24:13.851519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236976,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737744,"statfs":{"total":773094113280,"available":772851449856,"internally_reserved":0,"allocated":4169728,"data_stored":2301428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.439451"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570342+0000","last_change":"2024-09-17T05:24:10.469539+0000","last_active":"2024-09-17T05:24:11.570342+0000","last_peered":"2024-09-17T05:24:11.570342+0000","last_clean":"2024-09-17T05:24:11.570342+0000","last_became_active":"2024-09-17T05:24:10.469128+0000","last_became_peered":"2024-09-17T05:24:10.469128+0000","last_unstale":"2024-09-17T05:24:11.570342+0000","last_undegraded":"2024-09-17T05:24:11.570342+0000","last_fullsized":"2024-09-17T05:24:11.570342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:54.305463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453091+0000","last_change":"2024-09-17T05:24:10.449291+0000","last_active":"2024-09-17T05:24:11.453091+0000","last_peered":"2024-09-17T05:24:11.453091+0000","last_clean":"2024-09-17T05:24:11.453091+0000","last_became_active":"2024-09-17T05:24:10.448917+0000","last_became_peered":"2024-09-17T05:24:10.448917+0000","last_unstale":"2024-09-17T05:24:11.453091+0000","last_undegraded":"2024-09-17T05:24:11.453091+0000","last_fullsized":"2024-09-17T05:24:11.453091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570198+0000","last_change":"2024-09-17T05:24:10.479273+0000","last_active":"2024-09-17T05:24:11.570198+0000","last_peered":"2024-09-17T05:24:11.570198+0000","last_clean":"2024-09-17T05:24:11.570198+0000","last_became_active":"2024-09-17T05:24:10.479050+0000","last_became_peered":"2024-09-17T05:24:10.479050+0000","last_unstale":"2024-09-17T05:24:11.570198+0000","last_undegraded":"2024-09-17T05:24:11.570198+0000","last_fullsized":"2024-09-17T05:24:11.570198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:53.923931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444296+0000","last_change":"2024-09-17T05:24:10.478081+0000","last_active":"2024-09-17T05:24:11.444296+0000","last_peered":"2024-09-17T05:24:11.444296+0000","last_clean":"2024-09-17T05:24:11.444296+0000","last_became_active":"2024-09-17T05:24:10.477727+0000","last_became_peered":"2024-09-17T05:24:10.477727+0000","last_unstale":"2024-09-17T05:24:11.444296+0000","last_undegraded":"2024-09-17T05:24:11.444296+0000","last_fullsized":"2024-09-17T05:24:11.444296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:45.178639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452917+0000","last_change":"2024-09-17T05:24:10.465083+0000","last_active":"2024-09-17T05:24:11.452917+0000","last_peered":"2024-09-17T05:24:11.452917+0000","last_clean":"2024-09-17T05:24:11.452917+0000","last_became_active":"2024-09-17T05:24:10.464809+0000","last_became_peered":"2024-09-17T05:24:10.464809+0000","last_unstale":"2024-09-17T05:24:11.452917+0000","last_undegraded":"2024-09-17T05:24:11.452917+0000","last_fullsized":"2024-09-17T05:24:11.452917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456255+0000","last_change":"2024-09-17T05:24:10.471777+0000","last_active":"2024-09-17T05:24:11.456255+0000","last_peered":"2024-09-17T05:24:11.456255+0000","last_clean":"2024-09-17T05:24:11.456255+0000","last_became_active":"2024-09-17T05:24:10.471398+0000","last_became_peered":"2024-09-17T05:24:10.471398+0000","last_unstale":"2024-09-17T05:24:11.456255+0000","last_undegraded":"2024-09-17T05:24:11.456255+0000","last_fullsized":"2024-09-17T05:24:11.456255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:31.522207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570192+0000","last_change":"2024-09-17T05:24:10.480068+0000","last_active":"2024-09-17T05:24:11.570192+0000","last_peered":"2024-09-17T05:24:11.570192+0000","last_clean":"2024-09-17T05:24:11.570192+0000","last_became_active":"2024-09-17T05:24:10.475137+0000","last_became_peered":"2024-09-17T05:24:10.475137+0000","last_unstale":"2024-09-17T05:24:11.570192+0000","last_undegraded":"2024-09-17T05:24:11.570192+0000","last_fullsized":"2024-09-17T05:24:11.570192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:56.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569021+0000","last_change":"2024-09-17T05:24:10.473972+0000","last_active":"2024-09-17T05:24:11.569021+0000","last_peered":"2024-09-17T05:24:11.569021+0000","last_clean":"2024-09-17T05:24:11.569021+0000","last_became_active":"2024-09-17T05:24:10.472739+0000","last_became_peered":"2024-09-17T05:24:10.472739+0000","last_unstale":"2024-09-17T05:24:11.569021+0000","last_undegraded":"2024-09-17T05:24:11.569021+0000","last_fullsized":"2024-09-17T05:24:11.569021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:54.044785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570490+0000","last_change":"2024-09-17T05:24:10.447416+0000","last_active":"2024-09-17T05:24:11.570490+0000","last_peered":"2024-09-17T05:24:11.570490+0000","last_clean":"2024-09-17T05:24:11.570490+0000","last_became_active":"2024-09-17T05:24:10.447063+0000","last_became_peered":"2024-09-17T05:24:10.447063+0000","last_unstale":"2024-09-17T05:24:11.570490+0000","last_undegraded":"2024-09-17T05:24:11.570490+0000","last_fullsized":"2024-09-17T05:24:11.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:15.131322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445109+0000","last_change":"2024-09-17T05:24:10.455903+0000","last_active":"2024-09-17T05:24:11.445109+0000","last_peered":"2024-09-17T05:24:11.445109+0000","last_clean":"2024-09-17T05:24:11.445109+0000","last_became_active":"2024-09-17T05:24:10.453153+0000","last_became_peered":"2024-09-17T05:24:10.453153+0000","last_unstale":"2024-09-17T05:24:11.445109+0000","last_undegraded":"2024-09-17T05:24:11.445109+0000","last_fullsized":"2024-09-17T05:24:11.445109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:35.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445068+0000","last_change":"2024-09-17T05:24:10.471759+0000","last_active":"2024-09-17T05:24:11.445068+0000","last_peered":"2024-09-17T05:24:11.445068+0000","last_clean":"2024-09-17T05:24:11.445068+0000","last_became_active":"2024-09-17T05:24:10.471377+0000","last_became_peered":"2024-09-17T05:24:10.471377+0000","last_unstale":"2024-09-17T05:24:11.445068+0000","last_undegraded":"2024-09-17T05:24:11.445068+0000","last_fullsized":"2024-09-17T05:24:11.445068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:04.027995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569746+0000","last_change":"2024-09-17T05:24:10.466311+0000","last_active":"2024-09-17T05:24:11.569746+0000","last_peered":"2024-09-17T05:24:11.569746+0000","last_clean":"2024-09-17T05:24:11.569746+0000","last_became_active":"2024-09-17T05:24:10.465979+0000","last_became_peered":"2024-09-17T05:24:10.465979+0000","last_unstale":"2024-09-17T05:24:11.569746+0000","last_undegraded":"2024-09-17T05:24:11.569746+0000","last_fullsized":"2024-09-17T05:24:11.569746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:34.798120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568799+0000","last_change":"2024-09-17T05:24:10.458151+0000","last_active":"2024-09-17T05:24:11.568799+0000","last_peered":"2024-09-17T05:24:11.568799+0000","last_clean":"2024-09-17T05:24:11.568799+0000","last_became_active":"2024-09-17T05:24:10.457674+0000","last_became_peered":"2024-09-17T05:24:10.457674+0000","last_unstale":"2024-09-17T05:24:11.568799+0000","last_undegraded":"2024-09-17T05:24:11.568799+0000","last_fullsized":"2024-09-17T05:24:11.568799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:26.206130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444036+0000","last_change":"2024-09-17T05:24:10.449131+0000","last_active":"2024-09-17T05:24:11.444036+0000","last_peered":"2024-09-17T05:24:11.444036+0000","last_clean":"2024-09-17T05:24:11.444036+0000","last_became_active":"2024-09-17T05:24:10.448711+0000","last_became_peered":"2024-09-17T05:24:10.448711+0000","last_unstale":"2024-09-17T05:24:11.444036+0000","last_undegraded":"2024-09-17T05:24:11.444036+0000","last_fullsized":"2024-09-17T05:24:11.444036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443780+0000","last_change":"2024-09-17T05:24:10.482530+0000","last_active":"2024-09-17T05:24:11.443780+0000","last_peered":"2024-09-17T05:24:11.443780+0000","last_clean":"2024-09-17T05:24:11.443780+0000","last_became_active":"2024-09-17T05:24:10.482165+0000","last_became_peered":"2024-09-17T05:24:10.482165+0000","last_unstale":"2024-09-17T05:24:11.443780+0000","last_undegraded":"2024-09-17T05:24:11.443780+0000","last_fullsized":"2024-09-17T05:24:11.443780+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443466+0000","last_change":"2024-09-17T05:24:10.456929+0000","last_active":"2024-09-17T05:24:11.443466+0000","last_peered":"2024-09-17T05:24:11.443466+0000","last_clean":"2024-09-17T05:24:11.443466+0000","last_became_active":"2024-09-17T05:24:10.455650+0000","last_became_peered":"2024-09-17T05:24:10.455650+0000","last_unstale":"2024-09-17T05:24:11.443466+0000","last_undegraded":"2024-09-17T05:24:11.443466+0000","last_fullsized":"2024-09-17T05:24:11.443466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:13.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445349+0000","last_change":"2024-09-17T05:24:10.470333+0000","last_active":"2024-09-17T05:24:11.445349+0000","last_peered":"2024-09-17T05:24:11.445349+0000","last_clean":"2024-09-17T05:24:11.445349+0000","last_became_active":"2024-09-17T05:24:10.469931+0000","last_became_peered":"2024-09-17T05:24:10.469931+0000","last_unstale":"2024-09-17T05:24:11.445349+0000","last_undegraded":"2024-09-17T05:24:11.445349+0000","last_fullsized":"2024-09-17T05:24:11.445349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:32.436816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456633+0000","last_change":"2024-09-17T05:24:10.470783+0000","last_active":"2024-09-17T05:24:11.456633+0000","last_peered":"2024-09-17T05:24:11.456633+0000","last_clean":"2024-09-17T05:24:11.456633+0000","last_became_active":"2024-09-17T05:24:10.470506+0000","last_became_peered":"2024-09-17T05:24:10.470506+0000","last_unstale":"2024-09-17T05:24:11.456633+0000","last_undegraded":"2024-09-17T05:24:11.456633+0000","last_fullsized":"2024-09-17T05:24:11.456633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:53.923931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456462+0000","last_change":"2024-09-17T05:24:10.470443+0000","last_active":"2024-09-17T05:24:11.456462+0000","last_peered":"2024-09-17T05:24:11.456462+0000","last_clean":"2024-09-17T05:24:11.456462+0000","last_became_active":"2024-09-17T05:24:10.460044+0000","last_became_peered":"2024-09-17T05:24:10.460044+0000","last_unstale":"2024-09-17T05:24:11.456462+0000","last_undegraded":"2024-09-17T05:24:11.456462+0000","last_fullsized":"2024-09-17T05:24:11.456462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:56.610810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568994+0000","last_change":"2024-09-17T05:24:10.458337+0000","last_active":"2024-09-17T05:24:11.568994+0000","last_peered":"2024-09-17T05:24:11.568994+0000","last_clean":"2024-09-17T05:24:11.568994+0000","last_became_active":"2024-09-17T05:24:10.457444+0000","last_became_peered":"2024-09-17T05:24:10.457444+0000","last_unstale":"2024-09-17T05:24:11.568994+0000","last_undegraded":"2024-09-17T05:24:11.568994+0000","last_fullsized":"2024-09-17T05:24:11.568994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:38.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443572+0000","last_change":"2024-09-17T05:24:10.458779+0000","last_active":"2024-09-17T05:24:11.443572+0000","last_peered":"2024-09-17T05:24:11.443572+0000","last_clean":"2024-09-17T05:24:11.443572+0000","last_became_active":"2024-09-17T05:24:10.458464+0000","last_became_peered":"2024-09-17T05:24:10.458464+0000","last_unstale":"2024-09-17T05:24:11.443572+0000","last_undegraded":"2024-09-17T05:24:11.443572+0000","last_fullsized":"2024-09-17T05:24:11.443572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:45.178639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569616+0000","last_change":"2024-09-17T05:24:10.468578+0000","last_active":"2024-09-17T05:24:11.569616+0000","last_peered":"2024-09-17T05:24:11.569616+0000","last_clean":"2024-09-17T05:24:11.569616+0000","last_became_active":"2024-09-17T05:24:10.468146+0000","last_became_peered":"2024-09-17T05:24:10.468146+0000","last_unstale":"2024-09-17T05:24:11.569616+0000","last_undegraded":"2024-09-17T05:24:11.569616+0000","last_fullsized":"2024-09-17T05:24:11.569616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:13.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568460+0000","last_change":"2024-09-17T05:24:10.462357+0000","last_active":"2024-09-17T05:24:11.568460+0000","last_peered":"2024-09-17T05:24:11.568460+0000","last_clean":"2024-09-17T05:24:11.568460+0000","last_became_active":"2024-09-17T05:24:10.461583+0000","last_became_peered":"2024-09-17T05:24:10.461583+0000","last_unstale":"2024-09-17T05:24:11.568460+0000","last_undegraded":"2024-09-17T05:24:11.568460+0000","last_fullsized":"2024-09-17T05:24:11.568460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:41.944177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569192+0000","last_change":"2024-09-17T05:24:10.473577+0000","last_active":"2024-09-17T05:24:11.569192+0000","last_peered":"2024-09-17T05:24:11.569192+0000","last_clean":"2024-09-17T05:24:11.569192+0000","last_became_active":"2024-09-17T05:24:10.472394+0000","last_became_peered":"2024-09-17T05:24:10.472394+0000","last_unstale":"2024-09-17T05:24:11.569192+0000","last_undegraded":"2024-09-17T05:24:11.569192+0000","last_fullsized":"2024-09-17T05:24:11.569192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:10.079654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445228+0000","last_change":"2024-09-17T05:24:10.453065+0000","last_active":"2024-09-17T05:24:11.445228+0000","last_peered":"2024-09-17T05:24:11.445228+0000","last_clean":"2024-09-17T05:24:11.445228+0000","last_became_active":"2024-09-17T05:24:10.452302+0000","last_became_peered":"2024-09-17T05:24:10.452302+0000","last_unstale":"2024-09-17T05:24:11.445228+0000","last_undegraded":"2024-09-17T05:24:11.445228+0000","last_fullsized":"2024-09-17T05:24:11.445228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:07.516584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445207+0000","last_change":"2024-09-17T05:24:10.439352+0000","last_active":"2024-09-17T05:24:11.445207+0000","last_peered":"2024-09-17T05:24:11.445207+0000","last_clean":"2024-09-17T05:24:11.445207+0000","last_became_active":"2024-09-17T05:24:10.439027+0000","last_became_peered":"2024-09-17T05:24:10.439027+0000","last_unstale":"2024-09-17T05:24:11.445207+0000","last_undegraded":"2024-09-17T05:24:11.445207+0000","last_fullsized":"2024-09-17T05:24:11.445207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:41.596150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452927+0000","last_change":"2024-09-17T05:24:10.455105+0000","last_active":"2024-09-17T05:24:11.452927+0000","last_peered":"2024-09-17T05:24:11.452927+0000","last_clean":"2024-09-17T05:24:11.452927+0000","last_became_active":"2024-09-17T05:24:10.454654+0000","last_became_peered":"2024-09-17T05:24:10.454654+0000","last_unstale":"2024-09-17T05:24:11.452927+0000","last_undegraded":"2024-09-17T05:24:11.452927+0000","last_fullsized":"2024-09-17T05:24:11.452927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:45.721427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569774+0000","last_change":"2024-09-17T05:24:10.471535+0000","last_active":"2024-09-17T05:24:11.569774+0000","last_peered":"2024-09-17T05:24:11.569774+0000","last_clean":"2024-09-17T05:24:11.569774+0000","last_became_active":"2024-09-17T05:24:10.470985+0000","last_became_peered":"2024-09-17T05:24:10.470985+0000","last_unstale":"2024-09-17T05:24:11.569774+0000","last_undegraded":"2024-09-17T05:24:11.569774+0000","last_fullsized":"2024-09-17T05:24:11.569774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:26.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443868+0000","last_change":"2024-09-17T05:24:10.439417+0000","last_active":"2024-09-17T05:24:11.443868+0000","last_peered":"2024-09-17T05:24:11.443868+0000","last_clean":"2024-09-17T05:24:11.443868+0000","last_became_active":"2024-09-17T05:24:10.439058+0000","last_became_peered":"2024-09-17T05:24:10.439058+0000","last_unstale":"2024-09-17T05:24:11.443868+0000","last_undegraded":"2024-09-17T05:24:11.443868+0000","last_fullsized":"2024-09-17T05:24:11.443868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443452+0000","last_change":"2024-09-17T05:24:10.473985+0000","last_active":"2024-09-17T05:24:11.443452+0000","last_peered":"2024-09-17T05:24:11.443452+0000","last_clean":"2024-09-17T05:24:11.443452+0000","last_became_active":"2024-09-17T05:24:10.473600+0000","last_became_peered":"2024-09-17T05:24:10.473600+0000","last_unstale":"2024-09-17T05:24:11.443452+0000","last_undegraded":"2024-09-17T05:24:11.443452+0000","last_fullsized":"2024-09-17T05:24:11.443452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:06.724070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570418+0000","last_change":"2024-09-17T05:24:10.475020+0000","last_active":"2024-09-17T05:24:11.570418+0000","last_peered":"2024-09-17T05:24:11.570418+0000","last_clean":"2024-09-17T05:24:11.570418+0000","last_became_active":"2024-09-17T05:24:10.467584+0000","last_became_peered":"2024-09-17T05:24:10.467584+0000","last_unstale":"2024-09-17T05:24:11.570418+0000","last_undegraded":"2024-09-17T05:24:11.570418+0000","last_fullsized":"2024-09-17T05:24:11.570418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:29.260645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445019+0000","last_change":"2024-09-17T05:24:10.482824+0000","last_active":"2024-09-17T05:24:11.445019+0000","last_peered":"2024-09-17T05:24:11.445019+0000","last_clean":"2024-09-17T05:24:11.445019+0000","last_became_active":"2024-09-17T05:24:10.482452+0000","last_became_peered":"2024-09-17T05:24:10.482452+0000","last_unstale":"2024-09-17T05:24:11.445019+0000","last_undegraded":"2024-09-17T05:24:11.445019+0000","last_fullsized":"2024-09-17T05:24:11.445019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:29.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568944+0000","last_change":"2024-09-17T05:24:10.472519+0000","last_active":"2024-09-17T05:24:11.568944+0000","last_peered":"2024-09-17T05:24:11.568944+0000","last_clean":"2024-09-17T05:24:11.568944+0000","last_became_active":"2024-09-17T05:24:10.472074+0000","last_became_peered":"2024-09-17T05:24:10.472074+0000","last_unstale":"2024-09-17T05:24:11.568944+0000","last_undegraded":"2024-09-17T05:24:11.568944+0000","last_fullsized":"2024-09-17T05:24:11.568944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:41.429333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445538+0000","last_change":"2024-09-17T05:24:09.436901+0000","last_active":"2024-09-17T05:24:11.445538+0000","last_peered":"2024-09-17T05:24:11.445538+0000","last_clean":"2024-09-17T05:24:11.445538+0000","last_became_active":"2024-09-17T05:24:09.436544+0000","last_became_peered":"2024-09-17T05:24:09.436544+0000","last_unstale":"2024-09-17T05:24:11.445538+0000","last_undegraded":"2024-09-17T05:24:11.445538+0000","last_fullsized":"2024-09-17T05:24:11.445538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:57.252525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570546+0000","last_change":"2024-09-17T05:24:10.479447+0000","last_active":"2024-09-17T05:24:11.570546+0000","last_peered":"2024-09-17T05:24:11.570546+0000","last_clean":"2024-09-17T05:24:11.570546+0000","last_became_active":"2024-09-17T05:24:10.479282+0000","last_became_peered":"2024-09-17T05:24:10.479282+0000","last_unstale":"2024-09-17T05:24:11.570546+0000","last_undegraded":"2024-09-17T05:24:11.570546+0000","last_fullsized":"2024-09-17T05:24:11.570546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:39.535829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445171+0000","last_change":"2024-09-17T05:24:09.435574+0000","last_active":"2024-09-17T05:24:11.445171+0000","last_peered":"2024-09-17T05:24:11.445171+0000","last_clean":"2024-09-17T05:24:11.445171+0000","last_became_active":"2024-09-17T05:24:09.432948+0000","last_became_peered":"2024-09-17T05:24:09.432948+0000","last_unstale":"2024-09-17T05:24:11.445171+0000","last_undegraded":"2024-09-17T05:24:11.445171+0000","last_fullsized":"2024-09-17T05:24:11.445171+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:42.498255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570056+0000","last_change":"2024-09-17T05:24:10.459215+0000","last_active":"2024-09-17T05:24:11.570056+0000","last_peered":"2024-09-17T05:24:11.570056+0000","last_clean":"2024-09-17T05:24:11.570056+0000","last_became_active":"2024-09-17T05:24:10.458906+0000","last_became_peered":"2024-09-17T05:24:10.458906+0000","last_unstale":"2024-09-17T05:24:11.570056+0000","last_undegraded":"2024-09-17T05:24:11.570056+0000","last_fullsized":"2024-09-17T05:24:11.570056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:45.721427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456663+0000","last_change":"2024-09-17T05:24:09.443681+0000","last_active":"2024-09-17T05:24:11.456663+0000","last_peered":"2024-09-17T05:24:11.456663+0000","last_clean":"2024-09-17T05:24:11.456663+0000","last_became_active":"2024-09-17T05:24:09.443337+0000","last_became_peered":"2024-09-17T05:24:09.443337+0000","last_unstale":"2024-09-17T05:24:11.456663+0000","last_undegraded":"2024-09-17T05:24:11.456663+0000","last_fullsized":"2024-09-17T05:24:11.456663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:14.514733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445102+0000","last_change":"2024-09-17T05:24:10.457894+0000","last_active":"2024-09-17T05:24:11.445102+0000","last_peered":"2024-09-17T05:24:11.445102+0000","last_clean":"2024-09-17T05:24:11.445102+0000","last_became_active":"2024-09-17T05:24:10.457559+0000","last_became_peered":"2024-09-17T05:24:10.457559+0000","last_unstale":"2024-09-17T05:24:11.445102+0000","last_undegraded":"2024-09-17T05:24:11.445102+0000","last_fullsized":"2024-09-17T05:24:11.445102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:51.131527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444391+0000","last_change":"2024-09-17T05:24:09.437966+0000","last_active":"2024-09-17T05:24:11.444391+0000","last_peered":"2024-09-17T05:24:11.444391+0000","last_clean":"2024-09-17T05:24:11.444391+0000","last_became_active":"2024-09-17T05:24:09.437541+0000","last_became_peered":"2024-09-17T05:24:09.437541+0000","last_unstale":"2024-09-17T05:24:11.444391+0000","last_undegraded":"2024-09-17T05:24:11.444391+0000","last_fullsized":"2024-09-17T05:24:11.444391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:14.124577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569575+0000","last_change":"2024-09-17T05:24:10.447614+0000","last_active":"2024-09-17T05:24:11.569575+0000","last_peered":"2024-09-17T05:24:11.569575+0000","last_clean":"2024-09-17T05:24:11.569575+0000","last_became_active":"2024-09-17T05:24:10.447129+0000","last_became_peered":"2024-09-17T05:24:10.447129+0000","last_unstale":"2024-09-17T05:24:11.569575+0000","last_undegraded":"2024-09-17T05:24:11.569575+0000","last_fullsized":"2024-09-17T05:24:11.569575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:56.610810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569306+0000","last_change":"2024-09-17T05:24:09.441000+0000","last_active":"2024-09-17T05:24:11.569306+0000","last_peered":"2024-09-17T05:24:11.569306+0000","last_clean":"2024-09-17T05:24:11.569306+0000","last_became_active":"2024-09-17T05:24:09.439493+0000","last_became_peered":"2024-09-17T05:24:09.439493+0000","last_unstale":"2024-09-17T05:24:11.569306+0000","last_undegraded":"2024-09-17T05:24:11.569306+0000","last_fullsized":"2024-09-17T05:24:11.569306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:37.999908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569292+0000","last_change":"2024-09-17T05:24:10.470464+0000","last_active":"2024-09-17T05:24:11.569292+0000","last_peered":"2024-09-17T05:24:11.569292+0000","last_clean":"2024-09-17T05:24:11.569292+0000","last_became_active":"2024-09-17T05:24:10.467481+0000","last_became_peered":"2024-09-17T05:24:10.467481+0000","last_unstale":"2024-09-17T05:24:11.569292+0000","last_undegraded":"2024-09-17T05:24:11.569292+0000","last_fullsized":"2024-09-17T05:24:11.569292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:12.282949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569696+0000","last_change":"2024-09-17T05:24:09.436952+0000","last_active":"2024-09-17T05:24:11.569696+0000","last_peered":"2024-09-17T05:24:11.569696+0000","last_clean":"2024-09-17T05:24:11.569696+0000","last_became_active":"2024-09-17T05:24:09.436639+0000","last_became_peered":"2024-09-17T05:24:09.436639+0000","last_unstale":"2024-09-17T05:24:11.569696+0000","last_undegraded":"2024-09-17T05:24:11.569696+0000","last_fullsized":"2024-09-17T05:24:11.569696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:30.515462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569062+0000","last_change":"2024-09-17T05:24:10.439298+0000","last_active":"2024-09-17T05:24:11.569062+0000","last_peered":"2024-09-17T05:24:11.569062+0000","last_clean":"2024-09-17T05:24:11.569062+0000","last_became_active":"2024-09-17T05:24:10.438920+0000","last_became_peered":"2024-09-17T05:24:10.438920+0000","last_unstale":"2024-09-17T05:24:11.569062+0000","last_undegraded":"2024-09-17T05:24:11.569062+0000","last_fullsized":"2024-09-17T05:24:11.569062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:39.006653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456089+0000","last_change":"2024-09-17T05:24:09.445316+0000","last_active":"2024-09-17T05:24:11.456089+0000","last_peered":"2024-09-17T05:24:11.456089+0000","last_clean":"2024-09-17T05:24:11.456089+0000","last_became_active":"2024-09-17T05:24:09.444999+0000","last_became_peered":"2024-09-17T05:24:09.444999+0000","last_unstale":"2024-09-17T05:24:11.456089+0000","last_undegraded":"2024-09-17T05:24:11.456089+0000","last_fullsized":"2024-09-17T05:24:11.456089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:30.393590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443697+0000","last_change":"2024-09-17T05:24:10.465646+0000","last_active":"2024-09-17T05:24:11.443697+0000","last_peered":"2024-09-17T05:24:11.443697+0000","last_clean":"2024-09-17T05:24:11.443697+0000","last_became_active":"2024-09-17T05:24:10.465288+0000","last_became_peered":"2024-09-17T05:24:10.465288+0000","last_unstale":"2024-09-17T05:24:11.443697+0000","last_undegraded":"2024-09-17T05:24:11.443697+0000","last_fullsized":"2024-09-17T05:24:11.443697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:06.724070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569263+0000","last_change":"2024-09-17T05:24:09.440937+0000","last_active":"2024-09-17T05:24:11.569263+0000","last_peered":"2024-09-17T05:24:11.569263+0000","last_clean":"2024-09-17T05:24:11.569263+0000","last_became_active":"2024-09-17T05:24:09.440577+0000","last_became_peered":"2024-09-17T05:24:09.440577+0000","last_unstale":"2024-09-17T05:24:11.569263+0000","last_undegraded":"2024-09-17T05:24:11.569263+0000","last_fullsized":"2024-09-17T05:24:11.569263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:22.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445368+0000","last_change":"2024-09-17T05:24:10.466580+0000","last_active":"2024-09-17T05:24:11.445368+0000","last_peered":"2024-09-17T05:24:11.445368+0000","last_clean":"2024-09-17T05:24:11.445368+0000","last_became_active":"2024-09-17T05:24:10.466041+0000","last_became_peered":"2024-09-17T05:24:10.466041+0000","last_unstale":"2024-09-17T05:24:11.445368+0000","last_undegraded":"2024-09-17T05:24:11.445368+0000","last_fullsized":"2024-09-17T05:24:11.445368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:01.975736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452937+0000","last_change":"2024-09-17T05:24:09.435396+0000","last_active":"2024-09-17T05:24:11.452937+0000","last_peered":"2024-09-17T05:24:11.452937+0000","last_clean":"2024-09-17T05:24:11.452937+0000","last_became_active":"2024-09-17T05:24:09.434542+0000","last_became_peered":"2024-09-17T05:24:09.434542+0000","last_unstale":"2024-09-17T05:24:11.452937+0000","last_undegraded":"2024-09-17T05:24:11.452937+0000","last_fullsized":"2024-09-17T05:24:11.452937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:25.086177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445486+0000","last_change":"2024-09-17T05:24:10.452676+0000","last_active":"2024-09-17T05:24:11.445486+0000","last_peered":"2024-09-17T05:24:11.445486+0000","last_clean":"2024-09-17T05:24:11.445486+0000","last_became_active":"2024-09-17T05:24:10.451854+0000","last_became_peered":"2024-09-17T05:24:10.451854+0000","last_unstale":"2024-09-17T05:24:11.445486+0000","last_undegraded":"2024-09-17T05:24:11.445486+0000","last_fullsized":"2024-09-17T05:24:11.445486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:50.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453041+0000","last_change":"2024-09-17T05:24:09.442019+0000","last_active":"2024-09-17T05:24:11.453041+0000","last_peered":"2024-09-17T05:24:11.453041+0000","last_clean":"2024-09-17T05:24:11.453041+0000","last_became_active":"2024-09-17T05:24:09.441714+0000","last_became_peered":"2024-09-17T05:24:09.441714+0000","last_unstale":"2024-09-17T05:24:11.453041+0000","last_undegraded":"2024-09-17T05:24:11.453041+0000","last_fullsized":"2024-09-17T05:24:11.453041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:58.866053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444927+0000","last_change":"2024-09-17T05:24:10.478011+0000","last_active":"2024-09-17T05:24:11.444927+0000","last_peered":"2024-09-17T05:24:11.444927+0000","last_clean":"2024-09-17T05:24:11.444927+0000","last_became_active":"2024-09-17T05:24:10.477605+0000","last_became_peered":"2024-09-17T05:24:10.477605+0000","last_unstale":"2024-09-17T05:24:11.444927+0000","last_undegraded":"2024-09-17T05:24:11.444927+0000","last_fullsized":"2024-09-17T05:24:11.444927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445186+0000","last_change":"2024-09-17T05:24:09.439895+0000","last_active":"2024-09-17T05:24:11.445186+0000","last_peered":"2024-09-17T05:24:11.445186+0000","last_clean":"2024-09-17T05:24:11.445186+0000","last_became_active":"2024-09-17T05:24:09.439499+0000","last_became_peered":"2024-09-17T05:24:09.439499+0000","last_unstale":"2024-09-17T05:24:11.445186+0000","last_undegraded":"2024-09-17T05:24:11.445186+0000","last_fullsized":"2024-09-17T05:24:11.445186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:06.509839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444948+0000","last_change":"2024-09-17T05:24:10.463610+0000","last_active":"2024-09-17T05:24:11.444948+0000","last_peered":"2024-09-17T05:24:11.444948+0000","last_clean":"2024-09-17T05:24:11.444948+0000","last_became_active":"2024-09-17T05:24:10.463287+0000","last_became_peered":"2024-09-17T05:24:10.463287+0000","last_unstale":"2024-09-17T05:24:11.444948+0000","last_undegraded":"2024-09-17T05:24:11.444948+0000","last_fullsized":"2024-09-17T05:24:11.444948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:48.603532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568875+0000","last_change":"2024-09-17T05:24:09.434869+0000","last_active":"2024-09-17T05:24:11.568875+0000","last_peered":"2024-09-17T05:24:11.568875+0000","last_clean":"2024-09-17T05:24:11.568875+0000","last_became_active":"2024-09-17T05:24:09.434451+0000","last_became_peered":"2024-09-17T05:24:09.434451+0000","last_unstale":"2024-09-17T05:24:11.568875+0000","last_undegraded":"2024-09-17T05:24:11.568875+0000","last_fullsized":"2024-09-17T05:24:11.568875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:30.285720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568621+0000","last_change":"2024-09-17T05:24:10.457614+0000","last_active":"2024-09-17T05:24:11.568621+0000","last_peered":"2024-09-17T05:24:11.568621+0000","last_clean":"2024-09-17T05:24:11.568621+0000","last_became_active":"2024-09-17T05:24:10.457278+0000","last_became_peered":"2024-09-17T05:24:10.457278+0000","last_unstale":"2024-09-17T05:24:11.568621+0000","last_undegraded":"2024-09-17T05:24:11.568621+0000","last_fullsized":"2024-09-17T05:24:11.568621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:24.118863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569367+0000","last_change":"2024-09-17T05:24:09.444441+0000","last_active":"2024-09-17T05:24:11.569367+0000","last_peered":"2024-09-17T05:24:11.569367+0000","last_clean":"2024-09-17T05:24:11.569367+0000","last_became_active":"2024-09-17T05:24:09.443853+0000","last_became_peered":"2024-09-17T05:24:09.443853+0000","last_unstale":"2024-09-17T05:24:11.569367+0000","last_undegraded":"2024-09-17T05:24:11.569367+0000","last_fullsized":"2024-09-17T05:24:11.569367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:52.917186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444541+0000","last_change":"2024-09-17T05:24:10.464584+0000","last_active":"2024-09-17T05:24:11.444541+0000","last_peered":"2024-09-17T05:24:11.444541+0000","last_clean":"2024-09-17T05:24:11.444541+0000","last_became_active":"2024-09-17T05:24:10.460610+0000","last_became_peered":"2024-09-17T05:24:10.460610+0000","last_unstale":"2024-09-17T05:24:11.444541+0000","last_undegraded":"2024-09-17T05:24:11.444541+0000","last_fullsized":"2024-09-17T05:24:11.444541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444040+0000","last_change":"2024-09-17T05:24:09.436373+0000","last_active":"2024-09-17T05:24:11.444040+0000","last_peered":"2024-09-17T05:24:11.444040+0000","last_clean":"2024-09-17T05:24:11.444040+0000","last_became_active":"2024-09-17T05:24:09.436055+0000","last_became_peered":"2024-09-17T05:24:09.436055+0000","last_unstale":"2024-09-17T05:24:11.444040+0000","last_undegraded":"2024-09-17T05:24:11.444040+0000","last_fullsized":"2024-09-17T05:24:11.444040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:33.791376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569276+0000","last_change":"2024-09-17T05:24:10.473239+0000","last_active":"2024-09-17T05:24:11.569276+0000","last_peered":"2024-09-17T05:24:11.569276+0000","last_clean":"2024-09-17T05:24:11.569276+0000","last_became_active":"2024-09-17T05:24:10.472053+0000","last_became_peered":"2024-09-17T05:24:10.472053+0000","last_unstale":"2024-09-17T05:24:11.569276+0000","last_undegraded":"2024-09-17T05:24:11.569276+0000","last_fullsized":"2024-09-17T05:24:11.569276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:22.208293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443785+0000","last_change":"2024-09-17T05:24:09.431865+0000","last_active":"2024-09-17T05:24:11.443785+0000","last_peered":"2024-09-17T05:24:11.443785+0000","last_clean":"2024-09-17T05:24:11.443785+0000","last_became_active":"2024-09-17T05:24:09.431545+0000","last_became_peered":"2024-09-17T05:24:09.431545+0000","last_unstale":"2024-09-17T05:24:11.443785+0000","last_undegraded":"2024-09-17T05:24:11.443785+0000","last_fullsized":"2024-09-17T05:24:11.443785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:55.604065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444880+0000","last_change":"2024-09-17T05:24:10.483993+0000","last_active":"2024-09-17T05:24:11.444880+0000","last_peered":"2024-09-17T05:24:11.444880+0000","last_clean":"2024-09-17T05:24:11.444880+0000","last_became_active":"2024-09-17T05:24:10.482664+0000","last_became_peered":"2024-09-17T05:24:10.482664+0000","last_unstale":"2024-09-17T05:24:11.444880+0000","last_undegraded":"2024-09-17T05:24:11.444880+0000","last_fullsized":"2024-09-17T05:24:11.444880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:16.553451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.455944+0000","last_change":"2024-09-17T05:24:09.443609+0000","last_active":"2024-09-17T05:24:11.455944+0000","last_peered":"2024-09-17T05:24:11.455944+0000","last_clean":"2024-09-17T05:24:11.455944+0000","last_became_active":"2024-09-17T05:24:09.443210+0000","last_became_peered":"2024-09-17T05:24:09.443210+0000","last_unstale":"2024-09-17T05:24:11.455944+0000","last_undegraded":"2024-09-17T05:24:11.455944+0000","last_fullsized":"2024-09-17T05:24:11.455944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:43.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445462+0000","last_change":"2024-09-17T05:24:10.467882+0000","last_active":"2024-09-17T05:24:11.445462+0000","last_peered":"2024-09-17T05:24:11.445462+0000","last_clean":"2024-09-17T05:24:11.445462+0000","last_became_active":"2024-09-17T05:24:10.467444+0000","last_became_peered":"2024-09-17T05:24:10.467444+0000","last_unstale":"2024-09-17T05:24:11.445462+0000","last_undegraded":"2024-09-17T05:24:11.445462+0000","last_fullsized":"2024-09-17T05:24:11.445462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:12.969259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456505+0000","last_change":"2024-09-17T05:24:09.447465+0000","last_active":"2024-09-17T05:24:11.456505+0000","last_peered":"2024-09-17T05:24:11.456505+0000","last_clean":"2024-09-17T05:24:11.456505+0000","last_became_active":"2024-09-17T05:24:09.446941+0000","last_became_peered":"2024-09-17T05:24:09.446941+0000","last_unstale":"2024-09-17T05:24:11.456505+0000","last_undegraded":"2024-09-17T05:24:11.456505+0000","last_fullsized":"2024-09-17T05:24:11.456505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:58.544807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570499+0000","last_change":"2024-09-17T05:24:10.441511+0000","last_active":"2024-09-17T05:24:11.570499+0000","last_peered":"2024-09-17T05:24:11.570499+0000","last_clean":"2024-09-17T05:24:11.570499+0000","last_became_active":"2024-09-17T05:24:10.441225+0000","last_became_peered":"2024-09-17T05:24:10.441225+0000","last_unstale":"2024-09-17T05:24:11.570499+0000","last_undegraded":"2024-09-17T05:24:11.570499+0000","last_fullsized":"2024-09-17T05:24:11.570499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:28.879316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570508+0000","last_change":"2024-09-17T05:24:09.442314+0000","last_active":"2024-09-17T05:24:11.570508+0000","last_peered":"2024-09-17T05:24:11.570508+0000","last_clean":"2024-09-17T05:24:11.570508+0000","last_became_active":"2024-09-17T05:24:09.441967+0000","last_became_peered":"2024-09-17T05:24:09.441967+0000","last_unstale":"2024-09-17T05:24:11.570508+0000","last_undegraded":"2024-09-17T05:24:11.570508+0000","last_fullsized":"2024-09-17T05:24:11.570508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:24.057204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452976+0000","last_change":"2024-09-17T05:24:10.460678+0000","last_active":"2024-09-17T05:24:11.452976+0000","last_peered":"2024-09-17T05:24:11.452976+0000","last_clean":"2024-09-17T05:24:11.452976+0000","last_became_active":"2024-09-17T05:24:10.460186+0000","last_became_peered":"2024-09-17T05:24:10.460186+0000","last_unstale":"2024-09-17T05:24:11.452976+0000","last_undegraded":"2024-09-17T05:24:11.452976+0000","last_fullsized":"2024-09-17T05:24:11.452976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445030+0000","last_change":"2024-09-17T05:24:09.432947+0000","last_active":"2024-09-17T05:24:11.445030+0000","last_peered":"2024-09-17T05:24:11.445030+0000","last_clean":"2024-09-17T05:24:11.445030+0000","last_became_active":"2024-09-17T05:24:09.432619+0000","last_became_peered":"2024-09-17T05:24:09.432619+0000","last_unstale":"2024-09-17T05:24:11.445030+0000","last_undegraded":"2024-09-17T05:24:11.445030+0000","last_fullsized":"2024-09-17T05:24:11.445030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:40.589405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456549+0000","last_change":"2024-09-17T05:24:10.470780+0000","last_active":"2024-09-17T05:24:11.456549+0000","last_peered":"2024-09-17T05:24:11.456549+0000","last_clean":"2024-09-17T05:24:11.456549+0000","last_became_active":"2024-09-17T05:24:10.470080+0000","last_became_peered":"2024-09-17T05:24:10.470080+0000","last_unstale":"2024-09-17T05:24:11.456549+0000","last_undegraded":"2024-09-17T05:24:11.456549+0000","last_fullsized":"2024-09-17T05:24:11.456549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:34.798120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444123+0000","last_change":"2024-09-17T05:24:09.425489+0000","last_active":"2024-09-17T05:24:11.444123+0000","last_peered":"2024-09-17T05:24:11.444123+0000","last_clean":"2024-09-17T05:24:11.444123+0000","last_became_active":"2024-09-17T05:24:09.425164+0000","last_became_peered":"2024-09-17T05:24:09.425164+0000","last_unstale":"2024-09-17T05:24:11.444123+0000","last_undegraded":"2024-09-17T05:24:11.444123+0000","last_fullsized":"2024-09-17T05:24:11.444123+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:44.373590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568520+0000","last_change":"2024-09-17T05:24:10.463698+0000","last_active":"2024-09-17T05:24:11.568520+0000","last_peered":"2024-09-17T05:24:11.568520+0000","last_clean":"2024-09-17T05:24:11.568520+0000","last_became_active":"2024-09-17T05:24:10.462936+0000","last_became_peered":"2024-09-17T05:24:10.462936+0000","last_unstale":"2024-09-17T05:24:11.568520+0000","last_undegraded":"2024-09-17T05:24:11.568520+0000","last_fullsized":"2024-09-17T05:24:11.568520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:00.907853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453254+0000","last_change":"2024-09-17T05:24:09.431174+0000","last_active":"2024-09-17T05:24:11.453254+0000","last_peered":"2024-09-17T05:24:11.453254+0000","last_clean":"2024-09-17T05:24:11.453254+0000","last_became_active":"2024-09-17T05:24:09.430822+0000","last_became_peered":"2024-09-17T05:24:09.430822+0000","last_unstale":"2024-09-17T05:24:11.453254+0000","last_undegraded":"2024-09-17T05:24:11.453254+0000","last_fullsized":"2024-09-17T05:24:11.453254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:27.555116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453175+0000","last_change":"2024-09-17T05:24:10.452124+0000","last_active":"2024-09-17T05:24:11.453175+0000","last_peered":"2024-09-17T05:24:11.453175+0000","last_clean":"2024-09-17T05:24:11.453175+0000","last_became_active":"2024-09-17T05:24:10.451753+0000","last_became_peered":"2024-09-17T05:24:10.451753+0000","last_unstale":"2024-09-17T05:24:11.453175+0000","last_undegraded":"2024-09-17T05:24:11.453175+0000","last_fullsized":"2024-09-17T05:24:11.453175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569821+0000","last_change":"2024-09-17T05:24:09.438394+0000","last_active":"2024-09-17T05:24:11.569821+0000","last_peered":"2024-09-17T05:24:11.569821+0000","last_clean":"2024-09-17T05:24:11.569821+0000","last_became_active":"2024-09-17T05:24:09.438042+0000","last_became_peered":"2024-09-17T05:24:09.438042+0000","last_unstale":"2024-09-17T05:24:11.569821+0000","last_undegraded":"2024-09-17T05:24:11.569821+0000","last_fullsized":"2024-09-17T05:24:11.569821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:55.604065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568458+0000","last_change":"2024-09-17T05:24:10.459057+0000","last_active":"2024-09-17T05:24:11.568458+0000","last_peered":"2024-09-17T05:24:11.568458+0000","last_clean":"2024-09-17T05:24:11.568458+0000","last_became_active":"2024-09-17T05:24:10.458676+0000","last_became_peered":"2024-09-17T05:24:10.458676+0000","last_unstale":"2024-09-17T05:24:11.568458+0000","last_undegraded":"2024-09-17T05:24:11.568458+0000","last_fullsized":"2024-09-17T05:24:11.568458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:48.457021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569113+0000","last_change":"2024-09-17T05:24:09.437988+0000","last_active":"2024-09-17T05:24:11.569113+0000","last_peered":"2024-09-17T05:24:11.569113+0000","last_clean":"2024-09-17T05:24:11.569113+0000","last_became_active":"2024-09-17T05:24:09.437668+0000","last_became_peered":"2024-09-17T05:24:09.437668+0000","last_unstale":"2024-09-17T05:24:11.569113+0000","last_undegraded":"2024-09-17T05:24:11.569113+0000","last_fullsized":"2024-09-17T05:24:11.569113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:43.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569404+0000","last_change":"2024-09-17T05:24:10.440898+0000","last_active":"2024-09-17T05:24:11.569404+0000","last_peered":"2024-09-17T05:24:11.569404+0000","last_clean":"2024-09-17T05:24:11.569404+0000","last_became_active":"2024-09-17T05:24:10.440549+0000","last_became_peered":"2024-09-17T05:24:10.440549+0000","last_unstale":"2024-09-17T05:24:11.569404+0000","last_undegraded":"2024-09-17T05:24:11.569404+0000","last_fullsized":"2024-09-17T05:24:11.569404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:31.522207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444572+0000","last_change":"2024-09-17T05:24:09.440958+0000","last_active":"2024-09-17T05:24:11.444572+0000","last_peered":"2024-09-17T05:24:11.444572+0000","last_clean":"2024-09-17T05:24:11.444572+0000","last_became_active":"2024-09-17T05:24:09.435037+0000","last_became_peered":"2024-09-17T05:24:09.435037+0000","last_unstale":"2024-09-17T05:24:11.444572+0000","last_undegraded":"2024-09-17T05:24:11.444572+0000","last_fullsized":"2024-09-17T05:24:11.444572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:14.152601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456759+0000","last_change":"2024-09-17T05:24:10.468367+0000","last_active":"2024-09-17T05:24:11.456759+0000","last_peered":"2024-09-17T05:24:11.456759+0000","last_clean":"2024-09-17T05:24:11.456759+0000","last_became_active":"2024-09-17T05:24:10.467854+0000","last_became_peered":"2024-09-17T05:24:10.467854+0000","last_unstale":"2024-09-17T05:24:11.456759+0000","last_undegraded":"2024-09-17T05:24:11.456759+0000","last_fullsized":"2024-09-17T05:24:11.456759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:43.505000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568561+0000","last_change":"2024-09-17T05:24:09.440411+0000","last_active":"2024-09-17T05:24:11.568561+0000","last_peered":"2024-09-17T05:24:11.568561+0000","last_clean":"2024-09-17T05:24:11.568561+0000","last_became_active":"2024-09-17T05:24:09.440083+0000","last_became_peered":"2024-09-17T05:24:09.440083+0000","last_unstale":"2024-09-17T05:24:11.568561+0000","last_undegraded":"2024-09-17T05:24:11.568561+0000","last_fullsized":"2024-09-17T05:24:11.568561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:20.858099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445495+0000","last_change":"2024-09-17T05:24:10.459958+0000","last_active":"2024-09-17T05:24:11.445495+0000","last_peered":"2024-09-17T05:24:11.445495+0000","last_clean":"2024-09-17T05:24:11.445495+0000","last_became_active":"2024-09-17T05:24:10.457607+0000","last_became_peered":"2024-09-17T05:24:10.457607+0000","last_unstale":"2024-09-17T05:24:11.445495+0000","last_undegraded":"2024-09-17T05:24:11.445495+0000","last_fullsized":"2024-09-17T05:24:11.445495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:54.982346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453187+0000","last_change":"2024-09-17T05:24:09.441893+0000","last_active":"2024-09-17T05:24:11.453187+0000","last_peered":"2024-09-17T05:24:11.453187+0000","last_clean":"2024-09-17T05:24:11.453187+0000","last_became_active":"2024-09-17T05:24:09.441593+0000","last_became_peered":"2024-09-17T05:24:09.441593+0000","last_unstale":"2024-09-17T05:24:11.453187+0000","last_undegraded":"2024-09-17T05:24:11.453187+0000","last_fullsized":"2024-09-17T05:24:11.453187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:05.131621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456726+0000","last_change":"2024-09-17T05:24:10.451152+0000","last_active":"2024-09-17T05:24:11.456726+0000","last_peered":"2024-09-17T05:24:11.456726+0000","last_clean":"2024-09-17T05:24:11.456726+0000","last_became_active":"2024-09-17T05:24:10.450789+0000","last_became_peered":"2024-09-17T05:24:10.450789+0000","last_unstale":"2024-09-17T05:24:11.456726+0000","last_undegraded":"2024-09-17T05:24:11.456726+0000","last_fullsized":"2024-09-17T05:24:11.456726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:15.131322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568981+0000","last_change":"2024-09-17T05:24:09.434969+0000","last_active":"2024-09-17T05:24:11.568981+0000","last_peered":"2024-09-17T05:24:11.568981+0000","last_clean":"2024-09-17T05:24:11.568981+0000","last_became_active":"2024-09-17T05:24:09.433934+0000","last_became_peered":"2024-09-17T05:24:09.433934+0000","last_unstale":"2024-09-17T05:24:11.568981+0000","last_undegraded":"2024-09-17T05:24:11.568981+0000","last_fullsized":"2024-09-17T05:24:11.568981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:10.177328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443530+0000","last_change":"2024-09-17T05:24:10.455901+0000","last_active":"2024-09-17T05:24:11.443530+0000","last_peered":"2024-09-17T05:24:11.443530+0000","last_clean":"2024-09-17T05:24:11.443530+0000","last_became_active":"2024-09-17T05:24:10.455536+0000","last_became_peered":"2024-09-17T05:24:10.455536+0000","last_unstale":"2024-09-17T05:24:11.443530+0000","last_undegraded":"2024-09-17T05:24:11.443530+0000","last_fullsized":"2024-09-17T05:24:11.443530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568891+0000","last_change":"2024-09-17T05:24:09.439958+0000","last_active":"2024-09-17T05:24:11.568891+0000","last_peered":"2024-09-17T05:24:11.568891+0000","last_clean":"2024-09-17T05:24:11.568891+0000","last_became_active":"2024-09-17T05:24:09.439622+0000","last_became_peered":"2024-09-17T05:24:09.439622+0000","last_unstale":"2024-09-17T05:24:11.568891+0000","last_undegraded":"2024-09-17T05:24:11.568891+0000","last_fullsized":"2024-09-17T05:24:11.568891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.966754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444878+0000","last_change":"2024-09-17T05:24:10.461724+0000","last_active":"2024-09-17T05:24:11.444878+0000","last_peered":"2024-09-17T05:24:11.444878+0000","last_clean":"2024-09-17T05:24:11.444878+0000","last_became_active":"2024-09-17T05:24:10.460230+0000","last_became_peered":"2024-09-17T05:24:10.460230+0000","last_unstale":"2024-09-17T05:24:11.444878+0000","last_undegraded":"2024-09-17T05:24:11.444878+0000","last_fullsized":"2024-09-17T05:24:11.444878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568807+0000","last_change":"2024-09-17T05:24:09.437486+0000","last_active":"2024-09-17T05:24:11.568807+0000","last_peered":"2024-09-17T05:24:11.568807+0000","last_clean":"2024-09-17T05:24:11.568807+0000","last_became_active":"2024-09-17T05:24:09.437238+0000","last_became_peered":"2024-09-17T05:24:09.437238+0000","last_unstale":"2024-09-17T05:24:11.568807+0000","last_undegraded":"2024-09-17T05:24:11.568807+0000","last_fullsized":"2024-09-17T05:24:11.568807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:01.421393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444004+0000","last_change":"2024-09-17T05:24:10.478407+0000","last_active":"2024-09-17T05:24:11.444004+0000","last_peered":"2024-09-17T05:24:11.444004+0000","last_clean":"2024-09-17T05:24:11.444004+0000","last_became_active":"2024-09-17T05:24:10.477819+0000","last_became_peered":"2024-09-17T05:24:10.477819+0000","last_unstale":"2024-09-17T05:24:11.444004+0000","last_undegraded":"2024-09-17T05:24:11.444004+0000","last_fullsized":"2024-09-17T05:24:11.444004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:18.006097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570358+0000","last_change":"2024-09-17T05:24:09.439041+0000","last_active":"2024-09-17T05:24:11.570358+0000","last_peered":"2024-09-17T05:24:11.570358+0000","last_clean":"2024-09-17T05:24:11.570358+0000","last_became_active":"2024-09-17T05:24:09.438692+0000","last_became_peered":"2024-09-17T05:24:09.438692+0000","last_unstale":"2024-09-17T05:24:11.570358+0000","last_undegraded":"2024-09-17T05:24:11.570358+0000","last_fullsized":"2024-09-17T05:24:11.570358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:33.791376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444919+0000","last_change":"2024-09-17T05:24:10.453643+0000","last_active":"2024-09-17T05:24:11.444919+0000","last_peered":"2024-09-17T05:24:11.444919+0000","last_clean":"2024-09-17T05:24:11.444919+0000","last_became_active":"2024-09-17T05:24:10.452705+0000","last_became_peered":"2024-09-17T05:24:10.452705+0000","last_unstale":"2024-09-17T05:24:11.444919+0000","last_undegraded":"2024-09-17T05:24:11.444919+0000","last_fullsized":"2024-09-17T05:24:11.444919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:05.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443775+0000","last_change":"2024-09-17T05:24:09.431174+0000","last_active":"2024-09-17T05:24:11.443775+0000","last_peered":"2024-09-17T05:24:11.443775+0000","last_clean":"2024-09-17T05:24:11.443775+0000","last_became_active":"2024-09-17T05:24:09.430822+0000","last_became_peered":"2024-09-17T05:24:09.430822+0000","last_unstale":"2024-09-17T05:24:11.443775+0000","last_undegraded":"2024-09-17T05:24:11.443775+0000","last_fullsized":"2024-09-17T05:24:11.443775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:52.917186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444189+0000","last_change":"2024-09-17T05:24:10.455617+0000","last_active":"2024-09-17T05:24:11.444189+0000","last_peered":"2024-09-17T05:24:11.444189+0000","last_clean":"2024-09-17T05:24:11.444189+0000","last_became_active":"2024-09-17T05:24:10.455255+0000","last_became_peered":"2024-09-17T05:24:10.455255+0000","last_unstale":"2024-09-17T05:24:11.444189+0000","last_undegraded":"2024-09-17T05:24:11.444189+0000","last_fullsized":"2024-09-17T05:24:11.444189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445157+0000","last_change":"2024-09-17T05:24:09.432886+0000","last_active":"2024-09-17T05:24:11.445157+0000","last_peered":"2024-09-17T05:24:11.445157+0000","last_clean":"2024-09-17T05:24:11.445157+0000","last_became_active":"2024-09-17T05:24:09.432549+0000","last_became_peered":"2024-09-17T05:24:09.432549+0000","last_unstale":"2024-09-17T05:24:11.445157+0000","last_undegraded":"2024-09-17T05:24:11.445157+0000","last_fullsized":"2024-09-17T05:24:11.445157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:57.682790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":130,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569154+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:24:11.569154+0000","last_peered":"2024-09-17T05:24:11.569154+0000","last_clean":"2024-09-17T05:24:11.569154+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:24:11.569154+0000","last_undegraded":"2024-09-17T05:24:11.569154+0000","last_fullsized":"2024-09-17T05:24:11.569154+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":139506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":962560,"data_stored":730434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":141118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":380928,"data_stored":140260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":139518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":729922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":139574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:24:14.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:24:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 -- 172.21.15.43:0/3676130091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340adf80 msgr2=0x7fa9340ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 --2- 172.21.15.43:0/3676130091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340adf80 0x7fa9340ae390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa934069ae0 tx=0x7fa92001c830 comp rx=0 tx=0).stop 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 -- 172.21.15.43:0/3676130091 shutdown_connections 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 --2- 172.21.15.43:0/3676130091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340adf80 0x7fa9340ae390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 --2- 172.21.15.43:0/3676130091 >> v2:172.21.15.62:3300/0 conn(0x7fa9340b2f60 0x7fa93415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 -- 172.21.15.43:0/3676130091 >> 172.21.15.43:0/3676130091 conn(0x7fa9340d0010 msgr2=0x7fa9340d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 -- 172.21.15.43:0/3676130091 shutdown_connections 2024-09-17T05:24:14.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.846+0000 7fa939800640 1 -- 172.21.15.43:0/3676130091 wait complete. 2024-09-17T05:24:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 Processor -- start 2024-09-17T05:24:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 -- start start 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93406fe40 0x7fa9341fc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9340adf80 0x7fa9340ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa934168ef0 con 0x7fa9340b2f60 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa934168d70 con 0x7fa93406fe40 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa939800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa934169070 con 0x7fa9340adf80 2024-09-17T05:24:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56266/0 (socket says 172.21.15.43:56266) 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 -- 172.21.15.43:0/3235897055 learned_addr learned my addr 172.21.15.43:0/3235897055 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa933e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93406fe40 0x7fa9341fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa933400640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9340adf80 0x7fa9340ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 -- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9340adf80 msgr2=0x7fa9340ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9340adf80 0x7fa9340ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 -- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93406fe40 msgr2=0x7fa9341fc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93406fe40 0x7fa9341fc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 -- 172.21.15.43:0/3235897055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9340cfc00 con 0x7fa9340b2f60 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.850+0000 7fa938e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c6e0 tx=0x7fa92c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c003b80 con 0x7fa9340b2f60 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92c003d40 con 0x7fa9340b2f60 2024-09-17T05:24:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c005330 con 0x7fa9340b2f60 2024-09-17T05:24:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9341fcb50 con 0x7fa9340b2f60 2024-09-17T05:24:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341fd110 con 0x7fa9340b2f60 2024-09-17T05:24:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9341fc8c0 con 0x7fa9340b2f60 2024-09-17T05:24:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa92c0054d0 con 0x7fa9340b2f60 2024-09-17T05:24:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.854+0000 7fa92be00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 0x7fa914075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.858+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa92c093f10 con 0x7fa9340b2f60 2024-09-17T05:24:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.858+0000 7fa933e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 0x7fa914075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.858+0000 7fa933e00640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 0x7fa914075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa924004630 tx=0x7fa92400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:14.862+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa92c05ef20 con 0x7fa9340b2f60 2024-09-17T05:24:15.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.038+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9340ae390 con 0x7fa914073230 2024-09-17T05:24:15.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.042+0000 7fa92be00640 1 -- 172.21.15.43:0/3235897055 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fa9340ae390 con 0x7fa914073230 2024-09-17T05:24:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:15.047 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 msgr2=0x7fa914075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 0x7fa914075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa924004630 tx=0x7fa92400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 msgr2=0x7fa9340cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c6e0 tx=0x7fa92c00cba0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 shutdown_connections 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa914073230 0x7fa914075740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9340b2f60 0x7fa9340cf6c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9340adf80 0x7fa9340ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 --2- 172.21.15.43:0/3235897055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93406fe40 0x7fa9341fc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 >> 172.21.15.43:0/3235897055 conn(0x7fa9340d0010 msgr2=0x7fa934154720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:15.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 shutdown_connections 2024-09-17T05:24:15.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.046+0000 7fa939800640 1 -- 172.21.15.43:0/3235897055 wait complete. 2024-09-17T05:24:15.068 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-17T05:24:13.851519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236976,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737744,"statfs":{"total":773094113280,"available":772851449856,"internally_reserved":0,"allocated":4169728,"data_stored":2301428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.439451"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570342+0000","last_change":"2024-09-17T05:24:10.469539+0000","last_active":"2024-09-17T05:24:11.570342+0000","last_peered":"2024-09-17T05:24:11.570342+0000","last_clean":"2024-09-17T05:24:11.570342+0000","last_became_active":"2024-09-17T05:24:10.469128+0000","last_became_peered":"2024-09-17T05:24:10.469128+0000","last_unstale":"2024-09-17T05:24:11.570342+0000","last_undegraded":"2024-09-17T05:24:11.570342+0000","last_fullsized":"2024-09-17T05:24:11.570342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:54.305463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453091+0000","last_change":"2024-09-17T05:24:10.449291+0000","last_active":"2024-09-17T05:24:11.453091+0000","last_peered":"2024-09-17T05:24:11.453091+0000","last_clean":"2024-09-17T05:24:11.453091+0000","last_became_active":"2024-09-17T05:24:10.448917+0000","last_became_peered":"2024-09-17T05:24:10.448917+0000","last_unstale":"2024-09-17T05:24:11.453091+0000","last_undegraded":"2024-09-17T05:24:11.453091+0000","last_fullsized":"2024-09-17T05:24:11.453091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570198+0000","last_change":"2024-09-17T05:24:10.479273+0000","last_active":"2024-09-17T05:24:11.570198+0000","last_peered":"2024-09-17T05:24:11.570198+0000","last_clean":"2024-09-17T05:24:11.570198+0000","last_became_active":"2024-09-17T05:24:10.479050+0000","last_became_peered":"2024-09-17T05:24:10.479050+0000","last_unstale":"2024-09-17T05:24:11.570198+0000","last_undegraded":"2024-09-17T05:24:11.570198+0000","last_fullsized":"2024-09-17T05:24:11.570198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:53.923931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444296+0000","last_change":"2024-09-17T05:24:10.478081+0000","last_active":"2024-09-17T05:24:11.444296+0000","last_peered":"2024-09-17T05:24:11.444296+0000","last_clean":"2024-09-17T05:24:11.444296+0000","last_became_active":"2024-09-17T05:24:10.477727+0000","last_became_peered":"2024-09-17T05:24:10.477727+0000","last_unstale":"2024-09-17T05:24:11.444296+0000","last_undegraded":"2024-09-17T05:24:11.444296+0000","last_fullsized":"2024-09-17T05:24:11.444296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:45.178639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452917+0000","last_change":"2024-09-17T05:24:10.465083+0000","last_active":"2024-09-17T05:24:11.452917+0000","last_peered":"2024-09-17T05:24:11.452917+0000","last_clean":"2024-09-17T05:24:11.452917+0000","last_became_active":"2024-09-17T05:24:10.464809+0000","last_became_peered":"2024-09-17T05:24:10.464809+0000","last_unstale":"2024-09-17T05:24:11.452917+0000","last_undegraded":"2024-09-17T05:24:11.452917+0000","last_fullsized":"2024-09-17T05:24:11.452917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456255+0000","last_change":"2024-09-17T05:24:10.471777+0000","last_active":"2024-09-17T05:24:11.456255+0000","last_peered":"2024-09-17T05:24:11.456255+0000","last_clean":"2024-09-17T05:24:11.456255+0000","last_became_active":"2024-09-17T05:24:10.471398+0000","last_became_peered":"2024-09-17T05:24:10.471398+0000","last_unstale":"2024-09-17T05:24:11.456255+0000","last_undegraded":"2024-09-17T05:24:11.456255+0000","last_fullsized":"2024-09-17T05:24:11.456255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:31.522207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570192+0000","last_change":"2024-09-17T05:24:10.480068+0000","last_active":"2024-09-17T05:24:11.570192+0000","last_peered":"2024-09-17T05:24:11.570192+0000","last_clean":"2024-09-17T05:24:11.570192+0000","last_became_active":"2024-09-17T05:24:10.475137+0000","last_became_peered":"2024-09-17T05:24:10.475137+0000","last_unstale":"2024-09-17T05:24:11.570192+0000","last_undegraded":"2024-09-17T05:24:11.570192+0000","last_fullsized":"2024-09-17T05:24:11.570192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:56.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569021+0000","last_change":"2024-09-17T05:24:10.473972+0000","last_active":"2024-09-17T05:24:11.569021+0000","last_peered":"2024-09-17T05:24:11.569021+0000","last_clean":"2024-09-17T05:24:11.569021+0000","last_became_active":"2024-09-17T05:24:10.472739+0000","last_became_peered":"2024-09-17T05:24:10.472739+0000","last_unstale":"2024-09-17T05:24:11.569021+0000","last_undegraded":"2024-09-17T05:24:11.569021+0000","last_fullsized":"2024-09-17T05:24:11.569021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:54.044785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570490+0000","last_change":"2024-09-17T05:24:10.447416+0000","last_active":"2024-09-17T05:24:11.570490+0000","last_peered":"2024-09-17T05:24:11.570490+0000","last_clean":"2024-09-17T05:24:11.570490+0000","last_became_active":"2024-09-17T05:24:10.447063+0000","last_became_peered":"2024-09-17T05:24:10.447063+0000","last_unstale":"2024-09-17T05:24:11.570490+0000","last_undegraded":"2024-09-17T05:24:11.570490+0000","last_fullsized":"2024-09-17T05:24:11.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:15.131322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445109+0000","last_change":"2024-09-17T05:24:10.455903+0000","last_active":"2024-09-17T05:24:11.445109+0000","last_peered":"2024-09-17T05:24:11.445109+0000","last_clean":"2024-09-17T05:24:11.445109+0000","last_became_active":"2024-09-17T05:24:10.453153+0000","last_became_peered":"2024-09-17T05:24:10.453153+0000","last_unstale":"2024-09-17T05:24:11.445109+0000","last_undegraded":"2024-09-17T05:24:11.445109+0000","last_fullsized":"2024-09-17T05:24:11.445109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:35.164693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445068+0000","last_change":"2024-09-17T05:24:10.471759+0000","last_active":"2024-09-17T05:24:11.445068+0000","last_peered":"2024-09-17T05:24:11.445068+0000","last_clean":"2024-09-17T05:24:11.445068+0000","last_became_active":"2024-09-17T05:24:10.471377+0000","last_became_peered":"2024-09-17T05:24:10.471377+0000","last_unstale":"2024-09-17T05:24:11.445068+0000","last_undegraded":"2024-09-17T05:24:11.445068+0000","last_fullsized":"2024-09-17T05:24:11.445068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:04.027995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569746+0000","last_change":"2024-09-17T05:24:10.466311+0000","last_active":"2024-09-17T05:24:11.569746+0000","last_peered":"2024-09-17T05:24:11.569746+0000","last_clean":"2024-09-17T05:24:11.569746+0000","last_became_active":"2024-09-17T05:24:10.465979+0000","last_became_peered":"2024-09-17T05:24:10.465979+0000","last_unstale":"2024-09-17T05:24:11.569746+0000","last_undegraded":"2024-09-17T05:24:11.569746+0000","last_fullsized":"2024-09-17T05:24:11.569746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:34.798120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568799+0000","last_change":"2024-09-17T05:24:10.458151+0000","last_active":"2024-09-17T05:24:11.568799+0000","last_peered":"2024-09-17T05:24:11.568799+0000","last_clean":"2024-09-17T05:24:11.568799+0000","last_became_active":"2024-09-17T05:24:10.457674+0000","last_became_peered":"2024-09-17T05:24:10.457674+0000","last_unstale":"2024-09-17T05:24:11.568799+0000","last_undegraded":"2024-09-17T05:24:11.568799+0000","last_fullsized":"2024-09-17T05:24:11.568799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:26.206130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444036+0000","last_change":"2024-09-17T05:24:10.449131+0000","last_active":"2024-09-17T05:24:11.444036+0000","last_peered":"2024-09-17T05:24:11.444036+0000","last_clean":"2024-09-17T05:24:11.444036+0000","last_became_active":"2024-09-17T05:24:10.448711+0000","last_became_peered":"2024-09-17T05:24:10.448711+0000","last_unstale":"2024-09-17T05:24:11.444036+0000","last_undegraded":"2024-09-17T05:24:11.444036+0000","last_fullsized":"2024-09-17T05:24:11.444036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443780+0000","last_change":"2024-09-17T05:24:10.482530+0000","last_active":"2024-09-17T05:24:11.443780+0000","last_peered":"2024-09-17T05:24:11.443780+0000","last_clean":"2024-09-17T05:24:11.443780+0000","last_became_active":"2024-09-17T05:24:10.482165+0000","last_became_peered":"2024-09-17T05:24:10.482165+0000","last_unstale":"2024-09-17T05:24:11.443780+0000","last_undegraded":"2024-09-17T05:24:11.443780+0000","last_fullsized":"2024-09-17T05:24:11.443780+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443466+0000","last_change":"2024-09-17T05:24:10.456929+0000","last_active":"2024-09-17T05:24:11.443466+0000","last_peered":"2024-09-17T05:24:11.443466+0000","last_clean":"2024-09-17T05:24:11.443466+0000","last_became_active":"2024-09-17T05:24:10.455650+0000","last_became_peered":"2024-09-17T05:24:10.455650+0000","last_unstale":"2024-09-17T05:24:11.443466+0000","last_undegraded":"2024-09-17T05:24:11.443466+0000","last_fullsized":"2024-09-17T05:24:11.443466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:13.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445349+0000","last_change":"2024-09-17T05:24:10.470333+0000","last_active":"2024-09-17T05:24:11.445349+0000","last_peered":"2024-09-17T05:24:11.445349+0000","last_clean":"2024-09-17T05:24:11.445349+0000","last_became_active":"2024-09-17T05:24:10.469931+0000","last_became_peered":"2024-09-17T05:24:10.469931+0000","last_unstale":"2024-09-17T05:24:11.445349+0000","last_undegraded":"2024-09-17T05:24:11.445349+0000","last_fullsized":"2024-09-17T05:24:11.445349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:32.436816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456633+0000","last_change":"2024-09-17T05:24:10.470783+0000","last_active":"2024-09-17T05:24:11.456633+0000","last_peered":"2024-09-17T05:24:11.456633+0000","last_clean":"2024-09-17T05:24:11.456633+0000","last_became_active":"2024-09-17T05:24:10.470506+0000","last_became_peered":"2024-09-17T05:24:10.470506+0000","last_unstale":"2024-09-17T05:24:11.456633+0000","last_undegraded":"2024-09-17T05:24:11.456633+0000","last_fullsized":"2024-09-17T05:24:11.456633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:53.923931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456462+0000","last_change":"2024-09-17T05:24:10.470443+0000","last_active":"2024-09-17T05:24:11.456462+0000","last_peered":"2024-09-17T05:24:11.456462+0000","last_clean":"2024-09-17T05:24:11.456462+0000","last_became_active":"2024-09-17T05:24:10.460044+0000","last_became_peered":"2024-09-17T05:24:10.460044+0000","last_unstale":"2024-09-17T05:24:11.456462+0000","last_undegraded":"2024-09-17T05:24:11.456462+0000","last_fullsized":"2024-09-17T05:24:11.456462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:56.610810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568994+0000","last_change":"2024-09-17T05:24:10.458337+0000","last_active":"2024-09-17T05:24:11.568994+0000","last_peered":"2024-09-17T05:24:11.568994+0000","last_clean":"2024-09-17T05:24:11.568994+0000","last_became_active":"2024-09-17T05:24:10.457444+0000","last_became_peered":"2024-09-17T05:24:10.457444+0000","last_unstale":"2024-09-17T05:24:11.568994+0000","last_undegraded":"2024-09-17T05:24:11.568994+0000","last_fullsized":"2024-09-17T05:24:11.568994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:38.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443572+0000","last_change":"2024-09-17T05:24:10.458779+0000","last_active":"2024-09-17T05:24:11.443572+0000","last_peered":"2024-09-17T05:24:11.443572+0000","last_clean":"2024-09-17T05:24:11.443572+0000","last_became_active":"2024-09-17T05:24:10.458464+0000","last_became_peered":"2024-09-17T05:24:10.458464+0000","last_unstale":"2024-09-17T05:24:11.443572+0000","last_undegraded":"2024-09-17T05:24:11.443572+0000","last_fullsized":"2024-09-17T05:24:11.443572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:45.178639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569616+0000","last_change":"2024-09-17T05:24:10.468578+0000","last_active":"2024-09-17T05:24:11.569616+0000","last_peered":"2024-09-17T05:24:11.569616+0000","last_clean":"2024-09-17T05:24:11.569616+0000","last_became_active":"2024-09-17T05:24:10.468146+0000","last_became_peered":"2024-09-17T05:24:10.468146+0000","last_unstale":"2024-09-17T05:24:11.569616+0000","last_undegraded":"2024-09-17T05:24:11.569616+0000","last_fullsized":"2024-09-17T05:24:11.569616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:13.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568460+0000","last_change":"2024-09-17T05:24:10.462357+0000","last_active":"2024-09-17T05:24:11.568460+0000","last_peered":"2024-09-17T05:24:11.568460+0000","last_clean":"2024-09-17T05:24:11.568460+0000","last_became_active":"2024-09-17T05:24:10.461583+0000","last_became_peered":"2024-09-17T05:24:10.461583+0000","last_unstale":"2024-09-17T05:24:11.568460+0000","last_undegraded":"2024-09-17T05:24:11.568460+0000","last_fullsized":"2024-09-17T05:24:11.568460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:41.944177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569192+0000","last_change":"2024-09-17T05:24:10.473577+0000","last_active":"2024-09-17T05:24:11.569192+0000","last_peered":"2024-09-17T05:24:11.569192+0000","last_clean":"2024-09-17T05:24:11.569192+0000","last_became_active":"2024-09-17T05:24:10.472394+0000","last_became_peered":"2024-09-17T05:24:10.472394+0000","last_unstale":"2024-09-17T05:24:11.569192+0000","last_undegraded":"2024-09-17T05:24:11.569192+0000","last_fullsized":"2024-09-17T05:24:11.569192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:10.079654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445228+0000","last_change":"2024-09-17T05:24:10.453065+0000","last_active":"2024-09-17T05:24:11.445228+0000","last_peered":"2024-09-17T05:24:11.445228+0000","last_clean":"2024-09-17T05:24:11.445228+0000","last_became_active":"2024-09-17T05:24:10.452302+0000","last_became_peered":"2024-09-17T05:24:10.452302+0000","last_unstale":"2024-09-17T05:24:11.445228+0000","last_undegraded":"2024-09-17T05:24:11.445228+0000","last_fullsized":"2024-09-17T05:24:11.445228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:07.516584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445207+0000","last_change":"2024-09-17T05:24:10.439352+0000","last_active":"2024-09-17T05:24:11.445207+0000","last_peered":"2024-09-17T05:24:11.445207+0000","last_clean":"2024-09-17T05:24:11.445207+0000","last_became_active":"2024-09-17T05:24:10.439027+0000","last_became_peered":"2024-09-17T05:24:10.439027+0000","last_unstale":"2024-09-17T05:24:11.445207+0000","last_undegraded":"2024-09-17T05:24:11.445207+0000","last_fullsized":"2024-09-17T05:24:11.445207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:41.596150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452927+0000","last_change":"2024-09-17T05:24:10.455105+0000","last_active":"2024-09-17T05:24:11.452927+0000","last_peered":"2024-09-17T05:24:11.452927+0000","last_clean":"2024-09-17T05:24:11.452927+0000","last_became_active":"2024-09-17T05:24:10.454654+0000","last_became_peered":"2024-09-17T05:24:10.454654+0000","last_unstale":"2024-09-17T05:24:11.452927+0000","last_undegraded":"2024-09-17T05:24:11.452927+0000","last_fullsized":"2024-09-17T05:24:11.452927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:45.721427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569774+0000","last_change":"2024-09-17T05:24:10.471535+0000","last_active":"2024-09-17T05:24:11.569774+0000","last_peered":"2024-09-17T05:24:11.569774+0000","last_clean":"2024-09-17T05:24:11.569774+0000","last_became_active":"2024-09-17T05:24:10.470985+0000","last_became_peered":"2024-09-17T05:24:10.470985+0000","last_unstale":"2024-09-17T05:24:11.569774+0000","last_undegraded":"2024-09-17T05:24:11.569774+0000","last_fullsized":"2024-09-17T05:24:11.569774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:26.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443868+0000","last_change":"2024-09-17T05:24:10.439417+0000","last_active":"2024-09-17T05:24:11.443868+0000","last_peered":"2024-09-17T05:24:11.443868+0000","last_clean":"2024-09-17T05:24:11.443868+0000","last_became_active":"2024-09-17T05:24:10.439058+0000","last_became_peered":"2024-09-17T05:24:10.439058+0000","last_unstale":"2024-09-17T05:24:11.443868+0000","last_undegraded":"2024-09-17T05:24:11.443868+0000","last_fullsized":"2024-09-17T05:24:11.443868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443452+0000","last_change":"2024-09-17T05:24:10.473985+0000","last_active":"2024-09-17T05:24:11.443452+0000","last_peered":"2024-09-17T05:24:11.443452+0000","last_clean":"2024-09-17T05:24:11.443452+0000","last_became_active":"2024-09-17T05:24:10.473600+0000","last_became_peered":"2024-09-17T05:24:10.473600+0000","last_unstale":"2024-09-17T05:24:11.443452+0000","last_undegraded":"2024-09-17T05:24:11.443452+0000","last_fullsized":"2024-09-17T05:24:11.443452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:06.724070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570418+0000","last_change":"2024-09-17T05:24:10.475020+0000","last_active":"2024-09-17T05:24:11.570418+0000","last_peered":"2024-09-17T05:24:11.570418+0000","last_clean":"2024-09-17T05:24:11.570418+0000","last_became_active":"2024-09-17T05:24:10.467584+0000","last_became_peered":"2024-09-17T05:24:10.467584+0000","last_unstale":"2024-09-17T05:24:11.570418+0000","last_undegraded":"2024-09-17T05:24:11.570418+0000","last_fullsized":"2024-09-17T05:24:11.570418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:29.260645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445019+0000","last_change":"2024-09-17T05:24:10.482824+0000","last_active":"2024-09-17T05:24:11.445019+0000","last_peered":"2024-09-17T05:24:11.445019+0000","last_clean":"2024-09-17T05:24:11.445019+0000","last_became_active":"2024-09-17T05:24:10.482452+0000","last_became_peered":"2024-09-17T05:24:10.482452+0000","last_unstale":"2024-09-17T05:24:11.445019+0000","last_undegraded":"2024-09-17T05:24:11.445019+0000","last_fullsized":"2024-09-17T05:24:11.445019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:29.624742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568944+0000","last_change":"2024-09-17T05:24:10.472519+0000","last_active":"2024-09-17T05:24:11.568944+0000","last_peered":"2024-09-17T05:24:11.568944+0000","last_clean":"2024-09-17T05:24:11.568944+0000","last_became_active":"2024-09-17T05:24:10.472074+0000","last_became_peered":"2024-09-17T05:24:10.472074+0000","last_unstale":"2024-09-17T05:24:11.568944+0000","last_undegraded":"2024-09-17T05:24:11.568944+0000","last_fullsized":"2024-09-17T05:24:11.568944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:41.429333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445538+0000","last_change":"2024-09-17T05:24:09.436901+0000","last_active":"2024-09-17T05:24:11.445538+0000","last_peered":"2024-09-17T05:24:11.445538+0000","last_clean":"2024-09-17T05:24:11.445538+0000","last_became_active":"2024-09-17T05:24:09.436544+0000","last_became_peered":"2024-09-17T05:24:09.436544+0000","last_unstale":"2024-09-17T05:24:11.445538+0000","last_undegraded":"2024-09-17T05:24:11.445538+0000","last_fullsized":"2024-09-17T05:24:11.445538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:57.252525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570546+0000","last_change":"2024-09-17T05:24:10.479447+0000","last_active":"2024-09-17T05:24:11.570546+0000","last_peered":"2024-09-17T05:24:11.570546+0000","last_clean":"2024-09-17T05:24:11.570546+0000","last_became_active":"2024-09-17T05:24:10.479282+0000","last_became_peered":"2024-09-17T05:24:10.479282+0000","last_unstale":"2024-09-17T05:24:11.570546+0000","last_undegraded":"2024-09-17T05:24:11.570546+0000","last_fullsized":"2024-09-17T05:24:11.570546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:39.535829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445171+0000","last_change":"2024-09-17T05:24:09.435574+0000","last_active":"2024-09-17T05:24:11.445171+0000","last_peered":"2024-09-17T05:24:11.445171+0000","last_clean":"2024-09-17T05:24:11.445171+0000","last_became_active":"2024-09-17T05:24:09.432948+0000","last_became_peered":"2024-09-17T05:24:09.432948+0000","last_unstale":"2024-09-17T05:24:11.445171+0000","last_undegraded":"2024-09-17T05:24:11.445171+0000","last_fullsized":"2024-09-17T05:24:11.445171+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:42.498255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570056+0000","last_change":"2024-09-17T05:24:10.459215+0000","last_active":"2024-09-17T05:24:11.570056+0000","last_peered":"2024-09-17T05:24:11.570056+0000","last_clean":"2024-09-17T05:24:11.570056+0000","last_became_active":"2024-09-17T05:24:10.458906+0000","last_became_peered":"2024-09-17T05:24:10.458906+0000","last_unstale":"2024-09-17T05:24:11.570056+0000","last_undegraded":"2024-09-17T05:24:11.570056+0000","last_fullsized":"2024-09-17T05:24:11.570056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:45.721427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456663+0000","last_change":"2024-09-17T05:24:09.443681+0000","last_active":"2024-09-17T05:24:11.456663+0000","last_peered":"2024-09-17T05:24:11.456663+0000","last_clean":"2024-09-17T05:24:11.456663+0000","last_became_active":"2024-09-17T05:24:09.443337+0000","last_became_peered":"2024-09-17T05:24:09.443337+0000","last_unstale":"2024-09-17T05:24:11.456663+0000","last_undegraded":"2024-09-17T05:24:11.456663+0000","last_fullsized":"2024-09-17T05:24:11.456663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:14.514733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445102+0000","last_change":"2024-09-17T05:24:10.457894+0000","last_active":"2024-09-17T05:24:11.445102+0000","last_peered":"2024-09-17T05:24:11.445102+0000","last_clean":"2024-09-17T05:24:11.445102+0000","last_became_active":"2024-09-17T05:24:10.457559+0000","last_became_peered":"2024-09-17T05:24:10.457559+0000","last_unstale":"2024-09-17T05:24:11.445102+0000","last_undegraded":"2024-09-17T05:24:11.445102+0000","last_fullsized":"2024-09-17T05:24:11.445102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:51.131527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444391+0000","last_change":"2024-09-17T05:24:09.437966+0000","last_active":"2024-09-17T05:24:11.444391+0000","last_peered":"2024-09-17T05:24:11.444391+0000","last_clean":"2024-09-17T05:24:11.444391+0000","last_became_active":"2024-09-17T05:24:09.437541+0000","last_became_peered":"2024-09-17T05:24:09.437541+0000","last_unstale":"2024-09-17T05:24:11.444391+0000","last_undegraded":"2024-09-17T05:24:11.444391+0000","last_fullsized":"2024-09-17T05:24:11.444391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:14.124577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569575+0000","last_change":"2024-09-17T05:24:10.447614+0000","last_active":"2024-09-17T05:24:11.569575+0000","last_peered":"2024-09-17T05:24:11.569575+0000","last_clean":"2024-09-17T05:24:11.569575+0000","last_became_active":"2024-09-17T05:24:10.447129+0000","last_became_peered":"2024-09-17T05:24:10.447129+0000","last_unstale":"2024-09-17T05:24:11.569575+0000","last_undegraded":"2024-09-17T05:24:11.569575+0000","last_fullsized":"2024-09-17T05:24:11.569575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:56.610810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569306+0000","last_change":"2024-09-17T05:24:09.441000+0000","last_active":"2024-09-17T05:24:11.569306+0000","last_peered":"2024-09-17T05:24:11.569306+0000","last_clean":"2024-09-17T05:24:11.569306+0000","last_became_active":"2024-09-17T05:24:09.439493+0000","last_became_peered":"2024-09-17T05:24:09.439493+0000","last_unstale":"2024-09-17T05:24:11.569306+0000","last_undegraded":"2024-09-17T05:24:11.569306+0000","last_fullsized":"2024-09-17T05:24:11.569306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:37.999908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569292+0000","last_change":"2024-09-17T05:24:10.470464+0000","last_active":"2024-09-17T05:24:11.569292+0000","last_peered":"2024-09-17T05:24:11.569292+0000","last_clean":"2024-09-17T05:24:11.569292+0000","last_became_active":"2024-09-17T05:24:10.467481+0000","last_became_peered":"2024-09-17T05:24:10.467481+0000","last_unstale":"2024-09-17T05:24:11.569292+0000","last_undegraded":"2024-09-17T05:24:11.569292+0000","last_fullsized":"2024-09-17T05:24:11.569292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:12.282949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569696+0000","last_change":"2024-09-17T05:24:09.436952+0000","last_active":"2024-09-17T05:24:11.569696+0000","last_peered":"2024-09-17T05:24:11.569696+0000","last_clean":"2024-09-17T05:24:11.569696+0000","last_became_active":"2024-09-17T05:24:09.436639+0000","last_became_peered":"2024-09-17T05:24:09.436639+0000","last_unstale":"2024-09-17T05:24:11.569696+0000","last_undegraded":"2024-09-17T05:24:11.569696+0000","last_fullsized":"2024-09-17T05:24:11.569696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:30.515462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569062+0000","last_change":"2024-09-17T05:24:10.439298+0000","last_active":"2024-09-17T05:24:11.569062+0000","last_peered":"2024-09-17T05:24:11.569062+0000","last_clean":"2024-09-17T05:24:11.569062+0000","last_became_active":"2024-09-17T05:24:10.438920+0000","last_became_peered":"2024-09-17T05:24:10.438920+0000","last_unstale":"2024-09-17T05:24:11.569062+0000","last_undegraded":"2024-09-17T05:24:11.569062+0000","last_fullsized":"2024-09-17T05:24:11.569062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:39.006653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456089+0000","last_change":"2024-09-17T05:24:09.445316+0000","last_active":"2024-09-17T05:24:11.456089+0000","last_peered":"2024-09-17T05:24:11.456089+0000","last_clean":"2024-09-17T05:24:11.456089+0000","last_became_active":"2024-09-17T05:24:09.444999+0000","last_became_peered":"2024-09-17T05:24:09.444999+0000","last_unstale":"2024-09-17T05:24:11.456089+0000","last_undegraded":"2024-09-17T05:24:11.456089+0000","last_fullsized":"2024-09-17T05:24:11.456089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:30.393590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443697+0000","last_change":"2024-09-17T05:24:10.465646+0000","last_active":"2024-09-17T05:24:11.443697+0000","last_peered":"2024-09-17T05:24:11.443697+0000","last_clean":"2024-09-17T05:24:11.443697+0000","last_became_active":"2024-09-17T05:24:10.465288+0000","last_became_peered":"2024-09-17T05:24:10.465288+0000","last_unstale":"2024-09-17T05:24:11.443697+0000","last_undegraded":"2024-09-17T05:24:11.443697+0000","last_fullsized":"2024-09-17T05:24:11.443697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:06.724070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569263+0000","last_change":"2024-09-17T05:24:09.440937+0000","last_active":"2024-09-17T05:24:11.569263+0000","last_peered":"2024-09-17T05:24:11.569263+0000","last_clean":"2024-09-17T05:24:11.569263+0000","last_became_active":"2024-09-17T05:24:09.440577+0000","last_became_peered":"2024-09-17T05:24:09.440577+0000","last_unstale":"2024-09-17T05:24:11.569263+0000","last_undegraded":"2024-09-17T05:24:11.569263+0000","last_fullsized":"2024-09-17T05:24:11.569263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:22.502057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445368+0000","last_change":"2024-09-17T05:24:10.466580+0000","last_active":"2024-09-17T05:24:11.445368+0000","last_peered":"2024-09-17T05:24:11.445368+0000","last_clean":"2024-09-17T05:24:11.445368+0000","last_became_active":"2024-09-17T05:24:10.466041+0000","last_became_peered":"2024-09-17T05:24:10.466041+0000","last_unstale":"2024-09-17T05:24:11.445368+0000","last_undegraded":"2024-09-17T05:24:11.445368+0000","last_fullsized":"2024-09-17T05:24:11.445368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:01.975736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452937+0000","last_change":"2024-09-17T05:24:09.435396+0000","last_active":"2024-09-17T05:24:11.452937+0000","last_peered":"2024-09-17T05:24:11.452937+0000","last_clean":"2024-09-17T05:24:11.452937+0000","last_became_active":"2024-09-17T05:24:09.434542+0000","last_became_peered":"2024-09-17T05:24:09.434542+0000","last_unstale":"2024-09-17T05:24:11.452937+0000","last_undegraded":"2024-09-17T05:24:11.452937+0000","last_fullsized":"2024-09-17T05:24:11.452937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:25.086177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445486+0000","last_change":"2024-09-17T05:24:10.452676+0000","last_active":"2024-09-17T05:24:11.445486+0000","last_peered":"2024-09-17T05:24:11.445486+0000","last_clean":"2024-09-17T05:24:11.445486+0000","last_became_active":"2024-09-17T05:24:10.451854+0000","last_became_peered":"2024-09-17T05:24:10.451854+0000","last_unstale":"2024-09-17T05:24:11.445486+0000","last_undegraded":"2024-09-17T05:24:11.445486+0000","last_fullsized":"2024-09-17T05:24:11.445486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:50.183785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453041+0000","last_change":"2024-09-17T05:24:09.442019+0000","last_active":"2024-09-17T05:24:11.453041+0000","last_peered":"2024-09-17T05:24:11.453041+0000","last_clean":"2024-09-17T05:24:11.453041+0000","last_became_active":"2024-09-17T05:24:09.441714+0000","last_became_peered":"2024-09-17T05:24:09.441714+0000","last_unstale":"2024-09-17T05:24:11.453041+0000","last_undegraded":"2024-09-17T05:24:11.453041+0000","last_fullsized":"2024-09-17T05:24:11.453041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:58.866053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444927+0000","last_change":"2024-09-17T05:24:10.478011+0000","last_active":"2024-09-17T05:24:11.444927+0000","last_peered":"2024-09-17T05:24:11.444927+0000","last_clean":"2024-09-17T05:24:11.444927+0000","last_became_active":"2024-09-17T05:24:10.477605+0000","last_became_peered":"2024-09-17T05:24:10.477605+0000","last_unstale":"2024-09-17T05:24:11.444927+0000","last_undegraded":"2024-09-17T05:24:11.444927+0000","last_fullsized":"2024-09-17T05:24:11.444927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445186+0000","last_change":"2024-09-17T05:24:09.439895+0000","last_active":"2024-09-17T05:24:11.445186+0000","last_peered":"2024-09-17T05:24:11.445186+0000","last_clean":"2024-09-17T05:24:11.445186+0000","last_became_active":"2024-09-17T05:24:09.439499+0000","last_became_peered":"2024-09-17T05:24:09.439499+0000","last_unstale":"2024-09-17T05:24:11.445186+0000","last_undegraded":"2024-09-17T05:24:11.445186+0000","last_fullsized":"2024-09-17T05:24:11.445186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:06.509839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444948+0000","last_change":"2024-09-17T05:24:10.463610+0000","last_active":"2024-09-17T05:24:11.444948+0000","last_peered":"2024-09-17T05:24:11.444948+0000","last_clean":"2024-09-17T05:24:11.444948+0000","last_became_active":"2024-09-17T05:24:10.463287+0000","last_became_peered":"2024-09-17T05:24:10.463287+0000","last_unstale":"2024-09-17T05:24:11.444948+0000","last_undegraded":"2024-09-17T05:24:11.444948+0000","last_fullsized":"2024-09-17T05:24:11.444948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:48.603532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568875+0000","last_change":"2024-09-17T05:24:09.434869+0000","last_active":"2024-09-17T05:24:11.568875+0000","last_peered":"2024-09-17T05:24:11.568875+0000","last_clean":"2024-09-17T05:24:11.568875+0000","last_became_active":"2024-09-17T05:24:09.434451+0000","last_became_peered":"2024-09-17T05:24:09.434451+0000","last_unstale":"2024-09-17T05:24:11.568875+0000","last_undegraded":"2024-09-17T05:24:11.568875+0000","last_fullsized":"2024-09-17T05:24:11.568875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:30.285720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568621+0000","last_change":"2024-09-17T05:24:10.457614+0000","last_active":"2024-09-17T05:24:11.568621+0000","last_peered":"2024-09-17T05:24:11.568621+0000","last_clean":"2024-09-17T05:24:11.568621+0000","last_became_active":"2024-09-17T05:24:10.457278+0000","last_became_peered":"2024-09-17T05:24:10.457278+0000","last_unstale":"2024-09-17T05:24:11.568621+0000","last_undegraded":"2024-09-17T05:24:11.568621+0000","last_fullsized":"2024-09-17T05:24:11.568621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:24.118863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569367+0000","last_change":"2024-09-17T05:24:09.444441+0000","last_active":"2024-09-17T05:24:11.569367+0000","last_peered":"2024-09-17T05:24:11.569367+0000","last_clean":"2024-09-17T05:24:11.569367+0000","last_became_active":"2024-09-17T05:24:09.443853+0000","last_became_peered":"2024-09-17T05:24:09.443853+0000","last_unstale":"2024-09-17T05:24:11.569367+0000","last_undegraded":"2024-09-17T05:24:11.569367+0000","last_fullsized":"2024-09-17T05:24:11.569367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:52.917186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444541+0000","last_change":"2024-09-17T05:24:10.464584+0000","last_active":"2024-09-17T05:24:11.444541+0000","last_peered":"2024-09-17T05:24:11.444541+0000","last_clean":"2024-09-17T05:24:11.444541+0000","last_became_active":"2024-09-17T05:24:10.460610+0000","last_became_peered":"2024-09-17T05:24:10.460610+0000","last_unstale":"2024-09-17T05:24:11.444541+0000","last_undegraded":"2024-09-17T05:24:11.444541+0000","last_fullsized":"2024-09-17T05:24:11.444541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444040+0000","last_change":"2024-09-17T05:24:09.436373+0000","last_active":"2024-09-17T05:24:11.444040+0000","last_peered":"2024-09-17T05:24:11.444040+0000","last_clean":"2024-09-17T05:24:11.444040+0000","last_became_active":"2024-09-17T05:24:09.436055+0000","last_became_peered":"2024-09-17T05:24:09.436055+0000","last_unstale":"2024-09-17T05:24:11.444040+0000","last_undegraded":"2024-09-17T05:24:11.444040+0000","last_fullsized":"2024-09-17T05:24:11.444040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:33.791376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569276+0000","last_change":"2024-09-17T05:24:10.473239+0000","last_active":"2024-09-17T05:24:11.569276+0000","last_peered":"2024-09-17T05:24:11.569276+0000","last_clean":"2024-09-17T05:24:11.569276+0000","last_became_active":"2024-09-17T05:24:10.472053+0000","last_became_peered":"2024-09-17T05:24:10.472053+0000","last_unstale":"2024-09-17T05:24:11.569276+0000","last_undegraded":"2024-09-17T05:24:11.569276+0000","last_fullsized":"2024-09-17T05:24:11.569276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:22.208293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443785+0000","last_change":"2024-09-17T05:24:09.431865+0000","last_active":"2024-09-17T05:24:11.443785+0000","last_peered":"2024-09-17T05:24:11.443785+0000","last_clean":"2024-09-17T05:24:11.443785+0000","last_became_active":"2024-09-17T05:24:09.431545+0000","last_became_peered":"2024-09-17T05:24:09.431545+0000","last_unstale":"2024-09-17T05:24:11.443785+0000","last_undegraded":"2024-09-17T05:24:11.443785+0000","last_fullsized":"2024-09-17T05:24:11.443785+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:55.604065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444880+0000","last_change":"2024-09-17T05:24:10.483993+0000","last_active":"2024-09-17T05:24:11.444880+0000","last_peered":"2024-09-17T05:24:11.444880+0000","last_clean":"2024-09-17T05:24:11.444880+0000","last_became_active":"2024-09-17T05:24:10.482664+0000","last_became_peered":"2024-09-17T05:24:10.482664+0000","last_unstale":"2024-09-17T05:24:11.444880+0000","last_undegraded":"2024-09-17T05:24:11.444880+0000","last_fullsized":"2024-09-17T05:24:11.444880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:16.553451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.455944+0000","last_change":"2024-09-17T05:24:09.443609+0000","last_active":"2024-09-17T05:24:11.455944+0000","last_peered":"2024-09-17T05:24:11.455944+0000","last_clean":"2024-09-17T05:24:11.455944+0000","last_became_active":"2024-09-17T05:24:09.443210+0000","last_became_peered":"2024-09-17T05:24:09.443210+0000","last_unstale":"2024-09-17T05:24:11.455944+0000","last_undegraded":"2024-09-17T05:24:11.455944+0000","last_fullsized":"2024-09-17T05:24:11.455944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:43.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445462+0000","last_change":"2024-09-17T05:24:10.467882+0000","last_active":"2024-09-17T05:24:11.445462+0000","last_peered":"2024-09-17T05:24:11.445462+0000","last_clean":"2024-09-17T05:24:11.445462+0000","last_became_active":"2024-09-17T05:24:10.467444+0000","last_became_peered":"2024-09-17T05:24:10.467444+0000","last_unstale":"2024-09-17T05:24:11.445462+0000","last_undegraded":"2024-09-17T05:24:11.445462+0000","last_fullsized":"2024-09-17T05:24:11.445462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:12.969259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456505+0000","last_change":"2024-09-17T05:24:09.447465+0000","last_active":"2024-09-17T05:24:11.456505+0000","last_peered":"2024-09-17T05:24:11.456505+0000","last_clean":"2024-09-17T05:24:11.456505+0000","last_became_active":"2024-09-17T05:24:09.446941+0000","last_became_peered":"2024-09-17T05:24:09.446941+0000","last_unstale":"2024-09-17T05:24:11.456505+0000","last_undegraded":"2024-09-17T05:24:11.456505+0000","last_fullsized":"2024-09-17T05:24:11.456505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:58.544807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570499+0000","last_change":"2024-09-17T05:24:10.441511+0000","last_active":"2024-09-17T05:24:11.570499+0000","last_peered":"2024-09-17T05:24:11.570499+0000","last_clean":"2024-09-17T05:24:11.570499+0000","last_became_active":"2024-09-17T05:24:10.441225+0000","last_became_peered":"2024-09-17T05:24:10.441225+0000","last_unstale":"2024-09-17T05:24:11.570499+0000","last_undegraded":"2024-09-17T05:24:11.570499+0000","last_fullsized":"2024-09-17T05:24:11.570499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:28.879316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570508+0000","last_change":"2024-09-17T05:24:09.442314+0000","last_active":"2024-09-17T05:24:11.570508+0000","last_peered":"2024-09-17T05:24:11.570508+0000","last_clean":"2024-09-17T05:24:11.570508+0000","last_became_active":"2024-09-17T05:24:09.441967+0000","last_became_peered":"2024-09-17T05:24:09.441967+0000","last_unstale":"2024-09-17T05:24:11.570508+0000","last_undegraded":"2024-09-17T05:24:11.570508+0000","last_fullsized":"2024-09-17T05:24:11.570508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:24.057204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.452976+0000","last_change":"2024-09-17T05:24:10.460678+0000","last_active":"2024-09-17T05:24:11.452976+0000","last_peered":"2024-09-17T05:24:11.452976+0000","last_clean":"2024-09-17T05:24:11.452976+0000","last_became_active":"2024-09-17T05:24:10.460186+0000","last_became_peered":"2024-09-17T05:24:10.460186+0000","last_unstale":"2024-09-17T05:24:11.452976+0000","last_undegraded":"2024-09-17T05:24:11.452976+0000","last_fullsized":"2024-09-17T05:24:11.452976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:04.434394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445030+0000","last_change":"2024-09-17T05:24:09.432947+0000","last_active":"2024-09-17T05:24:11.445030+0000","last_peered":"2024-09-17T05:24:11.445030+0000","last_clean":"2024-09-17T05:24:11.445030+0000","last_became_active":"2024-09-17T05:24:09.432619+0000","last_became_peered":"2024-09-17T05:24:09.432619+0000","last_unstale":"2024-09-17T05:24:11.445030+0000","last_undegraded":"2024-09-17T05:24:11.445030+0000","last_fullsized":"2024-09-17T05:24:11.445030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:40.589405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456549+0000","last_change":"2024-09-17T05:24:10.470780+0000","last_active":"2024-09-17T05:24:11.456549+0000","last_peered":"2024-09-17T05:24:11.456549+0000","last_clean":"2024-09-17T05:24:11.456549+0000","last_became_active":"2024-09-17T05:24:10.470080+0000","last_became_peered":"2024-09-17T05:24:10.470080+0000","last_unstale":"2024-09-17T05:24:11.456549+0000","last_undegraded":"2024-09-17T05:24:11.456549+0000","last_fullsized":"2024-09-17T05:24:11.456549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:34.798120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444123+0000","last_change":"2024-09-17T05:24:09.425489+0000","last_active":"2024-09-17T05:24:11.444123+0000","last_peered":"2024-09-17T05:24:11.444123+0000","last_clean":"2024-09-17T05:24:11.444123+0000","last_became_active":"2024-09-17T05:24:09.425164+0000","last_became_peered":"2024-09-17T05:24:09.425164+0000","last_unstale":"2024-09-17T05:24:11.444123+0000","last_undegraded":"2024-09-17T05:24:11.444123+0000","last_fullsized":"2024-09-17T05:24:11.444123+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:44.373590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568520+0000","last_change":"2024-09-17T05:24:10.463698+0000","last_active":"2024-09-17T05:24:11.568520+0000","last_peered":"2024-09-17T05:24:11.568520+0000","last_clean":"2024-09-17T05:24:11.568520+0000","last_became_active":"2024-09-17T05:24:10.462936+0000","last_became_peered":"2024-09-17T05:24:10.462936+0000","last_unstale":"2024-09-17T05:24:11.568520+0000","last_undegraded":"2024-09-17T05:24:11.568520+0000","last_fullsized":"2024-09-17T05:24:11.568520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:00.907853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453254+0000","last_change":"2024-09-17T05:24:09.431174+0000","last_active":"2024-09-17T05:24:11.453254+0000","last_peered":"2024-09-17T05:24:11.453254+0000","last_clean":"2024-09-17T05:24:11.453254+0000","last_became_active":"2024-09-17T05:24:09.430822+0000","last_became_peered":"2024-09-17T05:24:09.430822+0000","last_unstale":"2024-09-17T05:24:11.453254+0000","last_undegraded":"2024-09-17T05:24:11.453254+0000","last_fullsized":"2024-09-17T05:24:11.453254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:27.555116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453175+0000","last_change":"2024-09-17T05:24:10.452124+0000","last_active":"2024-09-17T05:24:11.453175+0000","last_peered":"2024-09-17T05:24:11.453175+0000","last_clean":"2024-09-17T05:24:11.453175+0000","last_became_active":"2024-09-17T05:24:10.451753+0000","last_became_peered":"2024-09-17T05:24:10.451753+0000","last_unstale":"2024-09-17T05:24:11.453175+0000","last_undegraded":"2024-09-17T05:24:11.453175+0000","last_fullsized":"2024-09-17T05:24:11.453175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569821+0000","last_change":"2024-09-17T05:24:09.438394+0000","last_active":"2024-09-17T05:24:11.569821+0000","last_peered":"2024-09-17T05:24:11.569821+0000","last_clean":"2024-09-17T05:24:11.569821+0000","last_became_active":"2024-09-17T05:24:09.438042+0000","last_became_peered":"2024-09-17T05:24:09.438042+0000","last_unstale":"2024-09-17T05:24:11.569821+0000","last_undegraded":"2024-09-17T05:24:11.569821+0000","last_fullsized":"2024-09-17T05:24:11.569821+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:55.604065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568458+0000","last_change":"2024-09-17T05:24:10.459057+0000","last_active":"2024-09-17T05:24:11.568458+0000","last_peered":"2024-09-17T05:24:11.568458+0000","last_clean":"2024-09-17T05:24:11.568458+0000","last_became_active":"2024-09-17T05:24:10.458676+0000","last_became_peered":"2024-09-17T05:24:10.458676+0000","last_unstale":"2024-09-17T05:24:11.568458+0000","last_undegraded":"2024-09-17T05:24:11.568458+0000","last_fullsized":"2024-09-17T05:24:11.568458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:48.457021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569113+0000","last_change":"2024-09-17T05:24:09.437988+0000","last_active":"2024-09-17T05:24:11.569113+0000","last_peered":"2024-09-17T05:24:11.569113+0000","last_clean":"2024-09-17T05:24:11.569113+0000","last_became_active":"2024-09-17T05:24:09.437668+0000","last_became_peered":"2024-09-17T05:24:09.437668+0000","last_unstale":"2024-09-17T05:24:11.569113+0000","last_undegraded":"2024-09-17T05:24:11.569113+0000","last_fullsized":"2024-09-17T05:24:11.569113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:43.197372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569404+0000","last_change":"2024-09-17T05:24:10.440898+0000","last_active":"2024-09-17T05:24:11.569404+0000","last_peered":"2024-09-17T05:24:11.569404+0000","last_clean":"2024-09-17T05:24:11.569404+0000","last_became_active":"2024-09-17T05:24:10.440549+0000","last_became_peered":"2024-09-17T05:24:10.440549+0000","last_unstale":"2024-09-17T05:24:11.569404+0000","last_undegraded":"2024-09-17T05:24:11.569404+0000","last_fullsized":"2024-09-17T05:24:11.569404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:31.522207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444572+0000","last_change":"2024-09-17T05:24:09.440958+0000","last_active":"2024-09-17T05:24:11.444572+0000","last_peered":"2024-09-17T05:24:11.444572+0000","last_clean":"2024-09-17T05:24:11.444572+0000","last_became_active":"2024-09-17T05:24:09.435037+0000","last_became_peered":"2024-09-17T05:24:09.435037+0000","last_unstale":"2024-09-17T05:24:11.444572+0000","last_undegraded":"2024-09-17T05:24:11.444572+0000","last_fullsized":"2024-09-17T05:24:11.444572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:14.152601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456759+0000","last_change":"2024-09-17T05:24:10.468367+0000","last_active":"2024-09-17T05:24:11.456759+0000","last_peered":"2024-09-17T05:24:11.456759+0000","last_clean":"2024-09-17T05:24:11.456759+0000","last_became_active":"2024-09-17T05:24:10.467854+0000","last_became_peered":"2024-09-17T05:24:10.467854+0000","last_unstale":"2024-09-17T05:24:11.456759+0000","last_undegraded":"2024-09-17T05:24:11.456759+0000","last_fullsized":"2024-09-17T05:24:11.456759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:43.505000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568561+0000","last_change":"2024-09-17T05:24:09.440411+0000","last_active":"2024-09-17T05:24:11.568561+0000","last_peered":"2024-09-17T05:24:11.568561+0000","last_clean":"2024-09-17T05:24:11.568561+0000","last_became_active":"2024-09-17T05:24:09.440083+0000","last_became_peered":"2024-09-17T05:24:09.440083+0000","last_unstale":"2024-09-17T05:24:11.568561+0000","last_undegraded":"2024-09-17T05:24:11.568561+0000","last_fullsized":"2024-09-17T05:24:11.568561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:20.858099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445495+0000","last_change":"2024-09-17T05:24:10.459958+0000","last_active":"2024-09-17T05:24:11.445495+0000","last_peered":"2024-09-17T05:24:11.445495+0000","last_clean":"2024-09-17T05:24:11.445495+0000","last_became_active":"2024-09-17T05:24:10.457607+0000","last_became_peered":"2024-09-17T05:24:10.457607+0000","last_unstale":"2024-09-17T05:24:11.445495+0000","last_undegraded":"2024-09-17T05:24:11.445495+0000","last_fullsized":"2024-09-17T05:24:11.445495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:54.982346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.453187+0000","last_change":"2024-09-17T05:24:09.441893+0000","last_active":"2024-09-17T05:24:11.453187+0000","last_peered":"2024-09-17T05:24:11.453187+0000","last_clean":"2024-09-17T05:24:11.453187+0000","last_became_active":"2024-09-17T05:24:09.441593+0000","last_became_peered":"2024-09-17T05:24:09.441593+0000","last_unstale":"2024-09-17T05:24:11.453187+0000","last_undegraded":"2024-09-17T05:24:11.453187+0000","last_fullsized":"2024-09-17T05:24:11.453187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:05.131621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.456726+0000","last_change":"2024-09-17T05:24:10.451152+0000","last_active":"2024-09-17T05:24:11.456726+0000","last_peered":"2024-09-17T05:24:11.456726+0000","last_clean":"2024-09-17T05:24:11.456726+0000","last_became_active":"2024-09-17T05:24:10.450789+0000","last_became_peered":"2024-09-17T05:24:10.450789+0000","last_unstale":"2024-09-17T05:24:11.456726+0000","last_undegraded":"2024-09-17T05:24:11.456726+0000","last_fullsized":"2024-09-17T05:24:11.456726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:15.131322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568981+0000","last_change":"2024-09-17T05:24:09.434969+0000","last_active":"2024-09-17T05:24:11.568981+0000","last_peered":"2024-09-17T05:24:11.568981+0000","last_clean":"2024-09-17T05:24:11.568981+0000","last_became_active":"2024-09-17T05:24:09.433934+0000","last_became_peered":"2024-09-17T05:24:09.433934+0000","last_unstale":"2024-09-17T05:24:11.568981+0000","last_undegraded":"2024-09-17T05:24:11.568981+0000","last_fullsized":"2024-09-17T05:24:11.568981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:10.177328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443530+0000","last_change":"2024-09-17T05:24:10.455901+0000","last_active":"2024-09-17T05:24:11.443530+0000","last_peered":"2024-09-17T05:24:11.443530+0000","last_clean":"2024-09-17T05:24:11.443530+0000","last_became_active":"2024-09-17T05:24:10.455536+0000","last_became_peered":"2024-09-17T05:24:10.455536+0000","last_unstale":"2024-09-17T05:24:11.443530+0000","last_undegraded":"2024-09-17T05:24:11.443530+0000","last_fullsized":"2024-09-17T05:24:11.443530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:47.491680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568891+0000","last_change":"2024-09-17T05:24:09.439958+0000","last_active":"2024-09-17T05:24:11.568891+0000","last_peered":"2024-09-17T05:24:11.568891+0000","last_clean":"2024-09-17T05:24:11.568891+0000","last_became_active":"2024-09-17T05:24:09.439622+0000","last_became_peered":"2024-09-17T05:24:09.439622+0000","last_unstale":"2024-09-17T05:24:11.568891+0000","last_undegraded":"2024-09-17T05:24:11.568891+0000","last_fullsized":"2024-09-17T05:24:11.568891+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.966754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444878+0000","last_change":"2024-09-17T05:24:10.461724+0000","last_active":"2024-09-17T05:24:11.444878+0000","last_peered":"2024-09-17T05:24:11.444878+0000","last_clean":"2024-09-17T05:24:11.444878+0000","last_became_active":"2024-09-17T05:24:10.460230+0000","last_became_peered":"2024-09-17T05:24:10.460230+0000","last_unstale":"2024-09-17T05:24:11.444878+0000","last_undegraded":"2024-09-17T05:24:11.444878+0000","last_fullsized":"2024-09-17T05:24:11.444878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:58.259270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.568807+0000","last_change":"2024-09-17T05:24:09.437486+0000","last_active":"2024-09-17T05:24:11.568807+0000","last_peered":"2024-09-17T05:24:11.568807+0000","last_clean":"2024-09-17T05:24:11.568807+0000","last_became_active":"2024-09-17T05:24:09.437238+0000","last_became_peered":"2024-09-17T05:24:09.437238+0000","last_unstale":"2024-09-17T05:24:11.568807+0000","last_undegraded":"2024-09-17T05:24:11.568807+0000","last_fullsized":"2024-09-17T05:24:11.568807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:01.421393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444004+0000","last_change":"2024-09-17T05:24:10.478407+0000","last_active":"2024-09-17T05:24:11.444004+0000","last_peered":"2024-09-17T05:24:11.444004+0000","last_clean":"2024-09-17T05:24:11.444004+0000","last_became_active":"2024-09-17T05:24:10.477819+0000","last_became_peered":"2024-09-17T05:24:10.477819+0000","last_unstale":"2024-09-17T05:24:11.444004+0000","last_undegraded":"2024-09-17T05:24:11.444004+0000","last_fullsized":"2024-09-17T05:24:11.444004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:18.006097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.570358+0000","last_change":"2024-09-17T05:24:09.439041+0000","last_active":"2024-09-17T05:24:11.570358+0000","last_peered":"2024-09-17T05:24:11.570358+0000","last_clean":"2024-09-17T05:24:11.570358+0000","last_became_active":"2024-09-17T05:24:09.438692+0000","last_became_peered":"2024-09-17T05:24:09.438692+0000","last_unstale":"2024-09-17T05:24:11.570358+0000","last_undegraded":"2024-09-17T05:24:11.570358+0000","last_fullsized":"2024-09-17T05:24:11.570358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:33.791376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444919+0000","last_change":"2024-09-17T05:24:10.453643+0000","last_active":"2024-09-17T05:24:11.444919+0000","last_peered":"2024-09-17T05:24:11.444919+0000","last_clean":"2024-09-17T05:24:11.444919+0000","last_became_active":"2024-09-17T05:24:10.452705+0000","last_became_peered":"2024-09-17T05:24:10.452705+0000","last_unstale":"2024-09-17T05:24:11.444919+0000","last_undegraded":"2024-09-17T05:24:11.444919+0000","last_fullsized":"2024-09-17T05:24:11.444919+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:05.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.443775+0000","last_change":"2024-09-17T05:24:09.431174+0000","last_active":"2024-09-17T05:24:11.443775+0000","last_peered":"2024-09-17T05:24:11.443775+0000","last_clean":"2024-09-17T05:24:11.443775+0000","last_became_active":"2024-09-17T05:24:09.430822+0000","last_became_peered":"2024-09-17T05:24:09.430822+0000","last_unstale":"2024-09-17T05:24:11.443775+0000","last_undegraded":"2024-09-17T05:24:11.443775+0000","last_fullsized":"2024-09-17T05:24:11.443775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:52.917186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.444189+0000","last_change":"2024-09-17T05:24:10.455617+0000","last_active":"2024-09-17T05:24:11.444189+0000","last_peered":"2024-09-17T05:24:11.444189+0000","last_clean":"2024-09-17T05:24:11.444189+0000","last_became_active":"2024-09-17T05:24:10.455255+0000","last_became_peered":"2024-09-17T05:24:10.455255+0000","last_unstale":"2024-09-17T05:24:11.444189+0000","last_undegraded":"2024-09-17T05:24:11.444189+0000","last_fullsized":"2024-09-17T05:24:11.444189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:09.408154+0000","last_clean_scrub_stamp":"2024-09-17T05:24:09.408154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:15.159346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.445157+0000","last_change":"2024-09-17T05:24:09.432886+0000","last_active":"2024-09-17T05:24:11.445157+0000","last_peered":"2024-09-17T05:24:11.445157+0000","last_clean":"2024-09-17T05:24:11.445157+0000","last_became_active":"2024-09-17T05:24:09.432549+0000","last_became_peered":"2024-09-17T05:24:09.432549+0000","last_unstale":"2024-09-17T05:24:11.445157+0000","last_undegraded":"2024-09-17T05:24:11.445157+0000","last_fullsized":"2024-09-17T05:24:11.445157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:08.401409+0000","last_clean_scrub_stamp":"2024-09-17T05:24:08.401409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:57.682790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":130,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:24:11.569154+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:24:11.569154+0000","last_peered":"2024-09-17T05:24:11.569154+0000","last_clean":"2024-09-17T05:24:11.569154+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:24:11.569154+0000","last_undegraded":"2024-09-17T05:24:11.569154+0000","last_fullsized":"2024-09-17T05:24:11.569154+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":139506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":962560,"data_stored":730434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":141118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":380928,"data_stored":140260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":139518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":729922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":139574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:24:15.075 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:24:15.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- 172.21.15.43:0/1644465228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b5610 msgr2=0x7fb31c0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 --2- 172.21.15.43:0/1644465228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b5610 0x7fb31c0b5a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb304009840 tx=0x7fb30401c700 comp rx=0 tx=0).stop 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- 172.21.15.43:0/1644465228 shutdown_connections 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 --2- 172.21.15.43:0/1644465228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b5610 0x7fb31c0b5a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 --2- 172.21.15.43:0/1644465228 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31c15f470 0x7fb31c0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- 172.21.15.43:0/1644465228 >> 172.21.15.43:0/1644465228 conn(0x7fb31c0b2810 msgr2=0x7fb31c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- 172.21.15.43:0/1644465228 shutdown_connections 2024-09-17T05:24:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- 172.21.15.43:0/1644465228 wait complete. 2024-09-17T05:24:15.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 Processor -- start 2024-09-17T05:24:15.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.278+0000 7fb322e00640 1 -- start start 2024-09-17T05:24:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 0x7fb31c0afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31c0b5610 0x7fb31c1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c169520 con 0x7fb31c15f470 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1693a0 con 0x7fb31c0b3b10 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c1696a0 con 0x7fb31c0b5610 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56270/0 (socket says 172.21.15.43:56270) 2024-09-17T05:24:15.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 -- 172.21.15.43:0/4000749723 learned_addr learned my addr 172.21.15.43:0/4000749723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb31be00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 0x7fb31c0afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 -- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31c0b5610 msgr2=0x7fb31c1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31c0b5610 0x7fb31c1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 -- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 msgr2=0x7fb31c0afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 0x7fb31c0afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 -- 172.21.15.43:0/4000749723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c151c10 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb31be00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 0x7fb31c0afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb320e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c760 tx=0x7fb31400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314003db0 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb31400ce10 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140052d0 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1570a0 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1594f0 con 0x7fb31c15f470 2024-09-17T05:24:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.282+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c156e10 con 0x7fb31c15f470 2024-09-17T05:24:15.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.286+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb314005470 con 0x7fb31c15f470 2024-09-17T05:24:15.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.286+0000 7fb318c00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 0x7fb2f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.286+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb314093ff0 con 0x7fb31c15f470 2024-09-17T05:24:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.286+0000 7fb31be00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 0x7fb2f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.286+0000 7fb31be00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 0x7fb2f8075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb31c069610 tx=0x7fb30c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.290+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb31c0b5f60 con 0x7fb31c15f470 2024-09-17T05:24:15.335 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:24:15.334+0000 7fb451c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:24:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.526+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb31c0b5a20 con 0x7fb31c15f470 2024-09-17T05:24:15.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.530+0000 7fb318c00640 1 -- 172.21.15.43:0/4000749723 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fb31405e9d0 con 0x7fb31c15f470 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:24:15.533 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 msgr2=0x7fb2f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 0x7fb2f8075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb31c069610 tx=0x7fb30c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 msgr2=0x7fb31c15e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c760 tx=0x7fb31400cc20 comp rx=0 tx=0).stop 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 shutdown_connections 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2f8073200 0x7fb2f8075710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c15f470 0x7fb31c15e070 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb31c0b5610 0x7fb31c1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 --2- 172.21.15.43:0/4000749723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb31c0b3b10 0x7fb31c0afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 >> 172.21.15.43:0/4000749723 conn(0x7fb31c0b2810 msgr2=0x7fb31c0738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:15.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 shutdown_connections 2024-09-17T05:24:15.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.534+0000 7fb322e00640 1 -- 172.21.15.43:0/4000749723 wait complete. 2024-09-17T05:24:15.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 -- 172.21.15.43:0/2370687941 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 msgr2=0x7f13e00cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 --2- 172.21.15.43:0/2370687941 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e00cd380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009bc0 tx=0x7f13d801cac0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 -- 172.21.15.43:0/2370687941 shutdown_connections 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 --2- 172.21.15.43:0/2370687941 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e00cd380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 --2- 172.21.15.43:0/2370687941 >> v2:172.21.15.62:3300/0 conn(0x7f13e00b1ca0 0x7f13e00b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 -- 172.21.15.43:0/2370687941 >> 172.21.15.43:0/2370687941 conn(0x7f13e00cee70 msgr2=0x7f13e00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 -- 172.21.15.43:0/2370687941 shutdown_connections 2024-09-17T05:24:15.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.714+0000 7f13e6000640 1 -- 172.21.15.43:0/2370687941 wait complete. 2024-09-17T05:24:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 Processor -- start 2024-09-17T05:24:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 -- start start 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 0x7f13e0166060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e01665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00b25f0 con 0x7f13e00d0950 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00cdee0 con 0x7f13e00b1ca0 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13e6000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00b2770 con 0x7f13e00ccf70 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56272/0 (socket says 172.21.15.43:56272) 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13dfe00640 1 -- 172.21.15.43:0/2206279911 learned_addr learned my addr 172.21.15.43:0/2206279911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13d6a00640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e01665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.718+0000 7f13df400640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 0x7f13e0166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 -- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 msgr2=0x7f13e01665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e01665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 -- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 msgr2=0x7f13e0166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 0x7f13e0166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 -- 172.21.15.43:0/2206279911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00c8f30 con 0x7f13e00d0950 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13df400640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 0x7f13e0166060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13dfe00640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13d801b170 tx=0x7f13d8009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80bb070 con 0x7f13e00d0950 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d8004a50 con 0x7f13e00d0950 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00c6880 con 0x7f13e00d0950 2024-09-17T05:24:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80b2040 con 0x7f13e00d0950 2024-09-17T05:24:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01fa120 con 0x7f13e00d0950 2024-09-17T05:24:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e01fa380 con 0x7f13e00d0950 2024-09-17T05:24:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.722+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13d8004040 con 0x7f13e00d0950 2024-09-17T05:24:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.726+0000 7f13d7e00640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 0x7f13bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.726+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f13d812ddf0 con 0x7f13e00d0950 2024-09-17T05:24:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.726+0000 7f13df400640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 0x7f13bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.726+0000 7f13df400640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 0x7f13bc075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13e0074810 tx=0x7f13d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.730+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13d80f1db0 con 0x7f13e00d0950 2024-09-17T05:24:15.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.938+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e00cfa90 con 0x7f13e00d0950 2024-09-17T05:24:15.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.938+0000 7f13d7e00640 1 -- 172.21.15.43:0/2206279911 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f13d80166b0 con 0x7f13e00d0950 2024-09-17T05:24:15.942 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:15.942 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"btime":"2024-09-17T05:24:13:457749+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:12.876397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4860":{"gid":4860,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1404655827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1404655827},{"type":"v1","addr":"172.21.15.43:6835","nonce":1404655827}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4860,"qdb_cluster":[4860]},"id":2}]} 2024-09-17T05:24:15.943 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2024-09-17T05:24:15.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 msgr2=0x7f13bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 0x7f13bc075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13e0074810 tx=0x7f13d0009040 comp rx=0 tx=0).stop 2024-09-17T05:24:15.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 msgr2=0x7f13e015fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:15.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13d801b170 tx=0x7f13d8009320 comp rx=0 tx=0).stop 2024-09-17T05:24:15.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 shutdown_connections 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073200 0x7f13bc075710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00d0950 0x7f13e015fc40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00ccf70 0x7f13e01665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 --2- 172.21.15.43:0/2206279911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e00b1ca0 0x7f13e0166060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 >> 172.21.15.43:0/2206279911 conn(0x7f13e00cee70 msgr2=0x7f13e00b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 shutdown_connections 2024-09-17T05:24:15.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:15.942+0000 7f13e6000640 1 -- 172.21.15.43:0/2206279911 wait complete. 2024-09-17T05:24:16.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:24:09.841714+0000', 'modified': '2024-09-17T05:24:12.876397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4860}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4860': {'gid': 4860, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/1404655827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 1404655827}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1404655827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4860, 'qdb_cluster': [4860]} 2024-09-17T05:24:16.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:24:16.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- 172.21.15.43:0/1125347622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 msgr2=0x7f51a40cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 --2- 172.21.15.43:0/1125347622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a40cce70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f518c009840 tx=0x7f518c01c820 comp rx=0 tx=0).stop 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- 172.21.15.43:0/1125347622 shutdown_connections 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 --2- 172.21.15.43:0/1125347622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a40cce70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 --2- 172.21.15.43:0/1125347622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 0x7f51a40d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- 172.21.15.43:0/1125347622 >> 172.21.15.43:0/1125347622 conn(0x7f51a40cee80 msgr2=0x7f51a40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- 172.21.15.43:0/1125347622 shutdown_connections 2024-09-17T05:24:16.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- 172.21.15.43:0/1125347622 wait complete. 2024-09-17T05:24:16.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 Processor -- start 2024-09-17T05:24:16.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.530+0000 7f51a8a00640 1 -- start start 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 0x7f51a4151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a40cd3b0 0x7f51a4142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4169e10 con 0x7f51a40d0d80 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4169c90 con 0x7f51a40cd3b0 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f51a4169f90 con 0x7f51a40b7ce0 2024-09-17T05:24:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56300/0 (socket says 172.21.15.43:56300) 2024-09-17T05:24:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 -- 172.21.15.43:0/4228594014 learned_addr learned my addr 172.21.15.43:0/4228594014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f519be00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a40cd3b0 0x7f51a4142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 0x7f51a4151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 -- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 msgr2=0x7f51a4151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 0x7f51a4151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 -- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a40cd3b0 msgr2=0x7f51a4142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a40cd3b0 0x7f51a4142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 -- 172.21.15.43:0/4228594014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a415b160 con 0x7f51a40d0d80 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a2800640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c760 tx=0x7f519c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f519a000640 1 -- 172.21.15.43:0/4228594014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c003db0 con 0x7f51a40d0d80 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f519a000640 1 -- 172.21.15.43:0/4228594014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519c00ce10 con 0x7f51a40d0d80 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f519a000640 1 -- 172.21.15.43:0/4228594014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0052d0 con 0x7f51a40d0d80 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a415b440 con 0x7f51a40d0d80 2024-09-17T05:24:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.534+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a415ba00 con 0x7f51a40d0d80 2024-09-17T05:24:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f519a000640 1 -- 172.21.15.43:0/4228594014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f519c005470 con 0x7f51a40d0d80 2024-09-17T05:24:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f519a000640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 0x7f5180075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f519a000640 1 -- 172.21.15.43:0/4228594014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f519c093fe0 con 0x7f51a40d0d80 2024-09-17T05:24:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 0x7f5180075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f51b0211000 1 Processor -- start 2024-09-17T05:24:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f51b0211000 1 -- start start 2024-09-17T05:24:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.538+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 0x7f5180075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51a4071220 tx=0x7f5194005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b1fcf9410 con 0x555b1fcd3950 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555b1fc36b40 con 0x555b1fcbcca0 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555b1fc25f70 con 0x555b1f8e3460 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56308/0 (socket says 172.21.15.43:56308) 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 -- 172.21.15.43:0/1368784585 learned_addr learned my addr 172.21.15.43:0/1368784585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51a2800640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51a1e00640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 -- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 msgr2=0x555b1fe246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 -- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 msgr2=0x555b1fe25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 -- 172.21.15.43:0/1368784585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1fbed5b0 con 0x555b1fcd3950 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51a2800640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f519be00640 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f518c009790 tx=0x7f518c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c43cc1c9d1301c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f5192600640 1 -- 172.21.15.43:0/1368784585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0bc070 con 0x555b1fcd3950 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f5192600640 1 -- 172.21.15.43:0/1368784585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f518c004330 con 0x555b1fcd3950 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f5192600640 1 -- 172.21.15.43:0/1368784585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0b2040 con 0x555b1fcd3950 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- 172.21.15.43:0/1368784585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 msgr2=0x555b1fe24be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f518c009790 tx=0x7f518c0195c0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- 172.21.15.43:0/1368784585 shutdown_connections 2024-09-17T05:24:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcd3950 0x555b1fe24be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 --2- 172.21.15.43:0/1368784585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- 172.21.15.43:0/1368784585 >> 172.21.15.43:0/1368784585 conn(0x555b1fda9d90 msgr2=0x555b1fdd9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- 172.21.15.43:0/1368784585 shutdown_connections 2024-09-17T05:24:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- 172.21.15.43:0/1368784585 wait complete. 2024-09-17T05:24:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 -- start start 2024-09-17T05:24:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.542+0000 7f51b0211000 1 Processor -- start 2024-09-17T05:24:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe2b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcd3950 0x555b1fe2c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b1fbc05f0 con 0x555b1fcbcca0 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555b1fc29d80 con 0x555b1fcd3950 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555b1fbcc520 con 0x555b1f8e3460 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 10 client.0 _ll_register_callbacks cb 0x555b1fbd0830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56314/0 (socket says 172.21.15.43:56314) 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 -- 172.21.15.43:0/3835503163 learned_addr learned my addr 172.21.15.43:0/3835503163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe2b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51a2800640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcd3950 0x555b1fe2c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 -- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 msgr2=0x555b1fe2b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe2b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 -- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcd3950 msgr2=0x555b1fe2c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcd3950 0x555b1fe2c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 -- 172.21.15.43:0/3835503163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1fbc4190 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f519be00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f518c01aed0 tx=0x7f518c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d04c09bb229416 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0b2070 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b1fbbe110 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f518c0a5a00 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f518c0d0050 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 10 client.5040 fetch_fsmap learned FSMap version 17 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555b1fcaaf10 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555b1fbbe110 con 0x555b1fcbcca0 2024-09-17T05:24:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f518c0d3020 con 0x555b1fcbcca0 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 10 client.5040 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 10 client.5040 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 20 client.5040 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8c87d0 0x555b1fe35d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 4 client.5040 mds_command: new command op to 4860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 --> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1fc38a90 con 0x555b1f8c87d0 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.546+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8c87d0 0x555b1fe35d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f5186a00640 10 client.5040 ms_handle_connect on v2:172.21.15.43:6834/1404655827 2024-09-17T05:24:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8c87d0 0x555b1fe35d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a804996f6bdb2244 server_cookie=ede35e6f9d1020e1 in_seq=0 out_seq=0 2024-09-17T05:24:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f5186a00640 1 -- 172.21.15.43:0/3835503163 <== mds.0 v2:172.21.15.43:6834/1404655827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b1fc38a90 con 0x555b1f8c87d0 2024-09-17T05:24:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f5186a00640 10 client.5040 handle_command_reply: tid=0 2024-09-17T05:24:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 client.5040 shutdown 2024-09-17T05:24:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 msgr2=0x555b1fe2be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f518c01aed0 tx=0x7f518c0043b0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 shutdown_connections 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8c87d0 0x555b1fe35d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcd3950 0x555b1fe2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1fcbcca0 0x555b1fe2be60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 --2- 172.21.15.43:0/3835503163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1f8e3460 0x555b1fe2b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f5186a00640 0 client.5040 ms_handle_reset on v2:172.21.15.43:6834/1404655827 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 >> 172.21.15.43:0/3835503163 conn(0x555b1fda9d90 msgr2=0x555b1fde0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 shutdown_connections 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 1 -- 172.21.15.43:0/3835503163 wait complete. 2024-09-17T05:24:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.550+0000 7f51b0211000 20 client.5040 trim_cache size 0 max 16384 2024-09-17T05:24:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 Processor -- start 2024-09-17T05:24:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 -- start start 2024-09-17T05:24:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fdda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fddae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51a42030d0 con 0x555b1f8e3460 2024-09-17T05:24:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4202150 con 0x555b1fcbcca0 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f51a4159010 con 0x555b1fcd3950 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56328/0 (socket says 172.21.15.43:56328) 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 -- 172.21.15.43:0/4133056078 learned_addr learned my addr 172.21.15.43:0/4133056078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a2800640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fddae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f519be00640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fdda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 -- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 msgr2=0x555b1fddae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fddae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 -- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 msgr2=0x555b1fdda930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fdda930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 -- 172.21.15.43:0/4133056078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1fc81db0 con 0x555b1f8e3460 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a2800640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fddae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f51a1e00640 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x555b1fc37720 tx=0x7f5194014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc33e32634421de server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f5191c00640 1 -- 172.21.15.43:0/4133056078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519401dc00 con 0x555b1f8e3460 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f5191c00640 1 -- 172.21.15.43:0/4133056078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519401dda0 con 0x555b1f8e3460 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.570+0000 7f5191c00640 1 -- 172.21.15.43:0/4133056078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194004260 con 0x555b1f8e3460 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- 172.21.15.43:0/4133056078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 msgr2=0x555b1fdda3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x555b1fc37720 tx=0x7f5194014000 comp rx=0 tx=0).stop 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- 172.21.15.43:0/4133056078 shutdown_connections 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fdda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fddae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- 172.21.15.43:0/4133056078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fdda3f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- 172.21.15.43:0/4133056078 >> 172.21.15.43:0/4133056078 conn(0x555b1fda9d90 msgr2=0x555b1fdd8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- 172.21.15.43:0/4133056078 shutdown_connections 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- 172.21.15.43:0/4133056078 wait complete. 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- start start 2024-09-17T05:24:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 Processor -- start 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe9a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fe9aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b1fc29090 con 0x555b1f8e3460 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555b1fbf1ee0 con 0x555b1fcbcca0 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555b1fc35910 con 0x555b1fcd3950 2024-09-17T05:24:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51b0211000 10 client.0 _ll_register_callbacks cb 0x7f519c003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56344/0 (socket says 172.21.15.43:56344) 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 -- 172.21.15.43:0/3853188614 learned_addr learned my addr 172.21.15.43:0/3853188614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f519be00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe9a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 -- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 msgr2=0x555b1fe9aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a2800640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fe9aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fe9aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 -- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 msgr2=0x555b1fe9a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe9a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f51a1e00640 1 -- 172.21.15.43:0/3853188614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1fbecab0 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.574+0000 7f519be00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe9a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a1e00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5194032040 tx=0x7f519401da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5781f66ed295f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519403e020 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5194004220 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519401dda0 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5194045050 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 10 client.5046 fetch_fsmap learned FSMap version 17 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555b1fbcb070 con 0x555b1f8e3460 2024-09-17T05:24:16.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519401dda0 con 0x555b1f8e3460 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5194048020 con 0x555b1f8e3460 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 10 client.5046 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 10 client.5046 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 20 client.5046 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8b6830 0x555b1fe55770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 4 client.5046 mds_command: new command op to 4860 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 --> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] -- command(tid 0: {"prefix": "status"}) -- 0x555b1fc34540 con 0x555b1f8b6830 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f519be00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8b6830 0x555b1fe55770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 10 client.5046 ms_handle_connect on v2:172.21.15.43:6834/1404655827 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f519be00640 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8b6830 0x555b1fe55770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371a41d55d480409 server_cookie=8294904fc4ccb6f7 in_seq=0 out_seq=0 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 1 -- 172.21.15.43:0/3853188614 <== mds.0 v2:172.21.15.43:6834/1404655827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x555b1fc34540 con 0x555b1f8b6830 2024-09-17T05:24:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 10 client.5046 handle_command_reply: tid=0 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 client.5046 shutdown 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 msgr2=0x555b1fe9a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5194032040 tx=0x7f519401da80 comp rx=0 tx=0).stop 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 shutdown_connections 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:6834/1404655827,v1:172.21.15.43:6835/1404655827] conn(0x555b1f8b6830 0x555b1fe55770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b1fcbcca0 0x555b1fe9a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b1fcd3950 0x555b1fe9aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 --2- 172.21.15.43:0/3853188614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b1f8e3460 0x555b1fe9a030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f5186a00640 0 client.5046 ms_handle_reset on v2:172.21.15.43:6834/1404655827 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 >> 172.21.15.43:0/3853188614 conn(0x555b1fda9d90 msgr2=0x555b1fe356a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 shutdown_connections 2024-09-17T05:24:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 1 -- 172.21.15.43:0/3853188614 wait complete. 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51b0211000 20 client.5046 trim_cache size 0 max 16384 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4860, 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:24:16.588 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.1409631740000004, 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 17, 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 33, 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 33, 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 13.061654667999999 2024-09-17T05:24:16.589 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 msgr2=0x7f5180075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 0x7f5180075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51a4071220 tx=0x7f5194005f40 comp rx=0 tx=0).stop 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 msgr2=0x7f51a4157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c760 tx=0x7f519c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 shutdown_connections 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5180073230 0x7f5180075740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a40cd3b0 0x7f51a4142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51a40d0d80 0x7f51a4157720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 --2- 172.21.15.43:0/4228594014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51a40b7ce0 0x7f51a4151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 >> 172.21.15.43:0/4228594014 conn(0x7f51a40cee80 msgr2=0x7f51a40af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 shutdown_connections 2024-09-17T05:24:16.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.578+0000 7f51a8a00640 1 -- 172.21.15.43:0/4228594014 wait complete. 2024-09-17T05:24:16.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:24:16.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 -- 172.21.15.43:0/1379857515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 msgr2=0x7f2b680cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 --2- 172.21.15.43:0/1379857515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680cbc70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009c80 tx=0x7f2b6001cd10 comp rx=0 tx=0).stop 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 -- 172.21.15.43:0/1379857515 shutdown_connections 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 --2- 172.21.15.43:0/1379857515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680cbc70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 --2- 172.21.15.43:0/1379857515 >> v2:172.21.15.62:3300/0 conn(0x7f2b68070620 0x7f2b680b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 -- 172.21.15.43:0/1379857515 >> 172.21.15.43:0/1379857515 conn(0x7f2b680b23d0 msgr2=0x7f2b680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 -- 172.21.15.43:0/1379857515 shutdown_connections 2024-09-17T05:24:16.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.758+0000 7f2b6f800640 1 -- 172.21.15.43:0/1379857515 wait complete. 2024-09-17T05:24:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 Processor -- start 2024-09-17T05:24:16.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 -- start start 2024-09-17T05:24:16.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b68070620 0x7f2b680ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56354/0 (socket says 172.21.15.43:56354) 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3636983151 learned_addr learned my addr 172.21.15.43:0/3636983151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680ce5a0 con 0x7f2b6806ce60 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680ce420 con 0x7f2b68070620 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b680ce720 con 0x7f2b680cb860 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b67e00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b68070620 0x7f2b680ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6d800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 msgr2=0x7f2b680ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b68070620 msgr2=0x7f2b680ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b68070620 0x7f2b680ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.762+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680c9fe0 con 0x7f2b6806ce60 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea90 tx=0x7f2b5c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00cda0 con 0x7f2b6806ce60 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b5c004110 con 0x7f2b6806ce60 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b680c4eb0 con 0x7f2b6806ce60 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c003600 con 0x7f2b6806ce60 2024-09-17T05:24:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b680c5470 con 0x7f2b6806ce60 2024-09-17T05:24:16.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b680cecb0 con 0x7f2b6806ce60 2024-09-17T05:24:16.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b5c01d020 con 0x7f2b6806ce60 2024-09-17T05:24:16.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 0x7f2b48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:16.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.766+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2b5c094390 con 0x7f2b6806ce60 2024-09-17T05:24:16.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.770+0000 7f2b67e00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 0x7f2b48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:16.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.770+0000 7f2b67e00640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 0x7f2b48075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815a3d0 tx=0x7f2b54009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:16.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.774+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b5c05f3a0 con 0x7f2b6806ce60 2024-09-17T05:24:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.986+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b680b0530 con 0x7f2b6806ce60 2024-09-17T05:24:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.986+0000 7f2b64c00640 1 -- 172.21.15.43:0/3636983151 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f2b5c05eca0 con 0x7f2b6806ce60 2024-09-17T05:24:16.990 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 msgr2=0x7f2b48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 0x7f2b48075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815a3d0 tx=0x7f2b54009040 comp rx=0 tx=0).stop 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 msgr2=0x7f2b6809da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00ea90 tx=0x7f2b5c00ef50 comp rx=0 tx=0).stop 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 shutdown_connections 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073200 0x7f2b48075710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b68070620 0x7f2b680ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680cb860 0x7f2b680ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 --2- 172.21.15.43:0/3636983151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6806ce60 0x7f2b6809da20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 >> 172.21.15.43:0/3636983151 conn(0x7f2b680b23d0 msgr2=0x7f2b680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 shutdown_connections 2024-09-17T05:24:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:16.990+0000 7f2b6f800640 1 -- 172.21.15.43:0/3636983151 wait complete. 2024-09-17T05:24:17.012 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:24:17.012 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:24:17.012 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:24:17.012 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:24:17.012 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:17.013 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:24:17.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:24:17.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:24:17.013 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:17.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.013 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.023 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42387sec preferred_lft 42387sec 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:24:17.024 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:17.025 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:17.026 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:24:17.027 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:24:17.027 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:17.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.027 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:24:17.083 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:24:17.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.086 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:24:17.143 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:24:17.143 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:24:17.143 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:24:17.143 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:24:17.144 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:24:17.144 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:24:17.144 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:24:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:24:17.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:17.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.282 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:24:17.335 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:24:17.335 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:24:17.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:17.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:24:17.402 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:17.402 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:17.458 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:17.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:17.458 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:17.509 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:24:17.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:17.618 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:17.619 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:17.619 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:17.637 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:17.637 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:17.637 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:17.637 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:24:17.637 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:24:17.648 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:24:17.652 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:24:17.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:17.673 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:24:17.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:24:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:17.777 INFO:teuthology.orchestra.run.smithi043.stdout:client5052 2024-09-17T05:24:17.778 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:17.778 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:17.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.787 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-17T05:24:17.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.852 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-17T05:24:17.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:17.905 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-17T05:24:17.960 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:24:18.067 INFO:teuthology.orchestra.run.smithi043.stdout:1099511627776 2024-09-17T05:24:18.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:18.077 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:18.077 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:18.126 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:18.127 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:18.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 -- 172.21.15.43:0/461923469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c0cccb0 msgr2=0x7fc28c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 --2- 172.21.15.43:0/461923469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c0cccb0 0x7fc28c0b43f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc278009840 tx=0x7fc27801c790 comp rx=0 tx=0).stop 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 -- 172.21.15.43:0/461923469 shutdown_connections 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 --2- 172.21.15.43:0/461923469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c0cccb0 0x7fc28c0b43f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 --2- 172.21.15.43:0/461923469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c15cea0 0x7fc28c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 -- 172.21.15.43:0/461923469 >> 172.21.15.43:0/461923469 conn(0x7fc28c0691b0 msgr2=0x7fc28c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 -- 172.21.15.43:0/461923469 shutdown_connections 2024-09-17T05:24:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.386+0000 7fc291000640 1 -- 172.21.15.43:0/461923469 wait complete. 2024-09-17T05:24:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 Processor -- start 2024-09-17T05:24:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 -- start start 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c0cd0c0 0x7fc28c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 0x7fc28c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c16a390 con 0x7fc28c0cccb0 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c16a210 con 0x7fc28c15cea0 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc291000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c16a510 con 0x7fc28c0cd0c0 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56372/0 (socket says 172.21.15.43:56372) 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc28b400640 1 -- 172.21.15.43:0/1718681114 learned_addr learned my addr 172.21.15.43:0/1718681114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc28aa00640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c0cd0c0 0x7fc28c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.390+0000 7fc28be00640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 0x7fc28c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 -- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c0cd0c0 msgr2=0x7fc28c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c0cd0c0 0x7fc28c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 -- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 msgr2=0x7fc28c0b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 0x7fc28c0b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 -- 172.21.15.43:0/1718681114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c156910 con 0x7fc28c0cccb0 2024-09-17T05:24:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28be00640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 0x7fc28c0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc28b400640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e7f0 tx=0x7fc27c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00ccc0 con 0x7fc28c0cccb0 2024-09-17T05:24:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc27c00ce80 con 0x7fc28c0cccb0 2024-09-17T05:24:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c005280 con 0x7fc28c0cccb0 2024-09-17T05:24:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c0b5d20 con 0x7fc28c0cccb0 2024-09-17T05:24:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c0b79b0 con 0x7fc28c0cccb0 2024-09-17T05:24:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c0b43f0 con 0x7fc28c0cccb0 2024-09-17T05:24:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.394+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc27c005420 con 0x7fc28c0cccb0 2024-09-17T05:24:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.398+0000 7fc283400640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 0x7fc26c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.398+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc27c094c40 con 0x7fc28c0cccb0 2024-09-17T05:24:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.398+0000 7fc28aa00640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 0x7fc26c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.398+0000 7fc28aa00640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 0x7fc26c075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2780af000 tx=0x7fc27801afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.402+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc27c05fc50 con 0x7fc28c0cccb0 2024-09-17T05:24:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.590+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc28c06a1e0 con 0x7fc28c0cccb0 2024-09-17T05:24:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.590+0000 7fc283400640 1 -- 172.21.15.43:0/1718681114 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fc27c05f550 con 0x7fc28c0cccb0 2024-09-17T05:24:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 msgr2=0x7fc26c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 0x7fc26c075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc2780af000 tx=0x7fc27801afd0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 msgr2=0x7fc28c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e7f0 tx=0x7fc27c00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 shutdown_connections 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc26c073200 0x7fc26c075710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28c15cea0 0x7fc28c0b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc28c0cd0c0 0x7fc28c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 --2- 172.21.15.43:0/1718681114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc28c0cccb0 0x7fc28c1563d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 >> 172.21.15.43:0/1718681114 conn(0x7fc28c0691b0 msgr2=0x7fc28c15b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 shutdown_connections 2024-09-17T05:24:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.594+0000 7fc291000640 1 -- 172.21.15.43:0/1718681114 wait complete. 2024-09-17T05:24:18.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:24:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:18.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:18.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:24:18.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:18.676 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:24:18.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:18.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 -- 172.21.15.43:0/3395325768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0380155d70 msgr2=0x7f0364004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 --2- 172.21.15.43:0/3395325768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0380155d70 0x7f0364004810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0368006c00 tx=0x7f0368021940 comp rx=0 tx=0).stop 2024-09-17T05:24:18.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0385c00640 1 -- 172.21.15.43:0/3395325768 reap_dead start 2024-09-17T05:24:18.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 -- 172.21.15.43:0/3395325768 shutdown_connections 2024-09-17T05:24:18.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 -- 172.21.15.43:0/3395325768 >> 172.21.15.43:0/3395325768 conn(0x7f03800b23d0 msgr2=0x7f03800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:18.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 -- 172.21.15.43:0/3395325768 shutdown_connections 2024-09-17T05:24:18.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.898+0000 7f0387c00640 1 -- 172.21.15.43:0/3395325768 wait complete. 2024-09-17T05:24:18.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 Processor -- start 2024-09-17T05:24:18.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 -- start start 2024-09-17T05:24:18.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 0x7f03801f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03800ccf70 0x7f038015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03800ad5f0 con 0x7f03801f5f00 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f038015cb00 con 0x7f0380155d70 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0387c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f03800ad770 con 0x7f03800ccf70 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0385200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 0x7f03801f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0385200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 0x7f03801f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:33090/0 (socket says 172.21.15.43:33090) 2024-09-17T05:24:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56398/0 (socket says 172.21.15.43:56398) 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0385200640 1 -- 172.21.15.43:0/1160077608 learned_addr learned my addr 172.21.15.43:0/1160077608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0385c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03800ccf70 0x7f038015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 -- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03800ccf70 msgr2=0x7f038015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03800ccf70 0x7f038015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 -- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 msgr2=0x7f03801f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 0x7f03801f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.902+0000 7f0377e00640 1 -- 172.21.15.43:0/1160077608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03801f97c0 con 0x7f03801f5f00 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f0377e00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f03680af000 tx=0x7f036800f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680b0040 con 0x7f03801f5f00 2024-09-17T05:24:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0368006550 con 0x7f03801f5f00 2024-09-17T05:24:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368003860 con 0x7f03801f5f00 2024-09-17T05:24:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03801f2ca0 con 0x7f03801f5f00 2024-09-17T05:24:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03801f5420 con 0x7f03801f5f00 2024-09-17T05:24:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03800add00 con 0x7f03801f5f00 2024-09-17T05:24:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f03680b3070 con 0x7f03801f5f00 2024-09-17T05:24:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 0x7f035c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.906+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0368131d60 con 0x7f03801f5f00 2024-09-17T05:24:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.910+0000 7f0385200640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 0x7f035c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:18.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.910+0000 7f0385200640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 0x7f035c075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0370005bd0 tx=0x7f03700091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:18.914+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f03801f97c0 con 0x7f03801f5f00 2024-09-17T05:24:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.118+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f038006e070 con 0x7f03801f5f00 2024-09-17T05:24:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.118+0000 7f037d600640 1 -- 172.21.15.43:0/1160077608 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f03680ba070 con 0x7f03801f5f00 2024-09-17T05:24:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:19.121 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"btime":"2024-09-17T05:24:13:457749+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:12.876397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4860":{"gid":4860,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1404655827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1404655827},{"type":"v1","addr":"172.21.15.43:6835","nonce":1404655827}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4860,"qdb_cluster":[4860]},"id":2}]} 2024-09-17T05:24:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2024-09-17T05:24:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 msgr2=0x7f035c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 0x7f035c075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0370005bd0 tx=0x7f03700091e0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 msgr2=0x7f03801f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f03680af000 tx=0x7f036800f2f0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 shutdown_connections 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f035c073230 0x7f035c075740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f03800ccf70 0x7f038015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03801f5f00 0x7f03801f2760 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 --2- 172.21.15.43:0/1160077608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0380155d70 0x7f03801f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 >> 172.21.15.43:0/1160077608 conn(0x7f03800b23d0 msgr2=0x7f038015cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 shutdown_connections 2024-09-17T05:24:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.122+0000 7f0387c00640 1 -- 172.21.15.43:0/1160077608 wait complete. 2024-09-17T05:24:19.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T05:24:19.324 INFO:teuthology.orchestra.run.smithi043.stdout:{"message":"","return_code":0} 2024-09-17T05:24:19.325 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T05:24:19.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:24:19.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.482+0000 7fab52200640 1 -- 172.21.15.43:0/1694266362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab300099c0 msgr2=0x7fab3000bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.482+0000 7fab52200640 1 --2- 172.21.15.43:0/1694266362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab300099c0 0x7fab3000bde0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fab44001dc0 tx=0x7fab4401cb30 comp rx=0 tx=0).stop 2024-09-17T05:24:19.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.482+0000 7fab52200640 1 -- 172.21.15.43:0/1694266362 shutdown_connections 2024-09-17T05:24:19.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.482+0000 7fab52200640 1 --2- 172.21.15.43:0/1694266362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab300099c0 0x7fab3000bde0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.482+0000 7fab52200640 1 -- 172.21.15.43:0/1694266362 >> 172.21.15.43:0/1694266362 conn(0x7fab4c0b23d0 msgr2=0x7fab4c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- 172.21.15.43:0/1694266362 shutdown_connections 2024-09-17T05:24:19.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- 172.21.15.43:0/1694266362 wait complete. 2024-09-17T05:24:19.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 Processor -- start 2024-09-17T05:24:19.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- start start 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab300099c0 0x7fab4c158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab4c15c470 0x7fab4c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c0738a0 con 0x7fab4c0994c0 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c073720 con 0x7fab4c15c470 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.486+0000 7fab52200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fab4c073a20 con 0x7fab300099c0 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56422/0 (socket says 172.21.15.43:56422) 2024-09-17T05:24:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 -- 172.21.15.43:0/123634438 learned_addr learned my addr 172.21.15.43:0/123634438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab4b400640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab300099c0 0x7fab4c158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab4be00640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab4c15c470 0x7fab4c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 -- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab300099c0 msgr2=0x7fab4c158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab300099c0 0x7fab4c158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 -- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab4c15c470 msgr2=0x7fab4c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab4c15c470 0x7fab4c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 -- 172.21.15.43:0/123634438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c15cdc0 con 0x7fab4c0994c0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43e00640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab34011710 tx=0x7fab34011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3401f070 con 0x7fab4c0994c0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab3400fc30 con 0x7fab4c0994c0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c153580 con 0x7fab4c0994c0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34012450 con 0x7fab4c0994c0 2024-09-17T05:24:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c153b40 con 0x7fab4c0994c0 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4c0b4f60 con 0x7fab4c0994c0 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.490+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fab34022070 con 0x7fab4c0994c0 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.494+0000 7fab43400640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 0x7fab28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.494+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab34094c50 con 0x7fab4c0994c0 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.494+0000 7fab4b400640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 0x7fab28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.494+0000 7fab4b400640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 0x7fab28075740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab3c001870 tx=0x7fab3c006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.498+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fab4c15cdc0 con 0x7fab4c0994c0 2024-09-17T05:24:19.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.714+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fab4c15d0b0 con 0x7fab4c0994c0 2024-09-17T05:24:19.715 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:24:19.714+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:24:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.714+0000 7fab43400640 1 -- 172.21.15.43:0/123634438 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fab3405f580 con 0x7fab4c0994c0 2024-09-17T05:24:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:24:19.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 msgr2=0x7fab28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 0x7fab28075740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab3c001870 tx=0x7fab3c006040 comp rx=0 tx=0).stop 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 msgr2=0x7fab4c0998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab34011710 tx=0x7fab34011bd0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 shutdown_connections 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab28073230 0x7fab28075740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab4c15c470 0x7fab4c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4c0994c0 0x7fab4c0998d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 --2- 172.21.15.43:0/123634438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab300099c0 0x7fab4c158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 >> 172.21.15.43:0/123634438 conn(0x7fab4c0b23d0 msgr2=0x7fab4c0cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 shutdown_connections 2024-09-17T05:24:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.718+0000 7fab52200640 1 -- 172.21.15.43:0/123634438 wait complete. 2024-09-17T05:24:19.732 INFO:tasks.ceph.mds.c.smithi043.stderr: -5123> 2024-09-17T05:24:15.334+0000 7fb451c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:24:19.742 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:24:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 Processor -- start 2024-09-17T05:24:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 -- start start 2024-09-17T05:24:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee59d70 0x55b35ee57c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35efb7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b35eda7a00 con 0x55b35ee88cb0 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b35ee05f40 con 0x55b35ee59d70 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30becd5080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b35ef10890 con 0x55b35ef0fe40 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56424/0 (socket says 172.21.15.43:56424) 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 -- 172.21.15.43:0/3109750178 learned_addr learned my addr 172.21.15.43:0/3109750178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7e00640 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee59d70 0x55b35ee57c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b8800640 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35efb7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 -- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 msgr2=0x55b35efb7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35efb7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 -- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee59d70 msgr2=0x55b35ee57c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee59d70 0x55b35ee57c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.802+0000 7f30b7400640 1 -- 172.21.15.43:0/3109750178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35ed80c10 con 0x55b35ee88cb0 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30b7400640 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b1f0 tx=0x7f30a400b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c820927285b31a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30b6000640 1 -- 172.21.15.43:0/3109750178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a400a040 con 0x55b35ee88cb0 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30b6000640 1 -- 172.21.15.43:0/3109750178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a400f620 con 0x55b35ee88cb0 2024-09-17T05:24:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30b6000640 1 -- 172.21.15.43:0/3109750178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a400fa70 con 0x55b35ee88cb0 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- 172.21.15.43:0/3109750178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 msgr2=0x55b35efb6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b1f0 tx=0x7f30a400b6b0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- 172.21.15.43:0/3109750178 shutdown_connections 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35efb7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee88cb0 0x55b35efb6c30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- 172.21.15.43:0/3109750178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee59d70 0x55b35ee57c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- 172.21.15.43:0/3109750178 >> 172.21.15.43:0/3109750178 conn(0x55b35ee45d80 msgr2=0x55b35ee883c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- 172.21.15.43:0/3109750178 shutdown_connections 2024-09-17T05:24:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- 172.21.15.43:0/3109750178 wait complete. 2024-09-17T05:24:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 Processor -- start 2024-09-17T05:24:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 -- start start 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 0x55b35ef0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.806+0000 7f30becd5080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee88cb0 0x55b35ef18360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35ef1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b35ef2cc00 con 0x55b35ee59d70 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b35ef2f900 con 0x55b35ee88cb0 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b35ef2cea0 con 0x55b35ef0fe40 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 0x55b35ef0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 0x55b35ef0ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56428/0 (socket says 172.21.15.43:56428) 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 -- 172.21.15.43:0/367929758 learned_addr learned my addr 172.21.15.43:0/367929758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b8800640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35ef1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7400640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee88cb0 0x55b35ef18360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 -- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 msgr2=0x55b35ef1a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b35ef0fe40 0x55b35ef1a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 -- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee88cb0 msgr2=0x55b35ef18360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee88cb0 0x55b35ef18360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 -- 172.21.15.43:0/367929758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35ef1b230 con 0x55b35ee59d70 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7400640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b35ee88cb0 0x55b35ef18360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b7e00640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 0x55b35ef0ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f30a800a550 tx=0x7f30a80a7840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b4c00640 1 -- 172.21.15.43:0/367929758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b0880 con 0x55b35ee59d70 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b35ee9ef50 con 0x55b35ee59d70 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b4c00640 1 -- 172.21.15.43:0/367929758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a80b0a20 con 0x55b35ee59d70 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b4c00640 1 -- 172.21.15.43:0/367929758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b0e10 con 0x55b35ee59d70 2024-09-17T05:24:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35ee9f490 con 0x55b35ee59d70 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.810+0000 7f30b4c00640 1 -- 172.21.15.43:0/367929758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30a80bb3a0 con 0x55b35ee59d70 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b4c00640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3098071be0 0x7f30980740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b4c00640 1 -- 172.21.15.43:0/367929758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f30a8130a40 con 0x55b35ee59d70 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b7400640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3098071be0 0x7f30980740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30becd5080 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b35ee9e800 0x55b35efb7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34) -- 0x55b35efb7a70 con 0x55b35ee9e800 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b8800640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b35ee9e800 0x55b35efb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b7400640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3098071be0 0x7f30980740f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30a4003040 tx=0x7f30a4011040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.814+0000 7f30b8800640 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b35ee9e800 0x55b35efb7530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30b8800640 1 -- 172.21.15.43:0/367929758 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v34'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b0002040 con 0x55b35ee9e800 2024-09-17T05:24:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b35ee9e800 msgr2=0x55b35efb7530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b35ee9e800 0x55b35efb7530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3098071be0 msgr2=0x7f30980740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3098071be0 0x7f30980740f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30a4003040 tx=0x7f30a4011040 comp rx=0 tx=0).stop 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 msgr2=0x55b35ef0ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 --2- 172.21.15.43:0/367929758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b35ee59d70 0x55b35ef0ef60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f30a800a550 tx=0x7f30a80a7840 comp rx=0 tx=0).stop 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30b8800640 1 -- 172.21.15.43:0/367929758 reap_dead start 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 shutdown_connections 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 >> 172.21.15.43:0/367929758 conn(0x55b35ee45d80 msgr2=0x55b35eea20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 shutdown_connections 2024-09-17T05:24:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.818+0000 7f30becd5080 1 -- 172.21.15.43:0/367929758 wait complete. 2024-09-17T05:24:19.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:24:19.857 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:24:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 -- 172.21.15.43:0/792144803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc058155d70 msgr2=0x7fc05815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 --2- 172.21.15.43:0/792144803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc058155d70 0x7fc05815c7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc048009840 tx=0x7fc04801c760 comp rx=0 tx=0).stop 2024-09-17T05:24:19.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 -- 172.21.15.43:0/792144803 shutdown_connections 2024-09-17T05:24:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 --2- 172.21.15.43:0/792144803 >> v2:172.21.15.43:3300/0 conn(0x7fc0580b3af0 0x7fc0580ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 --2- 172.21.15.43:0/792144803 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc058155d70 0x7fc05815c7e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 -- 172.21.15.43:0/792144803 >> 172.21.15.43:0/792144803 conn(0x7fc0580b23d0 msgr2=0x7fc0580d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 -- 172.21.15.43:0/792144803 shutdown_connections 2024-09-17T05:24:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.986+0000 7fc05e400640 1 -- 172.21.15.43:0/792144803 wait complete. 2024-09-17T05:24:19.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 Processor -- start 2024-09-17T05:24:19.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- start start 2024-09-17T05:24:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc05806f110 0x7fc05809d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0580cf150 0x7fc0580cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc058168be0 con 0x7fc058155d70 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc058168a60 con 0x7fc0580cf150 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc058168d60 con 0x7fc05806f110 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56454/0 (socket says 172.21.15.43:56454) 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc057400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc05806f110 0x7fc05809d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 -- 172.21.15.43:0/3306873356 learned_addr learned my addr 172.21.15.43:0/3306873356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc057e00640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0580cf150 0x7fc0580cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 -- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc05806f110 msgr2=0x7fc05809d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc05806f110 0x7fc05809d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 -- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0580cf150 msgr2=0x7fc0580cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0580cf150 0x7fc0580cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 -- 172.21.15.43:0/3306873356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05809da30 con 0x7fc058155d70 2024-09-17T05:24:19.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04fe00640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c430 tx=0x7fc04000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:19.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040011790 con 0x7fc058155d70 2024-09-17T05:24:19.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc040017070 con 0x7fc058155d70 2024-09-17T05:24:19.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0580cfaa0 con 0x7fc058155d70 2024-09-17T05:24:19.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040011cc0 con 0x7fc058155d70 2024-09-17T05:24:19.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.990+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0580ca040 con 0x7fc058155d70 2024-09-17T05:24:19.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.994+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0580ad0b0 con 0x7fc058155d70 2024-09-17T05:24:19.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.994+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc04001c460 con 0x7fc058155d70 2024-09-17T05:24:19.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.994+0000 7fc04f400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 0x7fc034075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:19.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.994+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc040093ff0 con 0x7fc058155d70 2024-09-17T05:24:19.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.994+0000 7fc057400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 0x7fc034075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:19.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.998+0000 7fc057400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 0x7fc034075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc0480af000 tx=0x7fc04801afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:20.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:19.998+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc0580cfaa0 con 0x7fc058155d70 2024-09-17T05:24:20.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.214+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc05815bdb0 con 0x7fc058155d70 2024-09-17T05:24:20.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:20.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.718+0000 7fc04f400640 1 -- 172.21.15.43:0/3306873356 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7fc04005e8d0 con 0x7fc058155d70 2024-09-17T05:24:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:24:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 msgr2=0x7fc034075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 0x7fc034075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc0480af000 tx=0x7fc04801afa0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 msgr2=0x7fc0580c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c430 tx=0x7fc04000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 shutdown_connections 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc034073200 0x7fc034075710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0580cf150 0x7fc0580cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc058155d70 0x7fc0580c7c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 --2- 172.21.15.43:0/3306873356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc05806f110 0x7fc05809d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 >> 172.21.15.43:0/3306873356 conn(0x7fc0580b23d0 msgr2=0x7fc0580b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 shutdown_connections 2024-09-17T05:24:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.722+0000 7fc05e400640 1 -- 172.21.15.43:0/3306873356 wait complete. 2024-09-17T05:24:20.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 -- 172.21.15.43:0/1482118136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40737c0 msgr2=0x7f76e40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 --2- 172.21.15.43:0/1482118136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40737c0 0x7f76e40700a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009bf0 tx=0x7f76dc01ca80 comp rx=0 tx=0).stop 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 -- 172.21.15.43:0/1482118136 shutdown_connections 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 --2- 172.21.15.43:0/1482118136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40737c0 0x7f76e40700a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 -- 172.21.15.43:0/1482118136 >> 172.21.15.43:0/1482118136 conn(0x7f76e40b23d0 msgr2=0x7f76e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 -- 172.21.15.43:0/1482118136 shutdown_connections 2024-09-17T05:24:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.950+0000 7f76e8e00640 1 -- 172.21.15.43:0/1482118136 wait complete. 2024-09-17T05:24:20.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 Processor -- start 2024-09-17T05:24:20.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 -- start start 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 0x7f76e4099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 0x7f76e4074830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76e416b640 con 0x7f76e40cb2e0 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76e416b4c0 con 0x7f76e415fe10 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e8e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76e416b7c0 con 0x7f76e406fc90 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56470/0 (socket says 172.21.15.43:56470) 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 -- 172.21.15.43:0/225287293 learned_addr learned my addr 172.21.15.43:0/225287293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76dbe00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 0x7f76e4074830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 -- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 msgr2=0x7f76e4099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e1e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 0x7f76e4099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 0x7f76e4099890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 -- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 msgr2=0x7f76e4074830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.954+0000 7f76e2800640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 0x7f76e4074830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e2800640 1 -- 172.21.15.43:0/225287293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e415a380 con 0x7f76e40cb2e0 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76dbe00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 0x7f76e4074830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e1e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 0x7f76e4099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e2800640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76dc098580 tx=0x7f76dc005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0b2040 con 0x7f76e40cb2e0 2024-09-17T05:24:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76dc0bb070 con 0x7f76e40cb2e0 2024-09-17T05:24:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e415a660 con 0x7f76e40cb2e0 2024-09-17T05:24:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc004530 con 0x7f76e40cb2e0 2024-09-17T05:24:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e415ac20 con 0x7f76e40cb2e0 2024-09-17T05:24:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e415ae60 con 0x7f76e40cb2e0 2024-09-17T05:24:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76dc0046d0 con 0x7f76e40cb2e0 2024-09-17T05:24:20.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 0x7f76c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:20.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.958+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f76dc12eca0 con 0x7f76e40cb2e0 2024-09-17T05:24:20.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.962+0000 7f76e1e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 0x7f76c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:20.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.962+0000 7f76e1e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 0x7f76c0075710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76d40046c0 tx=0x7f76d4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:20.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:20.966+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76dc016790 con 0x7f76e40cb2e0 2024-09-17T05:24:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.174+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76e415bca0 con 0x7f76e40cb2e0 2024-09-17T05:24:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.174+0000 7f76da000640 1 -- 172.21.15.43:0/225287293 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5214 (secure 0 0 0) 0x7f76dc0f9c60 con 0x7f76e40cb2e0 2024-09-17T05:24:21.178 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:21.178 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"btime":"2024-09-17T05:24:20:725216+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:20.725196+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-17T05:24:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 20 2024-09-17T05:24:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 msgr2=0x7f76c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 0x7f76c0075710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76d40046c0 tx=0x7f76d4009040 comp rx=0 tx=0).stop 2024-09-17T05:24:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 msgr2=0x7f76e4074d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76dc098580 tx=0x7f76dc005a60 comp rx=0 tx=0).stop 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 shutdown_connections 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76c0073200 0x7f76c0075710 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e415fe10 0x7f76e4074830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e40cb2e0 0x7f76e4074d70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 --2- 172.21.15.43:0/225287293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76e406fc90 0x7f76e4099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 >> 172.21.15.43:0/225287293 conn(0x7f76e40b23d0 msgr2=0x7f76e4158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 shutdown_connections 2024-09-17T05:24:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:21.178+0000 7f76e8e00640 1 -- 172.21.15.43:0/225287293 wait complete. 2024-09-17T05:24:21.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:24:09.841714+0000', 'modified': '2024-09-17T05:24:20.725196+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5076}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5076': {'gid': 5076, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/331818553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 331818553}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 331818553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:24:21.603 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5076:up:replay 2024-09-17T05:24:22.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 -- 172.21.15.43:0/2717770154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 msgr2=0x7fb8e00b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/2717770154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e00b4b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009870 tx=0x7fb8d001c700 comp rx=0 tx=0).stop 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 -- 172.21.15.43:0/2717770154 shutdown_connections 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/2717770154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e00b4b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 -- 172.21.15.43:0/2717770154 >> 172.21.15.43:0/2717770154 conn(0x7fb8e00b23d0 msgr2=0x7fb8e00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 -- 172.21.15.43:0/2717770154 shutdown_connections 2024-09-17T05:24:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.774+0000 7fb8e5c00640 1 -- 172.21.15.43:0/2717770154 wait complete. 2024-09-17T05:24:22.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 Processor -- start 2024-09-17T05:24:22.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- start start 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006fc90 0x7fb8e015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e016aff0 con 0x7fb8e015c390 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e016ae70 con 0x7fb8e015fe10 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e016b170 con 0x7fb8e006fc90 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56472/0 (socket says 172.21.15.43:56472) 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 -- 172.21.15.43:0/578798374 learned_addr learned my addr 172.21.15.43:0/578798374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8dea00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006fc90 0x7fb8e015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8d6000640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 -- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006fc90 msgr2=0x7fb8e015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006fc90 0x7fb8e015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 -- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 msgr2=0x7fb8e015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e015dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 -- 172.21.15.43:0/578798374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00977a0 con 0x7fb8e015c390 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8df400640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800c760 tx=0x7fb8d800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8003b80 con 0x7fb8e015c390 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d8003d20 con 0x7fb8e015c390 2024-09-17T05:24:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e0156fb0 con 0x7fb8e015c390 2024-09-17T05:24:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800aab0 con 0x7fb8e015c390 2024-09-17T05:24:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.778+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e0157570 con 0x7fb8e015c390 2024-09-17T05:24:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.782+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e00b4b80 con 0x7fb8e015c390 2024-09-17T05:24:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.782+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb8d800ac50 con 0x7fb8e015c390 2024-09-17T05:24:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.782+0000 7fb8d7400640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 0x7fb8bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.782+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fb8d8093f80 con 0x7fb8e015c390 2024-09-17T05:24:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.782+0000 7fb8dea00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 0x7fb8bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.786+0000 7fb8dea00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 0x7fb8bc075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00af000 tx=0x7fb8d001af40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.786+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb8d805e8e0 con 0x7fb8e015c390 2024-09-17T05:24:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.994+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8e00ca0d0 con 0x7fb8e015c390 2024-09-17T05:24:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8d7400640 1 -- 172.21.15.43:0/578798374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5214 (secure 0 0 0) 0x7fb8d8063310 con 0x7fb8e015c390 2024-09-17T05:24:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:22.999 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"btime":"2024-09-17T05:24:22:728728+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:21.730124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-17T05:24:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 msgr2=0x7fb8bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 0x7fb8bc075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00af000 tx=0x7fb8d001af40 comp rx=0 tx=0).stop 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 msgr2=0x7fb8e015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800c760 tx=0x7fb8d800cc20 comp rx=0 tx=0).stop 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 shutdown_connections 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8bc073230 0x7fb8bc075740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8e015fe10 0x7fb8e015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8e015c390 0x7fb8e015c7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 --2- 172.21.15.43:0/578798374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8e006fc90 0x7fb8e015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 >> 172.21.15.43:0/578798374 conn(0x7fb8e00b23d0 msgr2=0x7fb8e00adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 shutdown_connections 2024-09-17T05:24:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:22.998+0000 7fb8e5c00640 1 -- 172.21.15.43:0/578798374 wait complete. 2024-09-17T05:24:23.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:24:09.841714+0000', 'modified': '2024-09-17T05:24:21.730124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5076}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5076': {'gid': 5076, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.43:6835/331818553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 331818553}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 331818553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:24:23.424 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5076:up:rejoin 2024-09-17T05:24:24.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:24.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 -- 172.21.15.43:0/2547860538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15c7e0 msgr2=0x7f369c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 --2- 172.21.15.43:0/2547860538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15c7e0 0x7f369c06f110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3694009bc0 tx=0x7f369401cb10 comp rx=0 tx=0).stop 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 -- 172.21.15.43:0/2547860538 shutdown_connections 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 --2- 172.21.15.43:0/2547860538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15c7e0 0x7f369c06f110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 -- 172.21.15.43:0/2547860538 >> 172.21.15.43:0/2547860538 conn(0x7f369c0b23d0 msgr2=0x7f369c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 -- 172.21.15.43:0/2547860538 shutdown_connections 2024-09-17T05:24:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.594+0000 7f36a3e00640 1 -- 172.21.15.43:0/2547860538 wait complete. 2024-09-17T05:24:24.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 Processor -- start 2024-09-17T05:24:24.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 -- start start 2024-09-17T05:24:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f369c155d70 0x7f369c09d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369c15c7e0 0x7f369c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f369c169dc0 con 0x7f369c15e8f0 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f369c169c40 con 0x7f369c15c7e0 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f369c169f40 con 0x7f369c155d70 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35804/0 (socket says 172.21.15.43:35804) 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 -- 172.21.15.43:0/932573650 learned_addr learned my addr 172.21.15.43:0/932573650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a0a00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369c15c7e0 0x7f369c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1400640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f369c155d70 0x7f369c09d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 -- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f369c155d70 msgr2=0x7f369c09d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f369c155d70 0x7f369c09d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 -- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369c15c7e0 msgr2=0x7f369c158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369c15c7e0 0x7f369c158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 -- 172.21.15.43:0/932573650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c09dc10 con 0x7f369c15e8f0 2024-09-17T05:24:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a1e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3694019720 tx=0x7f3694005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940bc070 con 0x7f369c15e8f0 2024-09-17T05:24:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3694004330 con 0x7f369c15e8f0 2024-09-17T05:24:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940b2040 con 0x7f369c15e8f0 2024-09-17T05:24:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.598+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c15ace0 con 0x7f369c15e8f0 2024-09-17T05:24:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c15af80 con 0x7f369c15e8f0 2024-09-17T05:24:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c15b150 con 0x7f369c15e8f0 2024-09-17T05:24:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36940ad440 con 0x7f369c15e8f0 2024-09-17T05:24:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f3699600640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f369412e4a0 con 0x7f369c15e8f0 2024-09-17T05:24:24.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.602+0000 7f36a1400640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:24.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.606+0000 7f36a1400640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f369c0715c0 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.606+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3694016790 con 0x7f369c15e8f0 2024-09-17T05:24:24.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.818+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f369c09d2f0 con 0x7f369c15e8f0 2024-09-17T05:24:24.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f3699600640 1 -- 172.21.15.43:0/932573650 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7f36940f9450 con 0x7f369c15e8f0 2024-09-17T05:24:24.823 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:24.823 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:24:24.823 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:24:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 msgr2=0x7f367c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f369c0715c0 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-09-17T05:24:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 msgr2=0x7f369c158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3694019720 tx=0x7f3694005b10 comp rx=0 tx=0).stop 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 shutdown_connections 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369c15c7e0 0x7f369c158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c15e8f0 0x7f369c158590 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 --2- 172.21.15.43:0/932573650 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f369c155d70 0x7f369c09d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 >> 172.21.15.43:0/932573650 conn(0x7f369c0b23d0 msgr2=0x7f369c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 shutdown_connections 2024-09-17T05:24:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:24.822+0000 7f36a3e00640 1 -- 172.21.15.43:0/932573650 wait complete. 2024-09-17T05:24:25.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:24:09.841714+0000', 'modified': '2024-09-17T05:24:23.737496+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 34, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5076}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5076': {'gid': 5076, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/331818553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 331818553}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 331818553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5076, 'qdb_cluster': [5076]} 2024-09-17T05:24:25.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:24:25.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- 172.21.15.43:0/3311051375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cc1f0 msgr2=0x7f06540cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 --2- 172.21.15.43:0/3311051375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cc1f0 0x7f06540cc600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f063c009900 tx=0x7f063c01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- 172.21.15.43:0/3311051375 shutdown_connections 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 --2- 172.21.15.43:0/3311051375 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cf220 0x7f06540cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 --2- 172.21.15.43:0/3311051375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cc1f0 0x7f06540cc600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- 172.21.15.43:0/3311051375 >> 172.21.15.43:0/3311051375 conn(0x7f06540b2de0 msgr2=0x7f06540b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- 172.21.15.43:0/3311051375 shutdown_connections 2024-09-17T05:24:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- 172.21.15.43:0/3311051375 wait complete. 2024-09-17T05:24:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 Processor -- start 2024-09-17T05:24:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.414+0000 7f0659800640 1 -- start start 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cc1f0 0x7f06541532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06540d0830 0x7f06541f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06540d0100 con 0x7f06540cf220 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06540cff80 con 0x7f06540d0830 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f06540b2700 con 0x7f06540cc1f0 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35822/0 (socket says 172.21.15.43:35822) 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 -- 172.21.15.43:0/2027050445 learned_addr learned my addr 172.21.15.43:0/2027050445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0653e00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cc1f0 0x7f06541532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0658e00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06540d0830 0x7f06541f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 -- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cc1f0 msgr2=0x7f06541532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cc1f0 0x7f06541532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 -- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06540d0830 msgr2=0x7f06541f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06540d0830 0x7f06541f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 -- 172.21.15.43:0/2027050445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06540caa20 con 0x7f06540cf220 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f064be00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f063c009850 tx=0x7f063c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0650c00640 1 -- 172.21.15.43:0/2027050445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0b2040 con 0x7f06540cf220 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0650c00640 1 -- 172.21.15.43:0/2027050445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f063c005320 con 0x7f06540cf220 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06540cad00 con 0x7f06540cf220 2024-09-17T05:24:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0650c00640 1 -- 172.21.15.43:0/2027050445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0046d0 con 0x7f06540cf220 2024-09-17T05:24:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.418+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541f86a0 con 0x7f06540cf220 2024-09-17T05:24:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0650c00640 1 -- 172.21.15.43:0/2027050445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f063c0bd060 con 0x7f06540cf220 2024-09-17T05:24:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0650c00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 0x7f0630075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0650c00640 1 -- 172.21.15.43:0/2027050445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f063c12dff0 con 0x7f06540cf220 2024-09-17T05:24:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0653e00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 0x7f0630075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 Processor -- start 2024-09-17T05:24:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 -- start start 2024-09-17T05:24:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0653e00640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 0x7f0630075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0644007fd0 tx=0x7f0644009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2dcd1460 0x55cc2e216b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0c1950 0x55cc2e217040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e0e7410 con 0x55cc2e0aaca0 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e024b40 con 0x55cc2e0c1950 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cc2e013f70 con 0x55cc2dcd1460 2024-09-17T05:24:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0653e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2dcd1460 0x55cc2e216b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35826/0 (socket says 172.21.15.43:35826) 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 -- 172.21.15.43:0/1642882902 learned_addr learned my addr 172.21.15.43:0/1642882902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f064be00640 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0c1950 0x55cc2e217040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 -- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2dcd1460 msgr2=0x55cc2e216b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2dcd1460 0x55cc2e216b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.422+0000 7f0658e00640 1 -- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0c1950 msgr2=0x55cc2e217040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0658e00640 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0c1950 0x55cc2e217040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0658e00640 1 -- 172.21.15.43:0/1642882902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2dfdb5b0 con 0x55cc2e0aaca0 2024-09-17T05:24:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0658e00640 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f064c009e90 tx=0x7f064c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8874e79b4452605 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0648c00640 1 -- 172.21.15.43:0/1642882902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c004d30 con 0x55cc2e0aaca0 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0648c00640 1 -- 172.21.15.43:0/1642882902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f064c003980 con 0x55cc2e0aaca0 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f0648c00640 1 -- 172.21.15.43:0/1642882902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c003db0 con 0x55cc2e0aaca0 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- 172.21.15.43:0/1642882902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 msgr2=0x55cc2e217580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f064c009e90 tx=0x7f064c0117b0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- 172.21.15.43:0/1642882902 shutdown_connections 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0c1950 0x55cc2e217040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e217580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 --2- 172.21.15.43:0/1642882902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2dcd1460 0x55cc2e216b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- 172.21.15.43:0/1642882902 >> 172.21.15.43:0/1642882902 conn(0x55cc2e197d90 msgr2=0x55cc2e1c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- 172.21.15.43:0/1642882902 shutdown_connections 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- 172.21.15.43:0/1642882902 wait complete. 2024-09-17T05:24:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 -- start start 2024-09-17T05:24:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.426+0000 7f06623d0000 1 Processor -- start 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e205ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc2dfae5f0 con 0x55cc2e0aaca0 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc2e017d80 con 0x55cc2dcd1460 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cc2dfba520 con 0x55cc2e0c1950 2024-09-17T05:24:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 10 client.0 _ll_register_callbacks cb 0x55cc2dfbe830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:55058/0 (socket says 172.21.15.43:55058) 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35836/0 (socket says 172.21.15.43:35836) 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 -- 172.21.15.43:0/2359481052 learned_addr learned my addr 172.21.15.43:0/2359481052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0658e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e205ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 -- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 msgr2=0x55cc2e205ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e205ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 -- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 msgr2=0x55cc2e205120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 -- 172.21.15.43:0/2359481052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2dfb2190 con 0x55cc2e0aaca0 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f064be00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f063c01cd90 tx=0x7f063c0f8310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b0b11b7db706e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0f8900 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f063c101de0 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc2dfac110 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f063c0fd330 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 10 client.5121 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55cc2e098f10 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55cc2dfac110 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f063c0f9b80 con 0x55cc2e0aaca0 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 10 client.5121 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 10 client.5121 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 20 client.5121 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dcb67d0 0x55cc2e2065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 4 client.5121 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc2e026a90 con 0x55cc2dcb67d0 2024-09-17T05:24:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dcb67d0 0x55cc2e2065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0641600640 10 client.5121 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.430+0000 7f0653e00640 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dcb67d0 0x55cc2e2065b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3e8ed80b318991 server_cookie=efb599b7b82b1d8f in_seq=0 out_seq=0 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f0641600640 1 -- 172.21.15.43:0/2359481052 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc2e026a90 con 0x55cc2dcb67d0 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f0641600640 10 client.5121 handle_command_reply: tid=0 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 client.5121 shutdown 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 msgr2=0x55cc2e205660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f063c01cd90 tx=0x7f063c0f8310 comp rx=0 tx=0).stop 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 shutdown_connections 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dcb67d0 0x55cc2e2065b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e205ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2e0aaca0 0x55cc2e205660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 --2- 172.21.15.43:0/2359481052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2dcd1460 0x55cc2e205120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f0641600640 0 client.5121 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 >> 172.21.15.43:0/2359481052 conn(0x55cc2e197d90 msgr2=0x55cc2e1cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 shutdown_connections 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 1 -- 172.21.15.43:0/2359481052 wait complete. 2024-09-17T05:24:25.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.434+0000 7f06623d0000 20 client.5121 trim_cache size 0 max 16384 2024-09-17T05:24:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 Processor -- start 2024-09-17T05:24:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 -- start start 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e1c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06540b5b30 con 0x55cc2dcd1460 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06540b4bb0 con 0x55cc2e0aaca0 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f06540ca7b0 con 0x55cc2e0c1950 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35842/0 (socket says 172.21.15.43:35842) 2024-09-17T05:24:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 -- 172.21.15.43:0/4146922861 learned_addr learned my addr 172.21.15.43:0/4146922861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f064be00640 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e1c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0658e00640 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 -- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 msgr2=0x55cc2e22d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 -- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 msgr2=0x55cc2e1c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e1c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 -- 172.21.15.43:0/4146922861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2e06fdb0 con 0x55cc2dcd1460 2024-09-17T05:24:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.454+0000 7f0653e00640 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f064400fd30 tx=0x7f0644012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc888980ddf50e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f0643e00640 1 -- 172.21.15.43:0/4146922861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644023e30 con 0x55cc2dcd1460 2024-09-17T05:24:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f0643e00640 1 -- 172.21.15.43:0/4146922861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f064401b040 con 0x55cc2dcd1460 2024-09-17T05:24:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f0643e00640 1 -- 172.21.15.43:0/4146922861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014450 con 0x55cc2dcd1460 2024-09-17T05:24:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- 172.21.15.43:0/4146922861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 msgr2=0x55cc2e1c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f064400fd30 tx=0x7f0644012000 comp rx=0 tx=0).stop 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- 172.21.15.43:0/4146922861 shutdown_connections 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e1c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- 172.21.15.43:0/4146922861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e1c6cd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- 172.21.15.43:0/4146922861 >> 172.21.15.43:0/4146922861 conn(0x55cc2e197d90 msgr2=0x55cc2e1cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- 172.21.15.43:0/4146922861 shutdown_connections 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- 172.21.15.43:0/4146922861 wait complete. 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- start start 2024-09-17T05:24:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 Processor -- start 2024-09-17T05:24:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e229b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc2dfdfee0 con 0x55cc2dcd1460 2024-09-17T05:24:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06540b4560 con 0x55cc2e0aaca0 2024-09-17T05:24:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.458+0000 7f06623d0000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cc2dfcd1e0 con 0x55cc2e0c1950 2024-09-17T05:24:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 10 client.0 _ll_register_callbacks cb 0x7f063c0f4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35846/0 (socket says 172.21.15.43:35846) 2024-09-17T05:24:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 -- 172.21.15.43:0/3904205404 learned_addr learned my addr 172.21.15.43:0/3904205404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0658e00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 -- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 msgr2=0x55cc2e22a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f064be00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e229b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 -- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 msgr2=0x55cc2e229b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e229b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 -- 172.21.15.43:0/3904205404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2dfdaab0 con 0x55cc2dcd1460 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0658e00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f064be00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e229b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0653e00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f064400fd30 tx=0x7f0644014700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9b6cc6cd98ac1d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644031070 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f0644014b00 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f064401b040 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f064403c050 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 10 client.5127 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55cc2dfb9070 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f064401b040 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f064403f020 con 0x55cc2dcd1460 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 10 client.5127 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 10 client.5127 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 20 client.5127 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dca4830 0x55cc2e22a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 4 client.5127 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "status"}) -- 0x55cc2e022540 con 0x55cc2dca4830 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f064be00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dca4830 0x55cc2e22a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 10 client.5127 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f064be00640 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dca4830 0x55cc2e22a5e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12425ff5be4af37e server_cookie=2f7579d6d54ef121 in_seq=0 out_seq=0 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 1 -- 172.21.15.43:0/3904205404 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cc2e022540 con 0x55cc2dca4830 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 10 client.5127 handle_command_reply: tid=0 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 client.5127 shutdown 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 msgr2=0x55cc2e229620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f064400fd30 tx=0x7f0644014700 comp rx=0 tx=0).stop 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 shutdown_connections 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55cc2dca4830 0x55cc2e22a5e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc2e0aaca0 0x55cc2e229b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc2e0c1950 0x55cc2e22a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 --2- 172.21.15.43:0/3904205404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc2dcd1460 0x55cc2e229620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f0641600640 0 client.5127 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 >> 172.21.15.43:0/3904205404 conn(0x55cc2e197d90 msgr2=0x55cc2e1cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 shutdown_connections 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 1 -- 172.21.15.43:0/3904205404 wait complete. 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.462+0000 7f06623d0000 20 client.5127 trim_cache size 0 max 16384 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:24:25.469 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5076, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.7362931469999996, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 34, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 34, 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 5.6045177449999999 2024-09-17T05:24:25.470 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 msgr2=0x7f0630075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 0x7f0630075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0644007fd0 tx=0x7f0644009040 comp rx=0 tx=0).stop 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 msgr2=0x7f06541f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f063c009850 tx=0x7f063c005760 comp rx=0 tx=0).stop 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 shutdown_connections 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0630073200 0x7f0630075710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06540d0830 0x7f06541f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06540cf220 0x7f06541f8ea0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 --2- 172.21.15.43:0/2027050445 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06540cc1f0 0x7f06541532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 >> 172.21.15.43:0/2027050445 conn(0x7f06540b2de0 msgr2=0x7f06540cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 shutdown_connections 2024-09-17T05:24:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.466+0000 7f0659800640 1 -- 172.21.15.43:0/2027050445 wait complete. 2024-09-17T05:24:25.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:24:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- 172.21.15.43:0/978114640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677406fce0 msgr2=0x7f67740700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 --2- 172.21.15.43:0/978114640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677406fce0 0x7f67740700f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f676c009bc0 tx=0x7f676c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- 172.21.15.43:0/978114640 shutdown_connections 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 --2- 172.21.15.43:0/978114640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677406fce0 0x7f67740700f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 --2- 172.21.15.43:0/978114640 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f677415f470 0x7f67740d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- 172.21.15.43:0/978114640 >> 172.21.15.43:0/978114640 conn(0x7f67740b2810 msgr2=0x7f67740b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- 172.21.15.43:0/978114640 shutdown_connections 2024-09-17T05:24:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- 172.21.15.43:0/978114640 wait complete. 2024-09-17T05:24:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 Processor -- start 2024-09-17T05:24:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.642+0000 7f6778c00640 1 -- start start 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 0x7f677414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67740b5610 0x7f677414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67741695c0 con 0x7f677415f470 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6774169440 con 0x7f677406fce0 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6774169740 con 0x7f67740b5610 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35868/0 (socket says 172.21.15.43:35868) 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 -- 172.21.15.43:0/407309561 learned_addr learned my addr 172.21.15.43:0/407309561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6771e00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 0x7f677414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f676be00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67740b5610 0x7f677414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 -- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67740b5610 msgr2=0x7f677414b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67740b5610 0x7f677414b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 -- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 msgr2=0x7f677414c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 0x7f677414c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 -- 172.21.15.43:0/407309561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677414bb90 con 0x7f677415f470 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6771e00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 0x7f677414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6772800640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f676c0b0040 tx=0x7f676c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0b2040 con 0x7f677415f470 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f676c004040 con 0x7f677415f470 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c004470 con 0x7f677415f470 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677414cbb0 con 0x7f677415f470 2024-09-17T05:24:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.646+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677414d170 con 0x7f677415f470 2024-09-17T05:24:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.650+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67740700f0 con 0x7f677415f470 2024-09-17T05:24:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.650+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f676c0bd030 con 0x7f677415f470 2024-09-17T05:24:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.650+0000 7f676a000640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 0x7f6750075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.650+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f676c12e3c0 con 0x7f677415f470 2024-09-17T05:24:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.650+0000 7f6771e00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 0x7f6750075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:25.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.654+0000 7f6771e00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 0x7f6750075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6774069610 tx=0x7f67640091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.654+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f676c0c4060 con 0x7f677415f470 2024-09-17T05:24:25.731 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:24:25.730+0000 7fcd24200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:24:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.870+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6774166010 con 0x7f677415f470 2024-09-17T05:24:25.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.870+0000 7f676a000640 1 -- 172.21.15.43:0/407309561 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f676c016790 con 0x7f677415f470 2024-09-17T05:24:25.872 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 msgr2=0x7f6750075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 0x7f6750075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6774069610 tx=0x7f67640091e0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 msgr2=0x7f67741fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f676c0b0040 tx=0x7f676c005a60 comp rx=0 tx=0).stop 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 shutdown_connections 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6750073200 0x7f6750075710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f677415f470 0x7f67741fa2c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67740b5610 0x7f677414b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 --2- 172.21.15.43:0/407309561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f677406fce0 0x7f677414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 >> 172.21.15.43:0/407309561 conn(0x7f67740b2810 msgr2=0x7f67740b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 shutdown_connections 2024-09-17T05:24:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:25.874+0000 7f6778c00640 1 -- 172.21.15.43:0/407309561 wait complete. 2024-09-17T05:24:25.895 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:24:25.895 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:24:25.895 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:24:25.895 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:24:25.896 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:25.896 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:24:25.896 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:24:25.896 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:24:25.896 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:25.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:25.897 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:24:25.907 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.908 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-09-17T05:24:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:24:25.911 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.912 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:24:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:24:25.913 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:25.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:25.914 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:24:25.972 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:24:25.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:25.974 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:24:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:24:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:24:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:24:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:24:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:24:26.029 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:24:26.029 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:24:26.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:24:26.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:26.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:26.166 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:24:26.176 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:24:26.176 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:24:26.176 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:26.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:26.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:24:26.238 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:26.239 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:26.291 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:26.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:26.291 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:26.346 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:24:26.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:26.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:26.452 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:26.453 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:26.453 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:26.472 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:26.473 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:26.473 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:26.473 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:24:26.473 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:24:26.484 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:24:26.485 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:24:26.489 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:24:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:26.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:26.509 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:24:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:24:26.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:client5139 2024-09-17T05:24:26.612 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:26.612 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:26.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:26.622 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-17T05:24:26.675 INFO:teuthology.orchestra.run.smithi043.stdout:file_undamaged 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-09-17T05:24:26.677 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:26.678 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T05:24:26.678 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-09-17T05:24:26.678 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T05:24:26.678 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-09-17T05:24:26.678 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:24:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:26 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-17T05:24:26.799 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T05:24:26.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 -- 172.21.15.43:0/1831189567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c0730b0 msgr2=0x7fa16c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 --2- 172.21.15.43:0/1831189567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c0730b0 0x7fa16c0734c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0d0190 tx=0x7fa16401cc60 comp rx=0 tx=0).stop 2024-09-17T05:24:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 -- 172.21.15.43:0/1831189567 shutdown_connections 2024-09-17T05:24:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 --2- 172.21.15.43:0/1831189567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c0730b0 0x7fa16c0734c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 --2- 172.21.15.43:0/1831189567 >> v2:172.21.15.62:3300/0 conn(0x7fa16c06fc90 0x7fa16c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 -- 172.21.15.43:0/1831189567 >> 172.21.15.43:0/1831189567 conn(0x7fa16c0b23d0 msgr2=0x7fa16c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 -- 172.21.15.43:0/1831189567 shutdown_connections 2024-09-17T05:24:27.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.002+0000 7fa171800640 1 -- 172.21.15.43:0/1831189567 wait complete. 2024-09-17T05:24:27.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 Processor -- start 2024-09-17T05:24:27.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- start start 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa16c06fc90 0x7fa16c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c0730b0 0x7fa16c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c169fd0 con 0x7fa16c15fe10 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c169e50 con 0x7fa16c0730b0 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c16a150 con 0x7fa16c06fc90 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35874/0 (socket says 172.21.15.43:35874) 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 -- 172.21.15.43:0/466358637 learned_addr learned my addr 172.21.15.43:0/466358637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16aa00640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa16c06fc90 0x7fa16c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa162000640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c0730b0 0x7fa16c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 -- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa16c06fc90 msgr2=0x7fa16c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa16c06fc90 0x7fa16c154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 -- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c0730b0 msgr2=0x7fa16c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c0730b0 0x7fa16c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 -- 172.21.15.43:0/466358637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c0b0550 con 0x7fa16c15fe10 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa16b400640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa164098760 tx=0x7fa164005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1640b2040 con 0x7fa16c15fe10 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1640bb070 con 0x7fa16c15fe10 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c0cd680 con 0x7fa16c15fe10 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164004240 con 0x7fa16c15fe10 2024-09-17T05:24:27.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.006+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c153d20 con 0x7fa16c15fe10 2024-09-17T05:24:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa16c0700a0 con 0x7fa16c15fe10 2024-09-17T05:24:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1640043e0 con 0x7fa16c15fe10 2024-09-17T05:24:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa163400640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 0x7fa148075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa16412f4c0 con 0x7fa16c15fe10 2024-09-17T05:24:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa16aa00640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 0x7fa148075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.010+0000 7fa16aa00640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 0x7fa148075710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa15c007fd0 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.014+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa16c0cd680 con 0x7fa16c15fe10 2024-09-17T05:24:27.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa16c0b17d0 con 0x7fa16c15fe10 2024-09-17T05:24:27.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa163400640 1 -- 172.21.15.43:0/466358637 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7fa164016970 con 0x7fa16c15fe10 2024-09-17T05:24:27.228 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:27.228 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:24:27.228 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 msgr2=0x7fa148075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 0x7fa148075710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa15c007fd0 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 msgr2=0x7fa16c0c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa164098760 tx=0x7fa164005c80 comp rx=0 tx=0).stop 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 shutdown_connections 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa148073200 0x7fa148075710 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa16c0730b0 0x7fa16c156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa16c15fe10 0x7fa16c0c9f90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 --2- 172.21.15.43:0/466358637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa16c06fc90 0x7fa16c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 >> 172.21.15.43:0/466358637 conn(0x7fa16c0b23d0 msgr2=0x7fa16c0ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 shutdown_connections 2024-09-17T05:24:27.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.226+0000 7fa171800640 1 -- 172.21.15.43:0/466358637 wait complete. 2024-09-17T05:24:27.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T05:24:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- 172.21.15.43:0/2975983837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c06fc90 msgr2=0x7fd80c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 --2- 172.21.15.43:0/2975983837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c06fc90 0x7fd80c0700a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0d0e90 tx=0x7fd7f401d200 comp rx=0 tx=0).stop 2024-09-17T05:24:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- 172.21.15.43:0/2975983837 shutdown_connections 2024-09-17T05:24:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 --2- 172.21.15.43:0/2975983837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c06fc90 0x7fd80c0700a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 --2- 172.21.15.43:0/2975983837 >> v2:172.21.15.62:3300/0 conn(0x7fd80c0730b0 0x7fd80c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 --2- 172.21.15.43:0/2975983837 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c15fe10 0x7fd80c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- 172.21.15.43:0/2975983837 >> 172.21.15.43:0/2975983837 conn(0x7fd80c0b23d0 msgr2=0x7fd80c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- 172.21.15.43:0/2975983837 shutdown_connections 2024-09-17T05:24:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- 172.21.15.43:0/2975983837 wait complete. 2024-09-17T05:24:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 Processor -- start 2024-09-17T05:24:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.410+0000 7fd811400640 1 -- start start 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c06fc90 0x7fd80c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c0730b0 0x7fd80c0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0cea50 con 0x7fd80c15fe10 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0ce8d0 con 0x7fd80c0730b0 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0cebd0 con 0x7fd80c06fc90 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35888/0 (socket says 172.21.15.43:35888) 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 -- 172.21.15.43:0/483514287 learned_addr learned my addr 172.21.15.43:0/483514287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd803e00640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c0730b0 0x7fd80c0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80a800640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c06fc90 0x7fd80c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 -- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c06fc90 msgr2=0x7fd80c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c06fc90 0x7fd80c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 -- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c0730b0 msgr2=0x7fd80c0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c0730b0 0x7fd80c0c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 -- 172.21.15.43:0/483514287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0c8f90 con 0x7fd80c15fe10 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd80b200640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b9e0 tx=0x7fd80400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd802a00640 1 -- 172.21.15.43:0/483514287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804019070 con 0x7fd80c15fe10 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd802a00640 1 -- 172.21.15.43:0/483514287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd80400cc30 con 0x7fd80c15fe10 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd802a00640 1 -- 172.21.15.43:0/483514287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8040146a0 con 0x7fd80c15fe10 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c157810 con 0x7fd80c15fe10 2024-09-17T05:24:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.414+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c0c8cb0 con 0x7fd80c15fe10 2024-09-17T05:24:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd802a00640 1 -- 172.21.15.43:0/483514287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd80401c070 con 0x7fd80c15fe10 2024-09-17T05:24:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd802a00640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 0x7fd7e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd802a00640 1 -- 172.21.15.43:0/483514287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fd804093f10 con 0x7fd80c15fe10 2024-09-17T05:24:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 Processor -- start 2024-09-17T05:24:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd80a800640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 0x7fd7e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 -- start start 2024-09-17T05:24:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd80a800640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 0x7fd7e8075740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0098f0 tx=0x7fd7fc006f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 0x558e623cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62268d90 0x558e623cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e6226dd00 con 0x558e6227f1c0 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558e621e1c90 con 0x558e62268d90 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558e621d10c0 con 0x558e62289b60 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35902/0 (socket says 172.21.15.43:35902) 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 -- 172.21.15.43:0/4242226011 learned_addr learned my addr 172.21.15.43:0/4242226011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd80b200640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62268d90 0x558e623cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd80a800640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 0x558e623cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 -- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 msgr2=0x558e623cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 0x558e623cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 -- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62268d90 msgr2=0x558e623cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62268d90 0x558e623cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd803e00640 1 -- 172.21.15.43:0/4242226011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e621ac290 con 0x558e6227f1c0 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.418+0000 7fd80a800640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 0x558e623cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd803e00640 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009230 tx=0x7fd7f4019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc4f3d9f9ba6d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd7fb000640 1 -- 172.21.15.43:0/4242226011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40b2070 con 0x558e6227f1c0 2024-09-17T05:24:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd7fb000640 1 -- 172.21.15.43:0/4242226011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7f40b3040 con 0x558e6227f1c0 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd7fb000640 1 -- 172.21.15.43:0/4242226011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4005040 con 0x558e6227f1c0 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- 172.21.15.43:0/4242226011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 msgr2=0x558e623cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009230 tx=0x7fd7f4019540 comp rx=0 tx=0).stop 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- 172.21.15.43:0/4242226011 shutdown_connections 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62268d90 0x558e623cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e6227f1c0 0x558e623cf820 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 --2- 172.21.15.43:0/4242226011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62289b60 0x558e623cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- 172.21.15.43:0/4242226011 >> 172.21.15.43:0/4242226011 conn(0x558e61e8e760 msgr2=0x558e623b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- 172.21.15.43:0/4242226011 shutdown_connections 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- 172.21.15.43:0/4242226011 wait complete. 2024-09-17T05:24:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 -- start start 2024-09-17T05:24:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.422+0000 7fd818985000 1 Processor -- start 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e6238b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e623b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e6216b940 con 0x558e62268d90 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558e621d4ed0 con 0x558e62289b60 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558e61e96820 con 0x558e6227f1c0 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 10 client.0 _ll_register_callbacks cb 0x558e621af7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35914/0 (socket says 172.21.15.43:35914) 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 -- 172.21.15.43:0/1286320387 learned_addr learned my addr 172.21.15.43:0/1286320387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd803e00640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e6238b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80b200640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e623b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 -- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 msgr2=0x558e6238b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e6238b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 -- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 msgr2=0x558e623b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e623b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 -- 172.21.15.43:0/1286320387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e62184550 con 0x558e62268d90 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd80a800640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0096d0 tx=0x7fd7fc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611a2817f4154923 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc025020 con 0x558e62268d90 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e62169400 con 0x558e62268d90 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fd7fc013690 con 0x558e62268d90 2024-09-17T05:24:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd7fc02f050 con 0x558e62268d90 2024-09-17T05:24:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 10 client.5160 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x558e6216f480 con 0x558e62268d90 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558e62169400 con 0x558e62268d90 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fd7fc032020 con 0x558e62268d90 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 10 client.5160 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 10 client.5160 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 20 client.5160 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e73ad0 0x558e623e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 4 client.5160 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e621e3be0 con 0x558e61e73ad0 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd803e00640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e73ad0 0x558e623e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd803e00640 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e73ad0 0x558e623e08a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99a78c632774874 server_cookie=9c09b900147d273f in_seq=0 out_seq=0 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 10 client.5160 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 1 -- 172.21.15.43:0/1286320387 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e621e3be0 con 0x558e61e73ad0 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd7ef400640 10 client.5160 handle_command_reply: tid=0 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.426+0000 7fd818985000 1 client.5160 shutdown 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 msgr2=0x558e6238b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0096d0 tx=0x7fd7fc0046d0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 shutdown_connections 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e73ad0 0x558e623e08a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e623b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e6238b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 --2- 172.21.15.43:0/1286320387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e6238b070 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 >> 172.21.15.43:0/1286320387 conn(0x558e61e8e760 msgr2=0x558e62383680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd7ef400640 0 client.5160 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 shutdown_connections 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 1 -- 172.21.15.43:0/1286320387 wait complete. 2024-09-17T05:24:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.430+0000 7fd818985000 20 client.5160 trim_cache size 0 max 16384 2024-09-17T05:24:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.442+0000 7fd818985000 1 Processor -- start 2024-09-17T05:24:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.442+0000 7fd818985000 1 -- start start 2024-09-17T05:24:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62268d90 0x558e624405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e6227f1c0 0x558e62440b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e62290fc0 con 0x558e62289b60 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558e6217c300 con 0x558e6227f1c0 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558e621e0a60 con 0x558e62268d90 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35916/0 (socket says 172.21.15.43:35916) 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 -- 172.21.15.43:0/2000473139 learned_addr learned my addr 172.21.15.43:0/2000473139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd803e00640 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e6227f1c0 0x558e62440b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80a800640 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62268d90 0x558e624405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 -- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62268d90 msgr2=0x558e624405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62268d90 0x558e624405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 -- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e6227f1c0 msgr2=0x558e62440b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e6227f1c0 0x558e62440b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 -- 172.21.15.43:0/2000473139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e62256d10 con 0x558e62289b60 2024-09-17T05:24:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd80b200640 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd80405e800 tx=0x7fd80405e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da86f0ea384d13ee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd7fa600640 1 -- 172.21.15.43:0/2000473139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80405e870 con 0x558e62289b60 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd7fa600640 1 -- 172.21.15.43:0/2000473139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8040632a0 con 0x558e62289b60 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd7fa600640 1 -- 172.21.15.43:0/2000473139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804067ee0 con 0x558e62289b60 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- 172.21.15.43:0/2000473139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 msgr2=0x558e62441040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd80405e800 tx=0x7fd80405e440 comp rx=0 tx=0).stop 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- 172.21.15.43:0/2000473139 shutdown_connections 2024-09-17T05:24:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e6227f1c0 0x558e62440b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62289b60 0x558e62441040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 --2- 172.21.15.43:0/2000473139 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e62268d90 0x558e624405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- 172.21.15.43:0/2000473139 >> 172.21.15.43:0/2000473139 conn(0x558e61e8e760 msgr2=0x558e62385690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- 172.21.15.43:0/2000473139 shutdown_connections 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- 172.21.15.43:0/2000473139 wait complete. 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 -- start start 2024-09-17T05:24:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.446+0000 7fd818985000 1 Processor -- start 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e62444b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e62445080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558e6219d300 con 0x558e62268d90 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558e6218e670 con 0x558e62289b60 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558e621d41e0 con 0x558e6227f1c0 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 10 client.0 _ll_register_callbacks cb 0x7fd7fc009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35928/0 (socket says 172.21.15.43:35928) 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 -- 172.21.15.43:0/2106679677 learned_addr learned my addr 172.21.15.43:0/2106679677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80b200640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e62445080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd803e00640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e62444b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 -- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 msgr2=0x558e62444b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e62444b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 -- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 msgr2=0x558e62445080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e62445080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 -- 172.21.15.43:0/2106679677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e62187e50 con 0x558e62268d90 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80b200640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e62445080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd80a800640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e700 tx=0x7fd7fc013450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746447b0f5d549 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc025070 con 0x558e62268d90 2024-09-17T05:24:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8040632a0 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fd7fc021880 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd7fc040050 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 10 client.5166 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x558e621762d0 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8040632a0 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fd7fc043020 con 0x558e62268d90 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 10 client.5166 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 10 client.5166 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 20 client.5166 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e61b30 0x558e623e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 4 client.5166 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x558e621df690 con 0x558e61e61b30 2024-09-17T05:24:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd803e00640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e61b30 0x558e623e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd7ef400640 10 client.5166 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.450+0000 7fd803e00640 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e61b30 0x558e623e2af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786ebfb77c201552 server_cookie=444fb7c18a146f1a in_seq=0 out_seq=0 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd7ef400640 1 -- 172.21.15.43:0/2106679677 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x558e621df690 con 0x558e61e61b30 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd7ef400640 10 client.5166 handle_command_reply: tid=0 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 client.5166 shutdown 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 msgr2=0x558e62444600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00e700 tx=0x7fd7fc013450 comp rx=0 tx=0).stop 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 shutdown_connections 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558e61e61b30 0x558e623e2af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e62289b60 0x558e62445080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558e6227f1c0 0x558e62444b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd7ef400640 0 client.5166 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 --2- 172.21.15.43:0/2106679677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558e62268d90 0x558e62444600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 >> 172.21.15.43:0/2106679677 conn(0x558e61e8e760 msgr2=0x558e623dbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 shutdown_connections 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 1 -- 172.21.15.43:0/2106679677 wait complete. 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd818985000 20 client.5166 trim_cache size 0 max 16384 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:24:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "dentry", 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 204375696, 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511627776, 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "frag": "*", 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "dname": "file_to_be_damaged", 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_id": "head", 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:27.460 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 msgr2=0x7fd7e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 0x7fd7e8075740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0098f0 tx=0x7fd7fc006f90 comp rx=0 tx=0).stop 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 msgr2=0x7fd80c0c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b9e0 tx=0x7fd80400bea0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 shutdown_connections 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e8073230 0x7fd7e8075740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c0730b0 0x7fd80c0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd80c15fe10 0x7fd80c0c80e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 --2- 172.21.15.43:0/483514287 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80c06fc90 0x7fd80c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 >> 172.21.15.43:0/483514287 conn(0x7fd80c0b23d0 msgr2=0x7fd80c073be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 shutdown_connections 2024-09-17T05:24:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27.454+0000 7fd811400640 1 -- 172.21.15.43:0/483514287 wait complete. 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> import errno 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:24:27.475 DEBUG:teuthology.orchestra.run.smithi043:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-17T05:24:27.476 DEBUG:teuthology.orchestra.run.smithi043:> f.close() 2024-09-17T05:24:27.476 DEBUG:teuthology.orchestra.run.smithi043:> except IOError as e: 2024-09-17T05:24:27.476 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(errno.EIO) 2024-09-17T05:24:27.476 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:24:27.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-17T05:24:27.551 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-17T05:24:27.552 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-09-17T05:24:27.553 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:24:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:27 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-17T05:24:27.675 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-17T05:24:27.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:27.676 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-17T05:24:27.732 INFO:teuthology.orchestra.run.smithi043.stdout:2 2024-09-17T05:24:27.732 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:27.732 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:27.783 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:24:27.783 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:24:27.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 -- 172.21.15.43:0/769237314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 msgr2=0x7facf00b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 --2- 172.21.15.43:0/769237314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf00b33e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7facf0071050 tx=0x7facdc01c800 comp rx=0 tx=0).stop 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 -- 172.21.15.43:0/769237314 shutdown_connections 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 --2- 172.21.15.43:0/769237314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf00b33e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 --2- 172.21.15.43:0/769237314 >> v2:172.21.15.62:3300/0 conn(0x7facf00b3920 0x7facf00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 --2- 172.21.15.43:0/769237314 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 0x7facf015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 -- 172.21.15.43:0/769237314 >> 172.21.15.43:0/769237314 conn(0x7facf00b23d0 msgr2=0x7facf00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 -- 172.21.15.43:0/769237314 shutdown_connections 2024-09-17T05:24:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.034+0000 7facf8600640 1 -- 172.21.15.43:0/769237314 wait complete. 2024-09-17T05:24:28.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 Processor -- start 2024-09-17T05:24:28.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 -- start start 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facf00b3920 0x7facf00c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 0x7facf0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facf015f870 con 0x7facf006f110 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7facf015f6f0 con 0x7facf00b3920 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf8600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7facf015f9f0 con 0x7facf0155d70 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35944/0 (socket says 172.21.15.43:35944) 2024-09-17T05:24:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 -- 172.21.15.43:0/4248154948 learned_addr learned my addr 172.21.15.43:0/4248154948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf6600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 0x7facf0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5200640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facf00b3920 0x7facf00c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 -- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 msgr2=0x7facf0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 0x7facf0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 -- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facf00b3920 msgr2=0x7facf00c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facf00b3920 0x7facf00c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.038+0000 7facf5c00640 1 -- 172.21.15.43:0/4248154948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf009d870 con 0x7facf006f110 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7facf5c00640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7face400d700 tx=0x7face400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4017070 con 0x7facf006f110 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7face4009b10 con 0x7facf006f110 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4010460 con 0x7facf006f110 2024-09-17T05:24:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf009db10 con 0x7facf006f110 2024-09-17T05:24:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf0155780 con 0x7facf006f110 2024-09-17T05:24:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf015fea0 con 0x7facf006f110 2024-09-17T05:24:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.042+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7face401a070 con 0x7facf006f110 2024-09-17T05:24:28.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.046+0000 7faced600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 0x7facd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.046+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7face4094450 con 0x7facf006f110 2024-09-17T05:24:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.046+0000 7facf5200640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 0x7facd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.046+0000 7facf5200640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 0x7facd0075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af000 tx=0x7facdc01b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.050+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7face405f400 con 0x7facf006f110 2024-09-17T05:24:28.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7facf0075350 con 0x7facf006f110 2024-09-17T05:24:28.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7faced600640 1 -- 172.21.15.43:0/4248154948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7face405ed00 con 0x7facf006f110 2024-09-17T05:24:28.236 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404655827 2024-09-18T05:24:19.717260+0000 2024-09-17T05:24:28.236 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/1404655827 2024-09-18T05:24:19.717260+0000 2024-09-17T05:24:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 msgr2=0x7facd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 0x7facd0075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af000 tx=0x7facdc01b040 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 msgr2=0x7facf008c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7face400d700 tx=0x7face400dbc0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 shutdown_connections 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7facd0073200 0x7facd0075710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facf00b3920 0x7facf00c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facf0155d70 0x7facf0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 --2- 172.21.15.43:0/4248154948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facf006f110 0x7facf008c1f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 >> 172.21.15.43:0/4248154948 conn(0x7facf00b23d0 msgr2=0x7facf00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 shutdown_connections 2024-09-17T05:24:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.234+0000 7facf8600640 1 -- 172.21.15.43:0/4248154948 wait complete. 2024-09-17T05:24:28.258 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:24:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:28.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:28.284 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:24:28.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:28.285 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:24:28.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:28.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-09-17T05:24:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 -- 172.21.15.43:0/2435354138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc003af0 msgr2=0x7f77bc003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 --2- 172.21.15.43:0/2435354138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc003af0 0x7f77bc003f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c260 tx=0x7f77c801cf20 comp rx=0 tx=0).stop 2024-09-17T05:24:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 -- 172.21.15.43:0/2435354138 shutdown_connections 2024-09-17T05:24:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 --2- 172.21.15.43:0/2435354138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc003af0 0x7f77bc003f20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 -- 172.21.15.43:0/2435354138 >> 172.21.15.43:0/2435354138 conn(0x7f77d806ce60 msgr2=0x7f77d806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 -- 172.21.15.43:0/2435354138 shutdown_connections 2024-09-17T05:24:28.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.538+0000 7f77dd800640 1 -- 172.21.15.43:0/2435354138 wait complete. 2024-09-17T05:24:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 Processor -- start 2024-09-17T05:24:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- start start 2024-09-17T05:24:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 0x7f77d8152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 0x7f77d8099a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77d81415e0 0x7f77d81419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80b5180 con 0x7f77d8151e60 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80b5000 con 0x7f77d81415e0 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77d806f0d0 con 0x7f77bc003af0 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 0x7f77d8152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 0x7f77d8099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 0x7f77d8152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:39936/0 (socket says 172.21.15.43:39936) 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77d6a00640 1 -- 172.21.15.43:0/2787701187 learned_addr learned my addr 172.21.15.43:0/2787701187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77d7400640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77d81415e0 0x7f77d81419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 -- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 msgr2=0x7f77d8152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 0x7f77d8152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 -- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77d81415e0 msgr2=0x7f77d81419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77d81415e0 0x7f77d81419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 -- 172.21.15.43:0/2787701187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d815c9d0 con 0x7f77d8151e60 2024-09-17T05:24:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cfe00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 0x7f77d8099a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f77c00011b0 tx=0x7f77c0001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cea00640 1 -- 172.21.15.43:0/2787701187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0015020 con 0x7f77d8151e60 2024-09-17T05:24:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cea00640 1 -- 172.21.15.43:0/2787701187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c0011750 con 0x7f77d8151e60 2024-09-17T05:24:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77cea00640 1 -- 172.21.15.43:0/2787701187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011ba0 con 0x7f77d8151e60 2024-09-17T05:24:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d8141f30 con 0x7f77d8151e60 2024-09-17T05:24:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.542+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d8141130 con 0x7f77d8151e60 2024-09-17T05:24:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77cea00640 1 -- 172.21.15.43:0/2787701187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77c0018020 con 0x7f77d8151e60 2024-09-17T05:24:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77cea00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 0x7f77b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77cea00640 1 -- 172.21.15.43:0/2787701187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f77c0094b30 con 0x7f77d8151e60 2024-09-17T05:24:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77d6a00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 0x7f77b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77e4ffa000 1 Processor -- start 2024-09-17T05:24:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.546+0000 7f77e4ffa000 1 -- start start 2024-09-17T05:24:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 0x7f77b4075740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77c801b7f0 tx=0x7f77c8019d60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 0x559edff2db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edff2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559edfd3fa60 con 0x559edfdce940 2024-09-17T05:24:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559edfd2ee90 con 0x559edfdbace0 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559edfd32ca0 con 0x559edfdb49a0 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35978/0 (socket says 172.21.15.43:35978) 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 -- 172.21.15.43:0/2623973362 learned_addr learned my addr 172.21.15.43:0/2623973362 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77cfe00640 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 0x559edff2db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 -- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 msgr2=0x559edff2e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edff2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 -- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 msgr2=0x559edff2db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 0x559edff2db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 -- 172.21.15.43:0/2623973362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edfd78060 con 0x559edfdce940 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77cfe00640 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 0x559edff2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77d6a00640 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77c801d400 tx=0x7f77c80a7050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c29281cf39fb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77c7400640 1 -- 172.21.15.43:0/2623973362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003800 con 0x559edfdce940 2024-09-17T05:24:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77c7400640 1 -- 172.21.15.43:0/2623973362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c80a5e50 con 0x559edfdce940 2024-09-17T05:24:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77c7400640 1 -- 172.21.15.43:0/2623973362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003bc0 con 0x559edfdce940 2024-09-17T05:24:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- 172.21.15.43:0/2623973362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 msgr2=0x559edff2d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77c801d400 tx=0x7f77c80a7050 comp rx=0 tx=0).stop 2024-09-17T05:24:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- 172.21.15.43:0/2623973362 shutdown_connections 2024-09-17T05:24:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdbace0 0x559edff2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edff2e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/2623973362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdce940 0x559edff2d620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- 172.21.15.43:0/2623973362 >> 172.21.15.43:0/2623973362 conn(0x559edf9ed330 msgr2=0x559edfee25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- 172.21.15.43:0/2623973362 shutdown_connections 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.550+0000 7f77e4ffa000 1 -- 172.21.15.43:0/2623973362 wait complete. 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- start start 2024-09-17T05:24:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 Processor -- start 2024-09-17T05:24:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edfee94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdce940 0x559edff1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559edfd72b80 con 0x559edfdbace0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559edfcd8970 con 0x559edfdce940 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559edfce4bd0 con 0x559edfdb49a0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 10 client.0 _ll_register_callbacks cb 0x559edfcd5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35980/0 (socket says 172.21.15.43:35980) 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 -- 172.21.15.43:0/3297481338 learned_addr learned my addr 172.21.15.43:0/3297481338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edfee94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77d7400640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdce940 0x559edff1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 -- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 msgr2=0x559edfee94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edfee94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 -- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdce940 msgr2=0x559edff1c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdce940 0x559edff1c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 -- 172.21.15.43:0/3297481338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edfcd7b50 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77cfe00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c0001320 tx=0x7f77c005eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fcd59612622899 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c005e7b0 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f77c004c870 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559edfcc7200 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f77c005f350 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 10 client.5184 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.554+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x559edfccd0c0 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559edfcc7200 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f77c005aca0 con 0x559edfdbace0 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 10 client.5184 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 10 client.5184 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 20 client.5184 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9d1b10 0x559edff348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 4 client.5184 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559edfd419b0 con 0x559edf9d1b10 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9d1b10 0x559edff348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9d1b10 0x559edff348a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1320c817cdde7436 server_cookie=e5b59d7dd61a7c89 in_seq=0 out_seq=0 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 10 client.5184 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 1 -- 172.21.15.43:0/3297481338 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559edfd419b0 con 0x559edf9d1b10 2024-09-17T05:24:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 10 client.5184 handle_command_reply: tid=0 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 client.5184 shutdown 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 msgr2=0x559edfee9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77c0001320 tx=0x7f77c005eb80 comp rx=0 tx=0).stop 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 shutdown_connections 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9d1b10 0x559edff348a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdce940 0x559edff1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edfdbace0 0x559edfee9a00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3297481338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdb49a0 0x559edfee94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77bbe00640 0 client.5184 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 >> 172.21.15.43:0/3297481338 conn(0x559edf9ed330 msgr2=0x559edfee2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 shutdown_connections 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3297481338 wait complete. 2024-09-17T05:24:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.558+0000 7f77e4ffa000 20 client.5184 trim_cache size 0 max 16384 2024-09-17T05:24:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 Processor -- start 2024-09-17T05:24:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 -- start start 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edff40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edfee0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8205240 con 0x559edf9ed330 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77d815b3f0 con 0x559edfdb49a0 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77d8140d90 con 0x559edfdbace0 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35990/0 (socket says 172.21.15.43:35990) 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 -- 172.21.15.43:0/3440924911 learned_addr learned my addr 172.21.15.43:0/3440924911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77cfe00640 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edff40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d7400640 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edfee0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 -- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 msgr2=0x559edfee0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edfee0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 -- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 msgr2=0x559edff40340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edff40340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 -- 172.21.15.43:0/3440924911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edfdb3de0 con 0x559edf9ed330 2024-09-17T05:24:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77d6a00640 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c801b620 tx=0x7f77c8006730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1ce5dedfff632d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77c6a00640 1 -- 172.21.15.43:0/3440924911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80d4020 con 0x559edf9ed330 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77c6a00640 1 -- 172.21.15.43:0/3440924911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c80038e0 con 0x559edf9ed330 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.578+0000 7f77c6a00640 1 -- 172.21.15.43:0/3440924911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003d10 con 0x559edf9ed330 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3440924911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 msgr2=0x559edff3fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f77c801b620 tx=0x7f77c8006730 comp rx=0 tx=0).stop 2024-09-17T05:24:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3440924911 shutdown_connections 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edff40340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edfee0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/3440924911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edff3fe00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3440924911 >> 172.21.15.43:0/3440924911 conn(0x559edfee9f80 msgr2=0x559edff1bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3440924911 shutdown_connections 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- 172.21.15.43:0/3440924911 wait complete. 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- start start 2024-09-17T05:24:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 Processor -- start 2024-09-17T05:24:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edfee4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edff5fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559edfcfaf60 con 0x559edf9ed330 2024-09-17T05:24:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559edfcfa780 con 0x559edfdb49a0 2024-09-17T05:24:28.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559edfd31fb0 con 0x559edfdbace0 2024-09-17T05:24:28.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77e4ffa000 10 client.0 _ll_register_callbacks cb 0x7f77c005b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:28.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36004/0 (socket says 172.21.15.43:36004) 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77d6a00640 1 -- 172.21.15.43:0/1007208087 learned_addr learned my addr 172.21.15.43:0/1007208087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77cfe00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edfee4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.582+0000 7f77d7400640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edff5fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 -- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 msgr2=0x559edff5fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edff5fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 -- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 msgr2=0x559edfee4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edfee4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 -- 172.21.15.43:0/1007208087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edfcc5640 con 0x559edf9ed330 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77d6a00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001930 tx=0x7f77c8006510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aef4598d9f226f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80d4070 con 0x559edf9ed330 2024-09-17T05:24:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f77c8003c90 con 0x559edf9ed330 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c80038e0 con 0x559edf9ed330 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f77c80b9460 con 0x559edf9ed330 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 10 client.5190 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x559edfcf7b10 con 0x559edf9ed330 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c80038e0 con 0x559edf9ed330 2024-09-17T05:24:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f77c80b9600 con 0x559edf9ed330 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 10 client.5190 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 10 client.5190 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 20 client.5190 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9eb1c0 0x559edff607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 4 client.5190 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x559edfd3d460 con 0x559edf9eb1c0 2024-09-17T05:24:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77cfe00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9eb1c0 0x559edff607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77cfe00640 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9eb1c0 0x559edff607a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410cbdc152c9e61c server_cookie=a2fd5e02445ae6f4 in_seq=0 out_seq=0 2024-09-17T05:24:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 10 client.5190 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 1 -- 172.21.15.43:0/1007208087 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559edfd3d460 con 0x559edf9eb1c0 2024-09-17T05:24:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 10 client.5190 handle_command_reply: tid=0 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 client.5190 shutdown 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 msgr2=0x559edfee4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001930 tx=0x7f77c8006510 comp rx=0 tx=0).stop 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 shutdown_connections 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x559edf9eb1c0 0x559edff607a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559edfdb49a0 0x559edfee4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559edfdbace0 0x559edff5fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77bbe00640 0 client.5190 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 --2- 172.21.15.43:0/1007208087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559edf9ed330 0x559edfee4240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 >> 172.21.15.43:0/1007208087 conn(0x559edfee9f80 msgr2=0x559edff3fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 shutdown_connections 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 1 -- 172.21.15.43:0/1007208087 wait complete. 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.586+0000 7f77e4ffa000 20 client.5190 trim_cache size 0 max 16384 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-17T05:24:28.596 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e7d18c5a-e367-435b-add6-01539dfc7b90", 2024-09-17T05:24:28.597 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-17T05:24:28.597 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 msgr2=0x7f77b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 0x7f77b4075740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77c801b7f0 tx=0x7f77c8019d60 comp rx=0 tx=0).stop 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 msgr2=0x7f77d8099a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 0x7f77d8099a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f77c00011b0 tx=0x7f77c0001720 comp rx=0 tx=0).stop 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 shutdown_connections 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77b4073230 0x7f77b4075740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77d81415e0 0x7f77d81419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d8151e60 0x7f77d8099a90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 --2- 172.21.15.43:0/2787701187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77bc003af0 0x7f77d8152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 >> 172.21.15.43:0/2787701187 conn(0x7f77d806ce60 msgr2=0x7f77d8075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 shutdown_connections 2024-09-17T05:24:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.590+0000 7f77dd800640 1 -- 172.21.15.43:0/2787701187 wait complete. 2024-09-17T05:24:28.611 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "e7d18c5a-e367-435b-add6-01539dfc7b90", "mode": "asynchronous" } 2024-09-17T05:24:28.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:28.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.770+0000 7fc5d3c00640 1 -- 172.21.15.43:0/3408797344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc0b3af0 msgr2=0x7fc5cc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.770+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/3408797344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc0b3af0 0x7fc5cc15c7e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009870 tx=0x7fc5b801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.770+0000 7fc5d3c00640 1 -- 172.21.15.43:0/3408797344 shutdown_connections 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.770+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/3408797344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc0b3af0 0x7fc5cc15c7e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.770+0000 7fc5d3c00640 1 -- 172.21.15.43:0/3408797344 >> 172.21.15.43:0/3408797344 conn(0x7fc5cc0b23d0 msgr2=0x7fc5cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- 172.21.15.43:0/3408797344 shutdown_connections 2024-09-17T05:24:28.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- 172.21.15.43:0/3408797344 wait complete. 2024-09-17T05:24:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 Processor -- start 2024-09-17T05:24:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- start start 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 0x7fc5cc1fc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc155d70 0x7fc5cc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc169ef0 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc169d70 con 0x7fc5cc06f110 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.774+0000 7fc5d3c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc16a070 con 0x7fc5cc155d70 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36020/0 (socket says 172.21.15.43:36020) 2024-09-17T05:24:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 -- 172.21.15.43:0/282285039 learned_addr learned my addr 172.21.15.43:0/282285039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1200640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 0x7fc5cc1fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc155d70 0x7fc5cc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 -- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc155d70 msgr2=0x7fc5cc153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc155d70 0x7fc5cc153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 -- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 msgr2=0x7fc5cc1fc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 0x7fc5cc1fc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 -- 172.21.15.43:0/282285039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1fbc70 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1200640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 0x7fc5cc1fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400cae0 tx=0x7fc5c400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5c8c00640 1 -- 172.21.15.43:0/282285039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4016020 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5c8c00640 1 -- 172.21.15.43:0/282285039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c400aae0 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc0cfc90 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5c8c00640 1 -- 172.21.15.43:0/282285039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c400ce20 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc0cff70 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.778+0000 7fc5c8c00640 1 -- 172.21.15.43:0/282285039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5c4019020 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5c8c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 0x7fc5ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5c8c00640 1 -- 172.21.15.43:0/282285039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc5c4094170 con 0x7fc5cc0c72b0 2024-09-17T05:24:28.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 0x7fc5ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 Processor -- start 2024-09-17T05:24:28.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 -- start start 2024-09-17T05:24:28.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 0x7fc5ac075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0000c10 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f6002e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6002e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f600182cd0 con 0x55f6001776d0 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f600098330 con 0x55f5ffe92aa0 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f6000f5bb0 con 0x55f5ffdce890 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36036/0 (socket says 172.21.15.43:36036) 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 -- 172.21.15.43:0/3669377624 learned_addr learned my addr 172.21.15.43:0/3669377624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f6002e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6002e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 -- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 msgr2=0x55f6002e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f6002e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 -- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 msgr2=0x55f6002e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6002e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 -- 172.21.15.43:0/3669377624 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f600099c80 con 0x55f6001776d0 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6002e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f6002e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.782+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000fd90 tx=0x7fc5c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932358d51eaf0f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5bd600640 1 -- 172.21.15.43:0/3669377624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0025070 con 0x55f6001776d0 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5bd600640 1 -- 172.21.15.43:0/3669377624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c001b050 con 0x55f6001776d0 2024-09-17T05:24:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5bd600640 1 -- 172.21.15.43:0/3669377624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0020610 con 0x55f6001776d0 2024-09-17T05:24:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- 172.21.15.43:0/3669377624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 msgr2=0x55f6002e39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000fd90 tx=0x7fc5c0012000 comp rx=0 tx=0).stop 2024-09-17T05:24:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- 172.21.15.43:0/3669377624 shutdown_connections 2024-09-17T05:24:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6002e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f6002e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3669377624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002e39f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- 172.21.15.43:0/3669377624 >> 172.21.15.43:0/3669377624 conn(0x55f5ffda25c0 msgr2=0x55f6002a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- 172.21.15.43:0/3669377624 shutdown_connections 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- 172.21.15.43:0/3669377624 wait complete. 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 -- start start 2024-09-17T05:24:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 Processor -- start 2024-09-17T05:24:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffdce890 0x55f60029fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.786+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffe92aa0 0x55f60029ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f6000e8df0 con 0x55f6001776d0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f6000e4fe0 con 0x55f5ffdce890 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f60008a4d0 con 0x55f5ffe92aa0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 10 client.0 _ll_register_callbacks cb 0x55f6000687f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36052/0 (socket says 172.21.15.43:36052) 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 -- 172.21.15.43:0/3898421996 learned_addr learned my addr 172.21.15.43:0/3898421996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffdce890 0x55f60029fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffe92aa0 0x55f60029ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 -- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffe92aa0 msgr2=0x55f60029ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffe92aa0 0x55f60029ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 -- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffdce890 msgr2=0x55f60029fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffdce890 0x55f60029fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 -- 172.21.15.43:0/3898421996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f600083210 con 0x55f6001776d0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4001ce0 tx=0x7fc5c405e260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2a277a6a63cb90 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c405a860 con 0x55f6001776d0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f60007d350 con 0x55f6001776d0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc5c407eda0 con 0x55f6001776d0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc5c4078e60 con 0x55f6001776d0 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 10 client.5202 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f600069fc0 con 0x55f6001776d0 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f60007d350 con 0x55f6001776d0 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fc5c405dc20 con 0x55f6001776d0 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 10 client.5202 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 10 client.5202 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 20 client.5202 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd87930 0x55f6002eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 4 client.5202 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6000f7b00 con 0x55f5ffd87930 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd87930 0x55f6002eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5d1200640 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd87930 0x55f6002eae80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709062b66c41a7f6 server_cookie=9562a3ec28678afe in_seq=0 out_seq=0 2024-09-17T05:24:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 10 client.5202 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 1 -- 172.21.15.43:0/3898421996 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6000f7b00 con 0x55f5ffd87930 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 10 client.5202 handle_command_reply: tid=0 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 client.5202 shutdown 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 msgr2=0x55f6002a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4001ce0 tx=0x7fc5c405e260 comp rx=0 tx=0).stop 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 shutdown_connections 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd87930 0x55f6002eae80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6002a65f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffe92aa0 0x55f60029ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 --2- 172.21.15.43:0/3898421996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffdce890 0x55f60029fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5b2000640 0 client.5202 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.790+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 >> 172.21.15.43:0/3898421996 conn(0x55f5ffda25c0 msgr2=0x55f600298d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.794+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 shutdown_connections 2024-09-17T05:24:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.794+0000 7fc5db19c000 1 -- 172.21.15.43:0/3898421996 wait complete. 2024-09-17T05:24:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.794+0000 7fc5db19c000 20 client.5202 trim_cache size 0 max 16384 2024-09-17T05:24:28.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.810+0000 7fc5db19c000 1 Processor -- start 2024-09-17T05:24:28.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.810+0000 7fc5db19c000 1 -- start start 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600354de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f600355320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f60029eb20 con 0x55f6001776d0 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc205380 con 0x55f5ffe92aa0 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc154940 con 0x55f5ffdce890 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36064/0 (socket says 172.21.15.43:36064) 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 -- 172.21.15.43:0/2159734404 learned_addr learned my addr 172.21.15.43:0/2159734404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1200640 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600354de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f600355320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 -- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 msgr2=0x55f600354de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600354de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 -- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 msgr2=0x55f600355320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f600355320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 -- 172.21.15.43:0/2159734404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f600142680 con 0x55f6001776d0 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc5c405e0c0 tx=0x7fc5c405eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8eb4ff45dfc470 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5bcc00640 1 -- 172.21.15.43:0/2159734404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c406c020 con 0x55f6001776d0 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5bcc00640 1 -- 172.21.15.43:0/2159734404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c4063070 con 0x55f6001776d0 2024-09-17T05:24:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5bcc00640 1 -- 172.21.15.43:0/2159734404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c407ede0 con 0x55f6001776d0 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- 172.21.15.43:0/2159734404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 msgr2=0x55f600355860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc5c405e0c0 tx=0x7fc5c405eb00 comp rx=0 tx=0).stop 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- 172.21.15.43:0/2159734404 shutdown_connections 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f600355320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f600355860 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 --2- 172.21.15.43:0/2159734404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600354de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- 172.21.15.43:0/2159734404 >> 172.21.15.43:0/2159734404 conn(0x55f5ffda25c0 msgr2=0x55f6002f4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- 172.21.15.43:0/2159734404 shutdown_connections 2024-09-17T05:24:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- 172.21.15.43:0/2159734404 wait complete. 2024-09-17T05:24:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 -- start start 2024-09-17T05:24:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.814+0000 7fc5db19c000 1 Processor -- start 2024-09-17T05:24:28.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600358f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6003594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc1542f0 con 0x55f6001776d0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc09abe0 con 0x55f5ffe92aa0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f5ffdaa680 con 0x55f5ffdce890 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 10 client.0 _ll_register_callbacks cb 0x7fc5c4001c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600358f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36072/0 (socket says 172.21.15.43:36072) 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 -- 172.21.15.43:0/1982776158 learned_addr learned my addr 172.21.15.43:0/1982776158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5cbe00640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6003594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 -- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 msgr2=0x55f600358f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600358f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 -- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 msgr2=0x55f6003594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6003594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 -- 172.21.15.43:0/1982776158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6000af0b0 con 0x55f6001776d0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1c00640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4001ce0 tx=0x7fc5c407edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b0159e5df80ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c406c040 con 0x55f6001776d0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c4063070 con 0x55f6001776d0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc5c4071b10 con 0x55f6001776d0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc5c40633f0 con 0x55f6001776d0 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 10 client.5208 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55f60007b610 con 0x55f6001776d0 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c4063070 con 0x55f6001776d0 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fc5c4074ac0 con 0x55f6001776d0 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 10 client.5208 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 10 client.5208 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 20 client.5208 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd75990 0x55f600314830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:28.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 4 client.5208 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6000f35b0 con 0x55f5ffd75990 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1200640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd75990 0x55f600314830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 10 client.5208 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5d1200640 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd75990 0x55f600314830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffbab6a07776ebf server_cookie=57d6b7b544145baa in_seq=0 out_seq=0 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 1 -- 172.21.15.43:0/1982776158 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+573 (crc 0 0 0) 0x55f6000f35b0 con 0x55f5ffd75990 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5b2000640 10 client.5208 handle_command_reply: tid=0 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.818+0000 7fc5db19c000 1 client.5208 shutdown 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 msgr2=0x55f6003599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4001ce0 tx=0x7fc5c407edd0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 shutdown_connections 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x55f5ffd75990 0x55f600314830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f5ffe92aa0 0x55f6003594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f6001776d0 0x55f6003599f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 --2- 172.21.15.43:0/1982776158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f5ffdce890 0x55f600358f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5b2000640 0 client.5208 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 >> 172.21.15.43:0/1982776158 conn(0x55f5ffda25c0 msgr2=0x55f6002a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 shutdown_connections 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 1 -- 172.21.15.43:0/1982776158 wait complete. 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5db19c000 20 client.5208 trim_cache size 0 max 16384 2024-09-17T05:24:28.826 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "e7d18c5a-e367-435b-add6-01539dfc7b90": { 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir", 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e7d18c5a-e367-435b-add6-01539dfc7b90", 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "repair" 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "e7d18c5a-e367-435b-add6-01539dfc7b90": { 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-09-17T05:24:28.586022+0000", 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir", 2024-09-17T05:24:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 0, 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 0, 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 msgr2=0x7fc5ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 0x7fc5ac075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0000c10 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 msgr2=0x7fc5cc0c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400cae0 tx=0x7fc5c400cbd0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 shutdown_connections 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ac073230 0x7fc5ac075740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5cc0c72b0 0x7fc5cc0c8cf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5cc155d70 0x7fc5cc153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 --2- 172.21.15.43:0/282285039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5cc06f110 0x7fc5cc1fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:28.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 >> 172.21.15.43:0/282285039 conn(0x7fc5cc0b23d0 msgr2=0x7fc5cc0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:28.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 shutdown_connections 2024-09-17T05:24:28.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:28.822+0000 7fc5d3c00640 1 -- 172.21.15.43:0/282285039 wait complete. 2024-09-17T05:24:28.843 INFO:tasks.cephfs.filesystem:scrub status for tag:e7d18c5a-e367-435b-add6-01539dfc7b90 - {'path': '/subdir', 'tag': 'e7d18c5a-e367-435b-add6-01539dfc7b90', 'options': 'repair'} 2024-09-17T05:24:50.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:50.740 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:58.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:58.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-17T05:24:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- 172.21.15.43:0/1765203185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fe10 msgr2=0x7f43740b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 --2- 172.21.15.43:0/1765203185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fe10 0x7f43740b4b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4364009870 tx=0x7f436401c700 comp rx=0 tx=0).stop 2024-09-17T05:24:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- 172.21.15.43:0/1765203185 shutdown_connections 2024-09-17T05:24:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 --2- 172.21.15.43:0/1765203185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f437415fe10 0x7f43740b4b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- 172.21.15.43:0/1765203185 >> 172.21.15.43:0/1765203185 conn(0x7f43740b23d0 msgr2=0x7f43740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- 172.21.15.43:0/1765203185 shutdown_connections 2024-09-17T05:24:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- 172.21.15.43:0/1765203185 wait complete. 2024-09-17T05:24:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 Processor -- start 2024-09-17T05:24:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.017+0000 7f4378c00640 1 -- start start 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f437406fc90 0x7f4374156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f437415fe10 0x7f43740afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f437416aff0 con 0x7f43741570f0 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f437416ae70 con 0x7f437415fe10 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f437416b170 con 0x7f437406fc90 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45780/0 (socket says 172.21.15.43:45780) 2024-09-17T05:24:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 -- 172.21.15.43:0/160648702 learned_addr learned my addr 172.21.15.43:0/160648702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4371e00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f437406fc90 0x7f4374156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f436be00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f437415fe10 0x7f43740afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 -- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f437406fc90 msgr2=0x7f4374156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f437406fc90 0x7f4374156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 -- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f437415fe10 msgr2=0x7f43740afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f437415fe10 0x7f43740afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 -- 172.21.15.43:0/160648702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437415f5d0 con 0x7f43741570f0 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4372800640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c760 tx=0x7f436c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f436a000640 1 -- 172.21.15.43:0/160648702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c003db0 con 0x7f43741570f0 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f436a000640 1 -- 172.21.15.43:0/160648702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436c00ce10 con 0x7f43741570f0 2024-09-17T05:24:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43740747c0 con 0x7f43741570f0 2024-09-17T05:24:59.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f436a000640 1 -- 172.21.15.43:0/160648702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0052d0 con 0x7f43741570f0 2024-09-17T05:24:59.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.021+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43740cb650 con 0x7f43741570f0 2024-09-17T05:24:59.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f436a000640 1 -- 172.21.15.43:0/160648702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f436c005470 con 0x7f43741570f0 2024-09-17T05:24:59.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f436a000640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 0x7f4350075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f436a000640 1 -- 172.21.15.43:0/160648702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f436c093e60 con 0x7f43741570f0 2024-09-17T05:24:59.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f438045d000 1 Processor -- start 2024-09-17T05:24:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f4371e00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 0x7f4350075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f438045d000 1 -- start start 2024-09-17T05:24:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.025+0000 7f4371e00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 0x7f4350075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43640af000 tx=0x7f436401af40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9fef6d0 0x558dda15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558dd9ffacd0 con 0x558dd9c46890 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558dd9f10330 con 0x558dd9fef6d0 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558dd9f6dbb0 con 0x558dd9d0aaa0 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45782/0 (socket says 172.21.15.43:45782) 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 -- 172.21.15.43:0/2470333637 learned_addr learned my addr 172.21.15.43:0/2470333637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f4371e00640 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9fef6d0 0x558dda15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f4372800640 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 -- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 msgr2=0x558dda15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 -- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9fef6d0 msgr2=0x558dda15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9fef6d0 0x558dda15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 -- 172.21.15.43:0/2470333637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd9f11c80 con 0x558dd9c46890 2024-09-17T05:24:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f436be00640 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f435c0017d0 tx=0x7f435c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cada269e44df22e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f4362600640 1 -- 172.21.15.43:0/2470333637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0052e0 con 0x558dd9c46890 2024-09-17T05:24:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f4362600640 1 -- 172.21.15.43:0/2470333637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f435c018070 con 0x558dd9c46890 2024-09-17T05:24:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f4362600640 1 -- 172.21.15.43:0/2470333637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00f9b0 con 0x558dd9c46890 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- 172.21.15.43:0/2470333637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 msgr2=0x558dda15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f435c0017d0 tx=0x7f435c007e00 comp rx=0 tx=0).stop 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- 172.21.15.43:0/2470333637 shutdown_connections 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda15bf30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 --2- 172.21.15.43:0/2470333637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9fef6d0 0x558dda15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- 172.21.15.43:0/2470333637 >> 172.21.15.43:0/2470333637 conn(0x558dd9c1a5c0 msgr2=0x558dda118ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- 172.21.15.43:0/2470333637 shutdown_connections 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- 172.21.15.43:0/2470333637 wait complete. 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 -- start start 2024-09-17T05:24:59.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.029+0000 7f438045d000 1 Processor -- start 2024-09-17T05:24:59.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda117a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda117f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558dd9f60df0 con 0x558dd9fef6d0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558dd9f5cfe0 con 0x558dd9c46890 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558dd9f024d0 con 0x558dd9d0aaa0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 10 client.0 _ll_register_callbacks cb 0x558dd9ee07f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45786/0 (socket says 172.21.15.43:45786) 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 -- 172.21.15.43:0/388654435 learned_addr learned my addr 172.21.15.43:0/388654435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4371e00640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda117a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f436be00640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda117f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 -- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 msgr2=0x558dda117f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda117f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 -- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 msgr2=0x558dda117a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda117a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 -- 172.21.15.43:0/388654435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd9efb210 con 0x558dd9fef6d0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f436be00640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda117f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4372800640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f436c002240 tx=0x7f436c05e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615e557d6b214cf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c063280 con 0x558dd9fef6d0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f436c05dab0 con 0x558dd9fef6d0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558dd9ef5350 con 0x558dd9fef6d0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f436c07f050 con 0x558dd9fef6d0 2024-09-17T05:24:59.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 10 client.5223 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x558dd9ee1fc0 con 0x558dd9fef6d0 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558dd9ef5350 con 0x558dd9fef6d0 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f436c05a1a0 con 0x558dd9fef6d0 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 10 client.5223 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 10 client.5223 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 20 client.5223 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bff930 0x558dda162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 4 client.5223 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dd9f6fb00 con 0x558dd9bff930 2024-09-17T05:24:59.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4371e00640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bff930 0x558dda162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4356a00640 10 client.5223 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.033+0000 7f4371e00640 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bff930 0x558dda162e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efeab250731a8500 server_cookie=defc99e6b7c926fe in_seq=0 out_seq=0 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f4356a00640 1 -- 172.21.15.43:0/388654435 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558dd9f6fb00 con 0x558dd9bff930 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f4356a00640 10 client.5223 handle_command_reply: tid=0 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 client.5223 shutdown 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 msgr2=0x558dda11e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f436c002240 tx=0x7f436c05e080 comp rx=0 tx=0).stop 2024-09-17T05:24:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 shutdown_connections 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bff930 0x558dda162e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9fef6d0 0x558dda11e5f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9d0aaa0 0x558dda117f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 --2- 172.21.15.43:0/388654435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda117a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 >> 172.21.15.43:0/388654435 conn(0x558dd9c1a5c0 msgr2=0x558dda110d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f4356a00640 0 client.5223 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 shutdown_connections 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 1 -- 172.21.15.43:0/388654435 wait complete. 2024-09-17T05:24:59.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.037+0000 7f438045d000 20 client.5223 trim_cache size 0 max 16384 2024-09-17T05:24:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.053+0000 7f438045d000 1 Processor -- start 2024-09-17T05:24:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- start start 2024-09-17T05:24:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda1ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558dda116b20 con 0x558dd9d0aaa0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43742053e0 con 0x558dd9c46890 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4374204460 con 0x558dd9fef6d0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45794/0 (socket says 172.21.15.43:45794) 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 -- 172.21.15.43:0/4250087700 learned_addr learned my addr 172.21.15.43:0/4250087700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f4372800640 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f4371e00640 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda1ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 -- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 msgr2=0x558dda1cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 -- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 msgr2=0x558dda1ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda1ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 -- 172.21.15.43:0/4250087700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dd9fba680 con 0x558dd9d0aaa0 2024-09-17T05:24:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f436be00640 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x558dda010760 tx=0x7f435c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3906b3af69e37d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f4361c00640 1 -- 172.21.15.43:0/4250087700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0069e0 con 0x558dd9d0aaa0 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f4361c00640 1 -- 172.21.15.43:0/4250087700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f435c021070 con 0x558dd9d0aaa0 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f4361c00640 1 -- 172.21.15.43:0/4250087700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c006e40 con 0x558dd9d0aaa0 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- 172.21.15.43:0/4250087700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 msgr2=0x558dda1cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x558dda010760 tx=0x7f435c001090 comp rx=0 tx=0).stop 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- 172.21.15.43:0/4250087700 shutdown_connections 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9d0aaa0 0x558dda1cd110 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 --2- 172.21.15.43:0/4250087700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9c46890 0x558dda1ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- 172.21.15.43:0/4250087700 >> 172.21.15.43:0/4250087700 conn(0x558dd9c1a5c0 msgr2=0x558dda16cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- 172.21.15.43:0/4250087700 shutdown_connections 2024-09-17T05:24:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.057+0000 7f438045d000 1 -- 172.21.15.43:0/4250087700 wait complete. 2024-09-17T05:24:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- start start 2024-09-17T05:24:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 Processor -- start 2024-09-17T05:24:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9d0aaa0 0x558dda1d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43740c9300 con 0x558dd9c46890 2024-09-17T05:24:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43740c8cb0 con 0x558dd9d0aaa0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558dd9c22680 con 0x558dd9fef6d0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 10 client.0 _ll_register_callbacks cb 0x7f436c00c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45804/0 (socket says 172.21.15.43:45804) 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 -- 172.21.15.43:0/1496649382 learned_addr learned my addr 172.21.15.43:0/1496649382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f436be00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9d0aaa0 0x558dda1d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4372800640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 -- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 msgr2=0x558dda1d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 -- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9d0aaa0 msgr2=0x558dda1d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9d0aaa0 0x558dda1d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 -- 172.21.15.43:0/1496649382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dd9f270b0 con 0x558dd9c46890 2024-09-17T05:24:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4371e00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43640b0040 tx=0x7f43640091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc5f6761c595b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4356a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:24:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43640043e0 con 0x558dd9c46890 2024-09-17T05:24:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4364003a50 con 0x558dd9c46890 2024-09-17T05:24:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f435c021070 con 0x558dd9c46890 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f43640dd050 con 0x558dd9c46890 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 10 client.5229 fetch_fsmap learned FSMap version 23 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.061+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x558dd9ef3610 con 0x558dd9c46890 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f435c021070 con 0x558dd9c46890 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f43640e0020 con 0x558dd9c46890 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 10 client.5229 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 10 client.5229 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 20 client.5229 populate_metadata read hostname 'smithi043' 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bed990 0x558dda18cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 4 client.5229 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dd9f6b5b0 con 0x558dd9bed990 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f436be00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bed990 0x558dda18cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 10 client.5229 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f436be00640 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bed990 0x558dda18cbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb21b9ecda1cee8 server_cookie=ebba76b88eaa7ec4 in_seq=0 out_seq=0 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 1 -- 172.21.15.43:0/1496649382 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x558dd9f6b5b0 con 0x558dd9bed990 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 10 client.5229 handle_command_reply: tid=0 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 client.5229 shutdown 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 msgr2=0x558dda1d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43640b0040 tx=0x7f43640091e0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 shutdown_connections 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x558dd9bed990 0x558dda18cbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558dd9d0aaa0 0x558dda1d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558dd9fef6d0 0x558dda1d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 --2- 172.21.15.43:0/1496649382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558dd9c46890 0x558dda1d0e20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4356a00640 0 client.5229 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 >> 172.21.15.43:0/1496649382 conn(0x558dd9c1a5c0 msgr2=0x558dda11d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 shutdown_connections 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 1 -- 172.21.15.43:0/1496649382 wait complete. 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f438045d000 20 client.5229 trim_cache size 0 max 16384 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-17T05:24:59.072 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {}, 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "e7d18c5a-e367-435b-add6-01539dfc7b90": { 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-09-17T05:24:28.586022+0000", 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir", 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 0, 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 0, 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-09-17T05:24:59.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:59.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:24:59.074 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:24:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 msgr2=0x7f4350075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 0x7f4350075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43640af000 tx=0x7f436401af40 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 msgr2=0x7f43740b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c760 tx=0x7f436c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 shutdown_connections 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4350073200 0x7f4350075710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f437415fe10 0x7f43740afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43741570f0 0x7f43740b0280 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 --2- 172.21.15.43:0/160648702 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f437406fc90 0x7f4374156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 >> 172.21.15.43:0/160648702 conn(0x7f43740b23d0 msgr2=0x7f43740c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 shutdown_connections 2024-09-17T05:24:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.065+0000 7f4378c00640 1 -- 172.21.15.43:0/160648702 wait complete. 2024-09-17T05:24:59.089 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T05:24:59.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 -- 172.21.15.43:0/786360171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa92415ee30 msgr2=0x7fa9240b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 --2- 172.21.15.43:0/786360171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa92415ee30 0x7fa9240b13a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b0130 tx=0x7fa91801c800 comp rx=0 tx=0).stop 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 -- 172.21.15.43:0/786360171 shutdown_connections 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 --2- 172.21.15.43:0/786360171 >> v2:172.21.15.62:3300/0 conn(0x7fa9240b69f0 0x7fa9240d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 --2- 172.21.15.43:0/786360171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa92415ee30 0x7fa9240b13a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 -- 172.21.15.43:0/786360171 >> 172.21.15.43:0/786360171 conn(0x7fa9240b2e80 msgr2=0x7fa9240b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 -- 172.21.15.43:0/786360171 shutdown_connections 2024-09-17T05:24:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.245+0000 7fa92be00640 1 -- 172.21.15.43:0/786360171 wait complete. 2024-09-17T05:24:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 Processor -- start 2024-09-17T05:24:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- start start 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 0x7fa924157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 0x7fa924141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa92416c3b0 con 0x7fa9240b18e0 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa92416c230 con 0x7fa92415ee30 2024-09-17T05:24:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa92416c530 con 0x7fa9240b69f0 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45816/0 (socket says 172.21.15.43:45816) 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 -- 172.21.15.43:0/124850155 learned_addr learned my addr 172.21.15.43:0/124850155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 -- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 msgr2=0x7fa924157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa928a00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 0x7fa924157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929e00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 0x7fa924141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 0x7fa924157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 -- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 msgr2=0x7fa924141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 0x7fa924141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 -- 172.21.15.43:0/124850155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9241fc600 con 0x7fa9240b18e0 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa928a00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 0x7fa924157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929e00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 0x7fa924141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa929400640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa9180982f0 tx=0x7fa918005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180b2040 con 0x7fa9240b18e0 2024-09-17T05:24:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9180bb070 con 0x7fa9240b18e0 2024-09-17T05:24:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9241fc8e0 con 0x7fa9240b18e0 2024-09-17T05:24:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918003600 con 0x7fa9240b18e0 2024-09-17T05:24:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.249+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9241fcea0 con 0x7fa9240b18e0 2024-09-17T05:24:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9240d0470 con 0x7fa9240b18e0 2024-09-17T05:24:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa9180037a0 con 0x7fa9240b18e0 2024-09-17T05:24:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa921600640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 0x7fa904075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fa91812e650 con 0x7fa9240b18e0 2024-09-17T05:24:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa928a00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 0x7fa904075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.253+0000 7fa928a00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 0x7fa904075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9240721d0 tx=0x7fa910009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.257+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa9180163f0 con 0x7fa9240b18e0 2024-09-17T05:24:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9240b3d50 con 0x7fa9240b18e0 2024-09-17T05:24:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa921600640 1 -- 172.21.15.43:0/124850155 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fa9180f9690 con 0x7fa9240b18e0 2024-09-17T05:24:59.473 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 msgr2=0x7fa904075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 0x7fa904075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9240721d0 tx=0x7fa910009040 comp rx=0 tx=0).stop 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 msgr2=0x7fa924157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa9180982f0 tx=0x7fa918005d20 comp rx=0 tx=0).stop 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 shutdown_connections 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa904073200 0x7fa904075710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa92415ee30 0x7fa924141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa9240b69f0 0x7fa924157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 --2- 172.21.15.43:0/124850155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9240b18e0 0x7fa924157100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 >> 172.21.15.43:0/124850155 conn(0x7fa9240b2e80 msgr2=0x7fa924070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 shutdown_connections 2024-09-17T05:24:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59.469+0000 7fa92be00640 1 -- 172.21.15.43:0/124850155 wait complete. 2024-09-17T05:24:59.494 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:24:59.495 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:59.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.496 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:24:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42344sec preferred_lft 42344sec 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:24:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.507 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:24:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:24:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:24:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:24:59.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.509 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:24:59.569 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:24:59.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.571 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:24:59.624 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:24:59.624 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:24:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:24:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:24:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:24:59.625 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:24:59.625 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:24:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:24:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:24:59.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.767 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:24:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:24:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:24:59.777 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:59.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:24:59.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:59.843 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:59.895 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:24:59.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:24:59.895 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:24:59.946 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:24:59.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:24:59.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:00.048 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:00.048 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:00.048 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:00.066 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:00.066 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:00.067 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:00.067 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:25:00.067 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:00.076 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:25:00.077 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:25:00.080 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:00.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:00.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:00.100 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:00.200 INFO:teuthology.orchestra.run.smithi043.stdout:client4976 2024-09-17T05:25:00.200 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:00.200 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:00.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:00.209 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-17T05:25:00.260 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:25:00.260 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:25:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.357+0000 7f846df23080 1 Processor -- start 2024-09-17T05:25:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.357+0000 7f846df23080 1 -- start start 2024-09-17T05:25:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 0x55f8098d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098d75b0 0x55f809861a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f80976ea00 con 0x55f809861f40 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8097ccf40 con 0x55f8098259c0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f809866e50 con 0x55f8098d75b0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 0x55f8098d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45842/0 (socket says 172.21.15.43:45842) 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467a00640 1 -- 172.21.15.43:0/1104087342 learned_addr learned my addr 172.21.15.43:0/1104087342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8466600640 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098d75b0 0x55f809861a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 -- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098d75b0 msgr2=0x55f809861a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098d75b0 0x55f809861a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 -- 172.21.15.43:0/1104087342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 msgr2=0x55f809989ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 -- 172.21.15.43:0/1104087342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f809747c10 con 0x55f8098259c0 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467a00640 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8467000640 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 0x55f8098d20b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55f809867fc0 tx=0x7f8458017ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6050b165221be2f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8465200640 1 -- 172.21.15.43:0/1104087342 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580ab570 con 0x55f8098259c0 2024-09-17T05:25:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8465200640 1 -- 172.21.15.43:0/1104087342 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84580aba70 con 0x55f8098259c0 2024-09-17T05:25:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f8465200640 1 -- 172.21.15.43:0/1104087342 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580abc80 con 0x55f8098259c0 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 msgr2=0x55f8098d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 0x55f8098d20b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55f809867fc0 tx=0x7f8458017ef0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- 172.21.15.43:0/1104087342 shutdown_connections 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f809989ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098d75b0 0x55f809861a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 --2- 172.21.15.43:0/1104087342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098259c0 0x55f8098d20b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- 172.21.15.43:0/1104087342 >> 172.21.15.43:0/1104087342 conn(0x55f80980cd80 msgr2=0x55f80986a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- 172.21.15.43:0/1104087342 shutdown_connections 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.361+0000 7f846df23080 1 -- 172.21.15.43:0/1104087342 wait complete. 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 Processor -- start 2024-09-17T05:25:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- start start 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098259c0 0x55f80986a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f8098fe610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098d75b0 0x55f809860f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8098d6ea0 con 0x55f809861f40 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8098d2850 con 0x55f8098d75b0 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f8098d6c00 con 0x55f8098259c0 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f8098fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f8098fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45850/0 (socket says 172.21.15.43:45850) 2024-09-17T05:25:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 -- 172.21.15.43:0/4210138808 learned_addr learned my addr 172.21.15.43:0/4210138808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8467a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098d75b0 0x55f809860f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8467000640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098259c0 0x55f80986a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 -- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098259c0 msgr2=0x55f80986a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8098259c0 0x55f80986a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 -- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098d75b0 msgr2=0x55f809860f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098d75b0 0x55f809860f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 -- 172.21.15.43:0/4210138808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8098fdc30 con 0x55f809861f40 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8467a00640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8098d75b0 0x55f809860f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f8466600640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f8098fe610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8454002f90 tx=0x7f845400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f845f400640 1 -- 172.21.15.43:0/4210138808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454015020 con 0x55f809861f40 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f845f400640 1 -- 172.21.15.43:0/4210138808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f845400ce40 con 0x55f809861f40 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8098fded0 con 0x55f809861f40 2024-09-17T05:25:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f845f400640 1 -- 172.21.15.43:0/4210138808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540118b0 con 0x55f809861f40 2024-09-17T05:25:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.365+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80986a770 con 0x55f809861f40 2024-09-17T05:25:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f845f400640 1 -- 172.21.15.43:0/4210138808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8454018020 con 0x55f809861f40 2024-09-17T05:25:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f845f400640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8448071be0 0x7f84480740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f845f400640 1 -- 172.21.15.43:0/4210138808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f8454092fc0 con 0x55f809861f40 2024-09-17T05:25:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f8467000640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8448071be0 0x7f84480740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f846df23080 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f8098d13f0 0x55f809820360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34) -- 0x55f8098208a0 con 0x55f8098d13f0 2024-09-17T05:25:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f8467a00640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f8098d13f0 0x55f809820360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.369+0000 7f8467000640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8448071be0 0x7f84480740f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55f8098d5620 tx=0x7f84580b6040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f8467a00640 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f8098d13f0 0x55f809820360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f8467a00640 1 -- 172.21.15.43:0/4210138808 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v34'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8460002040 con 0x55f8098d13f0 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f8098d13f0 msgr2=0x55f809820360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f8098d13f0 0x55f809820360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8448071be0 msgr2=0x7f84480740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8448071be0 0x7f84480740f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55f8098d5620 tx=0x7f84580b6040 comp rx=0 tx=0).stop 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 msgr2=0x55f8098fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 --2- 172.21.15.43:0/4210138808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f809861f40 0x55f8098fe610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8454002f90 tx=0x7f845400c6d0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f8467a00640 1 -- 172.21.15.43:0/4210138808 reap_dead start 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 shutdown_connections 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 >> 172.21.15.43:0/4210138808 conn(0x55f80980cd80 msgr2=0x55f8098d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 shutdown_connections 2024-09-17T05:25:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.373+0000 7f846df23080 1 -- 172.21.15.43:0/4210138808 wait complete. 2024-09-17T05:25:00.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 -- 172.21.15.43:0/2891211077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 msgr2=0x7fb8a00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/2891211077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a00ad0b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0072b60 tx=0x7fb89801ccc0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 -- 172.21.15.43:0/2891211077 shutdown_connections 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/2891211077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a00ad0b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/2891211077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a006f110 0x7fb8a00b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/2891211077 >> v2:172.21.15.62:3300/0 conn(0x7fb8a0155d70 0x7fb8a015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 -- 172.21.15.43:0/2891211077 >> 172.21.15.43:0/2891211077 conn(0x7fb8a00b23d0 msgr2=0x7fb8a00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 -- 172.21.15.43:0/2891211077 shutdown_connections 2024-09-17T05:25:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.537+0000 7fb8a5a00640 1 -- 172.21.15.43:0/2891211077 wait complete. 2024-09-17T05:25:00.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 Processor -- start 2024-09-17T05:25:00.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 -- start start 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8a006f110 0x7fb8a0165780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 0x7fb8a015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a015f870 con 0x7fb8a00b3920 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a015f6f0 con 0x7fb8a006f110 2024-09-17T05:25:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a015f9f0 con 0x7fb8a0155d70 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45860/0 (socket says 172.21.15.43:45860) 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 -- 172.21.15.43:0/747351653 learned_addr learned my addr 172.21.15.43:0/747351653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb89f400640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 0x7fb8a015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb89ea00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8a006f110 0x7fb8a0165780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 -- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 msgr2=0x7fb8a015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 0x7fb8a015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 -- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8a006f110 msgr2=0x7fb8a0165780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8a006f110 0x7fb8a0165780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 -- 172.21.15.43:0/747351653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00cfd60 con 0x7fb8a00b3920 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb89f400640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 0x7fb8a015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:00.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb897e00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e9a0 tx=0x7fb88800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88800cd80 con 0x7fb8a00b3920 2024-09-17T05:25:00.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb888004020 con 0x7fb8a00b3920 2024-09-17T05:25:00.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888004450 con 0x7fb8a00b3920 2024-09-17T05:25:00.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.541+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a00732b0 con 0x7fb8a00b3920 2024-09-17T05:25:00.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a0073870 con 0x7fb8a00b3920 2024-09-17T05:25:00.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a00cdb60 con 0x7fb8a00b3920 2024-09-17T05:25:00.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb888003600 con 0x7fb8a00b3920 2024-09-17T05:25:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb896a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 0x7fb87c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fb888094a50 con 0x7fb8a00b3920 2024-09-17T05:25:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb89ea00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 0x7fb87c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.545+0000 7fb89ea00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 0x7fb87c075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb890009600 tx=0x7fb8900091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.549+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb88805f450 con 0x7fb8a00b3920 2024-09-17T05:25:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a0073040 con 0x7fb8a00b3920 2024-09-17T05:25:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb896a00640 1 -- 172.21.15.43:0/747351653 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7fb888063e80 con 0x7fb8a00b3920 2024-09-17T05:25:00.764 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:00.764 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:25:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:25:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 msgr2=0x7fb87c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 0x7fb87c075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb890009600 tx=0x7fb8900091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 msgr2=0x7fb8a015a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e9a0 tx=0x7fb88800ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 shutdown_connections 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb87c073200 0x7fb87c075710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb8a0155d70 0x7fb8a015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a00b3920 0x7fb8a015a710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 --2- 172.21.15.43:0/747351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8a006f110 0x7fb8a0165780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 >> 172.21.15.43:0/747351653 conn(0x7fb8a00b23d0 msgr2=0x7fb8a00ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 shutdown_connections 2024-09-17T05:25:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.761+0000 7fb8a5a00640 1 -- 172.21.15.43:0/747351653 wait complete. 2024-09-17T05:25:00.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 204375696 2024-09-17T05:25:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- 172.21.15.43:0/2713774903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea015f470 msgr2=0x7f3ea00d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 --2- 172.21.15.43:0/2713774903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea015f470 0x7f3ea00d0390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009bc0 tx=0x7f3e9001ca80 comp rx=0 tx=0).stop 2024-09-17T05:25:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- 172.21.15.43:0/2713774903 shutdown_connections 2024-09-17T05:25:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 --2- 172.21.15.43:0/2713774903 >> v2:172.21.15.62:3300/0 conn(0x7f3ea006fce0 0x7f3ea00700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 --2- 172.21.15.43:0/2713774903 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea015f470 0x7f3ea00d0390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- 172.21.15.43:0/2713774903 >> 172.21.15.43:0/2713774903 conn(0x7f3ea00b2810 msgr2=0x7f3ea00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- 172.21.15.43:0/2713774903 shutdown_connections 2024-09-17T05:25:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- 172.21.15.43:0/2713774903 wait complete. 2024-09-17T05:25:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 Processor -- start 2024-09-17T05:25:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.945+0000 7f3ea6200640 1 -- start start 2024-09-17T05:25:00.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea00b5610 0x7f3ea01605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ea015f470 0x7f3ea0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0169660 con 0x7f3ea015f470 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01694e0 con 0x7f3ea006fce0 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea01697e0 con 0x7f3ea00b5610 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ea015f470 0x7f3ea0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:38602/0 (socket says 172.21.15.43:38602) 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 -- 172.21.15.43:0/1935457153 learned_addr learned my addr 172.21.15.43:0/1935457153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea00b5610 0x7f3ea01605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 -- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea00b5610 msgr2=0x7f3ea01605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea00b5610 0x7f3ea01605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 -- 172.21.15.43:0/1935457153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ea015f470 msgr2=0x7f3ea0160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ea015f470 0x7f3ea0160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 -- 172.21.15.43:0/1935457153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea0159810 con 0x7f3ea006fce0 2024-09-17T05:25:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e9001b130 tx=0x7f3e900092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e97e00640 1 -- 172.21.15.43:0/1935457153 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900bb070 con 0x7f3ea006fce0 2024-09-17T05:25:00.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e97e00640 1 -- 172.21.15.43:0/1935457153 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e90004d40 con 0x7f3ea006fce0 2024-09-17T05:25:00.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3e97e00640 1 -- 172.21.15.43:0/1935457153 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900b2040 con 0x7f3ea006fce0 2024-09-17T05:25:00.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea0158020 con 0x7f3ea006fce0 2024-09-17T05:25:00.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.949+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0158a50 con 0x7f3ea006fce0 2024-09-17T05:25:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3e97e00640 1 -- 172.21.15.43:0/1935457153 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e90004260 con 0x7f3ea006fce0 2024-09-17T05:25:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3e97e00640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 0x7f3e7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3e97e00640 1 -- 172.21.15.43:0/1935457153 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f3e9012e1f0 con 0x7f3ea006fce0 2024-09-17T05:25:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 0x7f3e7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 Processor -- start 2024-09-17T05:25:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 -- start start 2024-09-17T05:25:00.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 0x7f3e7c075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0152c00 tx=0x7f3e88009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910d370 0x565439287120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910db80 0x565439287660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565439125810 con 0x565439116430 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565439098980 con 0x56543910d370 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.953+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x565439087db0 con 0x56543910db80 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45872/0 (socket says 172.21.15.43:45872) 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 -- 172.21.15.43:0/3745001576 learned_addr learned my addr 172.21.15.43:0/3745001576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9fe00640 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910db80 0x565439287660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 -- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910db80 msgr2=0x565439287660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e96a00640 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910d370 0x565439287120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910db80 0x565439287660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 -- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910d370 msgr2=0x565439287120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910d370 0x565439287120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 -- 172.21.15.43:0/3745001576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565439055af0 con 0x565439116430 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e9f400640 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e900f5dd0 tx=0x7f3e900f8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92afad90cc9bc0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e8f400640 1 -- 172.21.15.43:0/3745001576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900f8670 con 0x565439116430 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e8f400640 1 -- 172.21.15.43:0/3745001576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e900f8b60 con 0x565439116430 2024-09-17T05:25:00.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3e8f400640 1 -- 172.21.15.43:0/3745001576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900fd590 con 0x565439116430 2024-09-17T05:25:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- 172.21.15.43:0/3745001576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 msgr2=0x565439286be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3e900f5dd0 tx=0x7f3e900f8930 comp rx=0 tx=0).stop 2024-09-17T05:25:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- 172.21.15.43:0/3745001576 shutdown_connections 2024-09-17T05:25:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910d370 0x565439287120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910db80 0x565439287660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 --2- 172.21.15.43:0/3745001576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565439116430 0x565439286be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- 172.21.15.43:0/3745001576 >> 172.21.15.43:0/3745001576 conn(0x56543920c3c0 msgr2=0x56543923bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- 172.21.15.43:0/3745001576 shutdown_connections 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- 172.21.15.43:0/3745001576 wait complete. 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 -- start start 2024-09-17T05:25:00.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.957+0000 7f3eada84000 1 Processor -- start 2024-09-17T05:25:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x5654392422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x565439242810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56543902d140 con 0x56543910d370 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56543908bbc0 con 0x56543910db80 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56543902e320 con 0x565439116430 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 10 client.0 _ll_register_callbacks cb 0x565439228b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45886/0 (socket says 172.21.15.43:45886) 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 -- 172.21.15.43:0/2631664108 learned_addr learned my addr 172.21.15.43:0/2631664108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e96a00640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x5654392422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9fe00640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x565439242810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 -- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 msgr2=0x565439242810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x565439242810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 -- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 msgr2=0x5654392422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x5654392422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 -- 172.21.15.43:0/2631664108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565439030ba0 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e9f400640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e900f8af0 tx=0x7f3e900fd3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a959f02dfa763a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90102440 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f3e9011e120 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56543901fe20 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3e900f8b60 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 10 client.5265 fetch_fsmap learned FSMap version 23 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x565439025f90 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56543901fe20 con 0x56543910d370 2024-09-17T05:25:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f3e900b5030 con 0x56543910d370 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 10 client.5265 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 10 client.5265 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 20 client.5265 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d2aa10 0x5654392981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 4 client.5265 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56543909a8d0 con 0x565438d2aa10 2024-09-17T05:25:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e96a00640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d2aa10 0x5654392981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e96a00640 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d2aa10 0x5654392981a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b048521df6efd8af server_cookie=2ba873d951fa7974 in_seq=0 out_seq=0 2024-09-17T05:25:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.961+0000 7f3e83e00640 10 client.5265 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:25:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3e83e00640 1 -- 172.21.15.43:0/2631664108 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56543909a8d0 con 0x565438d2aa10 2024-09-17T05:25:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3e83e00640 10 client.5265 handle_command_reply: tid=0 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 client.5265 shutdown 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 msgr2=0x565439241d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e900f8af0 tx=0x7f3e900fd3d0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 shutdown_connections 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d2aa10 0x5654392981a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x5654392422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x565439242810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 --2- 172.21.15.43:0/2631664108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439241d90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3e83e00640 0 client.5265 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 >> 172.21.15.43:0/2631664108 conn(0x56543920c3c0 msgr2=0x56543923a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 shutdown_connections 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 1 -- 172.21.15.43:0/2631664108 wait complete. 2024-09-17T05:25:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.965+0000 7f3eada84000 20 client.5265 trim_cache size 0 max 16384 2024-09-17T05:25:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 Processor -- start 2024-09-17T05:25:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- start start 2024-09-17T05:25:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x565439242770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x5654392991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56543902d140 con 0x56543910d370 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56543908bbc0 con 0x56543910db80 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56543902e320 con 0x565439116430 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45892/0 (socket says 172.21.15.43:45892) 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 -- 172.21.15.43:0/1504984009 learned_addr learned my addr 172.21.15.43:0/1504984009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9fe00640 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x5654392991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x565439242770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 -- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 msgr2=0x5654392991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x5654392991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 -- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 msgr2=0x565439242770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x565439242770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 -- 172.21.15.43:0/1504984009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565439051970 con 0x56543910d370 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3e9001cf60 tx=0x7f3e900f8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4d8438c512c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e8ea00640 1 -- 172.21.15.43:0/1504984009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9011e120 con 0x56543910d370 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e8ea00640 1 -- 172.21.15.43:0/1504984009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e9010ebd0 con 0x56543910d370 2024-09-17T05:25:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3e8ea00640 1 -- 172.21.15.43:0/1504984009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9011a0b0 con 0x56543910d370 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- 172.21.15.43:0/1504984009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 msgr2=0x565439242230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3e9001cf60 tx=0x7f3e900f8930 comp rx=0 tx=0).stop 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- 172.21.15.43:0/1504984009 shutdown_connections 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56543910db80 0x565439242770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565439116430 0x5654392991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 --2- 172.21.15.43:0/1504984009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910d370 0x565439242230 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- 172.21.15.43:0/1504984009 >> 172.21.15.43:0/1504984009 conn(0x56543920c3c0 msgr2=0x565439241b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- 172.21.15.43:0/1504984009 shutdown_connections 2024-09-17T05:25:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.981+0000 7f3eada84000 1 -- 172.21.15.43:0/1504984009 wait complete. 2024-09-17T05:25:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 -- start start 2024-09-17T05:25:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 Processor -- start 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910d370 0x56543929b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565439116430 0x5654392bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565439097750 con 0x56543910db80 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565439148710 con 0x565439116430 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56543900c5a0 con 0x56543910d370 2024-09-17T05:25:00.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 10 client.0 _ll_register_callbacks cb 0x7f3e900f52a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45906/0 (socket says 172.21.15.43:45906) 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 -- 172.21.15.43:0/1066760034 learned_addr learned my addr 172.21.15.43:0/1066760034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e9fe00640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565439116430 0x5654392bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910d370 0x56543929b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 -- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910d370 msgr2=0x56543929b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910d370 0x56543929b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 -- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565439116430 msgr2=0x5654392bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565439116430 0x5654392bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 -- 172.21.15.43:0/1066760034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543900cf50 con 0x56543910db80 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e96a00640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000f70 tx=0x7f3e88000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2960ae7c720a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8801a040 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f3e88014c70 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.985+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9010ebd0 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3e88035050 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 10 client.5271 fetch_fsmap learned FSMap version 23 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56543904aa60 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e9010ebd0 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f3e88038020 con 0x56543910db80 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 10 client.5271 fetch_fsmap finished waiting for FSMap version 23 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 10 client.5271 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 20 client.5271 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d18a70 0x5654392ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 4 client.5271 mds_command: new command op to 5076 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 204375696}] 2024-09-17T05:25:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 --> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] -- command(tid 0: {"prefix": "damage rm", "damage_id": 204375696}) -- 0x565439096380 con 0x565438d18a70 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d18a70 0x5654392ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e9f400640 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d18a70 0x5654392ffba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05156f91708579e server_cookie=11cf7ee3d3e9add4 in_seq=0 out_seq=0 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 10 client.5271 ms_handle_connect on v2:172.21.15.43:6834/331818553 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 1 -- 172.21.15.43:0/1066760034 <== mds.0 v2:172.21.15.43:6834/331818553 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x565439096380 con 0x565438d18a70 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 10 client.5271 handle_command_reply: tid=0 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 client.5271 shutdown 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 msgr2=0x56543929b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000f70 tx=0x7f3e88000ee0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 shutdown_connections 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:6834/331818553,v1:172.21.15.43:6835/331818553] conn(0x565438d18a70 0x5654392ffba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565439116430 0x5654392bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56543910db80 0x56543929b780 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 --2- 172.21.15.43:0/1066760034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56543910d370 0x56543929b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 >> 172.21.15.43:0/1066760034 conn(0x56543920c3c0 msgr2=0x5654392424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3e83e00640 0 client.5271 ms_handle_reset on v2:172.21.15.43:6834/331818553 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 shutdown_connections 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 1 -- 172.21.15.43:0/1066760034 wait complete. 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3eada84000 20 client.5271 trim_cache size 0 max 16384 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 msgr2=0x7f3e7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 0x7f3e7c075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0152c00 tx=0x7f3e88009040 comp rx=0 tx=0).stop 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 msgr2=0x7f3ea0157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e9001b130 tx=0x7f3e900092a0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 shutdown_connections 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e7c073200 0x7f3e7c075710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ea015f470 0x7f3ea0160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:00.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ea00b5610 0x7f3ea01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 --2- 172.21.15.43:0/1935457153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea006fce0 0x7f3ea0157ae0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 >> 172.21.15.43:0/1935457153 conn(0x7f3ea00b2810 msgr2=0x7f3ea00b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 shutdown_connections 2024-09-17T05:25:01.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:00.989+0000 7f3ea6200640 1 -- 172.21.15.43:0/1935457153 wait complete. 2024-09-17T05:25:01.013 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:25:01.013 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:25:01.013 DEBUG:teuthology.orchestra.run.smithi043:> import errno 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> f.close() 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> except IOError as e: 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(errno.EIO) 2024-09-17T05:25:01.014 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:25:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:01.093 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:25:01.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:01.221 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:01.303 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:25:01.303 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:01.303 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:01.304 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:01.314 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:01.314 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:01.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- 172.21.15.43:0/1368141305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92c4004d50 msgr2=0x7f92c4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 --2- 172.21.15.43:0/1368141305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92c4004d50 0x7f92c4005160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c610 tx=0x7f92d401d2e0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2000640 1 -- 172.21.15.43:0/1368141305 reap_dead start 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- 172.21.15.43:0/1368141305 shutdown_connections 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- 172.21.15.43:0/1368141305 >> 172.21.15.43:0/1368141305 conn(0x7f92dc0d1100 msgr2=0x7f92dc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- 172.21.15.43:0/1368141305 shutdown_connections 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- 172.21.15.43:0/1368141305 wait complete. 2024-09-17T05:25:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 Processor -- start 2024-09-17T05:25:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.565+0000 7f92e2a00640 1 -- start start 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92c4004d50 0x7f92dc1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc14ffb0 0x7f92dc14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc0cf230 con 0x7f92dc15cab0 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc0cf0b0 con 0x7f92dc14ffb0 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92dc0b49d0 con 0x7f92c4004d50 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45920/0 (socket says 172.21.15.43:45920) 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 -- 172.21.15.43:0/2804704404 learned_addr learned my addr 172.21.15.43:0/2804704404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e1600640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92c4004d50 0x7f92dc1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2000640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc14ffb0 0x7f92dc14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 -- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92c4004d50 msgr2=0x7f92dc1492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92c4004d50 0x7f92dc1492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 -- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc14ffb0 msgr2=0x7f92dc14bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc14ffb0 0x7f92dc14bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 -- 172.21.15.43:0/2804704404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc14cd20 con 0x7f92dc15cab0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e0c00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f92c8011b10 tx=0x7f92c8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c800f920 con 0x7f92dc15cab0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc14a120 con 0x7f92dc15cab0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92c800fac0 con 0x7f92dc15cab0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8005020 con 0x7f92dc15cab0 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc14bc80 con 0x7f92dc15cab0 2024-09-17T05:25:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.569+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92dc1562c0 con 0x7f92dc15cab0 2024-09-17T05:25:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.573+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92c80051c0 con 0x7f92dc15cab0 2024-09-17T05:25:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.573+0000 7f92d9600640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 0x7f92bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.573+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f92c8094fe0 con 0x7f92dc15cab0 2024-09-17T05:25:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.573+0000 7f92e1600640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 0x7f92bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.573+0000 7f92e1600640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 0x7f92bc075740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009c40 tx=0x7f92d00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.577+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92c805f850 con 0x7f92dc15cab0 2024-09-17T05:25:01.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.769+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92dc149f10 con 0x7f92dc15cab0 2024-09-17T05:25:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.769+0000 7f92d9600640 1 -- 172.21.15.43:0/2804704404 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v34) ==== 80+0+122 (secure 0 0 0) 0x7f92c8064280 con 0x7f92dc15cab0 2024-09-17T05:25:01.774 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404655827 2024-09-18T05:24:19.717260+0000 2024-09-17T05:25:01.774 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/1404655827 2024-09-18T05:24:19.717260+0000 2024-09-17T05:25:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 msgr2=0x7f92bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 0x7f92bc075740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009c40 tx=0x7f92d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 msgr2=0x7f92dc149800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f92c8011b10 tx=0x7f92c8011f10 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 shutdown_connections 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92bc073230 0x7f92bc075740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc14ffb0 0x7f92dc14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92dc15cab0 0x7f92dc149800 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 --2- 172.21.15.43:0/2804704404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92c4004d50 0x7f92dc1492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 >> 172.21.15.43:0/2804704404 conn(0x7f92dc0d1100 msgr2=0x7f92dc15a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 shutdown_connections 2024-09-17T05:25:01.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:01.773+0000 7f92e2a00640 1 -- 172.21.15.43:0/2804704404 wait complete. 2024-09-17T05:25:01.797 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:25:01.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:01.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:01.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:25:01.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:01.824 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:25:01.875 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:01.875 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:01.875 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:01.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:01.885 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:01.885 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:01.886 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:01.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:01.927 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:01.927 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:01.927 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:01.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:01.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 -- 172.21.15.43:0/2219939880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab480b1c40 msgr2=0x7fab4813ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 --2- 172.21.15.43:0/2219939880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab480b1c40 0x7fab4813ef50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fab48071210 tx=0x7fab4001cac0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 -- 172.21.15.43:0/2219939880 shutdown_connections 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 --2- 172.21.15.43:0/2219939880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab480b1c40 0x7fab4813ef50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 --2- 172.21.15.43:0/2219939880 >> v2:172.21.15.62:3300/0 conn(0x7fab4815fbb0 0x7fab48070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 -- 172.21.15.43:0/2219939880 >> 172.21.15.43:0/2219939880 conn(0x7fab480b24c0 msgr2=0x7fab480b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 -- 172.21.15.43:0/2219939880 shutdown_connections 2024-09-17T05:25:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 -- 172.21.15.43:0/2219939880 wait complete. 2024-09-17T05:25:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.129+0000 7fab4cc00640 1 Processor -- start 2024-09-17T05:25:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- start start 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48070b90 0x7fab4806f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 0x7fab4813dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab480b21d0 con 0x7fab4815fbb0 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab480b2050 con 0x7fab48070b90 2024-09-17T05:25:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fab4813fca0 con 0x7fab480b1c40 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45942/0 (socket says 172.21.15.43:45942) 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 -- 172.21.15.43:0/3656152958 learned_addr learned my addr 172.21.15.43:0/3656152958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab45e00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48070b90 0x7fab4806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab3fe00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 0x7fab4813dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 -- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 msgr2=0x7fab4813dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 0x7fab4813dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 -- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48070b90 msgr2=0x7fab4806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48070b90 0x7fab4806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 -- 172.21.15.43:0/3656152958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab48154320 con 0x7fab4815fbb0 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab3fe00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 0x7fab4813dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab46800640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab4001c530 tx=0x7fab40003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b2040 con 0x7fab4815fbb0 2024-09-17T05:25:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab48154600 con 0x7fab4815fbb0 2024-09-17T05:25:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab400bb070 con 0x7fab4815fbb0 2024-09-17T05:25:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40004330 con 0x7fab4815fbb0 2024-09-17T05:25:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab48154bc0 con 0x7fab4815fbb0 2024-09-17T05:25:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.133+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab48140910 con 0x7fab4815fbb0 2024-09-17T05:25:02.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.137+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fab400044d0 con 0x7fab4815fbb0 2024-09-17T05:25:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.137+0000 7fab3e000640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 0x7fab24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.137+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fab4012eca0 con 0x7fab4815fbb0 2024-09-17T05:25:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.137+0000 7fab45e00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 0x7fab24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.137+0000 7fab45e00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 0x7fab24075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab38009ad0 tx=0x7fab380091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.141+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fab40016770 con 0x7fab4815fbb0 2024-09-17T05:25:02.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab481540b0 con 0x7fab4815fbb0 2024-09-17T05:25:02.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab3e000640 1 -- 172.21.15.43:0/3656152958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7fab400f9c60 con 0x7fab4815fbb0 2024-09-17T05:25:02.357 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:02.357 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:25:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 msgr2=0x7fab24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 0x7fab24075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab38009ad0 tx=0x7fab380091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 msgr2=0x7fab4813e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab4001c530 tx=0x7fab40003f40 comp rx=0 tx=0).stop 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 shutdown_connections 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab24073200 0x7fab24075710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab4815fbb0 0x7fab4813e170 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab480b1c40 0x7fab4813dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 --2- 172.21.15.43:0/3656152958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48070b90 0x7fab4806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.353+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 >> 172.21.15.43:0/3656152958 conn(0x7fab480b24c0 msgr2=0x7fab4815b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.357+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 shutdown_connections 2024-09-17T05:25:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.357+0000 7fab4cc00640 1 -- 172.21.15.43:0/3656152958 wait complete. 2024-09-17T05:25:02.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 -- 172.21.15.43:0/3488325960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0ae6a0 msgr2=0x7fc97c0d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 --2- 172.21.15.43:0/3488325960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c0d0470 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc974009bc0 tx=0x7fc97401cba0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 -- 172.21.15.43:0/3488325960 shutdown_connections 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 --2- 172.21.15.43:0/3488325960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c0d0470 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 --2- 172.21.15.43:0/3488325960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c0b69f0 0x7fc97c0ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.533+0000 7fc980a00640 1 -- 172.21.15.43:0/3488325960 >> 172.21.15.43:0/3488325960 conn(0x7fc97c0b2e80 msgr2=0x7fc97c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- 172.21.15.43:0/3488325960 shutdown_connections 2024-09-17T05:25:02.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- 172.21.15.43:0/3488325960 wait complete. 2024-09-17T05:25:02.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 Processor -- start 2024-09-17T05:25:02.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- start start 2024-09-17T05:25:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 0x7fc97c150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c16c4d0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c16c350 con 0x7fc97c0ae6a0 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.537+0000 7fc980a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c16c650 con 0x7fc97c15ee30 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45960/0 (socket says 172.21.15.43:45960) 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 -- 172.21.15.43:0/3240866936 learned_addr learned my addr 172.21.15.43:0/3240866936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc979e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc97a800640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 0x7fc97c150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 -- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 msgr2=0x7fc97c150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 0x7fc97c150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 -- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 msgr2=0x7fc97c161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 -- 172.21.15.43:0/3240866936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c151500 con 0x7fc97c0b69f0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc97a800640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 0x7fc97c150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc979e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc973e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc96400d700 tx=0x7fc96400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964004230 con 0x7fc97c0b69f0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9640044c0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964009a00 con 0x7fc97c0b69f0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1621a0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c161ec0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0d0470 con 0x7fc97c0b69f0 2024-09-17T05:25:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.541+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc964009ba0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.545+0000 7fc972000640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 0x7fc958075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.545+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc964094500 con 0x7fc97c0b69f0 2024-09-17T05:25:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.545+0000 7fc979e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 0x7fc958075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.545+0000 7fc979e00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 0x7fc958075740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f70 tx=0x7fc96c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.549+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc964017020 con 0x7fc97c0b69f0 2024-09-17T05:25:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.761+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc97c0b3d50 con 0x7fc97c0b69f0 2024-09-17T05:25:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.761+0000 7fc972000640 1 -- 172.21.15.43:0/3240866936 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7fc96405f4b0 con 0x7fc97c0b69f0 2024-09-17T05:25:02.766 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:02.766 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:25:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:25:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 msgr2=0x7fc958075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 0x7fc958075740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f70 tx=0x7fc96c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 msgr2=0x7fc97c153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc96400d700 tx=0x7fc96400dbc0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 shutdown_connections 2024-09-17T05:25:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc958073230 0x7fc958075740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc97c15ee30 0x7fc97c150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc97c0b69f0 0x7fc97c153bc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 --2- 172.21.15.43:0/3240866936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc97c0ae6a0 0x7fc97c161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 >> 172.21.15.43:0/3240866936 conn(0x7fc97c0b2e80 msgr2=0x7fc97c071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 shutdown_connections 2024-09-17T05:25:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.765+0000 7fc980a00640 1 -- 172.21.15.43:0/3240866936 wait complete. 2024-09-17T05:25:02.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- 172.21.15.43:0/2396760909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c004da0 msgr2=0x7f485c005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 --2- 172.21.15.43:0/2396760909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f485c004da0 0x7f485c005250 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4874074b60 tx=0x7f486c01cc40 comp rx=0 tx=0).stop 2024-09-17T05:25:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487a000640 1 -- 172.21.15.43:0/2396760909 reap_dead start 2024-09-17T05:25:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- 172.21.15.43:0/2396760909 shutdown_connections 2024-09-17T05:25:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- 172.21.15.43:0/2396760909 >> 172.21.15.43:0/2396760909 conn(0x7f48740d0a60 msgr2=0x7f48740d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- 172.21.15.43:0/2396760909 shutdown_connections 2024-09-17T05:25:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- 172.21.15.43:0/2396760909 wait complete. 2024-09-17T05:25:02.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 Processor -- start 2024-09-17T05:25:02.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.945+0000 7f487c000640 1 -- start start 2024-09-17T05:25:02.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485c004da0 0x7f487406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4874156d60 0x7f4874157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4874068fc0 con 0x7f48741539a0 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48740d03f0 con 0x7f485c004da0 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4874069140 con 0x7f4874156d60 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45984/0 (socket says 172.21.15.43:45984) 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 -- 172.21.15.43:0/1197752931 learned_addr learned my addr 172.21.15.43:0/1197752931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4879600640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485c004da0 0x7f487406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487a000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4874156d60 0x7f4874157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 -- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4874156d60 msgr2=0x7f4874157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4874156d60 0x7f4874157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 -- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485c004da0 msgr2=0x7f487406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485c004da0 0x7f487406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 -- 172.21.15.43:0/1197752931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740ae980 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4878c00640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4860001980 tx=0x7f48600127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860013650 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48600137f0 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48740ae400 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860013c40 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4874149a20 con 0x7f48741539a0 2024-09-17T05:25:02.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.949+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4874160f60 con 0x7f48741539a0 2024-09-17T05:25:02.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.953+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f486001e480 con 0x7f48741539a0 2024-09-17T05:25:02.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.953+0000 7f4871600640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 0x7f4854075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.953+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f48600979a0 con 0x7f48741539a0 2024-09-17T05:25:02.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.953+0000 7f4879600640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 0x7f4854075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.953+0000 7f4879600640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 0x7f4854075740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c60 tx=0x7f48680091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:02.957+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48740d0880 con 0x7f48741539a0 2024-09-17T05:25:03.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.133+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4874154640 con 0x7f48741539a0 2024-09-17T05:25:03.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f4871600640 1 -- 172.21.15.43:0/1197752931 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16738 (secure 0 0 0) 0x7f48600629f0 con 0x7f48741539a0 2024-09-17T05:25:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:03.139 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:24:19.717292+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:24:08.162466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:24:08.803372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/1404655827":"2024-09-18T05:24:19.717260+0000","172.21.15.43:6834/1404655827":"2024-09-18T05:24:19.717260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 msgr2=0x7f4854075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 0x7f4854075740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c60 tx=0x7f48680091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 msgr2=0x7f48741494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4860001980 tx=0x7f48600127b0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 shutdown_connections 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4854073230 0x7f4854075740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4874156d60 0x7f4874157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48741539a0 0x7f48741494e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 --2- 172.21.15.43:0/1197752931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f485c004da0 0x7f487406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 >> 172.21.15.43:0/1197752931 conn(0x7f48740d0a60 msgr2=0x7f487415dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 shutdown_connections 2024-09-17T05:25:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.137+0000 7f487c000640 1 -- 172.21.15.43:0/1197752931 wait complete. 2024-09-17T05:25:03.162 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:25:03.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- 172.21.15.43:0/3169455144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc0b3af0 msgr2=0x7febdc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 --2- 172.21.15.43:0/3169455144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc0b3af0 0x7febdc0ad0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7febdc072b60 tx=0x7febd401cac0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- 172.21.15.43:0/3169455144 shutdown_connections 2024-09-17T05:25:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 --2- 172.21.15.43:0/3169455144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc0b3af0 0x7febdc0ad0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 --2- 172.21.15.43:0/3169455144 >> v2:172.21.15.62:3300/0 conn(0x7febdc155d70 0x7febdc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- 172.21.15.43:0/3169455144 >> 172.21.15.43:0/3169455144 conn(0x7febdc0b23d0 msgr2=0x7febdc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- 172.21.15.43:0/3169455144 shutdown_connections 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- 172.21.15.43:0/3169455144 wait complete. 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 Processor -- start 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.317+0000 7febe1e00640 1 -- start start 2024-09-17T05:25:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 0x7febdc09d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febdc09dca0 0x7febdc0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7febdc168c60 con 0x7febdc155d70 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febdc168ae0 con 0x7febdc09dca0 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7febdc168de0 con 0x7febdc06f110 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46014/0 (socket says 172.21.15.43:46014) 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 -- 172.21.15.43:0/2046910308 learned_addr learned my addr 172.21.15.43:0/2046910308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febdb400640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febdc09dca0 0x7febdc0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febdaa00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 0x7febdc09d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 -- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 msgr2=0x7febdc09d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 0x7febdc09d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 -- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febdc09dca0 msgr2=0x7febdc0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febdc09dca0 0x7febdc0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 -- 172.21.15.43:0/2046910308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc15af70 con 0x7febdc155d70 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febdaa00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 0x7febdc09d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febc400c3d0 tx=0x7febc400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4017070 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc0c6930 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc4003600 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4003a50 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc09d750 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.321+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc09da30 con 0x7febdc155d70 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.325+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7febc401a070 con 0x7febdc155d70 2024-09-17T05:25:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.325+0000 7febd3400640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 0x7febb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.325+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7febc40934f0 con 0x7febdc155d70 2024-09-17T05:25:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.325+0000 7febdaa00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 0x7febb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.325+0000 7febdaa00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 0x7febb8075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febdc0715c0 tx=0x7febcc0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.329+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7febc405de60 con 0x7febdc155d70 2024-09-17T05:25:03.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.537+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febdc0ad0b0 con 0x7febdc155d70 2024-09-17T05:25:03.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.537+0000 7febd3400640 1 -- 172.21.15.43:0/2046910308 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7febc4062890 con 0x7febdc155d70 2024-09-17T05:25:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:03.542 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:25:03.542 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:25:03.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 msgr2=0x7febb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 0x7febb8075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febdc0715c0 tx=0x7febcc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 msgr2=0x7febdc0b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febc400c3d0 tx=0x7febc400c890 comp rx=0 tx=0).stop 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 shutdown_connections 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7febb8073200 0x7febb8075710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febdc09dca0 0x7febdc0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febdc155d70 0x7febdc0b09c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 --2- 172.21.15.43:0/2046910308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7febdc06f110 0x7febdc09d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 >> 172.21.15.43:0/2046910308 conn(0x7febdc0b23d0 msgr2=0x7febdc15d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 shutdown_connections 2024-09-17T05:25:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.541+0000 7febe1e00640 1 -- 172.21.15.43:0/2046910308 wait complete. 2024-09-17T05:25:03.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 -- 172.21.15.43:0/2570974951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46600d0650 msgr2=0x7f46600cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 --2- 172.21.15.43:0/2570974951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46600d0650 0x7f46600cd750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f465c009c80 tx=0x7f465c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 -- 172.21.15.43:0/2570974951 shutdown_connections 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 --2- 172.21.15.43:0/2570974951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46600d0650 0x7f46600cd750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 --2- 172.21.15.43:0/2570974951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f466009ba60 0x7f46600b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 -- 172.21.15.43:0/2570974951 >> 172.21.15.43:0/2570974951 conn(0x7f46600cee60 msgr2=0x7f46600cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 -- 172.21.15.43:0/2570974951 shutdown_connections 2024-09-17T05:25:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.717+0000 7f4668800640 1 -- 172.21.15.43:0/2570974951 wait complete. 2024-09-17T05:25:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 Processor -- start 2024-09-17T05:25:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 -- start start 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46600d0240 0x7f46600c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 0x7f4660155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46600b2b80 con 0x7f466009ba60 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46600b2a00 con 0x7f46600d0650 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4668800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4660169e30 con 0x7f46600d0240 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4665e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4665e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54946/0 (socket says 172.21.15.43:54946) 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4665e00640 1 -- 172.21.15.43:0/1968120472 learned_addr learned my addr 172.21.15.43:0/1968120472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4666800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 0x7f4660155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.721+0000 7f4665400640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46600d0240 0x7f46600c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 -- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46600d0240 msgr2=0x7f46600c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46600d0240 0x7f46600c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 -- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 msgr2=0x7f4660155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 0x7f4660155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 -- 172.21.15.43:0/1968120472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466006ff10 con 0x7f466009ba60 2024-09-17T05:25:03.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4666800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 0x7f4660155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4665e00640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f465400d720 tx=0x7f465400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654017070 con 0x7f466009ba60 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4654009d20 con 0x7f466009ba60 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654005280 con 0x7f466009ba60 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46600b0c40 con 0x7f466009ba60 2024-09-17T05:25:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466015c0a0 con 0x7f466009ba60 2024-09-17T05:25:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466015c300 con 0x7f466009ba60 2024-09-17T05:25:03.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.725+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4654005420 con 0x7f466009ba60 2024-09-17T05:25:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.729+0000 7f465a000640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 0x7f4644075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.729+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f4654094370 con 0x7f466009ba60 2024-09-17T05:25:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.729+0000 7f4665400640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 0x7f4644075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.729+0000 7f4665400640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 0x7f4644075740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f465000b810 tx=0x7f4650008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.733+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f465405f320 con 0x7f466009ba60 2024-09-17T05:25:03.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.941+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46600b3130 con 0x7f466009ba60 2024-09-17T05:25:03.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.941+0000 7f465a000640 1 -- 172.21.15.43:0/1968120472 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5221 (secure 0 0 0) 0x7f465405ec20 con 0x7f466009ba60 2024-09-17T05:25:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:03.946 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-17T05:24:23:737508+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:24:09.841714+0000","modified":"2024-09-17T05:24:23.737496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5076},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5076":{"gid":5076,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/331818553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":331818553},{"type":"v1","addr":"172.21.15.43:6835","nonce":331818553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5076,"qdb_cluster":[5076]},"id":2}]} 2024-09-17T05:25:03.946 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 msgr2=0x7f4644075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 0x7f4644075740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f465000b810 tx=0x7f4650008040 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 msgr2=0x7f46600aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f465400d720 tx=0x7f465400dbe0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 shutdown_connections 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4644073230 0x7f4644075740 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46600d0650 0x7f4660155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46600d0240 0x7f46600c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 --2- 172.21.15.43:0/1968120472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f466009ba60 0x7f46600aef70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 >> 172.21.15.43:0/1968120472 conn(0x7f46600cee60 msgr2=0x7f46600724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 shutdown_connections 2024-09-17T05:25:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:03.945+0000 7f4668800640 1 -- 172.21.15.43:0/1968120472 wait complete. 2024-09-17T05:25:03.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- 172.21.15.43:0/1025879082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 msgr2=0x7f11840cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 --2- 172.21.15.43:0/1025879082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840cce70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f116c009900 tx=0x7f116c01c620 comp rx=0 tx=0).stop 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- 172.21.15.43:0/1025879082 shutdown_connections 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 --2- 172.21.15.43:0/1025879082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840cd3b0 0x7f11841609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 --2- 172.21.15.43:0/1025879082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840cce70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- 172.21.15.43:0/1025879082 >> 172.21.15.43:0/1025879082 conn(0x7f11840cee80 msgr2=0x7f11840cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- 172.21.15.43:0/1025879082 shutdown_connections 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- 172.21.15.43:0/1025879082 wait complete. 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 Processor -- start 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.129+0000 7f118aa00640 1 -- start start 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840b7ce0 0x7f11841f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840cd3b0 0x7f1184159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f118416a020 con 0x7f11840d0f50 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1184169ea0 con 0x7f11840cd3b0 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f118416a1a0 con 0x7f11840b7ce0 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54962/0 (socket says 172.21.15.43:54962) 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 -- 172.21.15.43:0/4246285770 learned_addr learned my addr 172.21.15.43:0/4246285770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1183e00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840b7ce0 0x7f11841f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f117be00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840cd3b0 0x7f1184159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 -- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840b7ce0 msgr2=0x7f11841f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840b7ce0 0x7f11841f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 -- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840cd3b0 msgr2=0x7f1184159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840cd3b0 0x7f1184159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 -- 172.21.15.43:0/4246285770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118415a1d0 con 0x7f11840d0f50 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1188a00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f117c00eab0 tx=0x7f117c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c00cea0 con 0x7f11840d0f50 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f117c004b30 con 0x7f11840d0f50 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11840b0670 con 0x7f11840d0f50 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c005020 con 0x7f11840d0f50 2024-09-17T05:25:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11841fa1c0 con 0x7f11840d0f50 2024-09-17T05:25:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11841609c0 con 0x7f11840d0f50 2024-09-17T05:25:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.133+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f117c0051c0 con 0x7f11840d0f50 2024-09-17T05:25:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.137+0000 7f1180c00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 0x7f1160075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.137+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5940+0+0 (secure 0 0 0) 0x7f117c094f20 con 0x7f11840d0f50 2024-09-17T05:25:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.137+0000 7f1183e00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 0x7f1160075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.137+0000 7f1183e00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 0x7f1160075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1184071220 tx=0x7f1174009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.141+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f117c05f760 con 0x7f11840d0f50 2024-09-17T05:25:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.313+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11840d0970 con 0x7f11840d0f50 2024-09-17T05:25:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f1180c00640 1 -- 172.21.15.43:0/4246285770 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+17034 (secure 0 0 0) 0x7f117c064190 con 0x7f11840d0f50 2024-09-17T05:25:04.323 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:04.323 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:03.988646+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:24:08.162466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:24:08.803372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.0","mappings":[{"from":4,"to":5}]},{"pgid":"5.14","mappings":[{"from":4,"to":5}]},{"pgid":"5.17","mappings":[{"from":4,"to":1}]},{"pgid":"5.36","mappings":[{"from":4,"to":3}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/1404655827":"2024-09-18T05:24:19.717260+0000","172.21.15.43:6834/1404655827":"2024-09-18T05:24:19.717260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 msgr2=0x7f1160075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 0x7f1160075710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1184071220 tx=0x7f1174009040 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 msgr2=0x7f11840af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f117c00eab0 tx=0x7f117c00ef70 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 shutdown_connections 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1160073200 0x7f1160075710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11840cd3b0 0x7f1184159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11840d0f50 0x7f11840af860 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 --2- 172.21.15.43:0/4246285770 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11840b7ce0 0x7f11841f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 >> 172.21.15.43:0/4246285770 conn(0x7f11840cee80 msgr2=0x7f118415ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 shutdown_connections 2024-09-17T05:25:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.317+0000 7f118aa00640 1 -- 172.21.15.43:0/4246285770 wait complete. 2024-09-17T05:25:04.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:25:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 -- 172.21.15.43:0/3458908194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0b3af0 msgr2=0x7f934c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 --2- 172.21.15.43:0/3458908194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0b3af0 0x7f934c15c7e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9334009840 tx=0x7f933401c6b0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 -- 172.21.15.43:0/3458908194 shutdown_connections 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 --2- 172.21.15.43:0/3458908194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0b3af0 0x7f934c15c7e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 -- 172.21.15.43:0/3458908194 >> 172.21.15.43:0/3458908194 conn(0x7f934c0b23d0 msgr2=0x7f934c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 -- 172.21.15.43:0/3458908194 shutdown_connections 2024-09-17T05:25:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.501+0000 7f9352400640 1 -- 172.21.15.43:0/3458908194 wait complete. 2024-09-17T05:25:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 Processor -- start 2024-09-17T05:25:04.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- start start 2024-09-17T05:25:04.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934c155d70 0x7f934c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0c9c10 0x7f934c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f934c169dc0 con 0x7f934c06f110 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f934c169c40 con 0x7f934c0c9c10 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f934c169f40 con 0x7f934c155d70 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54976/0 (socket says 172.21.15.43:54976) 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 -- 172.21.15.43:0/78019510 learned_addr learned my addr 172.21.15.43:0/78019510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934be00640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0c9c10 0x7f934c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9343e00640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934c155d70 0x7f934c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 -- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934c155d70 msgr2=0x7f934c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934c155d70 0x7f934c0c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 -- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0c9c10 msgr2=0x7f934c0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0c9c10 0x7f934c0c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 -- 172.21.15.43:0/78019510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c09cb20 con 0x7f934c06f110 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f934b400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f933c001230 tx=0x7f933c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c016020 con 0x7f934c06f110 2024-09-17T05:25:04.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f933c00a930 con 0x7f934c06f110 2024-09-17T05:25:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c09cdc0 con 0x7f934c06f110 2024-09-17T05:25:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c00ad80 con 0x7f934c06f110 2024-09-17T05:25:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.505+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c0c6ef0 con 0x7f934c06f110 2024-09-17T05:25:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.509+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c0d1150 con 0x7f934c06f110 2024-09-17T05:25:04.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.509+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f933c019020 con 0x7f934c06f110 2024-09-17T05:25:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.509+0000 7f9343400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 0x7f9328075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.509+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5940+0+0 (secure 0 0 0) 0x7f933c093f40 con 0x7f934c06f110 2024-09-17T05:25:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.509+0000 7f9343e00640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 0x7f9328075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.513+0000 7f9343e00640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 0x7f9328075740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93340ae000 tx=0x7f93340b3040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.513+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f934c0d1150 con 0x7f934c06f110 2024-09-17T05:25:04.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.725+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f934c15c7e0 con 0x7f934c06f110 2024-09-17T05:25:04.730 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:25:04.725+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:25:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9343400640 1 -- 172.21.15.43:0/78019510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7f933c020030 con 0x7f934c06f110 2024-09-17T05:25:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:25:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 msgr2=0x7f9328075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 0x7f9328075740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93340ae000 tx=0x7f93340b3040 comp rx=0 tx=0).stop 2024-09-17T05:25:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 msgr2=0x7f934c09d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f933c001230 tx=0x7f933c00cb70 comp rx=0 tx=0).stop 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 shutdown_connections 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9328073230 0x7f9328075740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c0c9c10 0x7f934c0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f934c155d70 0x7f934c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 --2- 172.21.15.43:0/78019510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f934c06f110 0x7f934c09d8a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 >> 172.21.15.43:0/78019510 conn(0x7f934c0b23d0 msgr2=0x7f934c0ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 shutdown_connections 2024-09-17T05:25:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.729+0000 7f9352400640 1 -- 172.21.15.43:0/78019510 wait complete. 2024-09-17T05:25:04.746 INFO:tasks.ceph.mds.c.smithi043.stderr: -3511> 2024-09-17T05:24:25.730+0000 7fcd24200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:04.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:25:04.877 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:25:04.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- 172.21.15.43:0/1224309013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc0b5d20 msgr2=0x7fa6bc0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 --2- 172.21.15.43:0/1224309013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc0b5d20 0x7fa6bc0d0390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009870 tx=0x7fa6a401c740 comp rx=0 tx=0).stop 2024-09-17T05:25:04.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- 172.21.15.43:0/1224309013 shutdown_connections 2024-09-17T05:25:04.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 --2- 172.21.15.43:0/1224309013 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc0b5d20 0x7fa6bc0d0390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- 172.21.15.43:0/1224309013 >> 172.21.15.43:0/1224309013 conn(0x7fa6bc0b2810 msgr2=0x7fa6bc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- 172.21.15.43:0/1224309013 shutdown_connections 2024-09-17T05:25:04.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- 172.21.15.43:0/1224309013 wait complete. 2024-09-17T05:25:04.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 Processor -- start 2024-09-17T05:25:04.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.925+0000 7fa6c3000640 1 -- start start 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc15f470 0x7fa6bc14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 0x7fa6bc1649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc16a4d0 con 0x7fa6bc06fce0 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc16a350 con 0x7fa6bc14f600 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc16a650 con 0x7fa6bc15f470 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55014/0 (socket says 172.21.15.43:55014) 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 -- 172.21.15.43:0/2029180639 learned_addr learned my addr 172.21.15.43:0/2029180639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6b3e00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc15f470 0x7fa6bc14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c1000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 0x7fa6bc1649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 -- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc15f470 msgr2=0x7fa6bc14e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc15f470 0x7fa6bc14e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 -- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 msgr2=0x7fa6bc1649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 0x7fa6bc1649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 -- 172.21.15.43:0/2029180639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc15b760 con 0x7fa6bc06fce0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c1000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 0x7fa6bc1649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6bbe00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c430 tx=0x7fa6ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac017070 con 0x7fa6bc06fce0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ac0053e0 con 0x7fa6bc06fce0 2024-09-17T05:25:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac003860 con 0x7fa6bc06fce0 2024-09-17T05:25:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc15ba40 con 0x7fa6bc06fce0 2024-09-17T05:25:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc159e20 con 0x7fa6bc06fce0 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.929+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc0d0390 con 0x7fa6bc06fce0 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.933+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa6ac01a070 con 0x7fa6bc06fce0 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.933+0000 7fa6b8c00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 0x7fa698075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.933+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6026+0+0 (secure 0 0 0) 0x7fa6ac093760 con 0x7fa6bc06fce0 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.933+0000 7fa6b3e00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 0x7fa698075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.933+0000 7fa6b3e00640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 0x7fa698075710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40af000 tx=0x7fa6a40b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:04.937+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6ac05df50 con 0x7fa6bc06fce0 2024-09-17T05:25:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.149+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa6bc0b3e90 con 0x7fa6bc06fce0 2024-09-17T05:25:05.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.729+0000 7fa6b8c00640 1 -- 172.21.15.43:0/2029180639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fa6ac062980 con 0x7fa6bc06fce0 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 msgr2=0x7fa698075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 0x7fa698075710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40af000 tx=0x7fa6a40b2040 comp rx=0 tx=0).stop 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 msgr2=0x7fa6bc15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c430 tx=0x7fa6ac00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 shutdown_connections 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa698073200 0x7fa698075710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6bc14f600 0x7fa6bc1649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6bc15f470 0x7fa6bc14e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 --2- 172.21.15.43:0/2029180639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6bc06fce0 0x7fa6bc15a620 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 >> 172.21.15.43:0/2029180639 conn(0x7fa6bc0b2810 msgr2=0x7fa6bc0b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:05.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 shutdown_connections 2024-09-17T05:25:05.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.733+0000 7fa6c3000640 1 -- 172.21.15.43:0/2029180639 wait complete. 2024-09-17T05:25:05.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:25:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- 172.21.15.43:0/2350671794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480d1160 msgr2=0x7efc480b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 --2- 172.21.15.43:0/2350671794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480d1160 0x7efc480b52e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc30009840 tx=0x7efc3001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- 172.21.15.43:0/2350671794 shutdown_connections 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 --2- 172.21.15.43:0/2350671794 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480b5820 0x7efc480b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 --2- 172.21.15.43:0/2350671794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480d1160 0x7efc480b52e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- 172.21.15.43:0/2350671794 >> 172.21.15.43:0/2350671794 conn(0x7efc480cef10 msgr2=0x7efc480cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- 172.21.15.43:0/2350671794 shutdown_connections 2024-09-17T05:25:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- 172.21.15.43:0/2350671794 wait complete. 2024-09-17T05:25:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 Processor -- start 2024-09-17T05:25:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.929+0000 7efc4d200640 1 -- start start 2024-09-17T05:25:05.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 0x7efc481659d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480d1160 0x7efc48165f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc4816a0a0 con 0x7efc480b5820 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc48169f20 con 0x7efc480d0060 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efc4816a220 con 0x7efc480d1160 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55024/0 (socket says 172.21.15.43:55024) 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 -- 172.21.15.43:0/1651277779 learned_addr learned my addr 172.21.15.43:0/1651277779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc3d400640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 0x7efc481659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc46800640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480d1160 0x7efc48165f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 -- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480d1160 msgr2=0x7efc48165f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480d1160 0x7efc48165f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 -- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 msgr2=0x7efc481659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 0x7efc481659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 -- 172.21.15.43:0/1651277779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc48165790 con 0x7efc480b5820 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc3d400640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 0x7efc481659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc45e00640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c430 tx=0x7efc3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38017070 con 0x7efc480b5820 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc38003600 con 0x7efc480b5820 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38003a30 con 0x7efc480b5820 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc480ca500 con 0x7efc480b5820 2024-09-17T05:25:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc480caac0 con 0x7efc480b5820 2024-09-17T05:25:05.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.933+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4806d9f0 con 0x7efc480b5820 2024-09-17T05:25:05.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.937+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc3801a070 con 0x7efc480b5820 2024-09-17T05:25:05.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.937+0000 7efc3ea00640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 0x7efc24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.937+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6026+0+0 (secure 0 0 0) 0x7efc380935c0 con 0x7efc480b5820 2024-09-17T05:25:05.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.937+0000 7efc3d400640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 0x7efc24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.937+0000 7efc3d400640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 0x7efc24075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc300af000 tx=0x7efc3001b030 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:05.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:05.941+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc3805e510 con 0x7efc480b5820 2024-09-17T05:25:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:06.117+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc480b5c30 con 0x7efc480b5820 2024-09-17T05:25:07.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.008+0000 7efc3ea00640 1 -- 172.21.15.43:0/1651277779 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7efc3805de10 con 0x7efc480b5820 2024-09-17T05:25:07.012 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:25:07.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 msgr2=0x7efc24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 0x7efc24075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc300af000 tx=0x7efc3001b030 comp rx=0 tx=0).stop 2024-09-17T05:25:07.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 msgr2=0x7efc48156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c430 tx=0x7efc3800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 shutdown_connections 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc24073200 0x7efc24075710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc480d0060 0x7efc481659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc480d1160 0x7efc48165f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 --2- 172.21.15.43:0/1651277779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc480b5820 0x7efc48156860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.012+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 >> 172.21.15.43:0/1651277779 conn(0x7efc480cef10 msgr2=0x7efc480728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.016+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 shutdown_connections 2024-09-17T05:25:07.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.016+0000 7efc4d200640 1 -- 172.21.15.43:0/1651277779 wait complete. 2024-09-17T05:25:07.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- 172.21.15.43:0/1424533509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 msgr2=0x7fa1240731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 --2- 172.21.15.43:0/1424533509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa1240731d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009bc0 tx=0x7fa11c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- 172.21.15.43:0/1424533509 shutdown_connections 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 --2- 172.21.15.43:0/1424533509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa1240731d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 --2- 172.21.15.43:0/1424533509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1240cf980 0x7fa1240b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- 172.21.15.43:0/1424533509 >> 172.21.15.43:0/1424533509 conn(0x7fa12406ce60 msgr2=0x7fa12406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- 172.21.15.43:0/1424533509 shutdown_connections 2024-09-17T05:25:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- 172.21.15.43:0/1424533509 wait complete. 2024-09-17T05:25:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 Processor -- start 2024-09-17T05:25:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.208+0000 7fa12b800640 1 -- start start 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 0x7fa12415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 0x7fa12415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa124168f00 con 0x7fa12415b7e0 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa124168d80 con 0x7fa124074c60 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa124169080 con 0x7fa124153450 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55046/0 (socket says 172.21.15.43:55046) 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 -- 172.21.15.43:0/1624204183 learned_addr learned my addr 172.21.15.43:0/1624204183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa129800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 0x7fa12415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa128e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 0x7fa12415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 -- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 msgr2=0x7fa12415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 0x7fa12415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 -- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 msgr2=0x7fa12415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 0x7fa12415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 -- 172.21.15.43:0/1624204183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa124141240 con 0x7fa12415b7e0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa129800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 0x7fa12415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa128e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 0x7fa12415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa123e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d950 tx=0x7fa11000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110017070 con 0x7fa12415b7e0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa110009d20 con 0x7fa12415b7e0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100106c0 con 0x7fa12415b7e0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12409e1e0 con 0x7fa12415b7e0 2024-09-17T05:25:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241417e0 con 0x7fa12415b7e0 2024-09-17T05:25:07.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.212+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1240b3b60 con 0x7fa12415b7e0 2024-09-17T05:25:07.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.216+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa11001a070 con 0x7fa12415b7e0 2024-09-17T05:25:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.216+0000 7fa120c00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 0x7fa104075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.216+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5588+0+0 (secure 0 0 0) 0x7fa1100943f0 con 0x7fa12415b7e0 2024-09-17T05:25:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.216+0000 7fa128e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 0x7fa104075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.216+0000 7fa128e00640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 0x7fa104075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa118001230 tx=0x7fa118009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.220+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa11005f470 con 0x7fa12415b7e0 2024-09-17T05:25:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:07.400+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa124070150 con 0x7fa12415b7e0 2024-09-17T05:25:08.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.012+0000 7fa120c00640 1 -- 172.21.15.43:0/1624204183 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7fa11005ed70 con 0x7fa12415b7e0 2024-09-17T05:25:08.018 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:25:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.020+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 msgr2=0x7fa104075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.020+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 0x7fa104075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa118001230 tx=0x7fa118009040 comp rx=0 tx=0).stop 2024-09-17T05:25:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.020+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 msgr2=0x7fa12409dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.020+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa11000d950 tx=0x7fa11000de10 comp rx=0 tx=0).stop 2024-09-17T05:25:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 shutdown_connections 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa104073200 0x7fa104075710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa124153450 0x7fa12415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12415b7e0 0x7fa12409dca0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 --2- 172.21.15.43:0/1624204183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa124074c60 0x7fa12415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 >> 172.21.15.43:0/1624204183 conn(0x7fa12406ce60 msgr2=0x7fa12406f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 shutdown_connections 2024-09-17T05:25:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.024+0000 7fa12b800640 1 -- 172.21.15.43:0/1624204183 wait complete. 2024-09-17T05:25:08.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:25:08.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 -- 172.21.15.43:0/2782323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 msgr2=0x7f99f4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 --2- 172.21.15.43:0/2782323634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f99f4003f90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009b70 tx=0x7f9a0001d270 comp rx=0 tx=0).stop 2024-09-17T05:25:08.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a14c00640 1 -- 172.21.15.43:0/2782323634 reap_dead start 2024-09-17T05:25:08.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 -- 172.21.15.43:0/2782323634 shutdown_connections 2024-09-17T05:25:08.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 -- 172.21.15.43:0/2782323634 >> 172.21.15.43:0/2782323634 conn(0x7f9a100b23d0 msgr2=0x7f9a100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 -- 172.21.15.43:0/2782323634 shutdown_connections 2024-09-17T05:25:08.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.220+0000 7f9a16c00640 1 -- 172.21.15.43:0/2782323634 wait complete. 2024-09-17T05:25:08.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 Processor -- start 2024-09-17T05:25:08.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- start start 2024-09-17T05:25:08.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a100cc100 0x7f9a101fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a101fc970 0x7f9a101fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10168e60 con 0x7f99f4003b20 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a100b51a0 con 0x7f9a100cc100 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9a10168fe0 con 0x7f9a101fc970 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55068/0 (socket says 172.21.15.43:55068) 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 -- 172.21.15.43:0/3977487264 learned_addr learned my addr 172.21.15.43:0/3977487264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a07e00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a100cc100 0x7f9a101fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a14c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a101fc970 0x7f9a101fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 -- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a101fc970 msgr2=0x7f9a101fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a101fc970 0x7f9a101fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 -- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a100cc100 msgr2=0x7f9a101fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a100cc100 0x7f9a101fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 -- 172.21.15.43:0/3977487264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10163fa0 con 0x7f99f4003b20 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0fe00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a0001cce0 tx=0x7f9a00005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000bb030 con 0x7f99f4003b20 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a000046d0 con 0x7f99f4003b20 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00004aa0 con 0x7f99f4003b20 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a10164280 con 0x7f99f4003b20 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.224+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a101666d0 con 0x7f99f4003b20 2024-09-17T05:25:08.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1006ce60 con 0x7f99f4003b20 2024-09-17T05:25:08.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a000be030 con 0x7f99f4003b20 2024-09-17T05:25:08.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a0cc00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 0x7f99ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f9a0012f3a0 con 0x7f99f4003b20 2024-09-17T05:25:08.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a07e00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 0x7f99ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.228+0000 7f9a07e00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 0x7f99ec075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a101fd4f0 tx=0x7f99f8006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.232+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9a00016fc0 con 0x7f99f4003b20 2024-09-17T05:25:08.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.408+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9a1015e1f0 con 0x7f99f4003b20 2024-09-17T05:25:08.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.412+0000 7f9a0cc00640 1 -- 172.21.15.43:0/3977487264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f9a000f9f70 con 0x7f99f4003b20 2024-09-17T05:25:08.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 msgr2=0x7f99ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 0x7f99ec075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a101fd4f0 tx=0x7f99f8006040 comp rx=0 tx=0).stop 2024-09-17T05:25:08.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 msgr2=0x7f9a10163410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a0001cce0 tx=0x7f9a00005500 comp rx=0 tx=0).stop 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 shutdown_connections 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99ec073230 0x7f99ec075740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a100cc100 0x7f9a101fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a101fc970 0x7f9a101fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 --2- 172.21.15.43:0/3977487264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99f4003b20 0x7f9a10163410 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 >> 172.21.15.43:0/3977487264 conn(0x7f9a100b23d0 msgr2=0x7f9a100ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 shutdown_connections 2024-09-17T05:25:08.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.416+0000 7f9a16c00640 1 -- 172.21.15.43:0/3977487264 wait complete. 2024-09-17T05:25:08.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-17T05:25:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- 172.21.15.43:0/4178711134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80ce6c0 msgr2=0x7f20d8166860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 --2- 172.21.15.43:0/4178711134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80ce6c0 0x7f20d8166860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bc0 tx=0x7f20d001cac0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- 172.21.15.43:0/4178711134 shutdown_connections 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 --2- 172.21.15.43:0/4178711134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80ce6c0 0x7f20d8166860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- 172.21.15.43:0/4178711134 >> 172.21.15.43:0/4178711134 conn(0x7f20d80b2e80 msgr2=0x7f20d80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- 172.21.15.43:0/4178711134 shutdown_connections 2024-09-17T05:25:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- 172.21.15.43:0/4178711134 wait complete. 2024-09-17T05:25:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 Processor -- start 2024-09-17T05:25:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- start start 2024-09-17T05:25:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80b6840 0x7f20d81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20d80ce6c0 0x7f20d8153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20d816a000 con 0x7f20d80b45a0 2024-09-17T05:25:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8169e80 con 0x7f20d80b6840 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.596+0000 7f20dde00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f20d816a180 con 0x7f20d80ce6c0 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55072/0 (socket says 172.21.15.43:55072) 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 -- 172.21.15.43:0/3648099703 learned_addr learned my addr 172.21.15.43:0/3648099703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20cfe00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80b6840 0x7f20d81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d7400640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20d80ce6c0 0x7f20d8153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 -- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20d80ce6c0 msgr2=0x7f20d8153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20d80ce6c0 0x7f20d8153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 -- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80b6840 msgr2=0x7f20d81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80b6840 0x7f20d81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 -- 172.21.15.43:0/3648099703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d815fca0 con 0x7f20d80b45a0 2024-09-17T05:25:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20d6a00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c800c450 tx=0x7f20c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80052d0 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c8005470 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d815f400 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8012870 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d815f9c0 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d80b6130 con 0x7f20d80b45a0 2024-09-17T05:25:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.600+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20c8003600 con 0x7f20d80b45a0 2024-09-17T05:25:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.604+0000 7f20cf400640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 0x7f20b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.604+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f20c8093940 con 0x7f20d80b45a0 2024-09-17T05:25:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.604+0000 7f20cfe00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 0x7f20b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.604+0000 7f20cfe00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 0x7f20b4075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20c0001ec0 tx=0x7f20c00016a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.608+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20c805e620 con 0x7f20d80b45a0 2024-09-17T05:25:08.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:08.840+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f20d80b7120 con 0x7f20d80b45a0 2024-09-17T05:25:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20cf400640 1 -- 172.21.15.43:0/3648099703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v113) ==== 134+0+0 (secure 0 0 0) 0x7f20c8063050 con 0x7f20d80b45a0 2024-09-17T05:25:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 msgr2=0x7f20b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 0x7f20b4075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20c0001ec0 tx=0x7f20c00016a0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 msgr2=0x7f20d81563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c800c450 tx=0x7f20c800c910 comp rx=0 tx=0).stop 2024-09-17T05:25:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 shutdown_connections 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f20b4073200 0x7f20b4075710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d80b6840 0x7f20d81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f20d80ce6c0 0x7f20d8153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 --2- 172.21.15.43:0/3648099703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d80b45a0 0x7f20d81563b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 >> 172.21.15.43:0/3648099703 conn(0x7f20d80b2e80 msgr2=0x7f20d8155280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 shutdown_connections 2024-09-17T05:25:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.016+0000 7f20dde00640 1 -- 172.21.15.43:0/3648099703 wait complete. 2024-09-17T05:25:09.042 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-09-17T05:25:09.042 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-09-17T05:25:09.042 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:25:09.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- 172.21.15.43:0/1539499140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be006ce60 msgr2=0x7f6be006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 --2- 172.21.15.43:0/1539499140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be006ce60 0x7f6be006d270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009840 tx=0x7f6bcc01c720 comp rx=0 tx=0).stop 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- 172.21.15.43:0/1539499140 shutdown_connections 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 --2- 172.21.15.43:0/1539499140 >> v2:172.21.15.62:3300/0 conn(0x7f6be00b58e0 0x7f6be00b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 --2- 172.21.15.43:0/1539499140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be006ce60 0x7f6be006d270 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- 172.21.15.43:0/1539499140 >> 172.21.15.43:0/1539499140 conn(0x7f6be00b2e80 msgr2=0x7f6be00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- 172.21.15.43:0/1539499140 shutdown_connections 2024-09-17T05:25:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- 172.21.15.43:0/1539499140 wait complete. 2024-09-17T05:25:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 Processor -- start 2024-09-17T05:25:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.200+0000 7f6be7600640 1 -- start start 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6be006ce60 0x7f6be01fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6be00b58e0 0x7f6be00d1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6be0169d20 con 0x7f6be00b4180 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6be0169ba0 con 0x7f6be00b58e0 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6be0169ea0 con 0x7f6be006ce60 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55092/0 (socket says 172.21.15.43:55092) 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 -- 172.21.15.43:0/879264229 learned_addr learned my addr 172.21.15.43:0/879264229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be5600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6be00b58e0 0x7f6be00d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be4c00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6be006ce60 0x7f6be01fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 -- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6be006ce60 msgr2=0x7f6be01fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6be006ce60 0x7f6be01fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 -- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6be00b58e0 msgr2=0x7f6be00d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6be00b58e0 0x7f6be00d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 -- 172.21.15.43:0/879264229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be01f8b90 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdfe00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc01cc00 tx=0x7f6bcc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0b2040 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bcc004130 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be00c7840 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0044e0 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be01f9120 con 0x7f6be00b4180 2024-09-17T05:25:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.204+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be006d270 con 0x7f6be00b4180 2024-09-17T05:25:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.208+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6bcc0bd030 con 0x7f6be00b4180 2024-09-17T05:25:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.208+0000 7f6bdcc00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 0x7f6bc00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.208+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f6bcc12dd00 con 0x7f6be00b4180 2024-09-17T05:25:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.208+0000 7f6be4c00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 0x7f6bc00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.208+0000 7f6be4c00640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 0x7f6bc00758b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40046c0 tx=0x7f6bd4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.212+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6bcc0c4020 con 0x7f6be00b4180 2024-09-17T05:25:09.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.388+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6be00c75d0 con 0x7f6be00b4180 2024-09-17T05:25:09.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6bdcc00640 1 -- 172.21.15.43:0/879264229 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13100 (secure 0 0 0) 0x7f6bcc0163d0 con 0x7f6be00b4180 2024-09-17T05:25:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:09.396 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:08.012696+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/331818553":"2024-09-18T05:25:04.729854+0000","172.21.15.43:6834/331818553":"2024-09-18T05:25:04.729854+0000","172.21.15.43:6835/1404655827":"2024-09-18T05:24:19.717260+0000","172.21.15.43:6834/1404655827":"2024-09-18T05:24:19.717260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 msgr2=0x7f6bc00758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 0x7f6bc00758b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40046c0 tx=0x7f6bd4009040 comp rx=0 tx=0).stop 2024-09-17T05:25:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 msgr2=0x7f6be0146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc01cc00 tx=0x7f6bcc005db0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 shutdown_connections 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bc00733a0 0x7f6bc00758b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6be00b58e0 0x7f6be00d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be00b4180 0x7f6be0146640 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 --2- 172.21.15.43:0/879264229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6be006ce60 0x7f6be01fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 >> 172.21.15.43:0/879264229 conn(0x7f6be00b2e80 msgr2=0x7f6be00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 shutdown_connections 2024-09-17T05:25:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.392+0000 7f6be7600640 1 -- 172.21.15.43:0/879264229 wait complete. 2024-09-17T05:25:09.418 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:25:09.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:25:09.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.572+0000 7f19cf200640 1 -- 172.21.15.43:0/3685392189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 msgr2=0x7f19c80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.572+0000 7f19cf200640 1 --2- 172.21.15.43:0/3685392189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c80ad0b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f19c8072b60 tx=0x7f19c001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- 172.21.15.43:0/3685392189 shutdown_connections 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- 172.21.15.43:0/3685392189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c80ad0b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- 172.21.15.43:0/3685392189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c806f110 0x7f19c80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- 172.21.15.43:0/3685392189 >> v2:172.21.15.62:3300/0 conn(0x7f19c8155d70 0x7f19c815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- 172.21.15.43:0/3685392189 >> 172.21.15.43:0/3685392189 conn(0x7f19c80b23d0 msgr2=0x7f19c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- 172.21.15.43:0/3685392189 shutdown_connections 2024-09-17T05:25:09.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- 172.21.15.43:0/3685392189 wait complete. 2024-09-17T05:25:09.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 Processor -- start 2024-09-17T05:25:09.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- start start 2024-09-17T05:25:09.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 0x7f19c80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c8155d70 0x7f19c809d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19c815f8d0 con 0x7f19c80b3920 2024-09-17T05:25:09.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19c815f750 con 0x7f19c806f110 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.576+0000 7f19cf200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f19c815fa50 con 0x7f19c8155d70 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55098/0 (socket says 172.21.15.43:55098) 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 -- 172.21.15.43:0/4124693692 learned_addr learned my addr 172.21.15.43:0/4124693692 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c7e00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 0x7f19c80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19cd200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c8155d70 0x7f19c809d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 -- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c8155d70 msgr2=0x7f19c809d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c8155d70 0x7f19c809d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 -- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 msgr2=0x7f19c80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 0x7f19c80cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 -- 172.21.15.43:0/4124693692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c80cdad0 con 0x7f19c80b3920 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c7e00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 0x7f19c80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19bfe00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d720 tx=0x7f19b000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017070 con 0x7f19c80b3920 2024-09-17T05:25:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b0009d20 con 0x7f19c80b3920 2024-09-17T05:25:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0005280 con 0x7f19c80b3920 2024-09-17T05:25:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c81524a0 con 0x7f19c80b3920 2024-09-17T05:25:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c8152a60 con 0x7f19c80b3920 2024-09-17T05:25:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c815ff20 con 0x7f19c80b3920 2024-09-17T05:25:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.580+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19b0005420 con 0x7f19c80b3920 2024-09-17T05:25:09.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.584+0000 7f19c4c00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 0x7f19a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.584+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f19b0093ee0 con 0x7f19c80b3920 2024-09-17T05:25:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.584+0000 7f19c7e00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 0x7f19a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.584+0000 7f19c7e00640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 0x7f19a4075710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19b8005950 tx=0x7f19b8005880 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.588+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19b005ec70 con 0x7f19c80b3920 2024-09-17T05:25:09.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.760+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19c80d02d0 con 0x7f19c80b3920 2024-09-17T05:25:09.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.760+0000 7f19c4c00640 1 -- 172.21.15.43:0/4124693692 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f19b00636a0 con 0x7f19c80b3920 2024-09-17T05:25:09.767 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 msgr2=0x7f19a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 0x7f19a4075710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19b8005950 tx=0x7f19b8005880 comp rx=0 tx=0).stop 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 msgr2=0x7f19c815de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d720 tx=0x7f19b000dbe0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 shutdown_connections 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19a4073200 0x7f19a4075710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19c8155d70 0x7f19c809d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b3920 0x7f19c815de10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 --2- 172.21.15.43:0/4124693692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c806f110 0x7f19c80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 >> 172.21.15.43:0/4124693692 conn(0x7f19c80b23d0 msgr2=0x7f19c815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 shutdown_connections 2024-09-17T05:25:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.764+0000 7f19cf200640 1 -- 172.21.15.43:0/4124693692 wait complete. 2024-09-17T05:25:09.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 -- 172.21.15.43:0/3351182185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc80b6670 msgr2=0x7fafc815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 --2- 172.21.15.43:0/3351182185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc80b6670 0x7fafc815f530 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fafc8073830 tx=0x7fafb001c830 comp rx=0 tx=0).stop 2024-09-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 -- 172.21.15.43:0/3351182185 shutdown_connections 2024-09-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 --2- 172.21.15.43:0/3351182185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc80b6670 0x7fafc815f530 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 --2- 172.21.15.43:0/3351182185 >> v2:172.21.15.62:3300/0 conn(0x7fafc80b45a0 0x7fafc80b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 -- 172.21.15.43:0/3351182185 >> 172.21.15.43:0/3351182185 conn(0x7fafc80b2e80 msgr2=0x7fafc80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 -- 172.21.15.43:0/3351182185 shutdown_connections 2024-09-17T05:25:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.940+0000 7fafcd200640 1 -- 172.21.15.43:0/3351182185 wait complete. 2024-09-17T05:25:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 Processor -- start 2024-09-17T05:25:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 -- start start 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafc80b45a0 0x7fafc8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc80b6670 0x7fafc80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8168c00 con 0x7fafc815fa70 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8168a80 con 0x7fafc80b6670 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafcd200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fafc8168d80 con 0x7fafc80b45a0 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.944+0000 7fafc6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55106/0 (socket says 172.21.15.43:55106) 2024-09-17T05:25:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafc80b45a0 0x7fafc8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafbfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc80b6670 0x7fafc80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 -- 172.21.15.43:0/1325027901 learned_addr learned my addr 172.21.15.43:0/1325027901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 -- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafc80b45a0 msgr2=0x7fafc8151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafc80b45a0 0x7fafc8151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 -- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc80b6670 msgr2=0x7fafc80c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc80b6670 0x7fafc80c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 -- 172.21.15.43:0/1325027901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc8151f80 con 0x7fafc815fa70 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafc6800640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafc000be60 tx=0x7fafc000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0019070 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafc0014440 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0014890 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc8151540 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc8151ca0 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc80b6130 con 0x7fafc815fa70 2024-09-17T05:25:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.948+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafc001c070 con 0x7fafc815fa70 2024-09-17T05:25:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.952+0000 7fafbe000640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 0x7fafa4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.952+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fafc0093f20 con 0x7fafc815fa70 2024-09-17T05:25:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.952+0000 7fafc5e00640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 0x7fafa4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.952+0000 7fafc5e00640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 0x7fafa4075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafb8004630 tx=0x7fafb800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:09.956+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafc005eb80 con 0x7fafc815fa70 2024-09-17T05:25:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:10.196+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fafc80b7120 con 0x7fafc815fa70 2024-09-17T05:25:11.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafbe000640 1 -- 172.21.15.43:0/1325027901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v115) ==== 144+0+0 (secure 0 0 0) 0x7fafc00635b0 con 0x7fafc815fa70 2024-09-17T05:25:11.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 msgr2=0x7fafa4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 0x7fafa4075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafb8004630 tx=0x7fafb800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 msgr2=0x7fafc80ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafc000be60 tx=0x7fafc000bf50 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 shutdown_connections 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fafa4073200 0x7fafa4075710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc80b6670 0x7fafc80c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc815fa70 0x7fafc80ca5b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 --2- 172.21.15.43:0/1325027901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafc80b45a0 0x7fafc8151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 >> 172.21.15.43:0/1325027901 conn(0x7fafc80b2e80 msgr2=0x7fafc8070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 shutdown_connections 2024-09-17T05:25:11.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.024+0000 7fafcd200640 1 -- 172.21.15.43:0/1325027901 wait complete. 2024-09-17T05:25:11.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:25:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 -- 172.21.15.43:0/2809247722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 msgr2=0x7f6cd80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 --2- 172.21.15.43:0/2809247722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd80cce70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6cd809bbe0 tx=0x7f6cc401c820 comp rx=0 tx=0).stop 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 -- 172.21.15.43:0/2809247722 shutdown_connections 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 --2- 172.21.15.43:0/2809247722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd80cce70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 -- 172.21.15.43:0/2809247722 >> 172.21.15.43:0/2809247722 conn(0x7f6cd80cee80 msgr2=0x7f6cd80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 -- 172.21.15.43:0/2809247722 shutdown_connections 2024-09-17T05:25:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.212+0000 7f6cdf600640 1 -- 172.21.15.43:0/2809247722 wait complete. 2024-09-17T05:25:11.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 Processor -- start 2024-09-17T05:25:11.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- start start 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd80b7ce0 0x7f6cd8159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd80cd3b0 0x7f6cd814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd816b030 con 0x7f6cd80d0f50 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd816aeb0 con 0x7f6cd80b7ce0 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd816b1b0 con 0x7f6cd80cd3b0 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55112/0 (socket says 172.21.15.43:55112) 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 -- 172.21.15.43:0/3808333119 learned_addr learned my addr 172.21.15.43:0/3808333119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdcc00640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd80b7ce0 0x7f6cd8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cd7e00640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd80cd3b0 0x7f6cd814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 -- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd80cd3b0 msgr2=0x7f6cd814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd80cd3b0 0x7f6cd814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 -- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd80b7ce0 msgr2=0x7f6cd8159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd80b7ce0 0x7f6cd8159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 -- 172.21.15.43:0/3808333119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd8156b80 con 0x7f6cd80d0f50 2024-09-17T05:25:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdd600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c730 tx=0x7f6cd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0003b80 con 0x7f6cd80d0f50 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cd0003d20 con 0x7f6cd80d0f50 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd8156e60 con 0x7f6cd80d0f50 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000aab0 con 0x7f6cd80d0f50 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.216+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd8157420 con 0x7f6cd80d0f50 2024-09-17T05:25:11.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd80cce70 con 0x7f6cd80d0f50 2024-09-17T05:25:11.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6cd000ac50 con 0x7f6cd80d0f50 2024-09-17T05:25:11.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cd4c00640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 0x7f6cb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f6cd0093c60 con 0x7f6cd80d0f50 2024-09-17T05:25:11.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cdcc00640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 0x7f6cb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.220+0000 7f6cdcc00640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 0x7f6cb8075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007fd0 tx=0x7f6ccc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:11.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.224+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6cd005e8d0 con 0x7f6cd80d0f50 2024-09-17T05:25:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.400+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f6cd80b7240 con 0x7f6cd80d0f50 2024-09-17T05:25:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.400+0000 7f6cd4c00640 1 -- 172.21.15.43:0/3808333119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7f6cd0063300 con 0x7f6cd80d0f50 2024-09-17T05:25:11.406 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:25:11.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:25:11.408 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:25:11.409 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:25:11.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 msgr2=0x7f6cb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 0x7f6cb8075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007fd0 tx=0x7f6ccc009040 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 msgr2=0x7f6cd81548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c730 tx=0x7f6cd000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 shutdown_connections 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6cb8073200 0x7f6cb8075710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cd80d0f50 0x7f6cd81548f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cd80cd3b0 0x7f6cd814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 --2- 172.21.15.43:0/3808333119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd80b7ce0 0x7f6cd8159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 >> 172.21.15.43:0/3808333119 conn(0x7f6cd80cee80 msgr2=0x7f6cd80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 shutdown_connections 2024-09-17T05:25:11.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.404+0000 7f6cdf600640 1 -- 172.21.15.43:0/3808333119 wait complete. 2024-09-17T05:25:11.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:25:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 -- 172.21.15.43:0/2412182436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4070620 msgr2=0x7f39d40b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 --2- 172.21.15.43:0/2412182436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4070620 0x7f39d40b30d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39d4073040 tx=0x7f39c401c770 comp rx=0 tx=0).stop 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 -- 172.21.15.43:0/2412182436 shutdown_connections 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 --2- 172.21.15.43:0/2412182436 >> v2:172.21.15.62:3300/0 conn(0x7f39d406d570 0x7f39d40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 --2- 172.21.15.43:0/2412182436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4070620 0x7f39d40b30d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 -- 172.21.15.43:0/2412182436 >> 172.21.15.43:0/2412182436 conn(0x7f39d40b23d0 msgr2=0x7f39d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 -- 172.21.15.43:0/2412182436 shutdown_connections 2024-09-17T05:25:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.584+0000 7f39d9600640 1 -- 172.21.15.43:0/2412182436 wait complete. 2024-09-17T05:25:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 Processor -- start 2024-09-17T05:25:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 -- start start 2024-09-17T05:25:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 0x7f39d409d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 0x7f39d41fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40ce6b0 con 0x7f39d415a3b0 2024-09-17T05:25:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40ce530 con 0x7f39d4070620 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d9600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f39d40ce830 con 0x7f39d40d0910 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55128/0 (socket says 172.21.15.43:55128) 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 -- 172.21.15.43:0/2653794736 learned_addr learned my addr 172.21.15.43:0/2653794736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d2a00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 0x7f39d409d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39cbe00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 0x7f39d41fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 -- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 msgr2=0x7f39d41fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 0x7f39d41fc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 -- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 msgr2=0x7f39d409d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 0x7f39d409d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d3400640 1 -- 172.21.15.43:0/2653794736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40c4460 con 0x7f39d415a3b0 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39d2a00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 0x7f39d409d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:11.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.588+0000 7f39cbe00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 0x7f39d41fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39d3400640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00eab0 tx=0x7f39cc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc00cea0 con 0x7f39d415a3b0 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39cc004b30 con 0x7f39d415a3b0 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d40c4740 con 0x7f39d415a3b0 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc005020 con 0x7f39d415a3b0 2024-09-17T05:25:11.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d40c4d00 con 0x7f39d415a3b0 2024-09-17T05:25:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d40cee00 con 0x7f39d415a3b0 2024-09-17T05:25:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f39cc0051c0 con 0x7f39d415a3b0 2024-09-17T05:25:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 0x7f39b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.592+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f39cc094ae0 con 0x7f39d415a3b0 2024-09-17T05:25:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.596+0000 7f39d2a00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 0x7f39b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.596+0000 7f39d2a00640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 0x7f39b0075740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39c40af000 tx=0x7f39c401afb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:11.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.596+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39cc05fec0 con 0x7f39d415a3b0 2024-09-17T05:25:11.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.772+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f39d409e190 con 0x7f39d415a3b0 2024-09-17T05:25:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.776+0000 7f39caa00640 1 -- 172.21.15.43:0/2653794736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f39cc05f7c0 con 0x7f39d415a3b0 2024-09-17T05:25:11.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 msgr2=0x7f39b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 0x7f39b0075740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39c40af000 tx=0x7f39c401afb0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 msgr2=0x7f39d41fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00eab0 tx=0x7f39cc00ef70 comp rx=0 tx=0).stop 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 shutdown_connections 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39b0073230 0x7f39b0075740 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d415a3b0 0x7f39d41fcc00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39d40d0910 0x7f39d41fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 --2- 172.21.15.43:0/2653794736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39d4070620 0x7f39d409d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 >> 172.21.15.43:0/2653794736 conn(0x7f39d40b23d0 msgr2=0x7f39d40af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 shutdown_connections 2024-09-17T05:25:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:11.780+0000 7f39d9600640 1 -- 172.21.15.43:0/2653794736 wait complete. 2024-09-17T05:25:11.804 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:25:11.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:11.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:11.955 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:25:11.956 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:11.956 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:11.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:11.965 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:11.965 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:11.974 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:11.974 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:11.974 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:12.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:12.015 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:12.015 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:12.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:12.029 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:25:12.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- 172.21.15.43:0/36491081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c0b3af0 msgr2=0x7fe90c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 --2- 172.21.15.43:0/36491081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c0b3af0 0x7fe90c15c7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009840 tx=0x7fe8f401c790 comp rx=0 tx=0).stop 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- 172.21.15.43:0/36491081 shutdown_connections 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 --2- 172.21.15.43:0/36491081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c0b3af0 0x7fe90c15c7e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- 172.21.15.43:0/36491081 >> 172.21.15.43:0/36491081 conn(0x7fe90c0b23d0 msgr2=0x7fe90c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- 172.21.15.43:0/36491081 shutdown_connections 2024-09-17T05:25:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- 172.21.15.43:0/36491081 wait complete. 2024-09-17T05:25:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 Processor -- start 2024-09-17T05:25:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.216+0000 7fe912800640 1 -- start start 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe90c06f110 0x7fe90c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90c155d70 0x7fe90c09c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c169e10 con 0x7fe90c155590 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c169c90 con 0x7fe90c155d70 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe90c169f90 con 0x7fe90c06f110 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55144/0 (socket says 172.21.15.43:55144) 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 -- 172.21.15.43:0/3087599473 learned_addr learned my addr 172.21.15.43:0/3087599473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90b400640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe90c06f110 0x7fe90c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe903e00640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90c155d70 0x7fe90c09c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 -- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe90c06f110 msgr2=0x7fe90c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe90c06f110 0x7fe90c154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 -- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90c155d70 msgr2=0x7fe90c09c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90c155d70 0x7fe90c09c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 -- 172.21.15.43:0/3087599473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c155330 con 0x7fe90c155590 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe90be00640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe90400cb10 tx=0x7fe90400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904016020 con 0x7fe90c155590 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90400ce50 con 0x7fe90c155590 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ada0 con 0x7fe90c155590 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c075100 con 0x7fe90c155590 2024-09-17T05:25:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c0c3170 con 0x7fe90c155590 2024-09-17T05:25:12.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.220+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c15c7e0 con 0x7fe90c155590 2024-09-17T05:25:12.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.224+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe904019020 con 0x7fe90c155590 2024-09-17T05:25:12.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.224+0000 7fe903400640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 0x7fe8e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.224+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7fe904093d40 con 0x7fe90c155590 2024-09-17T05:25:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.224+0000 7fe90b400640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 0x7fe8e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.224+0000 7fe90b400640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 0x7fe8e8075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0715c0 tx=0x7fe8fc009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.228+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe904020030 con 0x7fe90c155590 2024-09-17T05:25:12.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.436+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe90c0b00c0 con 0x7fe90c155590 2024-09-17T05:25:12.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.436+0000 7fe903400640 1 -- 172.21.15.43:0/3087599473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3908 (secure 0 0 0) 0x7fe90405ea30 con 0x7fe90c155590 2024-09-17T05:25:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:12.442 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":25,"btime":"2024-09-17T05:25:05:732361+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1373671179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1373671179},{"type":"v1","addr":"172.21.15.43:6833","nonce":1373671179}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2981766147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2981766147},{"type":"v1","addr":"172.21.15.62:6837","nonce":2981766147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3418470499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3418470499},{"type":"v1","addr":"172.21.15.62:6835","nonce":3418470499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/913635583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":913635583},{"type":"v1","addr":"172.21.15.43:6835","nonce":913635583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-09-17T05:25:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 25 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 msgr2=0x7fe8e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 0x7fe8e8075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0715c0 tx=0x7fe8fc009060 comp rx=0 tx=0).stop 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 msgr2=0x7fe90c09c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe90400cb10 tx=0x7fe90400cc00 comp rx=0 tx=0).stop 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 shutdown_connections 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe8e8073200 0x7fe8e8075710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe90c155d70 0x7fe90c09c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c155590 0x7fe90c09c640 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 --2- 172.21.15.43:0/3087599473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe90c06f110 0x7fe90c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 >> 172.21.15.43:0/3087599473 conn(0x7fe90c0b23d0 msgr2=0x7fe90c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 shutdown_connections 2024-09-17T05:25:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.440+0000 7fe912800640 1 -- 172.21.15.43:0/3087599473 wait complete. 2024-09-17T05:25:12.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:25:12.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:25:12.465 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:25:12.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:12.465 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:25:12.465 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:25:12.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:25:12.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:12.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:25:12.466 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:25:12.466 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:25:12.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:12.466 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:25:12.466 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:25:12.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:25:12.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:12.477 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:25:12.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:25:12.482 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:25:12.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:25:12.483 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:25:12.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:25:12.485 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:25:12.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:25:12.529 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:25:12.530 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:12.531 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:25:12.532 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:12.533 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:25:12.533 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:12.535 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:25:12.535 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:12.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:25:12.631 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:25:12.632 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:25:12.641 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:25:12.642 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:25:12.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- 172.21.15.43:0/1390125490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b540cb860 msgr2=0x7f2b540cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1390125490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b540cb860 0x7f2b540cbc70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009bc0 tx=0x7f2b4c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:25:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- 172.21.15.43:0/1390125490 shutdown_connections 2024-09-17T05:25:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1390125490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b540cb860 0x7f2b540cbc70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1390125490 >> v2:172.21.15.62:3300/0 conn(0x7f2b54070620 0x7f2b540b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- 172.21.15.43:0/1390125490 >> 172.21.15.43:0/1390125490 conn(0x7f2b540b23d0 msgr2=0x7f2b540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:12.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- 172.21.15.43:0/1390125490 shutdown_connections 2024-09-17T05:25:12.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- 172.21.15.43:0/1390125490 wait complete. 2024-09-17T05:25:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 Processor -- start 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.700+0000 7f2b5b000640 1 -- start start 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b54070620 0x7f2b540906a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b540cb860 0x7f2b540ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540ce680 con 0x7f2b54070620 2024-09-17T05:25:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540ce500 con 0x7f2b540cb860 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b540ce800 con 0x7f2b5406ce60 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b54070620 0x7f2b540906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:42362/0 (socket says 172.21.15.43:42362) 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 -- 172.21.15.43:0/1017351450 learned_addr learned my addr 172.21.15.43:0/1017351450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b59000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b540cb860 0x7f2b540ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 -- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b540cb860 msgr2=0x7f2b540ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b540cb860 0x7f2b540ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 -- 172.21.15.43:0/1017351450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b54070620 msgr2=0x7f2b540906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b54070620 0x7f2b540906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 -- 172.21.15.43:0/1017351450 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5409cf70 con 0x7f2b5406ce60 2024-09-17T05:25:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b53e00640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400eac0 tx=0x7f2b4400ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400cc80 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b44005020 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44005470 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5409c1c0 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5409c9b0 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.704+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b540cedb0 con 0x7f2b5406ce60 2024-09-17T05:25:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.708+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b440047b0 con 0x7f2b5406ce60 2024-09-17T05:25:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.708+0000 7f2b4be00640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 0x7f2b34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.708+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 5029+0+0 (secure 0 0 0) 0x7f2b44094370 con 0x7f2b5406ce60 2024-09-17T05:25:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.708+0000 7f2b53400640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 0x7f2b34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.708+0000 7f2b53400640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 0x7f2b34075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2b541581d0 tx=0x7f2b400013a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.712+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b4405efd0 con 0x7f2b5406ce60 2024-09-17T05:25:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:12.904+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b540d0580 con 0x7f2b5406ce60 2024-09-17T05:25:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b4be00640 1 -- 172.21.15.43:0/1017351450 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7f2b44063a00 con 0x7f2b5406ce60 2024-09-17T05:25:13.804 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:25:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 msgr2=0x7f2b34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 0x7f2b34075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2b541581d0 tx=0x7f2b400013a0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 msgr2=0x7f2b5409bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400eac0 tx=0x7f2b4400ef80 comp rx=0 tx=0).stop 2024-09-17T05:25:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 shutdown_connections 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b34073200 0x7f2b34075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b540cb860 0x7f2b540ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.800+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b54070620 0x7f2b540906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.804+0000 7f2b5b000640 1 --2- 172.21.15.43:0/1017351450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b5406ce60 0x7f2b5409bc80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.804+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 >> 172.21.15.43:0/1017351450 conn(0x7f2b540b23d0 msgr2=0x7f2b540b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.804+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 shutdown_connections 2024-09-17T05:25:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.804+0000 7f2b5b000640 1 -- 172.21.15.43:0/1017351450 wait complete. 2024-09-17T05:25:13.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:25:13.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- 172.21.15.43:0/1979308363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64940b3af0 msgr2=0x7f649415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- 172.21.15.43:0/1979308363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64940b3af0 0x7f649415c7e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f64940d0c40 tx=0x7f647c01c800 comp rx=0 tx=0).stop 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- 172.21.15.43:0/1979308363 shutdown_connections 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- 172.21.15.43:0/1979308363 >> v2:172.21.15.62:3300/0 conn(0x7f649406f110 0x7f64940ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- 172.21.15.43:0/1979308363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64940b3af0 0x7f649415c7e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- 172.21.15.43:0/1979308363 >> 172.21.15.43:0/1979308363 conn(0x7f64940b23d0 msgr2=0x7f64940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- 172.21.15.43:0/1979308363 shutdown_connections 2024-09-17T05:25:13.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- 172.21.15.43:0/1979308363 wait complete. 2024-09-17T05:25:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 Processor -- start 2024-09-17T05:25:13.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- start start 2024-09-17T05:25:13.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6494155d70 0x7f649415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f649409d860 0x7f64940cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6494168c30 con 0x7f649406f110 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6494168ab0 con 0x7f649409d860 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.992+0000 7f6498a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6494168db0 con 0x7f6494155d70 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51108/0 (socket says 172.21.15.43:51108) 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 -- 172.21.15.43:0/2326952497 learned_addr learned my addr 172.21.15.43:0/2326952497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6492800640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f649409d860 0x7f64940cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f648be00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6494155d70 0x7f649415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 -- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6494155d70 msgr2=0x7f649415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6494155d70 0x7f649415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 -- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f649409d860 msgr2=0x7f64940cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f649409d860 0x7f64940cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 -- 172.21.15.43:0/2326952497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64941f65f0 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6491e00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f648400b410 tx=0x7f648400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484015030 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6484004020 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484004470 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64941f68b0 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64941f6e70 con 0x7f649406f110 2024-09-17T05:25:14.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64940ad0b0 con 0x7f649406f110 2024-09-17T05:25:14.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:13.996+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6484018070 con 0x7f649406f110 2024-09-17T05:25:14.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.000+0000 7f648a000640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 0x7f6470075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.000+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f64840939c0 con 0x7f649406f110 2024-09-17T05:25:14.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.000+0000 7f648be00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 0x7f6470075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.000+0000 7f648be00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 0x7f6470075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f647c0af000 tx=0x7f647c0a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.004+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f648405e640 con 0x7f649406f110 2024-09-17T05:25:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f64940af030 con 0x7f649406f110 2024-09-17T05:25:14.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f648a000640 1 -- 172.21.15.43:0/2326952497 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f6484063070 con 0x7f649406f110 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.249 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:25:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:25:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:25:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:25:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:25:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:25:14.262 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 msgr2=0x7f6470075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 0x7f6470075710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f647c0af000 tx=0x7f647c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 msgr2=0x7f64941fa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f648400b410 tx=0x7f648400b8d0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 shutdown_connections 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6470073200 0x7f6470075710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f649409d860 0x7f64940cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6494155d70 0x7f649415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 --2- 172.21.15.43:0/2326952497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649406f110 0x7f64941fa5f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.244+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 >> 172.21.15.43:0/2326952497 conn(0x7f64940b23d0 msgr2=0x7f6494072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.248+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 shutdown_connections 2024-09-17T05:25:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.248+0000 7f6498a00640 1 -- 172.21.15.43:0/2326952497 wait complete. 2024-09-17T05:25:14.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:25:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 -- 172.21.15.43:0/2011188939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c09ba60 msgr2=0x7fd16c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 --2- 172.21.15.43:0/2011188939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c09ba60 0x7fd16c0b3130 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 -- 172.21.15.43:0/2011188939 shutdown_connections 2024-09-17T05:25:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 --2- 172.21.15.43:0/2011188939 >> v2:172.21.15.62:3300/0 conn(0x7fd16c0d0240 0x7fd16c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 --2- 172.21.15.43:0/2011188939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c09ba60 0x7fd16c0b3130 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 -- 172.21.15.43:0/2011188939 >> 172.21.15.43:0/2011188939 conn(0x7fd16c0cee60 msgr2=0x7fd16c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 -- 172.21.15.43:0/2011188939 shutdown_connections 2024-09-17T05:25:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.424+0000 7fd172c00640 1 -- 172.21.15.43:0/2011188939 wait complete. 2024-09-17T05:25:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 Processor -- start 2024-09-17T05:25:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 -- start start 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c09ba60 0x7fd16c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c0d0240 0x7fd16c0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b2b30 con 0x7fd16c0d0650 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b29b0 con 0x7fd16c09ba60 2024-09-17T05:25:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.428+0000 7fd172c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c169e30 con 0x7fd16c0d0240 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51110/0 (socket says 172.21.15.43:51110) 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 -- 172.21.15.43:0/3681968293 learned_addr learned my addr 172.21.15.43:0/3681968293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd16be00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c09ba60 0x7fd16c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd163400640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c0d0240 0x7fd16c0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 -- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c0d0240 msgr2=0x7fd16c0c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c0d0240 0x7fd16c0c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 -- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c09ba60 msgr2=0x7fd16c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c09ba60 0x7fd16c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 -- 172.21.15.43:0/3681968293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c073100 con 0x7fd16c0d0650 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd170c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c6b0 tx=0x7fd16400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164017070 con 0x7fd16c0d0650 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd164003c70 con 0x7fd16c0d0650 2024-09-17T05:25:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164005350 con 0x7fd16c0d0650 2024-09-17T05:25:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c0ca550 con 0x7fd16c0d0650 2024-09-17T05:25:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c1fc470 con 0x7fd16c0d0650 2024-09-17T05:25:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16c0c71f0 con 0x7fd16c0d0650 2024-09-17T05:25:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.432+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd16401a070 con 0x7fd16c0d0650 2024-09-17T05:25:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.436+0000 7fd168c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 0x7fd148075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.436+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fd164093510 con 0x7fd16c0d0650 2024-09-17T05:25:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.436+0000 7fd16be00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 0x7fd148075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.436+0000 7fd16be00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 0x7fd148075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af000 tx=0x7fd15c01b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.440+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd16405e230 con 0x7fd16c0d0650 2024-09-17T05:25:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd16c0ca720 con 0x7fd16c0d0650 2024-09-17T05:25:14.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd168c00640 1 -- 172.21.15.43:0/3681968293 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7fd164062c60 con 0x7fd16c0d0650 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:14.621 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 msgr2=0x7fd148075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 0x7fd148075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0af000 tx=0x7fd15c01b120 comp rx=0 tx=0).stop 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 msgr2=0x7fd16c15b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c6b0 tx=0x7fd16400cb70 comp rx=0 tx=0).stop 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 shutdown_connections 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd148073200 0x7fd148075710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd16c0d0650 0x7fd16c15b880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd16c0d0240 0x7fd16c0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 --2- 172.21.15.43:0/3681968293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd16c09ba60 0x7fd16c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 >> 172.21.15.43:0/3681968293 conn(0x7fd16c0cee60 msgr2=0x7fd16c0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 shutdown_connections 2024-09-17T05:25:14.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.616+0000 7fd172c00640 1 -- 172.21.15.43:0/3681968293 wait complete. 2024-09-17T05:25:14.643 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:25:14.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- 172.21.15.43:0/3727512303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700d10b0 msgr2=0x7f05700cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 --2- 172.21.15.43:0/3727512303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700d10b0 0x7f05700cd050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0560009840 tx=0x7f056001c790 comp rx=0 tx=0).stop 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- 172.21.15.43:0/3727512303 shutdown_connections 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 --2- 172.21.15.43:0/3727512303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700cd590 0x7f05700cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 --2- 172.21.15.43:0/3727512303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700d10b0 0x7f05700cd050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- 172.21.15.43:0/3727512303 >> 172.21.15.43:0/3727512303 conn(0x7f05700cef50 msgr2=0x7f05700cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- 172.21.15.43:0/3727512303 shutdown_connections 2024-09-17T05:25:14.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- 172.21.15.43:0/3727512303 wait complete. 2024-09-17T05:25:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 Processor -- start 2024-09-17T05:25:14.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.804+0000 7f0578800640 1 -- start start 2024-09-17T05:25:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700d0ca0 0x7f0570153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 0x7f057015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f05700d32a0 con 0x7f05700cd590 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f05700b8300 con 0x7f05700d0ca0 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f057015e310 con 0x7f05700d10b0 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51128/0 (socket says 172.21.15.43:51128) 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 -- 172.21.15.43:0/1442774085 learned_addr learned my addr 172.21.15.43:0/1442774085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575400640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700d0ca0 0x7f0570153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0576800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 0x7f057015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 -- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 msgr2=0x7f057015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 0x7f057015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 -- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700d0ca0 msgr2=0x7f0570153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700d0ca0 0x7f0570153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 -- 172.21.15.43:0/1442774085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057015cd00 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0576800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 0x7f057015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0575e00640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f056400c400 tx=0x7f056400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564011790 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0564017070 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701587d0 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564011cc0 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057015bfc0 con 0x7f05700cd590 2024-09-17T05:25:14.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.808+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05700b22e0 con 0x7f05700cd590 2024-09-17T05:25:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.812+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f056401c460 con 0x7f05700cd590 2024-09-17T05:25:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.812+0000 7f056a000640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.812+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f0564093c30 con 0x7f05700cd590 2024-09-17T05:25:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.812+0000 7f0575400640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.812+0000 7f0575400640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 0x7f0554075710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f05600af000 tx=0x7f056001afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.816+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f056405ef90 con 0x7f05700cd590 2024-09-17T05:25:14.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:14.992+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f057015ba90 con 0x7f05700cd590 2024-09-17T05:25:15.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.800+0000 7f056a000640 1 -- 172.21.15.43:0/1442774085 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f056405e890 con 0x7f05700cd590 2024-09-17T05:25:15.805 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 msgr2=0x7f0554075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 0x7f0554075710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f05600af000 tx=0x7f056001afd0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 msgr2=0x7f057015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f056400c400 tx=0x7f056400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 shutdown_connections 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700d0ca0 0x7f0570153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f05700d10b0 0x7f057015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 --2- 172.21.15.43:0/1442774085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05700cd590 0x7f057015c7c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 >> 172.21.15.43:0/1442774085 conn(0x7f05700cef50 msgr2=0x7f05700b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 shutdown_connections 2024-09-17T05:25:15.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:15.804+0000 7f0578800640 1 -- 172.21.15.43:0/1442774085 wait complete. 2024-09-17T05:25:15.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:25:16.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 -- 172.21.15.43:0/127318277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80cdfb0 msgr2=0x7f92f80cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 --2- 172.21.15.43:0/127318277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80cdfb0 0x7f92f80cd370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f92f80afb40 tx=0x7f92ec01c770 comp rx=0 tx=0).stop 2024-09-17T05:25:16.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 -- 172.21.15.43:0/127318277 shutdown_connections 2024-09-17T05:25:16.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 --2- 172.21.15.43:0/127318277 >> v2:172.21.15.62:3300/0 conn(0x7f92f80ce3c0 0x7f92f80cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 --2- 172.21.15.43:0/127318277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80cdfb0 0x7f92f80cd370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 -- 172.21.15.43:0/127318277 >> 172.21.15.43:0/127318277 conn(0x7f92f80b2810 msgr2=0x7f92f80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 -- 172.21.15.43:0/127318277 shutdown_connections 2024-09-17T05:25:16.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.004+0000 7f92ffa00640 1 -- 172.21.15.43:0/127318277 wait complete. 2024-09-17T05:25:16.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 Processor -- start 2024-09-17T05:25:16.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- start start 2024-09-17T05:25:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f80cdfb0 0x7f92f80afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 0x7f92f80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92f816a150 con 0x7f92f80ce3c0 2024-09-17T05:25:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92f8169fd0 con 0x7f92f81603e0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92f816a2d0 con 0x7f92f80cdfb0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51158/0 (socket says 172.21.15.43:51158) 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 -- 172.21.15.43:0/4205409004 learned_addr learned my addr 172.21.15.43:0/4205409004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92fd000640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f80cdfb0 0x7f92f80afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92fda00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 0x7f92f80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 -- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f80cdfb0 msgr2=0x7f92f80afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f80cdfb0 0x7f92f80afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 -- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 msgr2=0x7f92f80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 0x7f92f80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 -- 172.21.15.43:0/4205409004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8154740 con 0x7f92f80ce3c0 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92fda00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 0x7f92f80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f7e00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f92e400e7f0 tx=0x7f92e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400ccc0 con 0x7f92f80ce3c0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92e400ce80 con 0x7f92f80ce3c0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005280 con 0x7f92f80ce3c0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f814e990 con 0x7f92f80ce3c0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.008+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f8154180 con 0x7f92f80ce3c0 2024-09-17T05:25:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.012+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f81607f0 con 0x7f92f80ce3c0 2024-09-17T05:25:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.012+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92e4005420 con 0x7f92f80ce3c0 2024-09-17T05:25:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.012+0000 7f92f4c00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 0x7f92d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.012+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7f92e4094e00 con 0x7f92f80ce3c0 2024-09-17T05:25:16.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.016+0000 7f92fd000640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 0x7f92d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.016+0000 7f92fd000640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 0x7f92d8075710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0af000 tx=0x7f92ec0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.016+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92f814e990 con 0x7f92f80ce3c0 2024-09-17T05:25:16.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.192+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f92f80b00e0 con 0x7f92f80ce3c0 2024-09-17T05:25:16.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.808+0000 7f92f4c00640 1 -- 172.21.15.43:0/4205409004 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7f92f8154740 con 0x7f92f80ce3c0 2024-09-17T05:25:16.812 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:25:16.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.812+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 msgr2=0x7f92d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.812+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 0x7f92d8075710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0af000 tx=0x7f92ec0b2040 comp rx=0 tx=0).stop 2024-09-17T05:25:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.812+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 msgr2=0x7f92f814e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.812+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f92e400e7f0 tx=0x7f92e400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 shutdown_connections 2024-09-17T05:25:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f92d8073200 0x7f92d8075710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92f81603e0 0x7f92f80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92f80ce3c0 0x7f92f814e450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 --2- 172.21.15.43:0/4205409004 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92f80cdfb0 0x7f92f80afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 >> 172.21.15.43:0/4205409004 conn(0x7f92f80b2810 msgr2=0x7f92f815f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 shutdown_connections 2024-09-17T05:25:16.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:16.816+0000 7f92ffa00640 1 -- 172.21.15.43:0/4205409004 wait complete. 2024-09-17T05:25:16.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- 172.21.15.43:0/3749292925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 msgr2=0x7f2bfc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/3749292925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2bfc004c20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009b60 tx=0x7f2c0001d290 comp rx=0 tx=0).stop 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c17400640 1 -- 172.21.15.43:0/3749292925 reap_dead start 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- 172.21.15.43:0/3749292925 shutdown_connections 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- 172.21.15.43:0/3749292925 >> 172.21.15.43:0/3749292925 conn(0x7f2c180b23d0 msgr2=0x7f2c180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- 172.21.15.43:0/3749292925 shutdown_connections 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- 172.21.15.43:0/3749292925 wait complete. 2024-09-17T05:25:17.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 Processor -- start 2024-09-17T05:25:17.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.028+0000 7f2c1dc00640 1 -- start start 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c1815d730 0x7f2c181f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c1815ce70 0x7f2c180c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18167ec0 con 0x7f2bfc004810 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18167d40 con 0x7f2c1815ce70 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2c18168040 con 0x7f2c1815d730 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51176/0 (socket says 172.21.15.43:51176) 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 -- 172.21.15.43:0/2189808487 learned_addr learned my addr 172.21.15.43:0/2189808487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c0fe00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c1815d730 0x7f2c181f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c17400640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c1815ce70 0x7f2c180c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 -- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c1815d730 msgr2=0x7f2c181f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c1815d730 0x7f2c181f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 -- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c1815ce70 msgr2=0x7f2c180c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c1815ce70 0x7f2c180c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 -- 172.21.15.43:0/2189808487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181608c0 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c16a00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c08011b60 tx=0x7f2c0800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0801e070 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c0800e7c0 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800ebb0 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181fd4f0 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181f9760 con 0x7f2bfc004810 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.032+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c180b4b80 con 0x7f2bfc004810 2024-09-17T05:25:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.036+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c08021070 con 0x7f2bfc004810 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.036+0000 7f2c0ea00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 0x7f2bf4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.036+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7f2c08095970 con 0x7f2bfc004810 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.036+0000 7f2c0fe00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 0x7f2bf4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.036+0000 7f2c0fe00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 0x7f2bf4075740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ae000 tx=0x7f2c0001a030 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.040+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c08060410 con 0x7f2bfc004810 2024-09-17T05:25:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.252+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2c180764d0 con 0x7f2bfc004810 2024-09-17T05:25:17.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.808+0000 7f2c0ea00640 1 -- 172.21.15.43:0/2189808487 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v27) ==== 172+0+0 (secure 0 0 0) 0x7f2c08064e40 con 0x7f2bfc004810 2024-09-17T05:25:17.814 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 6 and data pool 7 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 msgr2=0x7f2bf4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 0x7f2bf4075740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ae000 tx=0x7f2c0001a030 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 msgr2=0x7f2c18160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c08011b60 tx=0x7f2c0800f790 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 shutdown_connections 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bf4073230 0x7f2bf4075740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c1815ce70 0x7f2c180c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c1815d730 0x7f2c181f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 --2- 172.21.15.43:0/2189808487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bfc004810 0x7f2c18160380 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2c08011b60 tx=0x7f2c0800f790 comp rx=0 tx=0).stop 2024-09-17T05:25:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 >> 172.21.15.43:0/2189808487 conn(0x7f2c180b23d0 msgr2=0x7f2c180b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 shutdown_connections 2024-09-17T05:25:17.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:17.812+0000 7f2c1dc00640 1 -- 172.21.15.43:0/2189808487 wait complete. 2024-09-17T05:25:17.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- 172.21.15.43:0/2382253418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf44155d70 msgr2=0x7fdf4415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 --2- 172.21.15.43:0/2382253418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf44155d70 0x7fdf4415c7e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf440af0a0 tx=0x7fdf3401c800 comp rx=0 tx=0).stop 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- 172.21.15.43:0/2382253418 shutdown_connections 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 --2- 172.21.15.43:0/2382253418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf440b3920 0x7fdf440ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 --2- 172.21.15.43:0/2382253418 >> v2:172.21.15.62:3300/0 conn(0x7fdf4406f110 0x7fdf440b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 --2- 172.21.15.43:0/2382253418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf44155d70 0x7fdf4415c7e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- 172.21.15.43:0/2382253418 >> 172.21.15.43:0/2382253418 conn(0x7fdf440b23d0 msgr2=0x7fdf440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- 172.21.15.43:0/2382253418 shutdown_connections 2024-09-17T05:25:18.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- 172.21.15.43:0/2382253418 wait complete. 2024-09-17T05:25:18.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 Processor -- start 2024-09-17T05:25:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.016+0000 7fdf48e00640 1 -- start start 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf4406f110 0x7fdf440c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf44155d70 0x7fdf4415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4415f8c0 con 0x7fdf440b3920 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4415f740 con 0x7fdf44155d70 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4415fa40 con 0x7fdf4406f110 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51198/0 (socket says 172.21.15.43:51198) 2024-09-17T05:25:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 -- 172.21.15.43:0/4132783398 learned_addr learned my addr 172.21.15.43:0/4132783398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf41e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf4406f110 0x7fdf440c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf42800640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf44155d70 0x7fdf4415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 -- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf4406f110 msgr2=0x7fdf440c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf4406f110 0x7fdf440c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 -- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf44155d70 msgr2=0x7fdf4415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf44155d70 0x7fdf4415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 -- 172.21.15.43:0/4132783398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf44098a90 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3be00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00d720 tx=0x7fdf2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c017070 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf2c009d20 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0106c0 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf44098d70 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf440c9610 con 0x7fdf440b3920 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.020+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4415ff30 con 0x7fdf440b3920 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.024+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf2c01a070 con 0x7fdf440b3920 2024-09-17T05:25:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.024+0000 7fdf3a000640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 0x7fdf20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.024+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fdf2c094280 con 0x7fdf440b3920 2024-09-17T05:25:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.024+0000 7fdf41e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 0x7fdf20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.024+0000 7fdf41e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 0x7fdf20075740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf340af000 tx=0x7fdf340a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.028+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf2c05ec50 con 0x7fdf440b3920 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.208+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf4406dba0 con 0x7fdf440b3920 2024-09-17T05:25:18.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.208+0000 7fdf3a000640 1 -- 172.21.15.43:0/4132783398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16622 (secure 0 0 0) 0x7fdf2c063680 con 0x7fdf440b3920 2024-09-17T05:25:18.213 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:18.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:17.258723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:15.006899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:16.208447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 msgr2=0x7fdf20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 0x7fdf20075740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf340af000 tx=0x7fdf340a7040 comp rx=0 tx=0).stop 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 msgr2=0x7fdf4415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00d720 tx=0x7fdf2c00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 shutdown_connections 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdf20073230 0x7fdf20075740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf44155d70 0x7fdf4415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf440b3920 0x7fdf4415a710 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 --2- 172.21.15.43:0/4132783398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf4406f110 0x7fdf440c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 >> 172.21.15.43:0/4132783398 conn(0x7fdf440b23d0 msgr2=0x7fdf44073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 shutdown_connections 2024-09-17T05:25:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.212+0000 7fdf48e00640 1 -- 172.21.15.43:0/4132783398 wait complete. 2024-09-17T05:25:18.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- 172.21.15.43:0/237003021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3406f110 msgr2=0x7fac340b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 --2- 172.21.15.43:0/237003021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3406f110 0x7fac340b33e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fac20009840 tx=0x7fac2001c760 comp rx=0 tx=0).stop 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- 172.21.15.43:0/237003021 shutdown_connections 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 --2- 172.21.15.43:0/237003021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3406f110 0x7fac340b33e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 --2- 172.21.15.43:0/237003021 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac34155d70 0x7fac3415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- 172.21.15.43:0/237003021 >> 172.21.15.43:0/237003021 conn(0x7fac340b23d0 msgr2=0x7fac340d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- 172.21.15.43:0/237003021 shutdown_connections 2024-09-17T05:25:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- 172.21.15.43:0/237003021 wait complete. 2024-09-17T05:25:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 Processor -- start 2024-09-17T05:25:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.392+0000 7fac3b600640 1 -- start start 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac340b3920 0x7fac340cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac34155d70 0x7fac340cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac34168a60 con 0x7fac3406f110 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac341688e0 con 0x7fac34155d70 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fac34168be0 con 0x7fac340b3920 2024-09-17T05:25:18.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51210/0 (socket says 172.21.15.43:51210) 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 -- 172.21.15.43:0/1642824490 learned_addr learned my addr 172.21.15.43:0/1642824490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac33e00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac340b3920 0x7fac340cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac39600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac34155d70 0x7fac340cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 -- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac340b3920 msgr2=0x7fac340cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac340b3920 0x7fac340cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 -- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac34155d70 msgr2=0x7fac340cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac34155d70 0x7fac340cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 -- 172.21.15.43:0/1642824490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3409df60 con 0x7fac3406f110 2024-09-17T05:25:18.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac38c00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e770 tx=0x7fac2800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ccc0 con 0x7fac3406f110 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac2800ee80 con 0x7fac3406f110 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341fc880 con 0x7fac3406f110 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ce60 con 0x7fac3406f110 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341fce40 con 0x7fac3406f110 2024-09-17T05:25:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.396+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac340b33e0 con 0x7fac3406f110 2024-09-17T05:25:18.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.400+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac28004260 con 0x7fac3406f110 2024-09-17T05:25:18.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.400+0000 7fac30c00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 0x7fac14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.400+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fac28094b10 con 0x7fac3406f110 2024-09-17T05:25:18.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.400+0000 7fac33e00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 0x7fac14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.400+0000 7fac33e00640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 0x7fac14075710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac200af000 tx=0x7fac2001afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.404+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac2805f450 con 0x7fac3406f110 2024-09-17T05:25:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.580+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac3415dc50 con 0x7fac3406f110 2024-09-17T05:25:18.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac30c00640 1 -- 172.21.15.43:0/1642824490 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16622 (secure 0 0 0) 0x7fac28063e80 con 0x7fac3406f110 2024-09-17T05:25:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:18.588 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:17.258723+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:15.006899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:16.208447+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:18.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 msgr2=0x7fac14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 0x7fac14075710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac200af000 tx=0x7fac2001afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 msgr2=0x7fac3409da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e770 tx=0x7fac2800ec30 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 shutdown_connections 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac14073200 0x7fac14075710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac34155d70 0x7fac340cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac340b3920 0x7fac340cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 --2- 172.21.15.43:0/1642824490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac3406f110 0x7fac3409da20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 >> 172.21.15.43:0/1642824490 conn(0x7fac340b23d0 msgr2=0x7fac34165b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 shutdown_connections 2024-09-17T05:25:18.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.584+0000 7fac3b600640 1 -- 172.21.15.43:0/1642824490 wait complete. 2024-09-17T05:25:18.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:25:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- 172.21.15.43:0/1551586362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c015fe10 msgr2=0x7f42c00b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 --2- 172.21.15.43:0/1551586362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c015fe10 0x7f42c00b4b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009870 tx=0x7f42b001c740 comp rx=0 tx=0).stop 2024-09-17T05:25:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- 172.21.15.43:0/1551586362 shutdown_connections 2024-09-17T05:25:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 --2- 172.21.15.43:0/1551586362 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c015fe10 0x7f42c00b4b80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- 172.21.15.43:0/1551586362 >> 172.21.15.43:0/1551586362 conn(0x7f42c00b23d0 msgr2=0x7f42c00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- 172.21.15.43:0/1551586362 shutdown_connections 2024-09-17T05:25:18.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- 172.21.15.43:0/1551586362 wait complete. 2024-09-17T05:25:18.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 Processor -- start 2024-09-17T05:25:18.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.768+0000 7f42c5e00640 1 -- start start 2024-09-17T05:25:18.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 0x7f42c00b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 0x7f42c00b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42c016aff0 con 0x7f42c006fc90 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42c016ae70 con 0x7f42c015fe10 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f42c016b170 con 0x7f42c0099720 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51220/0 (socket says 172.21.15.43:51220) 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 -- 172.21.15.43:0/2516811236 learned_addr learned my addr 172.21.15.43:0/2516811236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42b7e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 0x7f42c00b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bf400640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 0x7f42c00b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 -- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 msgr2=0x7f42c00b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 0x7f42c00b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 -- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 msgr2=0x7f42c00b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 0x7f42c00b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 -- 172.21.15.43:0/2516811236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c01572c0 con 0x7f42c006fc90 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42b7e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 0x7f42c00b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bf400640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 0x7f42c00b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42bea00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000c10 tx=0x7f42b00a5f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b00b2040 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42b0004240 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0005050 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c015b050 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c015b610 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.772+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c00b4b80 con 0x7f42c006fc90 2024-09-17T05:25:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.776+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f42b00051f0 con 0x7f42c006fc90 2024-09-17T05:25:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.776+0000 7f42b7400640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 0x7f429c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.776+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f42b012e220 con 0x7f42c006fc90 2024-09-17T05:25:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.776+0000 7f42b7e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 0x7f429c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.776+0000 7f42b7e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 0x7f429c075740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42a8001230 tx=0x7f42a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.780+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f42b0016450 con 0x7f42c006fc90 2024-09-17T05:25:18.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:18.992+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f42c006d1a0 con 0x7f42c006fc90 2024-09-17T05:25:19.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.832+0000 7f42b7400640 1 -- 172.21.15.43:0/2516811236 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f42b00f9230 con 0x7f42c006fc90 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 msgr2=0x7f429c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 0x7f429c075740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42a8001230 tx=0x7f42a8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 msgr2=0x7f42c00990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f42b0000c10 tx=0x7f42b00a5f70 comp rx=0 tx=0).stop 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 shutdown_connections 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f429c073230 0x7f429c075740 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42c015fe10 0x7f42c00b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42c0099720 0x7f42c00b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 --2- 172.21.15.43:0/2516811236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42c006fc90 0x7f42c00990b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 >> 172.21.15.43:0/2516811236 conn(0x7f42c00b23d0 msgr2=0x7f42c0074000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 shutdown_connections 2024-09-17T05:25:19.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:19.836+0000 7f42c5e00640 1 -- 172.21.15.43:0/2516811236 wait complete. 2024-09-17T05:25:19.863 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:25:19.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- 172.21.15.43:0/2386930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b5fe0 msgr2=0x7f2b0015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 --2- 172.21.15.43:0/2386930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b5fe0 0x7f2b0015f530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009900 tx=0x7f2af001c8d0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- 172.21.15.43:0/2386930183 shutdown_connections 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 --2- 172.21.15.43:0/2386930183 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b43c0 0x7f2b000b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 --2- 172.21.15.43:0/2386930183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b5fe0 0x7f2b0015f530 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- 172.21.15.43:0/2386930183 >> 172.21.15.43:0/2386930183 conn(0x7f2b000b2e80 msgr2=0x7f2b000b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- 172.21.15.43:0/2386930183 shutdown_connections 2024-09-17T05:25:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- 172.21.15.43:0/2386930183 wait complete. 2024-09-17T05:25:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 Processor -- start 2024-09-17T05:25:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.024+0000 7f2b06c00640 1 -- start start 2024-09-17T05:25:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 0x7f2b000cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b0015fa70 0x7f2b000cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00169160 con 0x7f2b000b43c0 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00168fe0 con 0x7f2b0015fa70 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b001692e0 con 0x7f2b000b5fe0 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51244/0 (socket says 172.21.15.43:51244) 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 -- 172.21.15.43:0/1076157678 learned_addr learned my addr 172.21.15.43:0/1076157678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2af7e00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 0x7f2b000cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b04c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b0015fa70 0x7f2b000cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 -- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 msgr2=0x7f2b000cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 0x7f2b000cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 -- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b0015fa70 msgr2=0x7f2b000cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b0015fa70 0x7f2b000cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 -- 172.21.15.43:0/1076157678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000ce920 con 0x7f2b000b43c0 2024-09-17T05:25:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2af7e00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 0x7f2b000cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2affe00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2af00b0040 tx=0x7f2af0004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af00b2040 con 0x7f2b000b43c0 2024-09-17T05:25:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af0003600 con 0x7f2b000b43c0 2024-09-17T05:25:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b000cec00 con 0x7f2b000b43c0 2024-09-17T05:25:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003a30 con 0x7f2b000b43c0 2024-09-17T05:25:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b00144730 con 0x7f2b000b43c0 2024-09-17T05:25:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.028+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b000b47d0 con 0x7f2b000b43c0 2024-09-17T05:25:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.032+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2af0003bd0 con 0x7f2b000b43c0 2024-09-17T05:25:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.032+0000 7f2afcc00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 0x7f2adc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.032+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2af012de50 con 0x7f2b000b43c0 2024-09-17T05:25:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.032+0000 7f2af7e00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 0x7f2adc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.032+0000 7f2af7e00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 0x7f2adc075710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b001fabd0 tx=0x7f2ae8008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.036+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2af00bd060 con 0x7f2b000b43c0 2024-09-17T05:25:20.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.244+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2b000cf380 con 0x7f2b000b43c0 2024-09-17T05:25:20.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:20.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2afcc00640 1 -- 172.21.15.43:0/1076157678 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f2af00164f0 con 0x7f2b000b43c0 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 msgr2=0x7f2adc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 0x7f2adc075710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b001fabd0 tx=0x7f2ae8008040 comp rx=0 tx=0).stop 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 msgr2=0x7f2b000ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2af00b0040 tx=0x7f2af0004150 comp rx=0 tx=0).stop 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 shutdown_connections 2024-09-17T05:25:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2adc073200 0x7f2adc075710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b0015fa70 0x7f2b000cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b000b5fe0 0x7f2b000cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 --2- 172.21.15.43:0/1076157678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b000b43c0 0x7f2b000ca980 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 >> 172.21.15.43:0/1076157678 conn(0x7f2b000b2e80 msgr2=0x7f2b0015c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 shutdown_connections 2024-09-17T05:25:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:20.840+0000 7f2b06c00640 1 -- 172.21.15.43:0/1076157678 wait complete. 2024-09-17T05:25:20.870 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:25:20.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:21.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 -- 172.21.15.43:0/562553592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 msgr2=0x7f5ecc004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 --2- 172.21.15.43:0/562553592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ecc004d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009b40 tx=0x7f5ed001d270 comp rx=0 tx=0).stop 2024-09-17T05:25:21.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eeda00640 1 -- 172.21.15.43:0/562553592 reap_dead start 2024-09-17T05:25:21.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 -- 172.21.15.43:0/562553592 shutdown_connections 2024-09-17T05:25:21.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 -- 172.21.15.43:0/562553592 >> 172.21.15.43:0/562553592 conn(0x7f5ee80b23d0 msgr2=0x7f5ee80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 -- 172.21.15.43:0/562553592 shutdown_connections 2024-09-17T05:25:21.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.080+0000 7f5eefa00640 1 -- 172.21.15.43:0/562553592 wait complete. 2024-09-17T05:25:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 Processor -- start 2024-09-17T05:25:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- start start 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ee8161000 0x7f5ee8162d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee8156090 0x7f5ee815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8168e80 con 0x7f5ecc0048b0 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8168d00 con 0x7f5ee8156090 2024-09-17T05:25:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee8169000 con 0x7f5ee8161000 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51258/0 (socket says 172.21.15.43:51258) 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 -- 172.21.15.43:0/3491650066 learned_addr learned my addr 172.21.15.43:0/3491650066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5edfe00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ee8161000 0x7f5ee8162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eeda00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee8156090 0x7f5ee815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 -- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ee8161000 msgr2=0x7f5ee8162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ee8161000 0x7f5ee8162d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 -- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee8156090 msgr2=0x7f5ee815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee8156090 0x7f5ee815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 -- 172.21.15.43:0/3491650066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee81f9770 con 0x7f5ecc0048b0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eed000640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8003ea0 tx=0x7f5ed8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8015070 con 0x7f5ecc0048b0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ed8011900 con 0x7f5ecc0048b0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee81f9a50 con 0x7f5ecc0048b0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8011d50 con 0x7f5ecc0048b0 2024-09-17T05:25:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.084+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee81fa010 con 0x7f5ecc0048b0 2024-09-17T05:25:21.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee80b4b80 con 0x7f5ecc0048b0 2024-09-17T05:25:21.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ed8018070 con 0x7f5ecc0048b0 2024-09-17T05:25:21.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5ee5600640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 0x7f5ec4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f5ed8094ff0 con 0x7f5ecc0048b0 2024-09-17T05:25:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5edfe00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 0x7f5ec4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.088+0000 7f5edfe00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 0x7f5ec4075710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00ad000 tx=0x7f5ed001a010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.092+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ed805f940 con 0x7f5ecc0048b0 2024-09-17T05:25:21.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.300+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ee80b4380 con 0x7f5ecc0048b0 2024-09-17T05:25:21.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5ee5600640 1 -- 172.21.15.43:0/3491650066 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5217 (secure 0 0 0) 0x7f5ed8064370 con 0x7f5ecc0048b0 2024-09-17T05:25:21.307 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:21.308 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-17T05:25:20:838273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:20.251677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/980890216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":980890216},{"type":"v1","addr":"172.21.15.62:6835","nonce":980890216}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5439,"qdb_cluster":[5439]},"id":3}]} 2024-09-17T05:25:21.308 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 msgr2=0x7f5ec4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 0x7f5ec4075710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00ad000 tx=0x7f5ed001a010 comp rx=0 tx=0).stop 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 msgr2=0x7f5ee81f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8003ea0 tx=0x7f5ed8003f90 comp rx=0 tx=0).stop 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 shutdown_connections 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ec4073200 0x7f5ec4075710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ee8156090 0x7f5ee815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ee8161000 0x7f5ee8162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 --2- 172.21.15.43:0/3491650066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0048b0 0x7f5ee81f9050 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 >> 172.21.15.43:0/3491650066 conn(0x7f5ee80b23d0 msgr2=0x7f5ee815ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 shutdown_connections 2024-09-17T05:25:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.304+0000 7f5eefa00640 1 -- 172.21.15.43:0/3491650066 wait complete. 2024-09-17T05:25:21.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:21.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 -- 172.21.15.43:0/2295255423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180d0f50 msgr2=0x7efd180cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 --2- 172.21.15.43:0/2295255423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180d0f50 0x7efd180cce70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efd1809bbe0 tx=0x7efd0401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 -- 172.21.15.43:0/2295255423 shutdown_connections 2024-09-17T05:25:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 --2- 172.21.15.43:0/2295255423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180d0f50 0x7efd180cce70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 -- 172.21.15.43:0/2295255423 >> 172.21.15.43:0/2295255423 conn(0x7efd180cee80 msgr2=0x7efd180cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 -- 172.21.15.43:0/2295255423 shutdown_connections 2024-09-17T05:25:21.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.484+0000 7efd1fe00640 1 -- 172.21.15.43:0/2295255423 wait complete. 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 Processor -- start 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 -- start start 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd180b7ce0 0x7efd18158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd180d0f50 0x7efd180b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd1816b030 con 0x7efd180cd3b0 2024-09-17T05:25:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd1816aeb0 con 0x7efd180b7ce0 2024-09-17T05:25:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1fe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efd1816b1b0 con 0x7efd180d0f50 2024-09-17T05:25:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51270/0 (socket says 172.21.15.43:51270) 2024-09-17T05:25:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 -- 172.21.15.43:0/175519100 learned_addr learned my addr 172.21.15.43:0/175519100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 -- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd180d0f50 msgr2=0x7efd180b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd180d0f50 0x7efd180b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 -- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd180b7ce0 msgr2=0x7efd18158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd180b7ce0 0x7efd18158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 -- 172.21.15.43:0/175519100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1815f510 con 0x7efd180cd3b0 2024-09-17T05:25:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.488+0000 7efd1ca00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efd0401a510 tx=0x7efd040a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd040bd070 con 0x7efd180cd3b0 2024-09-17T05:25:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd040044b0 con 0x7efd180cd3b0 2024-09-17T05:25:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1809c390 con 0x7efd180cd3b0 2024-09-17T05:25:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04005270 con 0x7efd180cd3b0 2024-09-17T05:25:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1815efa0 con 0x7efd180cd3b0 2024-09-17T05:25:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd180cce70 con 0x7efd180cd3b0 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd040c0070 con 0x7efd180cd3b0 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 0x7efcf8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7efd0412e330 con 0x7efd180cd3b0 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.492+0000 7efd1d400640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 0x7efcf8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.496+0000 7efd1d400640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 0x7efcf8075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fd0 tx=0x7efd0c0079a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.496+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd040164d0 con 0x7efd180cd3b0 2024-09-17T05:25:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.672+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd1815f230 con 0x7efd180cd3b0 2024-09-17T05:25:21.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.672+0000 7efd15600640 1 -- 172.21.15.43:0/175519100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16609 (secure 0 0 0) 0x7efd040f93d0 con 0x7efd180cd3b0 2024-09-17T05:25:21.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:21.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":47,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:18.825665+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:15.006899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:16.208447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 msgr2=0x7efcf8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 0x7efcf8075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fd0 tx=0x7efd0c0079a0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 msgr2=0x7efd1815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efd0401a510 tx=0x7efd040a6790 comp rx=0 tx=0).stop 2024-09-17T05:25:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 shutdown_connections 2024-09-17T05:25:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efcf8073200 0x7efcf8075710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd180d0f50 0x7efd180b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd180cd3b0 0x7efd1815de60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 --2- 172.21.15.43:0/175519100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd180b7ce0 0x7efd18158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 >> 172.21.15.43:0/175519100 conn(0x7efd180cee80 msgr2=0x7efd180724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 shutdown_connections 2024-09-17T05:25:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.676+0000 7efd1fe00640 1 -- 172.21.15.43:0/175519100 wait complete. 2024-09-17T05:25:21.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:25:21.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:21.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.852+0000 7f44e5000640 1 -- 172.21.15.43:0/3700900432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e0160860 msgr2=0x7f44e0160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.852+0000 7f44e5000640 1 --2- 172.21.15.43:0/3700900432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e0160860 0x7f44e0160c70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f44e0076c70 tx=0x7f44d801cd80 comp rx=0 tx=0).stop 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 -- 172.21.15.43:0/3700900432 shutdown_connections 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 --2- 172.21.15.43:0/3700900432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e0160860 0x7f44e0160c70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 --2- 172.21.15.43:0/3700900432 >> v2:172.21.15.62:3300/0 conn(0x7f44e00cffe0 0x7f44e0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 --2- 172.21.15.43:0/3700900432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e015c8c0 0x7f44e00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 -- 172.21.15.43:0/3700900432 >> 172.21.15.43:0/3700900432 conn(0x7f44e00d0a60 msgr2=0x7f44e00d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 -- 172.21.15.43:0/3700900432 shutdown_connections 2024-09-17T05:25:21.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 -- 172.21.15.43:0/3700900432 wait complete. 2024-09-17T05:25:21.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 Processor -- start 2024-09-17T05:25:21.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 -- start start 2024-09-17T05:25:21.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.856+0000 7f44e5000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e00cffe0 0x7f44e0150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 0x7f44e01669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00b4cf0 con 0x7f44e015c8c0 2024-09-17T05:25:21.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00b4b70 con 0x7f44e0160860 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f44e00b4e70 con 0x7f44e00cffe0 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51284/0 (socket says 172.21.15.43:51284) 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 -- 172.21.15.43:0/570873729 learned_addr learned my addr 172.21.15.43:0/570873729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44dde00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e00cffe0 0x7f44e0150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44de800640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 0x7f44e01669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 -- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e00cffe0 msgr2=0x7f44e0150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e00cffe0 0x7f44e0150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 -- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 msgr2=0x7f44e01669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 0x7f44e01669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 -- 172.21.15.43:0/570873729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e0151590 con 0x7f44e015c8c0 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44de800640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 0x7f44e01669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d7e00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f44c800d6a0 tx=0x7f44c800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8017070 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44c8004310 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8005280 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e0165990 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e006fb90 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e00cfaa0 con 0x7f44e015c8c0 2024-09-17T05:25:21.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.860+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f44c8005420 con 0x7f44e015c8c0 2024-09-17T05:25:21.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.864+0000 7f44d6000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 0x7f44bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.864+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f44c8094980 con 0x7f44e015c8c0 2024-09-17T05:25:21.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.864+0000 7f44dde00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 0x7f44bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.864+0000 7f44dde00640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 0x7f44bc075710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44d00058d0 tx=0x7f44d0005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:21.864+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f44c8020020 con 0x7f44e015c8c0 2024-09-17T05:25:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.036+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44e00d1730 con 0x7f44bc073200 2024-09-17T05:25:22.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.044+0000 7f44d6000640 1 -- 172.21.15.43:0/570873729 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251531 (secure 0 0 0) 0x7f44e00d1730 con 0x7f44bc073200 2024-09-17T05:25:22.050 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 msgr2=0x7f44bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 0x7f44bc075710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44d00058d0 tx=0x7f44d0005860 comp rx=0 tx=0).stop 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 msgr2=0x7f44e0166470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f44c800d6a0 tx=0x7f44c800db60 comp rx=0 tx=0).stop 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 shutdown_connections 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f44bc073200 0x7f44bc075710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44e0160860 0x7f44e01669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e015c8c0 0x7f44e0166470 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 --2- 172.21.15.43:0/570873729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f44e00cffe0 0x7f44e0150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 >> 172.21.15.43:0/570873729 conn(0x7f44e00d0a60 msgr2=0x7f44e01585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 shutdown_connections 2024-09-17T05:25:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.048+0000 7f44e5000640 1 -- 172.21.15.43:0/570873729 wait complete. 2024-09-17T05:25:22.072 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T05:25:21.868498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242428,"kb_used_data":5364,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732292,"statfs":{"total":773094113280,"available":772845867008,"internally_reserved":0,"allocated":5492736,"data_stored":2899253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836136+0000","last_change":"2024-09-17T05:25:17.830654+0000","last_active":"2024-09-17T05:25:17.836136+0000","last_peered":"2024-09-17T05:25:17.836136+0000","last_clean":"2024-09-17T05:25:17.836136+0000","last_became_active":"2024-09-17T05:25:17.830321+0000","last_became_peered":"2024-09-17T05:25:17.830321+0000","last_unstale":"2024-09-17T05:25:17.836136+0000","last_undegraded":"2024-09-17T05:25:17.836136+0000","last_fullsized":"2024-09-17T05:25:17.836136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:42.098022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848789+0000","last_change":"2024-09-17T05:25:17.844139+0000","last_active":"2024-09-17T05:25:17.848789+0000","last_peered":"2024-09-17T05:25:17.848789+0000","last_clean":"2024-09-17T05:25:17.848789+0000","last_became_active":"2024-09-17T05:25:17.843821+0000","last_became_peered":"2024-09-17T05:25:17.843821+0000","last_unstale":"2024-09-17T05:25:17.848789+0000","last_undegraded":"2024-09-17T05:25:17.848789+0000","last_fullsized":"2024-09-17T05:25:17.848789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:38.456234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850784+0000","last_change":"2024-09-17T05:25:17.847168+0000","last_active":"2024-09-17T05:25:17.850784+0000","last_peered":"2024-09-17T05:25:17.850784+0000","last_clean":"2024-09-17T05:25:17.850784+0000","last_became_active":"2024-09-17T05:25:17.846513+0000","last_became_peered":"2024-09-17T05:25:17.846513+0000","last_unstale":"2024-09-17T05:25:17.850784+0000","last_undegraded":"2024-09-17T05:25:17.850784+0000","last_fullsized":"2024-09-17T05:25:17.850784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:06.087190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859247+0000","last_change":"2024-09-17T05:25:17.845500+0000","last_active":"2024-09-17T05:25:17.859247+0000","last_peered":"2024-09-17T05:25:17.859247+0000","last_clean":"2024-09-17T05:25:17.859247+0000","last_became_active":"2024-09-17T05:25:17.845221+0000","last_became_peered":"2024-09-17T05:25:17.845221+0000","last_unstale":"2024-09-17T05:25:17.859247+0000","last_undegraded":"2024-09-17T05:25:17.859247+0000","last_fullsized":"2024-09-17T05:25:17.859247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:09.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859215+0000","last_change":"2024-09-17T05:25:17.858007+0000","last_active":"2024-09-17T05:25:17.859215+0000","last_peered":"2024-09-17T05:25:17.859215+0000","last_clean":"2024-09-17T05:25:17.859215+0000","last_became_active":"2024-09-17T05:25:17.855860+0000","last_became_peered":"2024-09-17T05:25:17.855860+0000","last_unstale":"2024-09-17T05:25:17.859215+0000","last_undegraded":"2024-09-17T05:25:17.859215+0000","last_fullsized":"2024-09-17T05:25:17.859215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817734+0000","last_change":"2024-09-17T05:25:16.817734+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817734+0000","last_undegraded":"2024-09-17T05:25:16.817734+0000","last_fullsized":"2024-09-17T05:25:16.817734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864857+0000","last_change":"2024-09-17T05:25:17.863014+0000","last_active":"2024-09-17T05:25:17.864857+0000","last_peered":"2024-09-17T05:25:17.864857+0000","last_clean":"2024-09-17T05:25:17.864857+0000","last_became_active":"2024-09-17T05:25:17.862710+0000","last_became_peered":"2024-09-17T05:25:17.862710+0000","last_unstale":"2024-09-17T05:25:17.864857+0000","last_undegraded":"2024-09-17T05:25:17.864857+0000","last_fullsized":"2024-09-17T05:25:17.864857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:54.789315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852055+0000","last_change":"2024-09-17T05:25:17.843269+0000","last_active":"2024-09-17T05:25:17.852055+0000","last_peered":"2024-09-17T05:25:17.852055+0000","last_clean":"2024-09-17T05:25:17.852055+0000","last_became_active":"2024-09-17T05:25:17.842790+0000","last_became_peered":"2024-09-17T05:25:17.842790+0000","last_unstale":"2024-09-17T05:25:17.852055+0000","last_undegraded":"2024-09-17T05:25:17.852055+0000","last_fullsized":"2024-09-17T05:25:17.852055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:09.373391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.818454+0000","last_change":"2024-09-17T05:25:16.818454+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.818454+0000","last_undegraded":"2024-09-17T05:25:16.818454+0000","last_fullsized":"2024-09-17T05:25:16.818454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843015+0000","last_change":"2024-09-17T05:25:17.834657+0000","last_active":"2024-09-17T05:25:17.843015+0000","last_peered":"2024-09-17T05:25:17.843015+0000","last_clean":"2024-09-17T05:25:17.843015+0000","last_became_active":"2024-09-17T05:25:17.834339+0000","last_became_peered":"2024-09-17T05:25:17.834339+0000","last_unstale":"2024-09-17T05:25:17.843015+0000","last_undegraded":"2024-09-17T05:25:17.843015+0000","last_fullsized":"2024-09-17T05:25:17.843015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862582+0000","last_change":"2024-09-17T05:25:17.857677+0000","last_active":"2024-09-17T05:25:17.862582+0000","last_peered":"2024-09-17T05:25:17.862582+0000","last_clean":"2024-09-17T05:25:17.862582+0000","last_became_active":"2024-09-17T05:25:17.857316+0000","last_became_peered":"2024-09-17T05:25:17.857316+0000","last_unstale":"2024-09-17T05:25:17.862582+0000","last_undegraded":"2024-09-17T05:25:17.862582+0000","last_fullsized":"2024-09-17T05:25:17.862582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.835972+0000","last_change":"2024-09-17T05:25:17.828984+0000","last_active":"2024-09-17T05:25:17.835972+0000","last_peered":"2024-09-17T05:25:17.835972+0000","last_clean":"2024-09-17T05:25:17.835972+0000","last_became_active":"2024-09-17T05:25:17.828631+0000","last_became_peered":"2024-09-17T05:25:17.828631+0000","last_unstale":"2024-09-17T05:25:17.835972+0000","last_undegraded":"2024-09-17T05:25:17.835972+0000","last_fullsized":"2024-09-17T05:25:17.835972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:25.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.819073+0000","last_change":"2024-09-17T05:25:16.819073+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.819073+0000","last_undegraded":"2024-09-17T05:25:16.819073+0000","last_fullsized":"2024-09-17T05:25:16.819073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859255+0000","last_change":"2024-09-17T05:25:17.847391+0000","last_active":"2024-09-17T05:25:17.859255+0000","last_peered":"2024-09-17T05:25:17.859255+0000","last_clean":"2024-09-17T05:25:17.859255+0000","last_became_active":"2024-09-17T05:25:17.847099+0000","last_became_peered":"2024-09-17T05:25:17.847099+0000","last_unstale":"2024-09-17T05:25:17.859255+0000","last_undegraded":"2024-09-17T05:25:17.859255+0000","last_fullsized":"2024-09-17T05:25:17.859255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:18.658668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862139+0000","last_change":"2024-09-17T05:25:17.852627+0000","last_active":"2024-09-17T05:25:17.862139+0000","last_peered":"2024-09-17T05:25:17.862139+0000","last_clean":"2024-09-17T05:25:17.862139+0000","last_became_active":"2024-09-17T05:25:17.849911+0000","last_became_peered":"2024-09-17T05:25:17.849911+0000","last_unstale":"2024-09-17T05:25:17.862139+0000","last_undegraded":"2024-09-17T05:25:17.862139+0000","last_fullsized":"2024-09-17T05:25:17.862139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:20.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852603+0000","last_change":"2024-09-17T05:25:17.849702+0000","last_active":"2024-09-17T05:25:17.852603+0000","last_peered":"2024-09-17T05:25:17.852603+0000","last_clean":"2024-09-17T05:25:17.852603+0000","last_became_active":"2024-09-17T05:25:17.849304+0000","last_became_peered":"2024-09-17T05:25:17.849304+0000","last_unstale":"2024-09-17T05:25:17.852603+0000","last_undegraded":"2024-09-17T05:25:17.852603+0000","last_fullsized":"2024-09-17T05:25:17.852603+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:31.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.844026+0000","last_change":"2024-09-17T05:25:17.838654+0000","last_active":"2024-09-17T05:25:17.844026+0000","last_peered":"2024-09-17T05:25:17.844026+0000","last_clean":"2024-09-17T05:25:17.844026+0000","last_became_active":"2024-09-17T05:25:17.838259+0000","last_became_peered":"2024-09-17T05:25:17.838259+0000","last_unstale":"2024-09-17T05:25:17.844026+0000","last_undegraded":"2024-09-17T05:25:17.844026+0000","last_fullsized":"2024-09-17T05:25:17.844026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:29.702298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863471+0000","last_change":"2024-09-17T05:25:17.862226+0000","last_active":"2024-09-17T05:25:17.863471+0000","last_peered":"2024-09-17T05:25:17.863471+0000","last_clean":"2024-09-17T05:25:17.863471+0000","last_became_active":"2024-09-17T05:25:17.860613+0000","last_became_peered":"2024-09-17T05:25:17.860613+0000","last_unstale":"2024-09-17T05:25:17.863471+0000","last_undegraded":"2024-09-17T05:25:17.863471+0000","last_fullsized":"2024-09-17T05:25:17.863471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:41.914430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859049+0000","last_change":"2024-09-17T05:25:17.855133+0000","last_active":"2024-09-17T05:25:17.859049+0000","last_peered":"2024-09-17T05:25:17.859049+0000","last_clean":"2024-09-17T05:25:17.859049+0000","last_became_active":"2024-09-17T05:25:17.854806+0000","last_became_peered":"2024-09-17T05:25:17.854806+0000","last_unstale":"2024-09-17T05:25:17.859049+0000","last_undegraded":"2024-09-17T05:25:17.859049+0000","last_fullsized":"2024-09-17T05:25:17.859049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:13.815396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.851242+0000","last_change":"2024-09-17T05:25:17.840899+0000","last_active":"2024-09-17T05:25:17.851242+0000","last_peered":"2024-09-17T05:25:17.851242+0000","last_clean":"2024-09-17T05:25:17.851242+0000","last_became_active":"2024-09-17T05:25:17.832241+0000","last_became_peered":"2024-09-17T05:25:17.832241+0000","last_unstale":"2024-09-17T05:25:17.851242+0000","last_undegraded":"2024-09-17T05:25:17.851242+0000","last_fullsized":"2024-09-17T05:25:17.851242+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:07.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859491+0000","last_change":"2024-09-17T05:25:17.846375+0000","last_active":"2024-09-17T05:25:17.859491+0000","last_peered":"2024-09-17T05:25:17.859491+0000","last_clean":"2024-09-17T05:25:17.859491+0000","last_became_active":"2024-09-17T05:25:17.846001+0000","last_became_peered":"2024-09-17T05:25:17.846001+0000","last_unstale":"2024-09-17T05:25:17.859491+0000","last_undegraded":"2024-09-17T05:25:17.859491+0000","last_fullsized":"2024-09-17T05:25:17.859491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:34.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855800+0000","last_change":"2024-09-17T05:25:17.849145+0000","last_active":"2024-09-17T05:25:17.855800+0000","last_peered":"2024-09-17T05:25:17.855800+0000","last_clean":"2024-09-17T05:25:17.855800+0000","last_became_active":"2024-09-17T05:25:17.848763+0000","last_became_peered":"2024-09-17T05:25:17.848763+0000","last_unstale":"2024-09-17T05:25:17.855800+0000","last_undegraded":"2024-09-17T05:25:17.855800+0000","last_fullsized":"2024-09-17T05:25:17.855800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:53.559077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.846762+0000","last_change":"2024-09-17T05:25:17.839774+0000","last_active":"2024-09-17T05:25:17.846762+0000","last_peered":"2024-09-17T05:25:17.846762+0000","last_clean":"2024-09-17T05:25:17.846762+0000","last_became_active":"2024-09-17T05:25:17.839418+0000","last_became_peered":"2024-09-17T05:25:17.839418+0000","last_unstale":"2024-09-17T05:25:17.846762+0000","last_undegraded":"2024-09-17T05:25:17.846762+0000","last_fullsized":"2024-09-17T05:25:17.846762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:59.137139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864301+0000","last_change":"2024-09-17T05:25:17.860865+0000","last_active":"2024-09-17T05:25:17.864301+0000","last_peered":"2024-09-17T05:25:17.864301+0000","last_clean":"2024-09-17T05:25:17.864301+0000","last_became_active":"2024-09-17T05:25:17.857489+0000","last_became_peered":"2024-09-17T05:25:17.857489+0000","last_unstale":"2024-09-17T05:25:17.864301+0000","last_undegraded":"2024-09-17T05:25:17.864301+0000","last_fullsized":"2024-09-17T05:25:17.864301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830904+0000","last_change":"2024-09-17T05:25:17.827002+0000","last_active":"2024-09-17T05:25:17.830904+0000","last_peered":"2024-09-17T05:25:17.830904+0000","last_clean":"2024-09-17T05:25:17.830904+0000","last_became_active":"2024-09-17T05:25:17.826648+0000","last_became_peered":"2024-09-17T05:25:17.826648+0000","last_unstale":"2024-09-17T05:25:17.830904+0000","last_undegraded":"2024-09-17T05:25:17.830904+0000","last_fullsized":"2024-09-17T05:25:17.830904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862280+0000","last_change":"2024-09-17T05:25:17.855977+0000","last_active":"2024-09-17T05:25:17.862280+0000","last_peered":"2024-09-17T05:25:17.862280+0000","last_clean":"2024-09-17T05:25:17.862280+0000","last_became_active":"2024-09-17T05:25:17.855650+0000","last_became_peered":"2024-09-17T05:25:17.855650+0000","last_unstale":"2024-09-17T05:25:17.862280+0000","last_undegraded":"2024-09-17T05:25:17.862280+0000","last_fullsized":"2024-09-17T05:25:17.862280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864199+0000","last_change":"2024-09-17T05:25:17.860605+0000","last_active":"2024-09-17T05:25:17.864199+0000","last_peered":"2024-09-17T05:25:17.864199+0000","last_clean":"2024-09-17T05:25:17.864199+0000","last_became_active":"2024-09-17T05:25:17.859963+0000","last_became_peered":"2024-09-17T05:25:17.859963+0000","last_unstale":"2024-09-17T05:25:17.864199+0000","last_undegraded":"2024-09-17T05:25:17.864199+0000","last_fullsized":"2024-09-17T05:25:17.864199+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:55.637768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852200+0000","last_change":"2024-09-17T05:25:17.841362+0000","last_active":"2024-09-17T05:25:17.852200+0000","last_peered":"2024-09-17T05:25:17.852200+0000","last_clean":"2024-09-17T05:25:17.852200+0000","last_became_active":"2024-09-17T05:25:17.840961+0000","last_became_peered":"2024-09-17T05:25:17.840961+0000","last_unstale":"2024-09-17T05:25:17.852200+0000","last_undegraded":"2024-09-17T05:25:17.852200+0000","last_fullsized":"2024-09-17T05:25:17.852200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.926288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837038+0000","last_change":"2024-09-17T05:25:17.829627+0000","last_active":"2024-09-17T05:25:17.837038+0000","last_peered":"2024-09-17T05:25:17.837038+0000","last_clean":"2024-09-17T05:25:17.837038+0000","last_became_active":"2024-09-17T05:25:17.829328+0000","last_became_peered":"2024-09-17T05:25:17.829328+0000","last_unstale":"2024-09-17T05:25:17.837038+0000","last_undegraded":"2024-09-17T05:25:17.837038+0000","last_fullsized":"2024-09-17T05:25:17.837038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:27.488602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.861137+0000","last_change":"2024-09-17T05:25:17.855557+0000","last_active":"2024-09-17T05:25:17.861137+0000","last_peered":"2024-09-17T05:25:17.861137+0000","last_clean":"2024-09-17T05:25:17.861137+0000","last_became_active":"2024-09-17T05:25:17.854842+0000","last_became_peered":"2024-09-17T05:25:17.854842+0000","last_unstale":"2024-09-17T05:25:17.861137+0000","last_undegraded":"2024-09-17T05:25:17.861137+0000","last_fullsized":"2024-09-17T05:25:17.861137+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:43.613646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817820+0000","last_change":"2024-09-17T05:25:16.817820+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817820+0000","last_undegraded":"2024-09-17T05:25:16.817820+0000","last_fullsized":"2024-09-17T05:25:16.817820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862079+0000","last_change":"2024-09-17T05:25:17.851832+0000","last_active":"2024-09-17T05:25:17.862079+0000","last_peered":"2024-09-17T05:25:17.862079+0000","last_clean":"2024-09-17T05:25:17.862079+0000","last_became_active":"2024-09-17T05:25:17.851503+0000","last_became_peered":"2024-09-17T05:25:17.851503+0000","last_unstale":"2024-09-17T05:25:17.862079+0000","last_undegraded":"2024-09-17T05:25:17.862079+0000","last_fullsized":"2024-09-17T05:25:17.862079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.055649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850746+0000","last_change":"2024-09-17T05:25:17.845069+0000","last_active":"2024-09-17T05:25:17.850746+0000","last_peered":"2024-09-17T05:25:17.850746+0000","last_clean":"2024-09-17T05:25:17.850746+0000","last_became_active":"2024-09-17T05:25:17.844766+0000","last_became_peered":"2024-09-17T05:25:17.844766+0000","last_unstale":"2024-09-17T05:25:17.850746+0000","last_undegraded":"2024-09-17T05:25:17.850746+0000","last_fullsized":"2024-09-17T05:25:17.850746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:18.184047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.838673+0000","last_change":"2024-09-17T05:25:16.830007+0000","last_active":"2024-09-17T05:25:16.838673+0000","last_peered":"2024-09-17T05:25:16.838673+0000","last_clean":"2024-09-17T05:25:16.838673+0000","last_became_active":"2024-09-17T05:25:16.828838+0000","last_became_peered":"2024-09-17T05:25:16.828838+0000","last_unstale":"2024-09-17T05:25:16.838673+0000","last_undegraded":"2024-09-17T05:25:16.838673+0000","last_fullsized":"2024-09-17T05:25:16.838673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:34.533131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858878+0000","last_change":"2024-09-17T05:25:17.855463+0000","last_active":"2024-09-17T05:25:17.858878+0000","last_peered":"2024-09-17T05:25:17.858878+0000","last_clean":"2024-09-17T05:25:17.858878+0000","last_became_active":"2024-09-17T05:25:17.855089+0000","last_became_peered":"2024-09-17T05:25:17.855089+0000","last_unstale":"2024-09-17T05:25:17.858878+0000","last_undegraded":"2024-09-17T05:25:17.858878+0000","last_fullsized":"2024-09-17T05:25:17.858878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.860692+0000","last_change":"2024-09-17T05:25:16.841386+0000","last_active":"2024-09-17T05:25:17.860692+0000","last_peered":"2024-09-17T05:25:17.860692+0000","last_clean":"2024-09-17T05:25:17.860692+0000","last_became_active":"2024-09-17T05:25:16.840942+0000","last_became_peered":"2024-09-17T05:25:16.840942+0000","last_unstale":"2024-09-17T05:25:17.860692+0000","last_undegraded":"2024-09-17T05:25:17.860692+0000","last_fullsized":"2024-09-17T05:25:17.860692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:09.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848773+0000","last_change":"2024-09-17T05:25:17.844192+0000","last_active":"2024-09-17T05:25:17.848773+0000","last_peered":"2024-09-17T05:25:17.848773+0000","last_clean":"2024-09-17T05:25:17.848773+0000","last_became_active":"2024-09-17T05:25:17.843753+0000","last_became_peered":"2024-09-17T05:25:17.843753+0000","last_unstale":"2024-09-17T05:25:17.848773+0000","last_undegraded":"2024-09-17T05:25:17.848773+0000","last_fullsized":"2024-09-17T05:25:17.848773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:45.282549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839405+0000","last_change":"2024-09-17T05:25:16.835803+0000","last_active":"2024-09-17T05:25:16.839405+0000","last_peered":"2024-09-17T05:25:16.839405+0000","last_clean":"2024-09-17T05:25:16.839405+0000","last_became_active":"2024-09-17T05:25:16.835332+0000","last_became_peered":"2024-09-17T05:25:16.835332+0000","last_unstale":"2024-09-17T05:25:16.839405+0000","last_undegraded":"2024-09-17T05:25:16.839405+0000","last_fullsized":"2024-09-17T05:25:16.839405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:57.056082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862435+0000","last_change":"2024-09-17T05:25:17.853747+0000","last_active":"2024-09-17T05:25:17.862435+0000","last_peered":"2024-09-17T05:25:17.862435+0000","last_clean":"2024-09-17T05:25:17.862435+0000","last_became_active":"2024-09-17T05:25:17.853420+0000","last_became_peered":"2024-09-17T05:25:17.853420+0000","last_unstale":"2024-09-17T05:25:17.862435+0000","last_undegraded":"2024-09-17T05:25:17.862435+0000","last_fullsized":"2024-09-17T05:25:17.862435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862840+0000","last_change":"2024-09-17T05:25:16.839690+0000","last_active":"2024-09-17T05:25:17.862840+0000","last_peered":"2024-09-17T05:25:17.862840+0000","last_clean":"2024-09-17T05:25:17.862840+0000","last_became_active":"2024-09-17T05:25:16.839208+0000","last_became_peered":"2024-09-17T05:25:16.839208+0000","last_unstale":"2024-09-17T05:25:17.862840+0000","last_undegraded":"2024-09-17T05:25:17.862840+0000","last_fullsized":"2024-09-17T05:25:17.862840+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:20.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864027+0000","last_change":"2024-09-17T05:25:17.860246+0000","last_active":"2024-09-17T05:25:17.864027+0000","last_peered":"2024-09-17T05:25:17.864027+0000","last_clean":"2024-09-17T05:25:17.864027+0000","last_became_active":"2024-09-17T05:25:17.859898+0000","last_became_peered":"2024-09-17T05:25:17.859898+0000","last_unstale":"2024-09-17T05:25:17.864027+0000","last_undegraded":"2024-09-17T05:25:17.864027+0000","last_fullsized":"2024-09-17T05:25:17.864027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:40.128894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.832185+0000","last_change":"2024-09-17T05:25:16.825193+0000","last_active":"2024-09-17T05:25:16.832185+0000","last_peered":"2024-09-17T05:25:16.832185+0000","last_clean":"2024-09-17T05:25:16.832185+0000","last_became_active":"2024-09-17T05:25:16.824705+0000","last_became_peered":"2024-09-17T05:25:16.824705+0000","last_unstale":"2024-09-17T05:25:16.832185+0000","last_undegraded":"2024-09-17T05:25:16.832185+0000","last_fullsized":"2024-09-17T05:25:16.832185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:23.844925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852903+0000","last_change":"2024-09-17T05:25:17.844440+0000","last_active":"2024-09-17T05:25:17.852903+0000","last_peered":"2024-09-17T05:25:17.852903+0000","last_clean":"2024-09-17T05:25:17.852903+0000","last_became_active":"2024-09-17T05:25:17.843574+0000","last_became_peered":"2024-09-17T05:25:17.843574+0000","last_unstale":"2024-09-17T05:25:17.852903+0000","last_undegraded":"2024-09-17T05:25:17.852903+0000","last_fullsized":"2024-09-17T05:25:17.852903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:27.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859395+0000","last_change":"2024-09-17T05:25:16.839269+0000","last_active":"2024-09-17T05:25:17.859395+0000","last_peered":"2024-09-17T05:25:17.859395+0000","last_clean":"2024-09-17T05:25:17.859395+0000","last_became_active":"2024-09-17T05:25:16.838891+0000","last_became_peered":"2024-09-17T05:25:16.838891+0000","last_unstale":"2024-09-17T05:25:17.859395+0000","last_undegraded":"2024-09-17T05:25:17.859395+0000","last_fullsized":"2024-09-17T05:25:17.859395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.956617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850851+0000","last_change":"2024-09-17T05:25:17.840191+0000","last_active":"2024-09-17T05:25:17.850851+0000","last_peered":"2024-09-17T05:25:17.850851+0000","last_clean":"2024-09-17T05:25:17.850851+0000","last_became_active":"2024-09-17T05:25:17.839875+0000","last_became_peered":"2024-09-17T05:25:17.839875+0000","last_unstale":"2024-09-17T05:25:17.850851+0000","last_undegraded":"2024-09-17T05:25:17.850851+0000","last_fullsized":"2024-09-17T05:25:17.850851+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818683+0000","last_change":"2024-09-17T05:25:16.834402+0000","last_active":"2024-09-17T05:25:17.818683+0000","last_peered":"2024-09-17T05:25:17.818683+0000","last_clean":"2024-09-17T05:25:17.818683+0000","last_became_active":"2024-09-17T05:25:16.834036+0000","last_became_peered":"2024-09-17T05:25:16.834036+0000","last_unstale":"2024-09-17T05:25:17.818683+0000","last_undegraded":"2024-09-17T05:25:17.818683+0000","last_fullsized":"2024-09-17T05:25:17.818683+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:37.453048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863162+0000","last_change":"2024-09-17T05:25:17.855835+0000","last_active":"2024-09-17T05:25:17.863162+0000","last_peered":"2024-09-17T05:25:17.863162+0000","last_clean":"2024-09-17T05:25:17.863162+0000","last_became_active":"2024-09-17T05:25:17.853327+0000","last_became_peered":"2024-09-17T05:25:17.853327+0000","last_unstale":"2024-09-17T05:25:17.863162+0000","last_undegraded":"2024-09-17T05:25:17.863162+0000","last_fullsized":"2024-09-17T05:25:17.863162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:41.256886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.823113+0000","last_change":"2024-09-17T05:25:16.839214+0000","last_active":"2024-09-17T05:25:17.823113+0000","last_peered":"2024-09-17T05:25:17.823113+0000","last_clean":"2024-09-17T05:25:17.823113+0000","last_became_active":"2024-09-17T05:25:16.838825+0000","last_became_peered":"2024-09-17T05:25:16.838825+0000","last_unstale":"2024-09-17T05:25:17.823113+0000","last_undegraded":"2024-09-17T05:25:17.823113+0000","last_fullsized":"2024-09-17T05:25:17.823113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:30.086751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852021+0000","last_change":"2024-09-17T05:25:17.850811+0000","last_active":"2024-09-17T05:25:17.852021+0000","last_peered":"2024-09-17T05:25:17.852021+0000","last_clean":"2024-09-17T05:25:17.852021+0000","last_became_active":"2024-09-17T05:25:17.850480+0000","last_became_peered":"2024-09-17T05:25:17.850480+0000","last_unstale":"2024-09-17T05:25:17.852021+0000","last_undegraded":"2024-09-17T05:25:17.852021+0000","last_fullsized":"2024-09-17T05:25:17.852021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:48.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836130+0000","last_change":"2024-09-17T05:25:16.841708+0000","last_active":"2024-09-17T05:25:17.836130+0000","last_peered":"2024-09-17T05:25:17.836130+0000","last_clean":"2024-09-17T05:25:17.836130+0000","last_became_active":"2024-09-17T05:25:16.841263+0000","last_became_peered":"2024-09-17T05:25:16.841263+0000","last_unstale":"2024-09-17T05:25:17.836130+0000","last_undegraded":"2024-09-17T05:25:17.836130+0000","last_fullsized":"2024-09-17T05:25:17.836130+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:21.553815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852485+0000","last_change":"2024-09-17T05:25:17.851388+0000","last_active":"2024-09-17T05:25:17.852485+0000","last_peered":"2024-09-17T05:25:17.852485+0000","last_clean":"2024-09-17T05:25:17.852485+0000","last_became_active":"2024-09-17T05:25:17.851036+0000","last_became_peered":"2024-09-17T05:25:17.851036+0000","last_unstale":"2024-09-17T05:25:17.852485+0000","last_undegraded":"2024-09-17T05:25:17.852485+0000","last_fullsized":"2024-09-17T05:25:17.852485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:14.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821624+0000","last_change":"2024-09-17T05:25:16.843023+0000","last_active":"2024-09-17T05:25:17.821624+0000","last_peered":"2024-09-17T05:25:17.821624+0000","last_clean":"2024-09-17T05:25:17.821624+0000","last_became_active":"2024-09-17T05:25:16.842671+0000","last_became_peered":"2024-09-17T05:25:16.842671+0000","last_unstale":"2024-09-17T05:25:17.821624+0000","last_undegraded":"2024-09-17T05:25:17.821624+0000","last_fullsized":"2024-09-17T05:25:17.821624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:40.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859214+0000","last_change":"2024-09-17T05:25:17.844156+0000","last_active":"2024-09-17T05:25:17.859214+0000","last_peered":"2024-09-17T05:25:17.859214+0000","last_clean":"2024-09-17T05:25:17.859214+0000","last_became_active":"2024-09-17T05:25:17.843870+0000","last_became_peered":"2024-09-17T05:25:17.843870+0000","last_unstale":"2024-09-17T05:25:17.859214+0000","last_undegraded":"2024-09-17T05:25:17.859214+0000","last_fullsized":"2024-09-17T05:25:17.859214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:02.380001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864696+0000","last_change":"2024-09-17T05:25:16.825444+0000","last_active":"2024-09-17T05:25:17.864696+0000","last_peered":"2024-09-17T05:25:17.864696+0000","last_clean":"2024-09-17T05:25:17.864696+0000","last_became_active":"2024-09-17T05:25:16.825122+0000","last_became_peered":"2024-09-17T05:25:16.825122+0000","last_unstale":"2024-09-17T05:25:17.864696+0000","last_undegraded":"2024-09-17T05:25:17.864696+0000","last_fullsized":"2024-09-17T05:25:17.864696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:38.050632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864876+0000","last_change":"2024-09-17T05:25:17.860864+0000","last_active":"2024-09-17T05:25:17.864876+0000","last_peered":"2024-09-17T05:25:17.864876+0000","last_clean":"2024-09-17T05:25:17.864876+0000","last_became_active":"2024-09-17T05:25:17.860380+0000","last_became_peered":"2024-09-17T05:25:17.860380+0000","last_unstale":"2024-09-17T05:25:17.864876+0000","last_undegraded":"2024-09-17T05:25:17.864876+0000","last_fullsized":"2024-09-17T05:25:17.864876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:41.208537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.866116+0000","last_change":"2024-09-17T05:25:16.845121+0000","last_active":"2024-09-17T05:25:17.866116+0000","last_peered":"2024-09-17T05:25:17.866116+0000","last_clean":"2024-09-17T05:25:17.866116+0000","last_became_active":"2024-09-17T05:25:16.844739+0000","last_became_peered":"2024-09-17T05:25:16.844739+0000","last_unstale":"2024-09-17T05:25:17.866116+0000","last_undegraded":"2024-09-17T05:25:17.866116+0000","last_fullsized":"2024-09-17T05:25:17.866116+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:26.485416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856703+0000","last_change":"2024-09-17T05:25:17.854295+0000","last_active":"2024-09-17T05:25:17.856703+0000","last_peered":"2024-09-17T05:25:17.856703+0000","last_clean":"2024-09-17T05:25:17.856703+0000","last_became_active":"2024-09-17T05:25:17.853814+0000","last_became_peered":"2024-09-17T05:25:17.853814+0000","last_unstale":"2024-09-17T05:25:17.856703+0000","last_undegraded":"2024-09-17T05:25:17.856703+0000","last_fullsized":"2024-09-17T05:25:17.856703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:38.195708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856501+0000","last_change":"2024-09-17T05:25:16.839573+0000","last_active":"2024-09-17T05:25:17.856501+0000","last_peered":"2024-09-17T05:25:17.856501+0000","last_clean":"2024-09-17T05:25:17.856501+0000","last_became_active":"2024-09-17T05:25:16.838379+0000","last_became_peered":"2024-09-17T05:25:16.838379+0000","last_unstale":"2024-09-17T05:25:17.856501+0000","last_undegraded":"2024-09-17T05:25:17.856501+0000","last_fullsized":"2024-09-17T05:25:17.856501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:00.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843182+0000","last_change":"2024-09-17T05:25:17.836620+0000","last_active":"2024-09-17T05:25:17.843182+0000","last_peered":"2024-09-17T05:25:17.843182+0000","last_clean":"2024-09-17T05:25:17.843182+0000","last_became_active":"2024-09-17T05:25:17.836304+0000","last_became_peered":"2024-09-17T05:25:17.836304+0000","last_unstale":"2024-09-17T05:25:17.843182+0000","last_undegraded":"2024-09-17T05:25:17.843182+0000","last_fullsized":"2024-09-17T05:25:17.843182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:46.933484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818843+0000","last_change":"2024-09-17T05:25:16.837793+0000","last_active":"2024-09-17T05:25:17.818843+0000","last_peered":"2024-09-17T05:25:17.818843+0000","last_clean":"2024-09-17T05:25:17.818843+0000","last_became_active":"2024-09-17T05:25:16.837076+0000","last_became_peered":"2024-09-17T05:25:16.837076+0000","last_unstale":"2024-09-17T05:25:17.818843+0000","last_undegraded":"2024-09-17T05:25:17.818843+0000","last_fullsized":"2024-09-17T05:25:17.818843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:44.279363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850823+0000","last_change":"2024-09-17T05:25:17.846642+0000","last_active":"2024-09-17T05:25:17.850823+0000","last_peered":"2024-09-17T05:25:17.850823+0000","last_clean":"2024-09-17T05:25:17.850823+0000","last_became_active":"2024-09-17T05:25:17.844111+0000","last_became_peered":"2024-09-17T05:25:17.844111+0000","last_unstale":"2024-09-17T05:25:17.850823+0000","last_undegraded":"2024-09-17T05:25:17.850823+0000","last_fullsized":"2024-09-17T05:25:17.850823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:40.896672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.844453+0000","last_change":"2024-09-17T05:25:16.842966+0000","last_active":"2024-09-17T05:25:16.844453+0000","last_peered":"2024-09-17T05:25:16.844453+0000","last_clean":"2024-09-17T05:25:16.844453+0000","last_became_active":"2024-09-17T05:25:16.842650+0000","last_became_peered":"2024-09-17T05:25:16.842650+0000","last_unstale":"2024-09-17T05:25:16.844453+0000","last_undegraded":"2024-09-17T05:25:16.844453+0000","last_fullsized":"2024-09-17T05:25:16.844453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:55:02.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.834870+0000","last_change":"2024-09-17T05:25:17.827842+0000","last_active":"2024-09-17T05:25:17.834870+0000","last_peered":"2024-09-17T05:25:17.834870+0000","last_clean":"2024-09-17T05:25:17.834870+0000","last_became_active":"2024-09-17T05:25:17.827506+0000","last_became_peered":"2024-09-17T05:25:17.827506+0000","last_unstale":"2024-09-17T05:25:17.834870+0000","last_undegraded":"2024-09-17T05:25:17.834870+0000","last_fullsized":"2024-09-17T05:25:17.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:52.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863866+0000","last_change":"2024-09-17T05:25:16.835808+0000","last_active":"2024-09-17T05:25:17.863866+0000","last_peered":"2024-09-17T05:25:17.863866+0000","last_clean":"2024-09-17T05:25:17.863866+0000","last_became_active":"2024-09-17T05:25:16.835452+0000","last_became_peered":"2024-09-17T05:25:16.835452+0000","last_unstale":"2024-09-17T05:25:17.863866+0000","last_undegraded":"2024-09-17T05:25:17.863866+0000","last_fullsized":"2024-09-17T05:25:17.863866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:12.812210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862083+0000","last_change":"2024-09-17T05:25:17.855358+0000","last_active":"2024-09-17T05:25:17.862083+0000","last_peered":"2024-09-17T05:25:17.862083+0000","last_clean":"2024-09-17T05:25:17.862083+0000","last_became_active":"2024-09-17T05:25:17.854829+0000","last_became_peered":"2024-09-17T05:25:17.854829+0000","last_unstale":"2024-09-17T05:25:17.862083+0000","last_undegraded":"2024-09-17T05:25:17.862083+0000","last_fullsized":"2024-09-17T05:25:17.862083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:41.242930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859036+0000","last_change":"2024-09-17T05:25:16.842074+0000","last_active":"2024-09-17T05:25:17.859036+0000","last_peered":"2024-09-17T05:25:17.859036+0000","last_clean":"2024-09-17T05:25:17.859036+0000","last_became_active":"2024-09-17T05:25:16.841700+0000","last_became_peered":"2024-09-17T05:25:16.841700+0000","last_unstale":"2024-09-17T05:25:17.859036+0000","last_undegraded":"2024-09-17T05:25:17.859036+0000","last_fullsized":"2024-09-17T05:25:17.859036+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:38.831286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850551+0000","last_change":"2024-09-17T05:25:17.843043+0000","last_active":"2024-09-17T05:25:17.850551+0000","last_peered":"2024-09-17T05:25:17.850551+0000","last_clean":"2024-09-17T05:25:17.850551+0000","last_became_active":"2024-09-17T05:25:17.842110+0000","last_became_peered":"2024-09-17T05:25:17.842110+0000","last_unstale":"2024-09-17T05:25:17.850551+0000","last_undegraded":"2024-09-17T05:25:17.850551+0000","last_fullsized":"2024-09-17T05:25:17.850551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:58.529182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.854795+0000","last_change":"2024-09-17T05:25:16.843994+0000","last_active":"2024-09-17T05:25:17.854795+0000","last_peered":"2024-09-17T05:25:17.854795+0000","last_clean":"2024-09-17T05:25:17.854795+0000","last_became_active":"2024-09-17T05:25:16.843335+0000","last_became_peered":"2024-09-17T05:25:16.843335+0000","last_unstale":"2024-09-17T05:25:17.854795+0000","last_undegraded":"2024-09-17T05:25:17.854795+0000","last_fullsized":"2024-09-17T05:25:17.854795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:10.828864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863238+0000","last_change":"2024-09-17T05:25:17.858165+0000","last_active":"2024-09-17T05:25:17.863238+0000","last_peered":"2024-09-17T05:25:17.863238+0000","last_clean":"2024-09-17T05:25:17.863238+0000","last_became_active":"2024-09-17T05:25:17.856741+0000","last_became_peered":"2024-09-17T05:25:17.856741+0000","last_unstale":"2024-09-17T05:25:17.863238+0000","last_undegraded":"2024-09-17T05:25:17.863238+0000","last_fullsized":"2024-09-17T05:25:17.863238+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:50.149002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856205+0000","last_change":"2024-09-17T05:25:16.842760+0000","last_active":"2024-09-17T05:25:17.856205+0000","last_peered":"2024-09-17T05:25:17.856205+0000","last_clean":"2024-09-17T05:25:17.856205+0000","last_became_active":"2024-09-17T05:25:16.842399+0000","last_became_peered":"2024-09-17T05:25:16.842399+0000","last_unstale":"2024-09-17T05:25:17.856205+0000","last_undegraded":"2024-09-17T05:25:17.856205+0000","last_fullsized":"2024-09-17T05:25:17.856205+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:04.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850747+0000","last_change":"2024-09-17T05:25:17.846811+0000","last_active":"2024-09-17T05:25:17.850747+0000","last_peered":"2024-09-17T05:25:17.850747+0000","last_clean":"2024-09-17T05:25:17.850747+0000","last_became_active":"2024-09-17T05:25:17.846191+0000","last_became_peered":"2024-09-17T05:25:17.846191+0000","last_unstale":"2024-09-17T05:25:17.850747+0000","last_undegraded":"2024-09-17T05:25:17.850747+0000","last_fullsized":"2024-09-17T05:25:17.850747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:42.562348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839603+0000","last_change":"2024-09-17T05:25:16.836689+0000","last_active":"2024-09-17T05:25:16.839603+0000","last_peered":"2024-09-17T05:25:16.839603+0000","last_clean":"2024-09-17T05:25:16.839603+0000","last_became_active":"2024-09-17T05:25:16.836345+0000","last_became_peered":"2024-09-17T05:25:16.836345+0000","last_unstale":"2024-09-17T05:25:16.839603+0000","last_undegraded":"2024-09-17T05:25:16.839603+0000","last_fullsized":"2024-09-17T05:25:16.839603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:09.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856658+0000","last_change":"2024-09-17T05:25:17.849888+0000","last_active":"2024-09-17T05:25:17.856658+0000","last_peered":"2024-09-17T05:25:17.856658+0000","last_clean":"2024-09-17T05:25:17.856658+0000","last_became_active":"2024-09-17T05:25:17.848605+0000","last_became_peered":"2024-09-17T05:25:17.848605+0000","last_unstale":"2024-09-17T05:25:17.856658+0000","last_undegraded":"2024-09-17T05:25:17.856658+0000","last_fullsized":"2024-09-17T05:25:17.856658+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:35.175603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852768+0000","last_change":"2024-09-17T05:25:16.836994+0000","last_active":"2024-09-17T05:25:17.852768+0000","last_peered":"2024-09-17T05:25:17.852768+0000","last_clean":"2024-09-17T05:25:17.852768+0000","last_became_active":"2024-09-17T05:25:16.836576+0000","last_became_peered":"2024-09-17T05:25:16.836576+0000","last_unstale":"2024-09-17T05:25:17.852768+0000","last_undegraded":"2024-09-17T05:25:17.852768+0000","last_fullsized":"2024-09-17T05:25:17.852768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:41.094836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817331+0000","last_change":"2024-09-17T05:25:16.817331+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817331+0000","last_undegraded":"2024-09-17T05:25:16.817331+0000","last_fullsized":"2024-09-17T05:25:16.817331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819191+0000","last_change":"2024-09-17T05:25:16.837411+0000","last_active":"2024-09-17T05:25:17.819191+0000","last_peered":"2024-09-17T05:25:17.819191+0000","last_clean":"2024-09-17T05:25:17.819191+0000","last_became_active":"2024-09-17T05:25:16.837058+0000","last_became_peered":"2024-09-17T05:25:16.837058+0000","last_unstale":"2024-09-17T05:25:17.819191+0000","last_undegraded":"2024-09-17T05:25:17.819191+0000","last_fullsized":"2024-09-17T05:25:17.819191+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:42.610461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859220+0000","last_change":"2024-09-17T05:25:17.850227+0000","last_active":"2024-09-17T05:25:17.859220+0000","last_peered":"2024-09-17T05:25:17.859220+0000","last_clean":"2024-09-17T05:25:17.859220+0000","last_became_active":"2024-09-17T05:25:17.849926+0000","last_became_peered":"2024-09-17T05:25:17.849926+0000","last_unstale":"2024-09-17T05:25:17.859220+0000","last_undegraded":"2024-09-17T05:25:17.859220+0000","last_fullsized":"2024-09-17T05:25:17.859220+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:35.791860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821447+0000","last_change":"2024-09-17T05:25:16.843105+0000","last_active":"2024-09-17T05:25:17.821447+0000","last_peered":"2024-09-17T05:25:17.821447+0000","last_clean":"2024-09-17T05:25:17.821447+0000","last_became_active":"2024-09-17T05:25:16.842668+0000","last_became_peered":"2024-09-17T05:25:16.842668+0000","last_unstale":"2024-09-17T05:25:17.821447+0000","last_undegraded":"2024-09-17T05:25:17.821447+0000","last_fullsized":"2024-09-17T05:25:17.821447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859135+0000","last_change":"2024-09-17T05:25:17.847878+0000","last_active":"2024-09-17T05:25:17.859135+0000","last_peered":"2024-09-17T05:25:17.859135+0000","last_clean":"2024-09-17T05:25:17.859135+0000","last_became_active":"2024-09-17T05:25:17.844937+0000","last_became_peered":"2024-09-17T05:25:17.844937+0000","last_unstale":"2024-09-17T05:25:17.859135+0000","last_undegraded":"2024-09-17T05:25:17.859135+0000","last_fullsized":"2024-09-17T05:25:17.859135+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:11.788487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837181+0000","last_change":"2024-09-17T05:25:16.839427+0000","last_active":"2024-09-17T05:25:17.837181+0000","last_peered":"2024-09-17T05:25:17.837181+0000","last_clean":"2024-09-17T05:25:17.837181+0000","last_became_active":"2024-09-17T05:25:16.838642+0000","last_became_peered":"2024-09-17T05:25:16.838642+0000","last_unstale":"2024-09-17T05:25:17.837181+0000","last_undegraded":"2024-09-17T05:25:17.837181+0000","last_fullsized":"2024-09-17T05:25:17.837181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.129441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859527+0000","last_change":"2024-09-17T05:25:17.854279+0000","last_active":"2024-09-17T05:25:17.859527+0000","last_peered":"2024-09-17T05:25:17.859527+0000","last_clean":"2024-09-17T05:25:17.859527+0000","last_became_active":"2024-09-17T05:25:17.853347+0000","last_became_peered":"2024-09-17T05:25:17.853347+0000","last_unstale":"2024-09-17T05:25:17.859527+0000","last_undegraded":"2024-09-17T05:25:17.859527+0000","last_fullsized":"2024-09-17T05:25:17.859527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:48.657810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.832834+0000","last_change":"2024-09-17T05:25:16.845000+0000","last_active":"2024-09-17T05:25:17.832834+0000","last_peered":"2024-09-17T05:25:17.832834+0000","last_clean":"2024-09-17T05:25:17.832834+0000","last_became_active":"2024-09-17T05:25:16.844264+0000","last_became_peered":"2024-09-17T05:25:16.844264+0000","last_unstale":"2024-09-17T05:25:17.832834+0000","last_undegraded":"2024-09-17T05:25:17.832834+0000","last_fullsized":"2024-09-17T05:25:17.832834+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:10.422465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852661+0000","last_change":"2024-09-17T05:25:17.848021+0000","last_active":"2024-09-17T05:25:17.852661+0000","last_peered":"2024-09-17T05:25:17.852661+0000","last_clean":"2024-09-17T05:25:17.852661+0000","last_became_active":"2024-09-17T05:25:17.847264+0000","last_became_peered":"2024-09-17T05:25:17.847264+0000","last_unstale":"2024-09-17T05:25:17.852661+0000","last_undegraded":"2024-09-17T05:25:17.852661+0000","last_fullsized":"2024-09-17T05:25:17.852661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:48.742377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.820417+0000","last_change":"2024-09-17T05:25:16.838187+0000","last_active":"2024-09-17T05:25:17.820417+0000","last_peered":"2024-09-17T05:25:17.820417+0000","last_clean":"2024-09-17T05:25:17.820417+0000","last_became_active":"2024-09-17T05:25:16.837770+0000","last_became_peered":"2024-09-17T05:25:16.837770+0000","last_unstale":"2024-09-17T05:25:17.820417+0000","last_undegraded":"2024-09-17T05:25:17.820417+0000","last_fullsized":"2024-09-17T05:25:17.820417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864215+0000","last_change":"2024-09-17T05:25:17.860851+0000","last_active":"2024-09-17T05:25:17.864215+0000","last_peered":"2024-09-17T05:25:17.864215+0000","last_clean":"2024-09-17T05:25:17.864215+0000","last_became_active":"2024-09-17T05:25:17.860262+0000","last_became_peered":"2024-09-17T05:25:17.860262+0000","last_unstale":"2024-09-17T05:25:17.864215+0000","last_undegraded":"2024-09-17T05:25:17.864215+0000","last_fullsized":"2024-09-17T05:25:17.864215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:15.173253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.827881+0000","last_change":"2024-09-17T05:25:16.841538+0000","last_active":"2024-09-17T05:25:17.827881+0000","last_peered":"2024-09-17T05:25:17.827881+0000","last_clean":"2024-09-17T05:25:17.827881+0000","last_became_active":"2024-09-17T05:25:16.840839+0000","last_became_peered":"2024-09-17T05:25:16.840839+0000","last_unstale":"2024-09-17T05:25:17.827881+0000","last_undegraded":"2024-09-17T05:25:17.827881+0000","last_fullsized":"2024-09-17T05:25:17.827881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:35.655115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856628+0000","last_change":"2024-09-17T05:25:17.849451+0000","last_active":"2024-09-17T05:25:17.856628+0000","last_peered":"2024-09-17T05:25:17.856628+0000","last_clean":"2024-09-17T05:25:17.856628+0000","last_became_active":"2024-09-17T05:25:17.848905+0000","last_became_peered":"2024-09-17T05:25:17.848905+0000","last_unstale":"2024-09-17T05:25:17.856628+0000","last_undegraded":"2024-09-17T05:25:17.856628+0000","last_fullsized":"2024-09-17T05:25:17.856628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:55.535278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819647+0000","last_change":"2024-09-17T05:25:16.841637+0000","last_active":"2024-09-17T05:25:17.819647+0000","last_peered":"2024-09-17T05:25:17.819647+0000","last_clean":"2024-09-17T05:25:17.819647+0000","last_became_active":"2024-09-17T05:25:16.840417+0000","last_became_peered":"2024-09-17T05:25:16.840417+0000","last_unstale":"2024-09-17T05:25:17.819647+0000","last_undegraded":"2024-09-17T05:25:17.819647+0000","last_fullsized":"2024-09-17T05:25:17.819647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:36.939696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862665+0000","last_change":"2024-09-17T05:25:17.858282+0000","last_active":"2024-09-17T05:25:17.862665+0000","last_peered":"2024-09-17T05:25:17.862665+0000","last_clean":"2024-09-17T05:25:17.862665+0000","last_became_active":"2024-09-17T05:25:17.856926+0000","last_became_peered":"2024-09-17T05:25:17.856926+0000","last_unstale":"2024-09-17T05:25:17.862665+0000","last_undegraded":"2024-09-17T05:25:17.862665+0000","last_fullsized":"2024-09-17T05:25:17.862665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:10.392801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837706+0000","last_change":"2024-09-17T05:25:16.843421+0000","last_active":"2024-09-17T05:25:17.837706+0000","last_peered":"2024-09-17T05:25:17.837706+0000","last_clean":"2024-09-17T05:25:17.837706+0000","last_became_active":"2024-09-17T05:25:16.843071+0000","last_became_peered":"2024-09-17T05:25:16.843071+0000","last_unstale":"2024-09-17T05:25:17.837706+0000","last_undegraded":"2024-09-17T05:25:17.837706+0000","last_fullsized":"2024-09-17T05:25:17.837706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855103+0000","last_change":"2024-09-17T05:25:17.851994+0000","last_active":"2024-09-17T05:25:17.855103+0000","last_peered":"2024-09-17T05:25:17.855103+0000","last_clean":"2024-09-17T05:25:17.855103+0000","last_became_active":"2024-09-17T05:25:17.851553+0000","last_became_peered":"2024-09-17T05:25:17.851553+0000","last_unstale":"2024-09-17T05:25:17.855103+0000","last_undegraded":"2024-09-17T05:25:17.855103+0000","last_fullsized":"2024-09-17T05:25:17.855103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:57.581440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839648+0000","last_change":"2024-09-17T05:25:16.838372+0000","last_active":"2024-09-17T05:25:16.839648+0000","last_peered":"2024-09-17T05:25:16.839648+0000","last_clean":"2024-09-17T05:25:16.839648+0000","last_became_active":"2024-09-17T05:25:16.837915+0000","last_became_peered":"2024-09-17T05:25:16.837915+0000","last_unstale":"2024-09-17T05:25:16.839648+0000","last_undegraded":"2024-09-17T05:25:16.839648+0000","last_fullsized":"2024-09-17T05:25:16.839648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:11.525857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863332+0000","last_change":"2024-09-17T05:25:17.857155+0000","last_active":"2024-09-17T05:25:17.863332+0000","last_peered":"2024-09-17T05:25:17.863332+0000","last_clean":"2024-09-17T05:25:17.863332+0000","last_became_active":"2024-09-17T05:25:17.855676+0000","last_became_peered":"2024-09-17T05:25:17.855676+0000","last_unstale":"2024-09-17T05:25:17.863332+0000","last_undegraded":"2024-09-17T05:25:17.863332+0000","last_fullsized":"2024-09-17T05:25:17.863332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:37.942881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.826545+0000","last_change":"2024-09-17T05:25:16.844953+0000","last_active":"2024-09-17T05:25:17.826545+0000","last_peered":"2024-09-17T05:25:17.826545+0000","last_clean":"2024-09-17T05:25:17.826545+0000","last_became_active":"2024-09-17T05:25:16.843607+0000","last_became_peered":"2024-09-17T05:25:16.843607+0000","last_unstale":"2024-09-17T05:25:17.826545+0000","last_undegraded":"2024-09-17T05:25:17.826545+0000","last_fullsized":"2024-09-17T05:25:17.826545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.778959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858722+0000","last_change":"2024-09-17T05:25:17.843412+0000","last_active":"2024-09-17T05:25:17.858722+0000","last_peered":"2024-09-17T05:25:17.858722+0000","last_clean":"2024-09-17T05:25:17.858722+0000","last_became_active":"2024-09-17T05:25:17.842545+0000","last_became_peered":"2024-09-17T05:25:17.842545+0000","last_unstale":"2024-09-17T05:25:17.858722+0000","last_undegraded":"2024-09-17T05:25:17.858722+0000","last_fullsized":"2024-09-17T05:25:17.858722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:20.366914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864068+0000","last_change":"2024-09-17T05:25:16.842097+0000","last_active":"2024-09-17T05:25:17.864068+0000","last_peered":"2024-09-17T05:25:17.864068+0000","last_clean":"2024-09-17T05:25:17.864068+0000","last_became_active":"2024-09-17T05:25:16.841570+0000","last_became_peered":"2024-09-17T05:25:16.841570+0000","last_unstale":"2024-09-17T05:25:17.864068+0000","last_undegraded":"2024-09-17T05:25:17.864068+0000","last_fullsized":"2024-09-17T05:25:17.864068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":154,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830646+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:17.830646+0000","last_peered":"2024-09-17T05:25:17.830646+0000","last_clean":"2024-09-17T05:25:17.830646+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:17.830646+0000","last_undegraded":"2024-09-17T05:25:17.830646+0000","last_fullsized":"2024-09-17T05:25:17.830646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":215170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":805526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":552960,"data_stored":216840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":216748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":215226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":805574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":215808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":208361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:22.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 -- 172.21.15.43:0/921667154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 msgr2=0x7fb0200d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 --2- 172.21.15.43:0/921667154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb0200d0390 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb020073870 tx=0x7fb01001c7a0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 -- 172.21.15.43:0/921667154 shutdown_connections 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 --2- 172.21.15.43:0/921667154 >> v2:172.21.15.62:3300/0 conn(0x7fb02006fce0 0x7fb0200700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 --2- 172.21.15.43:0/921667154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb0200d0390 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 -- 172.21.15.43:0/921667154 >> 172.21.15.43:0/921667154 conn(0x7fb0200b2810 msgr2=0x7fb0200b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 -- 172.21.15.43:0/921667154 shutdown_connections 2024-09-17T05:25:22.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.268+0000 7fb024a00640 1 -- 172.21.15.43:0/921667154 wait complete. 2024-09-17T05:25:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 Processor -- start 2024-09-17T05:25:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 -- start start 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02006fce0 0x7fb02015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0200b5610 0x7fb020159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb020169500 con 0x7fb02015f470 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb020169380 con 0x7fb02006fce0 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb024a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb020169680 con 0x7fb0200b5610 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51302/0 (socket says 172.21.15.43:51302) 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 -- 172.21.15.43:0/4276445278 learned_addr learned my addr 172.21.15.43:0/4276445278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01de00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02006fce0 0x7fb02015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb017e00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0200b5610 0x7fb020159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 -- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0200b5610 msgr2=0x7fb020159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0200b5610 0x7fb020159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 -- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02006fce0 msgr2=0x7fb02015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02006fce0 0x7fb02015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 -- 172.21.15.43:0/4276445278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02014e770 con 0x7fb02015f470 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.272+0000 7fb01e800640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e9a0 tx=0x7fb01800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004d30 con 0x7fb02015f470 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02015ed30 con 0x7fb02015f470 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb01800cb20 con 0x7fb02015f470 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018005020 con 0x7fb02015f470 2024-09-17T05:25:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02015a070 con 0x7fb02015f470 2024-09-17T05:25:22.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02015a2d0 con 0x7fb02015f470 2024-09-17T05:25:22.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0180051c0 con 0x7fb02015f470 2024-09-17T05:25:22.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 0x7faffc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fb018094b00 con 0x7fb02015f470 2024-09-17T05:25:22.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.276+0000 7fb01de00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 0x7faffc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.280+0000 7fb01de00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 0x7faffc075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0100af000 tx=0x7fb01001afe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.280+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb018016030 con 0x7fb02015f470 2024-09-17T05:25:22.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.452+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0200700f0 con 0x7faffc073200 2024-09-17T05:25:22.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.456+0000 7fb016000640 1 -- 172.21.15.43:0/4276445278 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251531 (secure 0 0 0) 0x7fb0200700f0 con 0x7faffc073200 2024-09-17T05:25:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:22.467 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 msgr2=0x7faffc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 0x7faffc075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0100af000 tx=0x7fb01001afe0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 msgr2=0x7fb02014ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e9a0 tx=0x7fb01800ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 shutdown_connections 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faffc073200 0x7faffc075710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb02015f470 0x7fb02014ef30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0200b5610 0x7fb020159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 --2- 172.21.15.43:0/4276445278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02006fce0 0x7fb02015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 >> 172.21.15.43:0/4276445278 conn(0x7fb0200b2810 msgr2=0x7fb0200cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 shutdown_connections 2024-09-17T05:25:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.460+0000 7fb024a00640 1 -- 172.21.15.43:0/4276445278 wait complete. 2024-09-17T05:25:22.487 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T05:25:21.868498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242428,"kb_used_data":5364,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732292,"statfs":{"total":773094113280,"available":772845867008,"internally_reserved":0,"allocated":5492736,"data_stored":2899253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836136+0000","last_change":"2024-09-17T05:25:17.830654+0000","last_active":"2024-09-17T05:25:17.836136+0000","last_peered":"2024-09-17T05:25:17.836136+0000","last_clean":"2024-09-17T05:25:17.836136+0000","last_became_active":"2024-09-17T05:25:17.830321+0000","last_became_peered":"2024-09-17T05:25:17.830321+0000","last_unstale":"2024-09-17T05:25:17.836136+0000","last_undegraded":"2024-09-17T05:25:17.836136+0000","last_fullsized":"2024-09-17T05:25:17.836136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:42.098022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848789+0000","last_change":"2024-09-17T05:25:17.844139+0000","last_active":"2024-09-17T05:25:17.848789+0000","last_peered":"2024-09-17T05:25:17.848789+0000","last_clean":"2024-09-17T05:25:17.848789+0000","last_became_active":"2024-09-17T05:25:17.843821+0000","last_became_peered":"2024-09-17T05:25:17.843821+0000","last_unstale":"2024-09-17T05:25:17.848789+0000","last_undegraded":"2024-09-17T05:25:17.848789+0000","last_fullsized":"2024-09-17T05:25:17.848789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:38.456234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850784+0000","last_change":"2024-09-17T05:25:17.847168+0000","last_active":"2024-09-17T05:25:17.850784+0000","last_peered":"2024-09-17T05:25:17.850784+0000","last_clean":"2024-09-17T05:25:17.850784+0000","last_became_active":"2024-09-17T05:25:17.846513+0000","last_became_peered":"2024-09-17T05:25:17.846513+0000","last_unstale":"2024-09-17T05:25:17.850784+0000","last_undegraded":"2024-09-17T05:25:17.850784+0000","last_fullsized":"2024-09-17T05:25:17.850784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:06.087190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859247+0000","last_change":"2024-09-17T05:25:17.845500+0000","last_active":"2024-09-17T05:25:17.859247+0000","last_peered":"2024-09-17T05:25:17.859247+0000","last_clean":"2024-09-17T05:25:17.859247+0000","last_became_active":"2024-09-17T05:25:17.845221+0000","last_became_peered":"2024-09-17T05:25:17.845221+0000","last_unstale":"2024-09-17T05:25:17.859247+0000","last_undegraded":"2024-09-17T05:25:17.859247+0000","last_fullsized":"2024-09-17T05:25:17.859247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:09.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859215+0000","last_change":"2024-09-17T05:25:17.858007+0000","last_active":"2024-09-17T05:25:17.859215+0000","last_peered":"2024-09-17T05:25:17.859215+0000","last_clean":"2024-09-17T05:25:17.859215+0000","last_became_active":"2024-09-17T05:25:17.855860+0000","last_became_peered":"2024-09-17T05:25:17.855860+0000","last_unstale":"2024-09-17T05:25:17.859215+0000","last_undegraded":"2024-09-17T05:25:17.859215+0000","last_fullsized":"2024-09-17T05:25:17.859215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817734+0000","last_change":"2024-09-17T05:25:16.817734+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817734+0000","last_undegraded":"2024-09-17T05:25:16.817734+0000","last_fullsized":"2024-09-17T05:25:16.817734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864857+0000","last_change":"2024-09-17T05:25:17.863014+0000","last_active":"2024-09-17T05:25:17.864857+0000","last_peered":"2024-09-17T05:25:17.864857+0000","last_clean":"2024-09-17T05:25:17.864857+0000","last_became_active":"2024-09-17T05:25:17.862710+0000","last_became_peered":"2024-09-17T05:25:17.862710+0000","last_unstale":"2024-09-17T05:25:17.864857+0000","last_undegraded":"2024-09-17T05:25:17.864857+0000","last_fullsized":"2024-09-17T05:25:17.864857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:54.789315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852055+0000","last_change":"2024-09-17T05:25:17.843269+0000","last_active":"2024-09-17T05:25:17.852055+0000","last_peered":"2024-09-17T05:25:17.852055+0000","last_clean":"2024-09-17T05:25:17.852055+0000","last_became_active":"2024-09-17T05:25:17.842790+0000","last_became_peered":"2024-09-17T05:25:17.842790+0000","last_unstale":"2024-09-17T05:25:17.852055+0000","last_undegraded":"2024-09-17T05:25:17.852055+0000","last_fullsized":"2024-09-17T05:25:17.852055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:09.373391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.818454+0000","last_change":"2024-09-17T05:25:16.818454+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.818454+0000","last_undegraded":"2024-09-17T05:25:16.818454+0000","last_fullsized":"2024-09-17T05:25:16.818454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843015+0000","last_change":"2024-09-17T05:25:17.834657+0000","last_active":"2024-09-17T05:25:17.843015+0000","last_peered":"2024-09-17T05:25:17.843015+0000","last_clean":"2024-09-17T05:25:17.843015+0000","last_became_active":"2024-09-17T05:25:17.834339+0000","last_became_peered":"2024-09-17T05:25:17.834339+0000","last_unstale":"2024-09-17T05:25:17.843015+0000","last_undegraded":"2024-09-17T05:25:17.843015+0000","last_fullsized":"2024-09-17T05:25:17.843015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862582+0000","last_change":"2024-09-17T05:25:17.857677+0000","last_active":"2024-09-17T05:25:17.862582+0000","last_peered":"2024-09-17T05:25:17.862582+0000","last_clean":"2024-09-17T05:25:17.862582+0000","last_became_active":"2024-09-17T05:25:17.857316+0000","last_became_peered":"2024-09-17T05:25:17.857316+0000","last_unstale":"2024-09-17T05:25:17.862582+0000","last_undegraded":"2024-09-17T05:25:17.862582+0000","last_fullsized":"2024-09-17T05:25:17.862582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.835972+0000","last_change":"2024-09-17T05:25:17.828984+0000","last_active":"2024-09-17T05:25:17.835972+0000","last_peered":"2024-09-17T05:25:17.835972+0000","last_clean":"2024-09-17T05:25:17.835972+0000","last_became_active":"2024-09-17T05:25:17.828631+0000","last_became_peered":"2024-09-17T05:25:17.828631+0000","last_unstale":"2024-09-17T05:25:17.835972+0000","last_undegraded":"2024-09-17T05:25:17.835972+0000","last_fullsized":"2024-09-17T05:25:17.835972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:25.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.819073+0000","last_change":"2024-09-17T05:25:16.819073+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.819073+0000","last_undegraded":"2024-09-17T05:25:16.819073+0000","last_fullsized":"2024-09-17T05:25:16.819073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859255+0000","last_change":"2024-09-17T05:25:17.847391+0000","last_active":"2024-09-17T05:25:17.859255+0000","last_peered":"2024-09-17T05:25:17.859255+0000","last_clean":"2024-09-17T05:25:17.859255+0000","last_became_active":"2024-09-17T05:25:17.847099+0000","last_became_peered":"2024-09-17T05:25:17.847099+0000","last_unstale":"2024-09-17T05:25:17.859255+0000","last_undegraded":"2024-09-17T05:25:17.859255+0000","last_fullsized":"2024-09-17T05:25:17.859255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:18.658668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862139+0000","last_change":"2024-09-17T05:25:17.852627+0000","last_active":"2024-09-17T05:25:17.862139+0000","last_peered":"2024-09-17T05:25:17.862139+0000","last_clean":"2024-09-17T05:25:17.862139+0000","last_became_active":"2024-09-17T05:25:17.849911+0000","last_became_peered":"2024-09-17T05:25:17.849911+0000","last_unstale":"2024-09-17T05:25:17.862139+0000","last_undegraded":"2024-09-17T05:25:17.862139+0000","last_fullsized":"2024-09-17T05:25:17.862139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:20.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852603+0000","last_change":"2024-09-17T05:25:17.849702+0000","last_active":"2024-09-17T05:25:17.852603+0000","last_peered":"2024-09-17T05:25:17.852603+0000","last_clean":"2024-09-17T05:25:17.852603+0000","last_became_active":"2024-09-17T05:25:17.849304+0000","last_became_peered":"2024-09-17T05:25:17.849304+0000","last_unstale":"2024-09-17T05:25:17.852603+0000","last_undegraded":"2024-09-17T05:25:17.852603+0000","last_fullsized":"2024-09-17T05:25:17.852603+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:31.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.844026+0000","last_change":"2024-09-17T05:25:17.838654+0000","last_active":"2024-09-17T05:25:17.844026+0000","last_peered":"2024-09-17T05:25:17.844026+0000","last_clean":"2024-09-17T05:25:17.844026+0000","last_became_active":"2024-09-17T05:25:17.838259+0000","last_became_peered":"2024-09-17T05:25:17.838259+0000","last_unstale":"2024-09-17T05:25:17.844026+0000","last_undegraded":"2024-09-17T05:25:17.844026+0000","last_fullsized":"2024-09-17T05:25:17.844026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:29.702298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863471+0000","last_change":"2024-09-17T05:25:17.862226+0000","last_active":"2024-09-17T05:25:17.863471+0000","last_peered":"2024-09-17T05:25:17.863471+0000","last_clean":"2024-09-17T05:25:17.863471+0000","last_became_active":"2024-09-17T05:25:17.860613+0000","last_became_peered":"2024-09-17T05:25:17.860613+0000","last_unstale":"2024-09-17T05:25:17.863471+0000","last_undegraded":"2024-09-17T05:25:17.863471+0000","last_fullsized":"2024-09-17T05:25:17.863471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:41.914430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859049+0000","last_change":"2024-09-17T05:25:17.855133+0000","last_active":"2024-09-17T05:25:17.859049+0000","last_peered":"2024-09-17T05:25:17.859049+0000","last_clean":"2024-09-17T05:25:17.859049+0000","last_became_active":"2024-09-17T05:25:17.854806+0000","last_became_peered":"2024-09-17T05:25:17.854806+0000","last_unstale":"2024-09-17T05:25:17.859049+0000","last_undegraded":"2024-09-17T05:25:17.859049+0000","last_fullsized":"2024-09-17T05:25:17.859049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:13.815396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.851242+0000","last_change":"2024-09-17T05:25:17.840899+0000","last_active":"2024-09-17T05:25:17.851242+0000","last_peered":"2024-09-17T05:25:17.851242+0000","last_clean":"2024-09-17T05:25:17.851242+0000","last_became_active":"2024-09-17T05:25:17.832241+0000","last_became_peered":"2024-09-17T05:25:17.832241+0000","last_unstale":"2024-09-17T05:25:17.851242+0000","last_undegraded":"2024-09-17T05:25:17.851242+0000","last_fullsized":"2024-09-17T05:25:17.851242+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:07.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859491+0000","last_change":"2024-09-17T05:25:17.846375+0000","last_active":"2024-09-17T05:25:17.859491+0000","last_peered":"2024-09-17T05:25:17.859491+0000","last_clean":"2024-09-17T05:25:17.859491+0000","last_became_active":"2024-09-17T05:25:17.846001+0000","last_became_peered":"2024-09-17T05:25:17.846001+0000","last_unstale":"2024-09-17T05:25:17.859491+0000","last_undegraded":"2024-09-17T05:25:17.859491+0000","last_fullsized":"2024-09-17T05:25:17.859491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:34.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855800+0000","last_change":"2024-09-17T05:25:17.849145+0000","last_active":"2024-09-17T05:25:17.855800+0000","last_peered":"2024-09-17T05:25:17.855800+0000","last_clean":"2024-09-17T05:25:17.855800+0000","last_became_active":"2024-09-17T05:25:17.848763+0000","last_became_peered":"2024-09-17T05:25:17.848763+0000","last_unstale":"2024-09-17T05:25:17.855800+0000","last_undegraded":"2024-09-17T05:25:17.855800+0000","last_fullsized":"2024-09-17T05:25:17.855800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:53.559077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.846762+0000","last_change":"2024-09-17T05:25:17.839774+0000","last_active":"2024-09-17T05:25:17.846762+0000","last_peered":"2024-09-17T05:25:17.846762+0000","last_clean":"2024-09-17T05:25:17.846762+0000","last_became_active":"2024-09-17T05:25:17.839418+0000","last_became_peered":"2024-09-17T05:25:17.839418+0000","last_unstale":"2024-09-17T05:25:17.846762+0000","last_undegraded":"2024-09-17T05:25:17.846762+0000","last_fullsized":"2024-09-17T05:25:17.846762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:59.137139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864301+0000","last_change":"2024-09-17T05:25:17.860865+0000","last_active":"2024-09-17T05:25:17.864301+0000","last_peered":"2024-09-17T05:25:17.864301+0000","last_clean":"2024-09-17T05:25:17.864301+0000","last_became_active":"2024-09-17T05:25:17.857489+0000","last_became_peered":"2024-09-17T05:25:17.857489+0000","last_unstale":"2024-09-17T05:25:17.864301+0000","last_undegraded":"2024-09-17T05:25:17.864301+0000","last_fullsized":"2024-09-17T05:25:17.864301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830904+0000","last_change":"2024-09-17T05:25:17.827002+0000","last_active":"2024-09-17T05:25:17.830904+0000","last_peered":"2024-09-17T05:25:17.830904+0000","last_clean":"2024-09-17T05:25:17.830904+0000","last_became_active":"2024-09-17T05:25:17.826648+0000","last_became_peered":"2024-09-17T05:25:17.826648+0000","last_unstale":"2024-09-17T05:25:17.830904+0000","last_undegraded":"2024-09-17T05:25:17.830904+0000","last_fullsized":"2024-09-17T05:25:17.830904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862280+0000","last_change":"2024-09-17T05:25:17.855977+0000","last_active":"2024-09-17T05:25:17.862280+0000","last_peered":"2024-09-17T05:25:17.862280+0000","last_clean":"2024-09-17T05:25:17.862280+0000","last_became_active":"2024-09-17T05:25:17.855650+0000","last_became_peered":"2024-09-17T05:25:17.855650+0000","last_unstale":"2024-09-17T05:25:17.862280+0000","last_undegraded":"2024-09-17T05:25:17.862280+0000","last_fullsized":"2024-09-17T05:25:17.862280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864199+0000","last_change":"2024-09-17T05:25:17.860605+0000","last_active":"2024-09-17T05:25:17.864199+0000","last_peered":"2024-09-17T05:25:17.864199+0000","last_clean":"2024-09-17T05:25:17.864199+0000","last_became_active":"2024-09-17T05:25:17.859963+0000","last_became_peered":"2024-09-17T05:25:17.859963+0000","last_unstale":"2024-09-17T05:25:17.864199+0000","last_undegraded":"2024-09-17T05:25:17.864199+0000","last_fullsized":"2024-09-17T05:25:17.864199+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:55.637768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852200+0000","last_change":"2024-09-17T05:25:17.841362+0000","last_active":"2024-09-17T05:25:17.852200+0000","last_peered":"2024-09-17T05:25:17.852200+0000","last_clean":"2024-09-17T05:25:17.852200+0000","last_became_active":"2024-09-17T05:25:17.840961+0000","last_became_peered":"2024-09-17T05:25:17.840961+0000","last_unstale":"2024-09-17T05:25:17.852200+0000","last_undegraded":"2024-09-17T05:25:17.852200+0000","last_fullsized":"2024-09-17T05:25:17.852200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.926288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837038+0000","last_change":"2024-09-17T05:25:17.829627+0000","last_active":"2024-09-17T05:25:17.837038+0000","last_peered":"2024-09-17T05:25:17.837038+0000","last_clean":"2024-09-17T05:25:17.837038+0000","last_became_active":"2024-09-17T05:25:17.829328+0000","last_became_peered":"2024-09-17T05:25:17.829328+0000","last_unstale":"2024-09-17T05:25:17.837038+0000","last_undegraded":"2024-09-17T05:25:17.837038+0000","last_fullsized":"2024-09-17T05:25:17.837038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:27.488602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.861137+0000","last_change":"2024-09-17T05:25:17.855557+0000","last_active":"2024-09-17T05:25:17.861137+0000","last_peered":"2024-09-17T05:25:17.861137+0000","last_clean":"2024-09-17T05:25:17.861137+0000","last_became_active":"2024-09-17T05:25:17.854842+0000","last_became_peered":"2024-09-17T05:25:17.854842+0000","last_unstale":"2024-09-17T05:25:17.861137+0000","last_undegraded":"2024-09-17T05:25:17.861137+0000","last_fullsized":"2024-09-17T05:25:17.861137+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:43.613646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817820+0000","last_change":"2024-09-17T05:25:16.817820+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817820+0000","last_undegraded":"2024-09-17T05:25:16.817820+0000","last_fullsized":"2024-09-17T05:25:16.817820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862079+0000","last_change":"2024-09-17T05:25:17.851832+0000","last_active":"2024-09-17T05:25:17.862079+0000","last_peered":"2024-09-17T05:25:17.862079+0000","last_clean":"2024-09-17T05:25:17.862079+0000","last_became_active":"2024-09-17T05:25:17.851503+0000","last_became_peered":"2024-09-17T05:25:17.851503+0000","last_unstale":"2024-09-17T05:25:17.862079+0000","last_undegraded":"2024-09-17T05:25:17.862079+0000","last_fullsized":"2024-09-17T05:25:17.862079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.055649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850746+0000","last_change":"2024-09-17T05:25:17.845069+0000","last_active":"2024-09-17T05:25:17.850746+0000","last_peered":"2024-09-17T05:25:17.850746+0000","last_clean":"2024-09-17T05:25:17.850746+0000","last_became_active":"2024-09-17T05:25:17.844766+0000","last_became_peered":"2024-09-17T05:25:17.844766+0000","last_unstale":"2024-09-17T05:25:17.850746+0000","last_undegraded":"2024-09-17T05:25:17.850746+0000","last_fullsized":"2024-09-17T05:25:17.850746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:18.184047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.838673+0000","last_change":"2024-09-17T05:25:16.830007+0000","last_active":"2024-09-17T05:25:16.838673+0000","last_peered":"2024-09-17T05:25:16.838673+0000","last_clean":"2024-09-17T05:25:16.838673+0000","last_became_active":"2024-09-17T05:25:16.828838+0000","last_became_peered":"2024-09-17T05:25:16.828838+0000","last_unstale":"2024-09-17T05:25:16.838673+0000","last_undegraded":"2024-09-17T05:25:16.838673+0000","last_fullsized":"2024-09-17T05:25:16.838673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:34.533131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858878+0000","last_change":"2024-09-17T05:25:17.855463+0000","last_active":"2024-09-17T05:25:17.858878+0000","last_peered":"2024-09-17T05:25:17.858878+0000","last_clean":"2024-09-17T05:25:17.858878+0000","last_became_active":"2024-09-17T05:25:17.855089+0000","last_became_peered":"2024-09-17T05:25:17.855089+0000","last_unstale":"2024-09-17T05:25:17.858878+0000","last_undegraded":"2024-09-17T05:25:17.858878+0000","last_fullsized":"2024-09-17T05:25:17.858878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.860692+0000","last_change":"2024-09-17T05:25:16.841386+0000","last_active":"2024-09-17T05:25:17.860692+0000","last_peered":"2024-09-17T05:25:17.860692+0000","last_clean":"2024-09-17T05:25:17.860692+0000","last_became_active":"2024-09-17T05:25:16.840942+0000","last_became_peered":"2024-09-17T05:25:16.840942+0000","last_unstale":"2024-09-17T05:25:17.860692+0000","last_undegraded":"2024-09-17T05:25:17.860692+0000","last_fullsized":"2024-09-17T05:25:17.860692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:09.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848773+0000","last_change":"2024-09-17T05:25:17.844192+0000","last_active":"2024-09-17T05:25:17.848773+0000","last_peered":"2024-09-17T05:25:17.848773+0000","last_clean":"2024-09-17T05:25:17.848773+0000","last_became_active":"2024-09-17T05:25:17.843753+0000","last_became_peered":"2024-09-17T05:25:17.843753+0000","last_unstale":"2024-09-17T05:25:17.848773+0000","last_undegraded":"2024-09-17T05:25:17.848773+0000","last_fullsized":"2024-09-17T05:25:17.848773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:45.282549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839405+0000","last_change":"2024-09-17T05:25:16.835803+0000","last_active":"2024-09-17T05:25:16.839405+0000","last_peered":"2024-09-17T05:25:16.839405+0000","last_clean":"2024-09-17T05:25:16.839405+0000","last_became_active":"2024-09-17T05:25:16.835332+0000","last_became_peered":"2024-09-17T05:25:16.835332+0000","last_unstale":"2024-09-17T05:25:16.839405+0000","last_undegraded":"2024-09-17T05:25:16.839405+0000","last_fullsized":"2024-09-17T05:25:16.839405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:57.056082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862435+0000","last_change":"2024-09-17T05:25:17.853747+0000","last_active":"2024-09-17T05:25:17.862435+0000","last_peered":"2024-09-17T05:25:17.862435+0000","last_clean":"2024-09-17T05:25:17.862435+0000","last_became_active":"2024-09-17T05:25:17.853420+0000","last_became_peered":"2024-09-17T05:25:17.853420+0000","last_unstale":"2024-09-17T05:25:17.862435+0000","last_undegraded":"2024-09-17T05:25:17.862435+0000","last_fullsized":"2024-09-17T05:25:17.862435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862840+0000","last_change":"2024-09-17T05:25:16.839690+0000","last_active":"2024-09-17T05:25:17.862840+0000","last_peered":"2024-09-17T05:25:17.862840+0000","last_clean":"2024-09-17T05:25:17.862840+0000","last_became_active":"2024-09-17T05:25:16.839208+0000","last_became_peered":"2024-09-17T05:25:16.839208+0000","last_unstale":"2024-09-17T05:25:17.862840+0000","last_undegraded":"2024-09-17T05:25:17.862840+0000","last_fullsized":"2024-09-17T05:25:17.862840+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:20.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864027+0000","last_change":"2024-09-17T05:25:17.860246+0000","last_active":"2024-09-17T05:25:17.864027+0000","last_peered":"2024-09-17T05:25:17.864027+0000","last_clean":"2024-09-17T05:25:17.864027+0000","last_became_active":"2024-09-17T05:25:17.859898+0000","last_became_peered":"2024-09-17T05:25:17.859898+0000","last_unstale":"2024-09-17T05:25:17.864027+0000","last_undegraded":"2024-09-17T05:25:17.864027+0000","last_fullsized":"2024-09-17T05:25:17.864027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:40.128894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.832185+0000","last_change":"2024-09-17T05:25:16.825193+0000","last_active":"2024-09-17T05:25:16.832185+0000","last_peered":"2024-09-17T05:25:16.832185+0000","last_clean":"2024-09-17T05:25:16.832185+0000","last_became_active":"2024-09-17T05:25:16.824705+0000","last_became_peered":"2024-09-17T05:25:16.824705+0000","last_unstale":"2024-09-17T05:25:16.832185+0000","last_undegraded":"2024-09-17T05:25:16.832185+0000","last_fullsized":"2024-09-17T05:25:16.832185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:23.844925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852903+0000","last_change":"2024-09-17T05:25:17.844440+0000","last_active":"2024-09-17T05:25:17.852903+0000","last_peered":"2024-09-17T05:25:17.852903+0000","last_clean":"2024-09-17T05:25:17.852903+0000","last_became_active":"2024-09-17T05:25:17.843574+0000","last_became_peered":"2024-09-17T05:25:17.843574+0000","last_unstale":"2024-09-17T05:25:17.852903+0000","last_undegraded":"2024-09-17T05:25:17.852903+0000","last_fullsized":"2024-09-17T05:25:17.852903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:27.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859395+0000","last_change":"2024-09-17T05:25:16.839269+0000","last_active":"2024-09-17T05:25:17.859395+0000","last_peered":"2024-09-17T05:25:17.859395+0000","last_clean":"2024-09-17T05:25:17.859395+0000","last_became_active":"2024-09-17T05:25:16.838891+0000","last_became_peered":"2024-09-17T05:25:16.838891+0000","last_unstale":"2024-09-17T05:25:17.859395+0000","last_undegraded":"2024-09-17T05:25:17.859395+0000","last_fullsized":"2024-09-17T05:25:17.859395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.956617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850851+0000","last_change":"2024-09-17T05:25:17.840191+0000","last_active":"2024-09-17T05:25:17.850851+0000","last_peered":"2024-09-17T05:25:17.850851+0000","last_clean":"2024-09-17T05:25:17.850851+0000","last_became_active":"2024-09-17T05:25:17.839875+0000","last_became_peered":"2024-09-17T05:25:17.839875+0000","last_unstale":"2024-09-17T05:25:17.850851+0000","last_undegraded":"2024-09-17T05:25:17.850851+0000","last_fullsized":"2024-09-17T05:25:17.850851+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818683+0000","last_change":"2024-09-17T05:25:16.834402+0000","last_active":"2024-09-17T05:25:17.818683+0000","last_peered":"2024-09-17T05:25:17.818683+0000","last_clean":"2024-09-17T05:25:17.818683+0000","last_became_active":"2024-09-17T05:25:16.834036+0000","last_became_peered":"2024-09-17T05:25:16.834036+0000","last_unstale":"2024-09-17T05:25:17.818683+0000","last_undegraded":"2024-09-17T05:25:17.818683+0000","last_fullsized":"2024-09-17T05:25:17.818683+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:37.453048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863162+0000","last_change":"2024-09-17T05:25:17.855835+0000","last_active":"2024-09-17T05:25:17.863162+0000","last_peered":"2024-09-17T05:25:17.863162+0000","last_clean":"2024-09-17T05:25:17.863162+0000","last_became_active":"2024-09-17T05:25:17.853327+0000","last_became_peered":"2024-09-17T05:25:17.853327+0000","last_unstale":"2024-09-17T05:25:17.863162+0000","last_undegraded":"2024-09-17T05:25:17.863162+0000","last_fullsized":"2024-09-17T05:25:17.863162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:41.256886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.823113+0000","last_change":"2024-09-17T05:25:16.839214+0000","last_active":"2024-09-17T05:25:17.823113+0000","last_peered":"2024-09-17T05:25:17.823113+0000","last_clean":"2024-09-17T05:25:17.823113+0000","last_became_active":"2024-09-17T05:25:16.838825+0000","last_became_peered":"2024-09-17T05:25:16.838825+0000","last_unstale":"2024-09-17T05:25:17.823113+0000","last_undegraded":"2024-09-17T05:25:17.823113+0000","last_fullsized":"2024-09-17T05:25:17.823113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:30.086751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852021+0000","last_change":"2024-09-17T05:25:17.850811+0000","last_active":"2024-09-17T05:25:17.852021+0000","last_peered":"2024-09-17T05:25:17.852021+0000","last_clean":"2024-09-17T05:25:17.852021+0000","last_became_active":"2024-09-17T05:25:17.850480+0000","last_became_peered":"2024-09-17T05:25:17.850480+0000","last_unstale":"2024-09-17T05:25:17.852021+0000","last_undegraded":"2024-09-17T05:25:17.852021+0000","last_fullsized":"2024-09-17T05:25:17.852021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:48.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836130+0000","last_change":"2024-09-17T05:25:16.841708+0000","last_active":"2024-09-17T05:25:17.836130+0000","last_peered":"2024-09-17T05:25:17.836130+0000","last_clean":"2024-09-17T05:25:17.836130+0000","last_became_active":"2024-09-17T05:25:16.841263+0000","last_became_peered":"2024-09-17T05:25:16.841263+0000","last_unstale":"2024-09-17T05:25:17.836130+0000","last_undegraded":"2024-09-17T05:25:17.836130+0000","last_fullsized":"2024-09-17T05:25:17.836130+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:21.553815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852485+0000","last_change":"2024-09-17T05:25:17.851388+0000","last_active":"2024-09-17T05:25:17.852485+0000","last_peered":"2024-09-17T05:25:17.852485+0000","last_clean":"2024-09-17T05:25:17.852485+0000","last_became_active":"2024-09-17T05:25:17.851036+0000","last_became_peered":"2024-09-17T05:25:17.851036+0000","last_unstale":"2024-09-17T05:25:17.852485+0000","last_undegraded":"2024-09-17T05:25:17.852485+0000","last_fullsized":"2024-09-17T05:25:17.852485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:14.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821624+0000","last_change":"2024-09-17T05:25:16.843023+0000","last_active":"2024-09-17T05:25:17.821624+0000","last_peered":"2024-09-17T05:25:17.821624+0000","last_clean":"2024-09-17T05:25:17.821624+0000","last_became_active":"2024-09-17T05:25:16.842671+0000","last_became_peered":"2024-09-17T05:25:16.842671+0000","last_unstale":"2024-09-17T05:25:17.821624+0000","last_undegraded":"2024-09-17T05:25:17.821624+0000","last_fullsized":"2024-09-17T05:25:17.821624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:40.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859214+0000","last_change":"2024-09-17T05:25:17.844156+0000","last_active":"2024-09-17T05:25:17.859214+0000","last_peered":"2024-09-17T05:25:17.859214+0000","last_clean":"2024-09-17T05:25:17.859214+0000","last_became_active":"2024-09-17T05:25:17.843870+0000","last_became_peered":"2024-09-17T05:25:17.843870+0000","last_unstale":"2024-09-17T05:25:17.859214+0000","last_undegraded":"2024-09-17T05:25:17.859214+0000","last_fullsized":"2024-09-17T05:25:17.859214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:02.380001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864696+0000","last_change":"2024-09-17T05:25:16.825444+0000","last_active":"2024-09-17T05:25:17.864696+0000","last_peered":"2024-09-17T05:25:17.864696+0000","last_clean":"2024-09-17T05:25:17.864696+0000","last_became_active":"2024-09-17T05:25:16.825122+0000","last_became_peered":"2024-09-17T05:25:16.825122+0000","last_unstale":"2024-09-17T05:25:17.864696+0000","last_undegraded":"2024-09-17T05:25:17.864696+0000","last_fullsized":"2024-09-17T05:25:17.864696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:38.050632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864876+0000","last_change":"2024-09-17T05:25:17.860864+0000","last_active":"2024-09-17T05:25:17.864876+0000","last_peered":"2024-09-17T05:25:17.864876+0000","last_clean":"2024-09-17T05:25:17.864876+0000","last_became_active":"2024-09-17T05:25:17.860380+0000","last_became_peered":"2024-09-17T05:25:17.860380+0000","last_unstale":"2024-09-17T05:25:17.864876+0000","last_undegraded":"2024-09-17T05:25:17.864876+0000","last_fullsized":"2024-09-17T05:25:17.864876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:41.208537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.866116+0000","last_change":"2024-09-17T05:25:16.845121+0000","last_active":"2024-09-17T05:25:17.866116+0000","last_peered":"2024-09-17T05:25:17.866116+0000","last_clean":"2024-09-17T05:25:17.866116+0000","last_became_active":"2024-09-17T05:25:16.844739+0000","last_became_peered":"2024-09-17T05:25:16.844739+0000","last_unstale":"2024-09-17T05:25:17.866116+0000","last_undegraded":"2024-09-17T05:25:17.866116+0000","last_fullsized":"2024-09-17T05:25:17.866116+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:26.485416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856703+0000","last_change":"2024-09-17T05:25:17.854295+0000","last_active":"2024-09-17T05:25:17.856703+0000","last_peered":"2024-09-17T05:25:17.856703+0000","last_clean":"2024-09-17T05:25:17.856703+0000","last_became_active":"2024-09-17T05:25:17.853814+0000","last_became_peered":"2024-09-17T05:25:17.853814+0000","last_unstale":"2024-09-17T05:25:17.856703+0000","last_undegraded":"2024-09-17T05:25:17.856703+0000","last_fullsized":"2024-09-17T05:25:17.856703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:38.195708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856501+0000","last_change":"2024-09-17T05:25:16.839573+0000","last_active":"2024-09-17T05:25:17.856501+0000","last_peered":"2024-09-17T05:25:17.856501+0000","last_clean":"2024-09-17T05:25:17.856501+0000","last_became_active":"2024-09-17T05:25:16.838379+0000","last_became_peered":"2024-09-17T05:25:16.838379+0000","last_unstale":"2024-09-17T05:25:17.856501+0000","last_undegraded":"2024-09-17T05:25:17.856501+0000","last_fullsized":"2024-09-17T05:25:17.856501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:00.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843182+0000","last_change":"2024-09-17T05:25:17.836620+0000","last_active":"2024-09-17T05:25:17.843182+0000","last_peered":"2024-09-17T05:25:17.843182+0000","last_clean":"2024-09-17T05:25:17.843182+0000","last_became_active":"2024-09-17T05:25:17.836304+0000","last_became_peered":"2024-09-17T05:25:17.836304+0000","last_unstale":"2024-09-17T05:25:17.843182+0000","last_undegraded":"2024-09-17T05:25:17.843182+0000","last_fullsized":"2024-09-17T05:25:17.843182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:46.933484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818843+0000","last_change":"2024-09-17T05:25:16.837793+0000","last_active":"2024-09-17T05:25:17.818843+0000","last_peered":"2024-09-17T05:25:17.818843+0000","last_clean":"2024-09-17T05:25:17.818843+0000","last_became_active":"2024-09-17T05:25:16.837076+0000","last_became_peered":"2024-09-17T05:25:16.837076+0000","last_unstale":"2024-09-17T05:25:17.818843+0000","last_undegraded":"2024-09-17T05:25:17.818843+0000","last_fullsized":"2024-09-17T05:25:17.818843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:44.279363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850823+0000","last_change":"2024-09-17T05:25:17.846642+0000","last_active":"2024-09-17T05:25:17.850823+0000","last_peered":"2024-09-17T05:25:17.850823+0000","last_clean":"2024-09-17T05:25:17.850823+0000","last_became_active":"2024-09-17T05:25:17.844111+0000","last_became_peered":"2024-09-17T05:25:17.844111+0000","last_unstale":"2024-09-17T05:25:17.850823+0000","last_undegraded":"2024-09-17T05:25:17.850823+0000","last_fullsized":"2024-09-17T05:25:17.850823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:40.896672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.844453+0000","last_change":"2024-09-17T05:25:16.842966+0000","last_active":"2024-09-17T05:25:16.844453+0000","last_peered":"2024-09-17T05:25:16.844453+0000","last_clean":"2024-09-17T05:25:16.844453+0000","last_became_active":"2024-09-17T05:25:16.842650+0000","last_became_peered":"2024-09-17T05:25:16.842650+0000","last_unstale":"2024-09-17T05:25:16.844453+0000","last_undegraded":"2024-09-17T05:25:16.844453+0000","last_fullsized":"2024-09-17T05:25:16.844453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:55:02.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.834870+0000","last_change":"2024-09-17T05:25:17.827842+0000","last_active":"2024-09-17T05:25:17.834870+0000","last_peered":"2024-09-17T05:25:17.834870+0000","last_clean":"2024-09-17T05:25:17.834870+0000","last_became_active":"2024-09-17T05:25:17.827506+0000","last_became_peered":"2024-09-17T05:25:17.827506+0000","last_unstale":"2024-09-17T05:25:17.834870+0000","last_undegraded":"2024-09-17T05:25:17.834870+0000","last_fullsized":"2024-09-17T05:25:17.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:52.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863866+0000","last_change":"2024-09-17T05:25:16.835808+0000","last_active":"2024-09-17T05:25:17.863866+0000","last_peered":"2024-09-17T05:25:17.863866+0000","last_clean":"2024-09-17T05:25:17.863866+0000","last_became_active":"2024-09-17T05:25:16.835452+0000","last_became_peered":"2024-09-17T05:25:16.835452+0000","last_unstale":"2024-09-17T05:25:17.863866+0000","last_undegraded":"2024-09-17T05:25:17.863866+0000","last_fullsized":"2024-09-17T05:25:17.863866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:12.812210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862083+0000","last_change":"2024-09-17T05:25:17.855358+0000","last_active":"2024-09-17T05:25:17.862083+0000","last_peered":"2024-09-17T05:25:17.862083+0000","last_clean":"2024-09-17T05:25:17.862083+0000","last_became_active":"2024-09-17T05:25:17.854829+0000","last_became_peered":"2024-09-17T05:25:17.854829+0000","last_unstale":"2024-09-17T05:25:17.862083+0000","last_undegraded":"2024-09-17T05:25:17.862083+0000","last_fullsized":"2024-09-17T05:25:17.862083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:41.242930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859036+0000","last_change":"2024-09-17T05:25:16.842074+0000","last_active":"2024-09-17T05:25:17.859036+0000","last_peered":"2024-09-17T05:25:17.859036+0000","last_clean":"2024-09-17T05:25:17.859036+0000","last_became_active":"2024-09-17T05:25:16.841700+0000","last_became_peered":"2024-09-17T05:25:16.841700+0000","last_unstale":"2024-09-17T05:25:17.859036+0000","last_undegraded":"2024-09-17T05:25:17.859036+0000","last_fullsized":"2024-09-17T05:25:17.859036+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:38.831286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850551+0000","last_change":"2024-09-17T05:25:17.843043+0000","last_active":"2024-09-17T05:25:17.850551+0000","last_peered":"2024-09-17T05:25:17.850551+0000","last_clean":"2024-09-17T05:25:17.850551+0000","last_became_active":"2024-09-17T05:25:17.842110+0000","last_became_peered":"2024-09-17T05:25:17.842110+0000","last_unstale":"2024-09-17T05:25:17.850551+0000","last_undegraded":"2024-09-17T05:25:17.850551+0000","last_fullsized":"2024-09-17T05:25:17.850551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:58.529182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.854795+0000","last_change":"2024-09-17T05:25:16.843994+0000","last_active":"2024-09-17T05:25:17.854795+0000","last_peered":"2024-09-17T05:25:17.854795+0000","last_clean":"2024-09-17T05:25:17.854795+0000","last_became_active":"2024-09-17T05:25:16.843335+0000","last_became_peered":"2024-09-17T05:25:16.843335+0000","last_unstale":"2024-09-17T05:25:17.854795+0000","last_undegraded":"2024-09-17T05:25:17.854795+0000","last_fullsized":"2024-09-17T05:25:17.854795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:10.828864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863238+0000","last_change":"2024-09-17T05:25:17.858165+0000","last_active":"2024-09-17T05:25:17.863238+0000","last_peered":"2024-09-17T05:25:17.863238+0000","last_clean":"2024-09-17T05:25:17.863238+0000","last_became_active":"2024-09-17T05:25:17.856741+0000","last_became_peered":"2024-09-17T05:25:17.856741+0000","last_unstale":"2024-09-17T05:25:17.863238+0000","last_undegraded":"2024-09-17T05:25:17.863238+0000","last_fullsized":"2024-09-17T05:25:17.863238+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:50.149002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856205+0000","last_change":"2024-09-17T05:25:16.842760+0000","last_active":"2024-09-17T05:25:17.856205+0000","last_peered":"2024-09-17T05:25:17.856205+0000","last_clean":"2024-09-17T05:25:17.856205+0000","last_became_active":"2024-09-17T05:25:16.842399+0000","last_became_peered":"2024-09-17T05:25:16.842399+0000","last_unstale":"2024-09-17T05:25:17.856205+0000","last_undegraded":"2024-09-17T05:25:17.856205+0000","last_fullsized":"2024-09-17T05:25:17.856205+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:04.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850747+0000","last_change":"2024-09-17T05:25:17.846811+0000","last_active":"2024-09-17T05:25:17.850747+0000","last_peered":"2024-09-17T05:25:17.850747+0000","last_clean":"2024-09-17T05:25:17.850747+0000","last_became_active":"2024-09-17T05:25:17.846191+0000","last_became_peered":"2024-09-17T05:25:17.846191+0000","last_unstale":"2024-09-17T05:25:17.850747+0000","last_undegraded":"2024-09-17T05:25:17.850747+0000","last_fullsized":"2024-09-17T05:25:17.850747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:42.562348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839603+0000","last_change":"2024-09-17T05:25:16.836689+0000","last_active":"2024-09-17T05:25:16.839603+0000","last_peered":"2024-09-17T05:25:16.839603+0000","last_clean":"2024-09-17T05:25:16.839603+0000","last_became_active":"2024-09-17T05:25:16.836345+0000","last_became_peered":"2024-09-17T05:25:16.836345+0000","last_unstale":"2024-09-17T05:25:16.839603+0000","last_undegraded":"2024-09-17T05:25:16.839603+0000","last_fullsized":"2024-09-17T05:25:16.839603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:09.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856658+0000","last_change":"2024-09-17T05:25:17.849888+0000","last_active":"2024-09-17T05:25:17.856658+0000","last_peered":"2024-09-17T05:25:17.856658+0000","last_clean":"2024-09-17T05:25:17.856658+0000","last_became_active":"2024-09-17T05:25:17.848605+0000","last_became_peered":"2024-09-17T05:25:17.848605+0000","last_unstale":"2024-09-17T05:25:17.856658+0000","last_undegraded":"2024-09-17T05:25:17.856658+0000","last_fullsized":"2024-09-17T05:25:17.856658+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:35.175603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852768+0000","last_change":"2024-09-17T05:25:16.836994+0000","last_active":"2024-09-17T05:25:17.852768+0000","last_peered":"2024-09-17T05:25:17.852768+0000","last_clean":"2024-09-17T05:25:17.852768+0000","last_became_active":"2024-09-17T05:25:16.836576+0000","last_became_peered":"2024-09-17T05:25:16.836576+0000","last_unstale":"2024-09-17T05:25:17.852768+0000","last_undegraded":"2024-09-17T05:25:17.852768+0000","last_fullsized":"2024-09-17T05:25:17.852768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:41.094836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817331+0000","last_change":"2024-09-17T05:25:16.817331+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817331+0000","last_undegraded":"2024-09-17T05:25:16.817331+0000","last_fullsized":"2024-09-17T05:25:16.817331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819191+0000","last_change":"2024-09-17T05:25:16.837411+0000","last_active":"2024-09-17T05:25:17.819191+0000","last_peered":"2024-09-17T05:25:17.819191+0000","last_clean":"2024-09-17T05:25:17.819191+0000","last_became_active":"2024-09-17T05:25:16.837058+0000","last_became_peered":"2024-09-17T05:25:16.837058+0000","last_unstale":"2024-09-17T05:25:17.819191+0000","last_undegraded":"2024-09-17T05:25:17.819191+0000","last_fullsized":"2024-09-17T05:25:17.819191+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:42.610461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859220+0000","last_change":"2024-09-17T05:25:17.850227+0000","last_active":"2024-09-17T05:25:17.859220+0000","last_peered":"2024-09-17T05:25:17.859220+0000","last_clean":"2024-09-17T05:25:17.859220+0000","last_became_active":"2024-09-17T05:25:17.849926+0000","last_became_peered":"2024-09-17T05:25:17.849926+0000","last_unstale":"2024-09-17T05:25:17.859220+0000","last_undegraded":"2024-09-17T05:25:17.859220+0000","last_fullsized":"2024-09-17T05:25:17.859220+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:35.791860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821447+0000","last_change":"2024-09-17T05:25:16.843105+0000","last_active":"2024-09-17T05:25:17.821447+0000","last_peered":"2024-09-17T05:25:17.821447+0000","last_clean":"2024-09-17T05:25:17.821447+0000","last_became_active":"2024-09-17T05:25:16.842668+0000","last_became_peered":"2024-09-17T05:25:16.842668+0000","last_unstale":"2024-09-17T05:25:17.821447+0000","last_undegraded":"2024-09-17T05:25:17.821447+0000","last_fullsized":"2024-09-17T05:25:17.821447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859135+0000","last_change":"2024-09-17T05:25:17.847878+0000","last_active":"2024-09-17T05:25:17.859135+0000","last_peered":"2024-09-17T05:25:17.859135+0000","last_clean":"2024-09-17T05:25:17.859135+0000","last_became_active":"2024-09-17T05:25:17.844937+0000","last_became_peered":"2024-09-17T05:25:17.844937+0000","last_unstale":"2024-09-17T05:25:17.859135+0000","last_undegraded":"2024-09-17T05:25:17.859135+0000","last_fullsized":"2024-09-17T05:25:17.859135+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:11.788487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837181+0000","last_change":"2024-09-17T05:25:16.839427+0000","last_active":"2024-09-17T05:25:17.837181+0000","last_peered":"2024-09-17T05:25:17.837181+0000","last_clean":"2024-09-17T05:25:17.837181+0000","last_became_active":"2024-09-17T05:25:16.838642+0000","last_became_peered":"2024-09-17T05:25:16.838642+0000","last_unstale":"2024-09-17T05:25:17.837181+0000","last_undegraded":"2024-09-17T05:25:17.837181+0000","last_fullsized":"2024-09-17T05:25:17.837181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.129441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859527+0000","last_change":"2024-09-17T05:25:17.854279+0000","last_active":"2024-09-17T05:25:17.859527+0000","last_peered":"2024-09-17T05:25:17.859527+0000","last_clean":"2024-09-17T05:25:17.859527+0000","last_became_active":"2024-09-17T05:25:17.853347+0000","last_became_peered":"2024-09-17T05:25:17.853347+0000","last_unstale":"2024-09-17T05:25:17.859527+0000","last_undegraded":"2024-09-17T05:25:17.859527+0000","last_fullsized":"2024-09-17T05:25:17.859527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:48.657810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.832834+0000","last_change":"2024-09-17T05:25:16.845000+0000","last_active":"2024-09-17T05:25:17.832834+0000","last_peered":"2024-09-17T05:25:17.832834+0000","last_clean":"2024-09-17T05:25:17.832834+0000","last_became_active":"2024-09-17T05:25:16.844264+0000","last_became_peered":"2024-09-17T05:25:16.844264+0000","last_unstale":"2024-09-17T05:25:17.832834+0000","last_undegraded":"2024-09-17T05:25:17.832834+0000","last_fullsized":"2024-09-17T05:25:17.832834+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:10.422465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852661+0000","last_change":"2024-09-17T05:25:17.848021+0000","last_active":"2024-09-17T05:25:17.852661+0000","last_peered":"2024-09-17T05:25:17.852661+0000","last_clean":"2024-09-17T05:25:17.852661+0000","last_became_active":"2024-09-17T05:25:17.847264+0000","last_became_peered":"2024-09-17T05:25:17.847264+0000","last_unstale":"2024-09-17T05:25:17.852661+0000","last_undegraded":"2024-09-17T05:25:17.852661+0000","last_fullsized":"2024-09-17T05:25:17.852661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:48.742377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.820417+0000","last_change":"2024-09-17T05:25:16.838187+0000","last_active":"2024-09-17T05:25:17.820417+0000","last_peered":"2024-09-17T05:25:17.820417+0000","last_clean":"2024-09-17T05:25:17.820417+0000","last_became_active":"2024-09-17T05:25:16.837770+0000","last_became_peered":"2024-09-17T05:25:16.837770+0000","last_unstale":"2024-09-17T05:25:17.820417+0000","last_undegraded":"2024-09-17T05:25:17.820417+0000","last_fullsized":"2024-09-17T05:25:17.820417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864215+0000","last_change":"2024-09-17T05:25:17.860851+0000","last_active":"2024-09-17T05:25:17.864215+0000","last_peered":"2024-09-17T05:25:17.864215+0000","last_clean":"2024-09-17T05:25:17.864215+0000","last_became_active":"2024-09-17T05:25:17.860262+0000","last_became_peered":"2024-09-17T05:25:17.860262+0000","last_unstale":"2024-09-17T05:25:17.864215+0000","last_undegraded":"2024-09-17T05:25:17.864215+0000","last_fullsized":"2024-09-17T05:25:17.864215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:15.173253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.827881+0000","last_change":"2024-09-17T05:25:16.841538+0000","last_active":"2024-09-17T05:25:17.827881+0000","last_peered":"2024-09-17T05:25:17.827881+0000","last_clean":"2024-09-17T05:25:17.827881+0000","last_became_active":"2024-09-17T05:25:16.840839+0000","last_became_peered":"2024-09-17T05:25:16.840839+0000","last_unstale":"2024-09-17T05:25:17.827881+0000","last_undegraded":"2024-09-17T05:25:17.827881+0000","last_fullsized":"2024-09-17T05:25:17.827881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:35.655115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856628+0000","last_change":"2024-09-17T05:25:17.849451+0000","last_active":"2024-09-17T05:25:17.856628+0000","last_peered":"2024-09-17T05:25:17.856628+0000","last_clean":"2024-09-17T05:25:17.856628+0000","last_became_active":"2024-09-17T05:25:17.848905+0000","last_became_peered":"2024-09-17T05:25:17.848905+0000","last_unstale":"2024-09-17T05:25:17.856628+0000","last_undegraded":"2024-09-17T05:25:17.856628+0000","last_fullsized":"2024-09-17T05:25:17.856628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:55.535278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819647+0000","last_change":"2024-09-17T05:25:16.841637+0000","last_active":"2024-09-17T05:25:17.819647+0000","last_peered":"2024-09-17T05:25:17.819647+0000","last_clean":"2024-09-17T05:25:17.819647+0000","last_became_active":"2024-09-17T05:25:16.840417+0000","last_became_peered":"2024-09-17T05:25:16.840417+0000","last_unstale":"2024-09-17T05:25:17.819647+0000","last_undegraded":"2024-09-17T05:25:17.819647+0000","last_fullsized":"2024-09-17T05:25:17.819647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:36.939696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862665+0000","last_change":"2024-09-17T05:25:17.858282+0000","last_active":"2024-09-17T05:25:17.862665+0000","last_peered":"2024-09-17T05:25:17.862665+0000","last_clean":"2024-09-17T05:25:17.862665+0000","last_became_active":"2024-09-17T05:25:17.856926+0000","last_became_peered":"2024-09-17T05:25:17.856926+0000","last_unstale":"2024-09-17T05:25:17.862665+0000","last_undegraded":"2024-09-17T05:25:17.862665+0000","last_fullsized":"2024-09-17T05:25:17.862665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:10.392801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837706+0000","last_change":"2024-09-17T05:25:16.843421+0000","last_active":"2024-09-17T05:25:17.837706+0000","last_peered":"2024-09-17T05:25:17.837706+0000","last_clean":"2024-09-17T05:25:17.837706+0000","last_became_active":"2024-09-17T05:25:16.843071+0000","last_became_peered":"2024-09-17T05:25:16.843071+0000","last_unstale":"2024-09-17T05:25:17.837706+0000","last_undegraded":"2024-09-17T05:25:17.837706+0000","last_fullsized":"2024-09-17T05:25:17.837706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855103+0000","last_change":"2024-09-17T05:25:17.851994+0000","last_active":"2024-09-17T05:25:17.855103+0000","last_peered":"2024-09-17T05:25:17.855103+0000","last_clean":"2024-09-17T05:25:17.855103+0000","last_became_active":"2024-09-17T05:25:17.851553+0000","last_became_peered":"2024-09-17T05:25:17.851553+0000","last_unstale":"2024-09-17T05:25:17.855103+0000","last_undegraded":"2024-09-17T05:25:17.855103+0000","last_fullsized":"2024-09-17T05:25:17.855103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:57.581440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839648+0000","last_change":"2024-09-17T05:25:16.838372+0000","last_active":"2024-09-17T05:25:16.839648+0000","last_peered":"2024-09-17T05:25:16.839648+0000","last_clean":"2024-09-17T05:25:16.839648+0000","last_became_active":"2024-09-17T05:25:16.837915+0000","last_became_peered":"2024-09-17T05:25:16.837915+0000","last_unstale":"2024-09-17T05:25:16.839648+0000","last_undegraded":"2024-09-17T05:25:16.839648+0000","last_fullsized":"2024-09-17T05:25:16.839648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:11.525857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863332+0000","last_change":"2024-09-17T05:25:17.857155+0000","last_active":"2024-09-17T05:25:17.863332+0000","last_peered":"2024-09-17T05:25:17.863332+0000","last_clean":"2024-09-17T05:25:17.863332+0000","last_became_active":"2024-09-17T05:25:17.855676+0000","last_became_peered":"2024-09-17T05:25:17.855676+0000","last_unstale":"2024-09-17T05:25:17.863332+0000","last_undegraded":"2024-09-17T05:25:17.863332+0000","last_fullsized":"2024-09-17T05:25:17.863332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:37.942881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.826545+0000","last_change":"2024-09-17T05:25:16.844953+0000","last_active":"2024-09-17T05:25:17.826545+0000","last_peered":"2024-09-17T05:25:17.826545+0000","last_clean":"2024-09-17T05:25:17.826545+0000","last_became_active":"2024-09-17T05:25:16.843607+0000","last_became_peered":"2024-09-17T05:25:16.843607+0000","last_unstale":"2024-09-17T05:25:17.826545+0000","last_undegraded":"2024-09-17T05:25:17.826545+0000","last_fullsized":"2024-09-17T05:25:17.826545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.778959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858722+0000","last_change":"2024-09-17T05:25:17.843412+0000","last_active":"2024-09-17T05:25:17.858722+0000","last_peered":"2024-09-17T05:25:17.858722+0000","last_clean":"2024-09-17T05:25:17.858722+0000","last_became_active":"2024-09-17T05:25:17.842545+0000","last_became_peered":"2024-09-17T05:25:17.842545+0000","last_unstale":"2024-09-17T05:25:17.858722+0000","last_undegraded":"2024-09-17T05:25:17.858722+0000","last_fullsized":"2024-09-17T05:25:17.858722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:20.366914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864068+0000","last_change":"2024-09-17T05:25:16.842097+0000","last_active":"2024-09-17T05:25:17.864068+0000","last_peered":"2024-09-17T05:25:17.864068+0000","last_clean":"2024-09-17T05:25:17.864068+0000","last_became_active":"2024-09-17T05:25:16.841570+0000","last_became_peered":"2024-09-17T05:25:16.841570+0000","last_unstale":"2024-09-17T05:25:17.864068+0000","last_undegraded":"2024-09-17T05:25:17.864068+0000","last_fullsized":"2024-09-17T05:25:17.864068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":154,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830646+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:17.830646+0000","last_peered":"2024-09-17T05:25:17.830646+0000","last_clean":"2024-09-17T05:25:17.830646+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:17.830646+0000","last_undegraded":"2024-09-17T05:25:17.830646+0000","last_fullsized":"2024-09-17T05:25:17.830646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":215170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":805526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":552960,"data_stored":216840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":216748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":215226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":805574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":215808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":208361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:22.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:22.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1142315446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0b7ce0 msgr2=0x7f8eec0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1142315446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec0d0840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0739f0 tx=0x7f8edc01c5e0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1142315446 shutdown_connections 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1142315446 >> v2:172.21.15.62:3300/0 conn(0x7f8eec0cd3b0 0x7f8eec0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1142315446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1142315446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec0d0840 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1142315446 >> 172.21.15.43:0/1142315446 conn(0x7f8eec0cee80 msgr2=0x7f8eec0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1142315446 shutdown_connections 2024-09-17T05:25:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1142315446 wait complete. 2024-09-17T05:25:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 Processor -- start 2024-09-17T05:25:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.696+0000 7f8ef2c00640 1 -- start start 2024-09-17T05:25:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1609c0 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec160b40 con 0x7f8eec0b7ce0 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec160cc0 con 0x7f8eec0d0d80 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51324/0 (socket says 172.21.15.43:51324) 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 -- 172.21.15.43:0/1718682471 learned_addr learned my addr 172.21.15.43:0/1718682471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8eebe00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef0c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 -- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 msgr2=0x7f8eec158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 -- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eec0b7ce0 msgr2=0x7f8eec150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 -- 172.21.15.43:0/1718682471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec1506a0 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef0c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee3e00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d720 tx=0x7f8ed400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4017070 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ed4005310 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4009820 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec1f9090 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1f9650 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.700+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec0b1c30 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.704+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ed40099c0 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.704+0000 7f8ee8c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 0x7f8ec8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.704+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f8ed40943b0 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.704+0000 7f8eebe00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 0x7f8ec8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.704+0000 7f8eebe00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 0x7f8ec8075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc0b5040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.708+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ed405ece0 con 0x7f8eec0cd3b0 2024-09-17T05:25:22.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.884+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8eec1fd1a0 con 0x7f8ec8073200 2024-09-17T05:25:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.892+0000 7f8ee8c00640 1 -- 172.21.15.43:0/1718682471 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251531 (secure 0 0 0) 0x7f8eec1fd1a0 con 0x7f8ec8073200 2024-09-17T05:25:22.898 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 msgr2=0x7f8ec8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 0x7f8ec8075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc0b5040 comp rx=0 tx=0).stop 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 msgr2=0x7f8eec158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d720 tx=0x7f8ed400dbe0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 shutdown_connections 2024-09-17T05:25:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ec8073200 0x7f8ec8075710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8eec0d0d80 0x7f8eec158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8eec0cd3b0 0x7f8eec158580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 --2- 172.21.15.43:0/1718682471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8eec0b7ce0 0x7f8eec150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 >> 172.21.15.43:0/1718682471 conn(0x7f8eec0cee80 msgr2=0x7f8eec15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 shutdown_connections 2024-09-17T05:25:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:22.896+0000 7f8ef2c00640 1 -- 172.21.15.43:0/1718682471 wait complete. 2024-09-17T05:25:22.920 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-17T05:25:21.868498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242428,"kb_used_data":5364,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732292,"statfs":{"total":773094113280,"available":772845867008,"internally_reserved":0,"allocated":5492736,"data_stored":2899253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836136+0000","last_change":"2024-09-17T05:25:17.830654+0000","last_active":"2024-09-17T05:25:17.836136+0000","last_peered":"2024-09-17T05:25:17.836136+0000","last_clean":"2024-09-17T05:25:17.836136+0000","last_became_active":"2024-09-17T05:25:17.830321+0000","last_became_peered":"2024-09-17T05:25:17.830321+0000","last_unstale":"2024-09-17T05:25:17.836136+0000","last_undegraded":"2024-09-17T05:25:17.836136+0000","last_fullsized":"2024-09-17T05:25:17.836136+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:42.098022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848789+0000","last_change":"2024-09-17T05:25:17.844139+0000","last_active":"2024-09-17T05:25:17.848789+0000","last_peered":"2024-09-17T05:25:17.848789+0000","last_clean":"2024-09-17T05:25:17.848789+0000","last_became_active":"2024-09-17T05:25:17.843821+0000","last_became_peered":"2024-09-17T05:25:17.843821+0000","last_unstale":"2024-09-17T05:25:17.848789+0000","last_undegraded":"2024-09-17T05:25:17.848789+0000","last_fullsized":"2024-09-17T05:25:17.848789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:38.456234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850784+0000","last_change":"2024-09-17T05:25:17.847168+0000","last_active":"2024-09-17T05:25:17.850784+0000","last_peered":"2024-09-17T05:25:17.850784+0000","last_clean":"2024-09-17T05:25:17.850784+0000","last_became_active":"2024-09-17T05:25:17.846513+0000","last_became_peered":"2024-09-17T05:25:17.846513+0000","last_unstale":"2024-09-17T05:25:17.850784+0000","last_undegraded":"2024-09-17T05:25:17.850784+0000","last_fullsized":"2024-09-17T05:25:17.850784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:06.087190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859247+0000","last_change":"2024-09-17T05:25:17.845500+0000","last_active":"2024-09-17T05:25:17.859247+0000","last_peered":"2024-09-17T05:25:17.859247+0000","last_clean":"2024-09-17T05:25:17.859247+0000","last_became_active":"2024-09-17T05:25:17.845221+0000","last_became_peered":"2024-09-17T05:25:17.845221+0000","last_unstale":"2024-09-17T05:25:17.859247+0000","last_undegraded":"2024-09-17T05:25:17.859247+0000","last_fullsized":"2024-09-17T05:25:17.859247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:09.723432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859215+0000","last_change":"2024-09-17T05:25:17.858007+0000","last_active":"2024-09-17T05:25:17.859215+0000","last_peered":"2024-09-17T05:25:17.859215+0000","last_clean":"2024-09-17T05:25:17.859215+0000","last_became_active":"2024-09-17T05:25:17.855860+0000","last_became_peered":"2024-09-17T05:25:17.855860+0000","last_unstale":"2024-09-17T05:25:17.859215+0000","last_undegraded":"2024-09-17T05:25:17.859215+0000","last_fullsized":"2024-09-17T05:25:17.859215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817734+0000","last_change":"2024-09-17T05:25:16.817734+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817734+0000","last_undegraded":"2024-09-17T05:25:16.817734+0000","last_fullsized":"2024-09-17T05:25:16.817734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864857+0000","last_change":"2024-09-17T05:25:17.863014+0000","last_active":"2024-09-17T05:25:17.864857+0000","last_peered":"2024-09-17T05:25:17.864857+0000","last_clean":"2024-09-17T05:25:17.864857+0000","last_became_active":"2024-09-17T05:25:17.862710+0000","last_became_peered":"2024-09-17T05:25:17.862710+0000","last_unstale":"2024-09-17T05:25:17.864857+0000","last_undegraded":"2024-09-17T05:25:17.864857+0000","last_fullsized":"2024-09-17T05:25:17.864857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:54.789315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852055+0000","last_change":"2024-09-17T05:25:17.843269+0000","last_active":"2024-09-17T05:25:17.852055+0000","last_peered":"2024-09-17T05:25:17.852055+0000","last_clean":"2024-09-17T05:25:17.852055+0000","last_became_active":"2024-09-17T05:25:17.842790+0000","last_became_peered":"2024-09-17T05:25:17.842790+0000","last_unstale":"2024-09-17T05:25:17.852055+0000","last_undegraded":"2024-09-17T05:25:17.852055+0000","last_fullsized":"2024-09-17T05:25:17.852055+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:09.373391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.818454+0000","last_change":"2024-09-17T05:25:16.818454+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.818454+0000","last_undegraded":"2024-09-17T05:25:16.818454+0000","last_fullsized":"2024-09-17T05:25:16.818454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843015+0000","last_change":"2024-09-17T05:25:17.834657+0000","last_active":"2024-09-17T05:25:17.843015+0000","last_peered":"2024-09-17T05:25:17.843015+0000","last_clean":"2024-09-17T05:25:17.843015+0000","last_became_active":"2024-09-17T05:25:17.834339+0000","last_became_peered":"2024-09-17T05:25:17.834339+0000","last_unstale":"2024-09-17T05:25:17.843015+0000","last_undegraded":"2024-09-17T05:25:17.843015+0000","last_fullsized":"2024-09-17T05:25:17.843015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862582+0000","last_change":"2024-09-17T05:25:17.857677+0000","last_active":"2024-09-17T05:25:17.862582+0000","last_peered":"2024-09-17T05:25:17.862582+0000","last_clean":"2024-09-17T05:25:17.862582+0000","last_became_active":"2024-09-17T05:25:17.857316+0000","last_became_peered":"2024-09-17T05:25:17.857316+0000","last_unstale":"2024-09-17T05:25:17.862582+0000","last_undegraded":"2024-09-17T05:25:17.862582+0000","last_fullsized":"2024-09-17T05:25:17.862582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.835972+0000","last_change":"2024-09-17T05:25:17.828984+0000","last_active":"2024-09-17T05:25:17.835972+0000","last_peered":"2024-09-17T05:25:17.835972+0000","last_clean":"2024-09-17T05:25:17.835972+0000","last_became_active":"2024-09-17T05:25:17.828631+0000","last_became_peered":"2024-09-17T05:25:17.828631+0000","last_unstale":"2024-09-17T05:25:17.835972+0000","last_undegraded":"2024-09-17T05:25:17.835972+0000","last_fullsized":"2024-09-17T05:25:17.835972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:25.873513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.819073+0000","last_change":"2024-09-17T05:25:16.819073+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.819073+0000","last_undegraded":"2024-09-17T05:25:16.819073+0000","last_fullsized":"2024-09-17T05:25:16.819073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859255+0000","last_change":"2024-09-17T05:25:17.847391+0000","last_active":"2024-09-17T05:25:17.859255+0000","last_peered":"2024-09-17T05:25:17.859255+0000","last_clean":"2024-09-17T05:25:17.859255+0000","last_became_active":"2024-09-17T05:25:17.847099+0000","last_became_peered":"2024-09-17T05:25:17.847099+0000","last_unstale":"2024-09-17T05:25:17.859255+0000","last_undegraded":"2024-09-17T05:25:17.859255+0000","last_fullsized":"2024-09-17T05:25:17.859255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:18.658668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862139+0000","last_change":"2024-09-17T05:25:17.852627+0000","last_active":"2024-09-17T05:25:17.862139+0000","last_peered":"2024-09-17T05:25:17.862139+0000","last_clean":"2024-09-17T05:25:17.862139+0000","last_became_active":"2024-09-17T05:25:17.849911+0000","last_became_peered":"2024-09-17T05:25:17.849911+0000","last_unstale":"2024-09-17T05:25:17.862139+0000","last_undegraded":"2024-09-17T05:25:17.862139+0000","last_fullsized":"2024-09-17T05:25:17.862139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:20.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852603+0000","last_change":"2024-09-17T05:25:17.849702+0000","last_active":"2024-09-17T05:25:17.852603+0000","last_peered":"2024-09-17T05:25:17.852603+0000","last_clean":"2024-09-17T05:25:17.852603+0000","last_became_active":"2024-09-17T05:25:17.849304+0000","last_became_peered":"2024-09-17T05:25:17.849304+0000","last_unstale":"2024-09-17T05:25:17.852603+0000","last_undegraded":"2024-09-17T05:25:17.852603+0000","last_fullsized":"2024-09-17T05:25:17.852603+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:31.089937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.844026+0000","last_change":"2024-09-17T05:25:17.838654+0000","last_active":"2024-09-17T05:25:17.844026+0000","last_peered":"2024-09-17T05:25:17.844026+0000","last_clean":"2024-09-17T05:25:17.844026+0000","last_became_active":"2024-09-17T05:25:17.838259+0000","last_became_peered":"2024-09-17T05:25:17.838259+0000","last_unstale":"2024-09-17T05:25:17.844026+0000","last_undegraded":"2024-09-17T05:25:17.844026+0000","last_fullsized":"2024-09-17T05:25:17.844026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:29.702298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863471+0000","last_change":"2024-09-17T05:25:17.862226+0000","last_active":"2024-09-17T05:25:17.863471+0000","last_peered":"2024-09-17T05:25:17.863471+0000","last_clean":"2024-09-17T05:25:17.863471+0000","last_became_active":"2024-09-17T05:25:17.860613+0000","last_became_peered":"2024-09-17T05:25:17.860613+0000","last_unstale":"2024-09-17T05:25:17.863471+0000","last_undegraded":"2024-09-17T05:25:17.863471+0000","last_fullsized":"2024-09-17T05:25:17.863471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:41.914430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859049+0000","last_change":"2024-09-17T05:25:17.855133+0000","last_active":"2024-09-17T05:25:17.859049+0000","last_peered":"2024-09-17T05:25:17.859049+0000","last_clean":"2024-09-17T05:25:17.859049+0000","last_became_active":"2024-09-17T05:25:17.854806+0000","last_became_peered":"2024-09-17T05:25:17.854806+0000","last_unstale":"2024-09-17T05:25:17.859049+0000","last_undegraded":"2024-09-17T05:25:17.859049+0000","last_fullsized":"2024-09-17T05:25:17.859049+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:13.815396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.851242+0000","last_change":"2024-09-17T05:25:17.840899+0000","last_active":"2024-09-17T05:25:17.851242+0000","last_peered":"2024-09-17T05:25:17.851242+0000","last_clean":"2024-09-17T05:25:17.851242+0000","last_became_active":"2024-09-17T05:25:17.832241+0000","last_became_peered":"2024-09-17T05:25:17.832241+0000","last_unstale":"2024-09-17T05:25:17.851242+0000","last_undegraded":"2024-09-17T05:25:17.851242+0000","last_fullsized":"2024-09-17T05:25:17.851242+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:07.272236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859491+0000","last_change":"2024-09-17T05:25:17.846375+0000","last_active":"2024-09-17T05:25:17.859491+0000","last_peered":"2024-09-17T05:25:17.859491+0000","last_clean":"2024-09-17T05:25:17.859491+0000","last_became_active":"2024-09-17T05:25:17.846001+0000","last_became_peered":"2024-09-17T05:25:17.846001+0000","last_unstale":"2024-09-17T05:25:17.859491+0000","last_undegraded":"2024-09-17T05:25:17.859491+0000","last_fullsized":"2024-09-17T05:25:17.859491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:34.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855800+0000","last_change":"2024-09-17T05:25:17.849145+0000","last_active":"2024-09-17T05:25:17.855800+0000","last_peered":"2024-09-17T05:25:17.855800+0000","last_clean":"2024-09-17T05:25:17.855800+0000","last_became_active":"2024-09-17T05:25:17.848763+0000","last_became_peered":"2024-09-17T05:25:17.848763+0000","last_unstale":"2024-09-17T05:25:17.855800+0000","last_undegraded":"2024-09-17T05:25:17.855800+0000","last_fullsized":"2024-09-17T05:25:17.855800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:53.559077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.846762+0000","last_change":"2024-09-17T05:25:17.839774+0000","last_active":"2024-09-17T05:25:17.846762+0000","last_peered":"2024-09-17T05:25:17.846762+0000","last_clean":"2024-09-17T05:25:17.846762+0000","last_became_active":"2024-09-17T05:25:17.839418+0000","last_became_peered":"2024-09-17T05:25:17.839418+0000","last_unstale":"2024-09-17T05:25:17.846762+0000","last_undegraded":"2024-09-17T05:25:17.846762+0000","last_fullsized":"2024-09-17T05:25:17.846762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:59.137139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864301+0000","last_change":"2024-09-17T05:25:17.860865+0000","last_active":"2024-09-17T05:25:17.864301+0000","last_peered":"2024-09-17T05:25:17.864301+0000","last_clean":"2024-09-17T05:25:17.864301+0000","last_became_active":"2024-09-17T05:25:17.857489+0000","last_became_peered":"2024-09-17T05:25:17.857489+0000","last_unstale":"2024-09-17T05:25:17.864301+0000","last_undegraded":"2024-09-17T05:25:17.864301+0000","last_fullsized":"2024-09-17T05:25:17.864301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830904+0000","last_change":"2024-09-17T05:25:17.827002+0000","last_active":"2024-09-17T05:25:17.830904+0000","last_peered":"2024-09-17T05:25:17.830904+0000","last_clean":"2024-09-17T05:25:17.830904+0000","last_became_active":"2024-09-17T05:25:17.826648+0000","last_became_peered":"2024-09-17T05:25:17.826648+0000","last_unstale":"2024-09-17T05:25:17.830904+0000","last_undegraded":"2024-09-17T05:25:17.830904+0000","last_fullsized":"2024-09-17T05:25:17.830904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:41.756792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862280+0000","last_change":"2024-09-17T05:25:17.855977+0000","last_active":"2024-09-17T05:25:17.862280+0000","last_peered":"2024-09-17T05:25:17.862280+0000","last_clean":"2024-09-17T05:25:17.862280+0000","last_became_active":"2024-09-17T05:25:17.855650+0000","last_became_peered":"2024-09-17T05:25:17.855650+0000","last_unstale":"2024-09-17T05:25:17.862280+0000","last_undegraded":"2024-09-17T05:25:17.862280+0000","last_fullsized":"2024-09-17T05:25:17.862280+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864199+0000","last_change":"2024-09-17T05:25:17.860605+0000","last_active":"2024-09-17T05:25:17.864199+0000","last_peered":"2024-09-17T05:25:17.864199+0000","last_clean":"2024-09-17T05:25:17.864199+0000","last_became_active":"2024-09-17T05:25:17.859963+0000","last_became_peered":"2024-09-17T05:25:17.859963+0000","last_unstale":"2024-09-17T05:25:17.864199+0000","last_undegraded":"2024-09-17T05:25:17.864199+0000","last_fullsized":"2024-09-17T05:25:17.864199+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:55.637768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852200+0000","last_change":"2024-09-17T05:25:17.841362+0000","last_active":"2024-09-17T05:25:17.852200+0000","last_peered":"2024-09-17T05:25:17.852200+0000","last_clean":"2024-09-17T05:25:17.852200+0000","last_became_active":"2024-09-17T05:25:17.840961+0000","last_became_peered":"2024-09-17T05:25:17.840961+0000","last_unstale":"2024-09-17T05:25:17.852200+0000","last_undegraded":"2024-09-17T05:25:17.852200+0000","last_fullsized":"2024-09-17T05:25:17.852200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.926288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837038+0000","last_change":"2024-09-17T05:25:17.829627+0000","last_active":"2024-09-17T05:25:17.837038+0000","last_peered":"2024-09-17T05:25:17.837038+0000","last_clean":"2024-09-17T05:25:17.837038+0000","last_became_active":"2024-09-17T05:25:17.829328+0000","last_became_peered":"2024-09-17T05:25:17.829328+0000","last_unstale":"2024-09-17T05:25:17.837038+0000","last_undegraded":"2024-09-17T05:25:17.837038+0000","last_fullsized":"2024-09-17T05:25:17.837038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:27.488602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.861137+0000","last_change":"2024-09-17T05:25:17.855557+0000","last_active":"2024-09-17T05:25:17.861137+0000","last_peered":"2024-09-17T05:25:17.861137+0000","last_clean":"2024-09-17T05:25:17.861137+0000","last_became_active":"2024-09-17T05:25:17.854842+0000","last_became_peered":"2024-09-17T05:25:17.854842+0000","last_unstale":"2024-09-17T05:25:17.861137+0000","last_undegraded":"2024-09-17T05:25:17.861137+0000","last_fullsized":"2024-09-17T05:25:17.861137+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:43.613646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817820+0000","last_change":"2024-09-17T05:25:16.817820+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817820+0000","last_undegraded":"2024-09-17T05:25:16.817820+0000","last_fullsized":"2024-09-17T05:25:16.817820+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862079+0000","last_change":"2024-09-17T05:25:17.851832+0000","last_active":"2024-09-17T05:25:17.862079+0000","last_peered":"2024-09-17T05:25:17.862079+0000","last_clean":"2024-09-17T05:25:17.862079+0000","last_became_active":"2024-09-17T05:25:17.851503+0000","last_became_peered":"2024-09-17T05:25:17.851503+0000","last_unstale":"2024-09-17T05:25:17.862079+0000","last_undegraded":"2024-09-17T05:25:17.862079+0000","last_fullsized":"2024-09-17T05:25:17.862079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.055649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850746+0000","last_change":"2024-09-17T05:25:17.845069+0000","last_active":"2024-09-17T05:25:17.850746+0000","last_peered":"2024-09-17T05:25:17.850746+0000","last_clean":"2024-09-17T05:25:17.850746+0000","last_became_active":"2024-09-17T05:25:17.844766+0000","last_became_peered":"2024-09-17T05:25:17.844766+0000","last_unstale":"2024-09-17T05:25:17.850746+0000","last_undegraded":"2024-09-17T05:25:17.850746+0000","last_fullsized":"2024-09-17T05:25:17.850746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:18.184047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.838673+0000","last_change":"2024-09-17T05:25:16.830007+0000","last_active":"2024-09-17T05:25:16.838673+0000","last_peered":"2024-09-17T05:25:16.838673+0000","last_clean":"2024-09-17T05:25:16.838673+0000","last_became_active":"2024-09-17T05:25:16.828838+0000","last_became_peered":"2024-09-17T05:25:16.828838+0000","last_unstale":"2024-09-17T05:25:16.838673+0000","last_undegraded":"2024-09-17T05:25:16.838673+0000","last_fullsized":"2024-09-17T05:25:16.838673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:34.533131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858878+0000","last_change":"2024-09-17T05:25:17.855463+0000","last_active":"2024-09-17T05:25:17.858878+0000","last_peered":"2024-09-17T05:25:17.858878+0000","last_clean":"2024-09-17T05:25:17.858878+0000","last_became_active":"2024-09-17T05:25:17.855089+0000","last_became_peered":"2024-09-17T05:25:17.855089+0000","last_unstale":"2024-09-17T05:25:17.858878+0000","last_undegraded":"2024-09-17T05:25:17.858878+0000","last_fullsized":"2024-09-17T05:25:17.858878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:58.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.860692+0000","last_change":"2024-09-17T05:25:16.841386+0000","last_active":"2024-09-17T05:25:17.860692+0000","last_peered":"2024-09-17T05:25:17.860692+0000","last_clean":"2024-09-17T05:25:17.860692+0000","last_became_active":"2024-09-17T05:25:16.840942+0000","last_became_peered":"2024-09-17T05:25:16.840942+0000","last_unstale":"2024-09-17T05:25:17.860692+0000","last_undegraded":"2024-09-17T05:25:17.860692+0000","last_fullsized":"2024-09-17T05:25:17.860692+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:09.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.848773+0000","last_change":"2024-09-17T05:25:17.844192+0000","last_active":"2024-09-17T05:25:17.848773+0000","last_peered":"2024-09-17T05:25:17.848773+0000","last_clean":"2024-09-17T05:25:17.848773+0000","last_became_active":"2024-09-17T05:25:17.843753+0000","last_became_peered":"2024-09-17T05:25:17.843753+0000","last_unstale":"2024-09-17T05:25:17.848773+0000","last_undegraded":"2024-09-17T05:25:17.848773+0000","last_fullsized":"2024-09-17T05:25:17.848773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:45.282549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839405+0000","last_change":"2024-09-17T05:25:16.835803+0000","last_active":"2024-09-17T05:25:16.839405+0000","last_peered":"2024-09-17T05:25:16.839405+0000","last_clean":"2024-09-17T05:25:16.839405+0000","last_became_active":"2024-09-17T05:25:16.835332+0000","last_became_peered":"2024-09-17T05:25:16.835332+0000","last_unstale":"2024-09-17T05:25:16.839405+0000","last_undegraded":"2024-09-17T05:25:16.839405+0000","last_fullsized":"2024-09-17T05:25:16.839405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:57.056082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862435+0000","last_change":"2024-09-17T05:25:17.853747+0000","last_active":"2024-09-17T05:25:17.862435+0000","last_peered":"2024-09-17T05:25:17.862435+0000","last_clean":"2024-09-17T05:25:17.862435+0000","last_became_active":"2024-09-17T05:25:17.853420+0000","last_became_peered":"2024-09-17T05:25:17.853420+0000","last_unstale":"2024-09-17T05:25:17.862435+0000","last_undegraded":"2024-09-17T05:25:17.862435+0000","last_fullsized":"2024-09-17T05:25:17.862435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862840+0000","last_change":"2024-09-17T05:25:16.839690+0000","last_active":"2024-09-17T05:25:17.862840+0000","last_peered":"2024-09-17T05:25:17.862840+0000","last_clean":"2024-09-17T05:25:17.862840+0000","last_became_active":"2024-09-17T05:25:16.839208+0000","last_became_peered":"2024-09-17T05:25:16.839208+0000","last_unstale":"2024-09-17T05:25:17.862840+0000","last_undegraded":"2024-09-17T05:25:17.862840+0000","last_fullsized":"2024-09-17T05:25:17.862840+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:20.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864027+0000","last_change":"2024-09-17T05:25:17.860246+0000","last_active":"2024-09-17T05:25:17.864027+0000","last_peered":"2024-09-17T05:25:17.864027+0000","last_clean":"2024-09-17T05:25:17.864027+0000","last_became_active":"2024-09-17T05:25:17.859898+0000","last_became_peered":"2024-09-17T05:25:17.859898+0000","last_unstale":"2024-09-17T05:25:17.864027+0000","last_undegraded":"2024-09-17T05:25:17.864027+0000","last_fullsized":"2024-09-17T05:25:17.864027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:40.128894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.832185+0000","last_change":"2024-09-17T05:25:16.825193+0000","last_active":"2024-09-17T05:25:16.832185+0000","last_peered":"2024-09-17T05:25:16.832185+0000","last_clean":"2024-09-17T05:25:16.832185+0000","last_became_active":"2024-09-17T05:25:16.824705+0000","last_became_peered":"2024-09-17T05:25:16.824705+0000","last_unstale":"2024-09-17T05:25:16.832185+0000","last_undegraded":"2024-09-17T05:25:16.832185+0000","last_fullsized":"2024-09-17T05:25:16.832185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:23.844925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852903+0000","last_change":"2024-09-17T05:25:17.844440+0000","last_active":"2024-09-17T05:25:17.852903+0000","last_peered":"2024-09-17T05:25:17.852903+0000","last_clean":"2024-09-17T05:25:17.852903+0000","last_became_active":"2024-09-17T05:25:17.843574+0000","last_became_peered":"2024-09-17T05:25:17.843574+0000","last_unstale":"2024-09-17T05:25:17.852903+0000","last_undegraded":"2024-09-17T05:25:17.852903+0000","last_fullsized":"2024-09-17T05:25:17.852903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:27.659295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859395+0000","last_change":"2024-09-17T05:25:16.839269+0000","last_active":"2024-09-17T05:25:17.859395+0000","last_peered":"2024-09-17T05:25:17.859395+0000","last_clean":"2024-09-17T05:25:17.859395+0000","last_became_active":"2024-09-17T05:25:16.838891+0000","last_became_peered":"2024-09-17T05:25:16.838891+0000","last_unstale":"2024-09-17T05:25:17.859395+0000","last_undegraded":"2024-09-17T05:25:17.859395+0000","last_fullsized":"2024-09-17T05:25:17.859395+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.956617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850851+0000","last_change":"2024-09-17T05:25:17.840191+0000","last_active":"2024-09-17T05:25:17.850851+0000","last_peered":"2024-09-17T05:25:17.850851+0000","last_clean":"2024-09-17T05:25:17.850851+0000","last_became_active":"2024-09-17T05:25:17.839875+0000","last_became_peered":"2024-09-17T05:25:17.839875+0000","last_unstale":"2024-09-17T05:25:17.850851+0000","last_undegraded":"2024-09-17T05:25:17.850851+0000","last_fullsized":"2024-09-17T05:25:17.850851+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:16.807035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818683+0000","last_change":"2024-09-17T05:25:16.834402+0000","last_active":"2024-09-17T05:25:17.818683+0000","last_peered":"2024-09-17T05:25:17.818683+0000","last_clean":"2024-09-17T05:25:17.818683+0000","last_became_active":"2024-09-17T05:25:16.834036+0000","last_became_peered":"2024-09-17T05:25:16.834036+0000","last_unstale":"2024-09-17T05:25:17.818683+0000","last_undegraded":"2024-09-17T05:25:17.818683+0000","last_fullsized":"2024-09-17T05:25:17.818683+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:37.453048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863162+0000","last_change":"2024-09-17T05:25:17.855835+0000","last_active":"2024-09-17T05:25:17.863162+0000","last_peered":"2024-09-17T05:25:17.863162+0000","last_clean":"2024-09-17T05:25:17.863162+0000","last_became_active":"2024-09-17T05:25:17.853327+0000","last_became_peered":"2024-09-17T05:25:17.853327+0000","last_unstale":"2024-09-17T05:25:17.863162+0000","last_undegraded":"2024-09-17T05:25:17.863162+0000","last_fullsized":"2024-09-17T05:25:17.863162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:41.256886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.823113+0000","last_change":"2024-09-17T05:25:16.839214+0000","last_active":"2024-09-17T05:25:17.823113+0000","last_peered":"2024-09-17T05:25:17.823113+0000","last_clean":"2024-09-17T05:25:17.823113+0000","last_became_active":"2024-09-17T05:25:16.838825+0000","last_became_peered":"2024-09-17T05:25:16.838825+0000","last_unstale":"2024-09-17T05:25:17.823113+0000","last_undegraded":"2024-09-17T05:25:17.823113+0000","last_fullsized":"2024-09-17T05:25:17.823113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:30.086751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852021+0000","last_change":"2024-09-17T05:25:17.850811+0000","last_active":"2024-09-17T05:25:17.852021+0000","last_peered":"2024-09-17T05:25:17.852021+0000","last_clean":"2024-09-17T05:25:17.852021+0000","last_became_active":"2024-09-17T05:25:17.850480+0000","last_became_peered":"2024-09-17T05:25:17.850480+0000","last_unstale":"2024-09-17T05:25:17.852021+0000","last_undegraded":"2024-09-17T05:25:17.852021+0000","last_fullsized":"2024-09-17T05:25:17.852021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:48.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.836130+0000","last_change":"2024-09-17T05:25:16.841708+0000","last_active":"2024-09-17T05:25:17.836130+0000","last_peered":"2024-09-17T05:25:17.836130+0000","last_clean":"2024-09-17T05:25:17.836130+0000","last_became_active":"2024-09-17T05:25:16.841263+0000","last_became_peered":"2024-09-17T05:25:16.841263+0000","last_unstale":"2024-09-17T05:25:17.836130+0000","last_undegraded":"2024-09-17T05:25:17.836130+0000","last_fullsized":"2024-09-17T05:25:17.836130+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:21.553815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852485+0000","last_change":"2024-09-17T05:25:17.851388+0000","last_active":"2024-09-17T05:25:17.852485+0000","last_peered":"2024-09-17T05:25:17.852485+0000","last_clean":"2024-09-17T05:25:17.852485+0000","last_became_active":"2024-09-17T05:25:17.851036+0000","last_became_peered":"2024-09-17T05:25:17.851036+0000","last_unstale":"2024-09-17T05:25:17.852485+0000","last_undegraded":"2024-09-17T05:25:17.852485+0000","last_fullsized":"2024-09-17T05:25:17.852485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:14.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821624+0000","last_change":"2024-09-17T05:25:16.843023+0000","last_active":"2024-09-17T05:25:17.821624+0000","last_peered":"2024-09-17T05:25:17.821624+0000","last_clean":"2024-09-17T05:25:17.821624+0000","last_became_active":"2024-09-17T05:25:16.842671+0000","last_became_peered":"2024-09-17T05:25:16.842671+0000","last_unstale":"2024-09-17T05:25:17.821624+0000","last_undegraded":"2024-09-17T05:25:17.821624+0000","last_fullsized":"2024-09-17T05:25:17.821624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:40.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859214+0000","last_change":"2024-09-17T05:25:17.844156+0000","last_active":"2024-09-17T05:25:17.859214+0000","last_peered":"2024-09-17T05:25:17.859214+0000","last_clean":"2024-09-17T05:25:17.859214+0000","last_became_active":"2024-09-17T05:25:17.843870+0000","last_became_peered":"2024-09-17T05:25:17.843870+0000","last_unstale":"2024-09-17T05:25:17.859214+0000","last_undegraded":"2024-09-17T05:25:17.859214+0000","last_fullsized":"2024-09-17T05:25:17.859214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:02.380001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864696+0000","last_change":"2024-09-17T05:25:16.825444+0000","last_active":"2024-09-17T05:25:17.864696+0000","last_peered":"2024-09-17T05:25:17.864696+0000","last_clean":"2024-09-17T05:25:17.864696+0000","last_became_active":"2024-09-17T05:25:16.825122+0000","last_became_peered":"2024-09-17T05:25:16.825122+0000","last_unstale":"2024-09-17T05:25:17.864696+0000","last_undegraded":"2024-09-17T05:25:17.864696+0000","last_fullsized":"2024-09-17T05:25:17.864696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:38.050632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864876+0000","last_change":"2024-09-17T05:25:17.860864+0000","last_active":"2024-09-17T05:25:17.864876+0000","last_peered":"2024-09-17T05:25:17.864876+0000","last_clean":"2024-09-17T05:25:17.864876+0000","last_became_active":"2024-09-17T05:25:17.860380+0000","last_became_peered":"2024-09-17T05:25:17.860380+0000","last_unstale":"2024-09-17T05:25:17.864876+0000","last_undegraded":"2024-09-17T05:25:17.864876+0000","last_fullsized":"2024-09-17T05:25:17.864876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:41.208537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.866116+0000","last_change":"2024-09-17T05:25:16.845121+0000","last_active":"2024-09-17T05:25:17.866116+0000","last_peered":"2024-09-17T05:25:17.866116+0000","last_clean":"2024-09-17T05:25:17.866116+0000","last_became_active":"2024-09-17T05:25:16.844739+0000","last_became_peered":"2024-09-17T05:25:16.844739+0000","last_unstale":"2024-09-17T05:25:17.866116+0000","last_undegraded":"2024-09-17T05:25:17.866116+0000","last_fullsized":"2024-09-17T05:25:17.866116+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:26.485416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856703+0000","last_change":"2024-09-17T05:25:17.854295+0000","last_active":"2024-09-17T05:25:17.856703+0000","last_peered":"2024-09-17T05:25:17.856703+0000","last_clean":"2024-09-17T05:25:17.856703+0000","last_became_active":"2024-09-17T05:25:17.853814+0000","last_became_peered":"2024-09-17T05:25:17.853814+0000","last_unstale":"2024-09-17T05:25:17.856703+0000","last_undegraded":"2024-09-17T05:25:17.856703+0000","last_fullsized":"2024-09-17T05:25:17.856703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:38.195708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856501+0000","last_change":"2024-09-17T05:25:16.839573+0000","last_active":"2024-09-17T05:25:17.856501+0000","last_peered":"2024-09-17T05:25:17.856501+0000","last_clean":"2024-09-17T05:25:17.856501+0000","last_became_active":"2024-09-17T05:25:16.838379+0000","last_became_peered":"2024-09-17T05:25:16.838379+0000","last_unstale":"2024-09-17T05:25:17.856501+0000","last_undegraded":"2024-09-17T05:25:17.856501+0000","last_fullsized":"2024-09-17T05:25:17.856501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:00.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.843182+0000","last_change":"2024-09-17T05:25:17.836620+0000","last_active":"2024-09-17T05:25:17.843182+0000","last_peered":"2024-09-17T05:25:17.843182+0000","last_clean":"2024-09-17T05:25:17.843182+0000","last_became_active":"2024-09-17T05:25:17.836304+0000","last_became_peered":"2024-09-17T05:25:17.836304+0000","last_unstale":"2024-09-17T05:25:17.843182+0000","last_undegraded":"2024-09-17T05:25:17.843182+0000","last_fullsized":"2024-09-17T05:25:17.843182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:46.933484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.818843+0000","last_change":"2024-09-17T05:25:16.837793+0000","last_active":"2024-09-17T05:25:17.818843+0000","last_peered":"2024-09-17T05:25:17.818843+0000","last_clean":"2024-09-17T05:25:17.818843+0000","last_became_active":"2024-09-17T05:25:16.837076+0000","last_became_peered":"2024-09-17T05:25:16.837076+0000","last_unstale":"2024-09-17T05:25:17.818843+0000","last_undegraded":"2024-09-17T05:25:17.818843+0000","last_fullsized":"2024-09-17T05:25:17.818843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:44.279363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850823+0000","last_change":"2024-09-17T05:25:17.846642+0000","last_active":"2024-09-17T05:25:17.850823+0000","last_peered":"2024-09-17T05:25:17.850823+0000","last_clean":"2024-09-17T05:25:17.850823+0000","last_became_active":"2024-09-17T05:25:17.844111+0000","last_became_peered":"2024-09-17T05:25:17.844111+0000","last_unstale":"2024-09-17T05:25:17.850823+0000","last_undegraded":"2024-09-17T05:25:17.850823+0000","last_fullsized":"2024-09-17T05:25:17.850823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:40.896672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.844453+0000","last_change":"2024-09-17T05:25:16.842966+0000","last_active":"2024-09-17T05:25:16.844453+0000","last_peered":"2024-09-17T05:25:16.844453+0000","last_clean":"2024-09-17T05:25:16.844453+0000","last_became_active":"2024-09-17T05:25:16.842650+0000","last_became_peered":"2024-09-17T05:25:16.842650+0000","last_unstale":"2024-09-17T05:25:16.844453+0000","last_undegraded":"2024-09-17T05:25:16.844453+0000","last_fullsized":"2024-09-17T05:25:16.844453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:55:02.668141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.834870+0000","last_change":"2024-09-17T05:25:17.827842+0000","last_active":"2024-09-17T05:25:17.834870+0000","last_peered":"2024-09-17T05:25:17.834870+0000","last_clean":"2024-09-17T05:25:17.834870+0000","last_became_active":"2024-09-17T05:25:17.827506+0000","last_became_peered":"2024-09-17T05:25:17.827506+0000","last_unstale":"2024-09-17T05:25:17.834870+0000","last_undegraded":"2024-09-17T05:25:17.834870+0000","last_fullsized":"2024-09-17T05:25:17.834870+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:52.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863866+0000","last_change":"2024-09-17T05:25:16.835808+0000","last_active":"2024-09-17T05:25:17.863866+0000","last_peered":"2024-09-17T05:25:17.863866+0000","last_clean":"2024-09-17T05:25:17.863866+0000","last_became_active":"2024-09-17T05:25:16.835452+0000","last_became_peered":"2024-09-17T05:25:16.835452+0000","last_unstale":"2024-09-17T05:25:17.863866+0000","last_undegraded":"2024-09-17T05:25:17.863866+0000","last_fullsized":"2024-09-17T05:25:17.863866+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:12.812210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862083+0000","last_change":"2024-09-17T05:25:17.855358+0000","last_active":"2024-09-17T05:25:17.862083+0000","last_peered":"2024-09-17T05:25:17.862083+0000","last_clean":"2024-09-17T05:25:17.862083+0000","last_became_active":"2024-09-17T05:25:17.854829+0000","last_became_peered":"2024-09-17T05:25:17.854829+0000","last_unstale":"2024-09-17T05:25:17.862083+0000","last_undegraded":"2024-09-17T05:25:17.862083+0000","last_fullsized":"2024-09-17T05:25:17.862083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:41.242930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859036+0000","last_change":"2024-09-17T05:25:16.842074+0000","last_active":"2024-09-17T05:25:17.859036+0000","last_peered":"2024-09-17T05:25:17.859036+0000","last_clean":"2024-09-17T05:25:17.859036+0000","last_became_active":"2024-09-17T05:25:16.841700+0000","last_became_peered":"2024-09-17T05:25:16.841700+0000","last_unstale":"2024-09-17T05:25:17.859036+0000","last_undegraded":"2024-09-17T05:25:17.859036+0000","last_fullsized":"2024-09-17T05:25:17.859036+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:38.831286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850551+0000","last_change":"2024-09-17T05:25:17.843043+0000","last_active":"2024-09-17T05:25:17.850551+0000","last_peered":"2024-09-17T05:25:17.850551+0000","last_clean":"2024-09-17T05:25:17.850551+0000","last_became_active":"2024-09-17T05:25:17.842110+0000","last_became_peered":"2024-09-17T05:25:17.842110+0000","last_unstale":"2024-09-17T05:25:17.850551+0000","last_undegraded":"2024-09-17T05:25:17.850551+0000","last_fullsized":"2024-09-17T05:25:17.850551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:58.529182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.854795+0000","last_change":"2024-09-17T05:25:16.843994+0000","last_active":"2024-09-17T05:25:17.854795+0000","last_peered":"2024-09-17T05:25:17.854795+0000","last_clean":"2024-09-17T05:25:17.854795+0000","last_became_active":"2024-09-17T05:25:16.843335+0000","last_became_peered":"2024-09-17T05:25:16.843335+0000","last_unstale":"2024-09-17T05:25:17.854795+0000","last_undegraded":"2024-09-17T05:25:17.854795+0000","last_fullsized":"2024-09-17T05:25:17.854795+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:10.828864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863238+0000","last_change":"2024-09-17T05:25:17.858165+0000","last_active":"2024-09-17T05:25:17.863238+0000","last_peered":"2024-09-17T05:25:17.863238+0000","last_clean":"2024-09-17T05:25:17.863238+0000","last_became_active":"2024-09-17T05:25:17.856741+0000","last_became_peered":"2024-09-17T05:25:17.856741+0000","last_unstale":"2024-09-17T05:25:17.863238+0000","last_undegraded":"2024-09-17T05:25:17.863238+0000","last_fullsized":"2024-09-17T05:25:17.863238+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:50.149002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856205+0000","last_change":"2024-09-17T05:25:16.842760+0000","last_active":"2024-09-17T05:25:17.856205+0000","last_peered":"2024-09-17T05:25:17.856205+0000","last_clean":"2024-09-17T05:25:17.856205+0000","last_became_active":"2024-09-17T05:25:16.842399+0000","last_became_peered":"2024-09-17T05:25:16.842399+0000","last_unstale":"2024-09-17T05:25:17.856205+0000","last_undegraded":"2024-09-17T05:25:17.856205+0000","last_fullsized":"2024-09-17T05:25:17.856205+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:04.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.850747+0000","last_change":"2024-09-17T05:25:17.846811+0000","last_active":"2024-09-17T05:25:17.850747+0000","last_peered":"2024-09-17T05:25:17.850747+0000","last_clean":"2024-09-17T05:25:17.850747+0000","last_became_active":"2024-09-17T05:25:17.846191+0000","last_became_peered":"2024-09-17T05:25:17.846191+0000","last_unstale":"2024-09-17T05:25:17.850747+0000","last_undegraded":"2024-09-17T05:25:17.850747+0000","last_fullsized":"2024-09-17T05:25:17.850747+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:42.562348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839603+0000","last_change":"2024-09-17T05:25:16.836689+0000","last_active":"2024-09-17T05:25:16.839603+0000","last_peered":"2024-09-17T05:25:16.839603+0000","last_clean":"2024-09-17T05:25:16.839603+0000","last_became_active":"2024-09-17T05:25:16.836345+0000","last_became_peered":"2024-09-17T05:25:16.836345+0000","last_unstale":"2024-09-17T05:25:16.839603+0000","last_undegraded":"2024-09-17T05:25:16.839603+0000","last_fullsized":"2024-09-17T05:25:16.839603+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:09.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856658+0000","last_change":"2024-09-17T05:25:17.849888+0000","last_active":"2024-09-17T05:25:17.856658+0000","last_peered":"2024-09-17T05:25:17.856658+0000","last_clean":"2024-09-17T05:25:17.856658+0000","last_became_active":"2024-09-17T05:25:17.848605+0000","last_became_peered":"2024-09-17T05:25:17.848605+0000","last_unstale":"2024-09-17T05:25:17.856658+0000","last_undegraded":"2024-09-17T05:25:17.856658+0000","last_fullsized":"2024-09-17T05:25:17.856658+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:35.175603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852768+0000","last_change":"2024-09-17T05:25:16.836994+0000","last_active":"2024-09-17T05:25:17.852768+0000","last_peered":"2024-09-17T05:25:17.852768+0000","last_clean":"2024-09-17T05:25:17.852768+0000","last_became_active":"2024-09-17T05:25:16.836576+0000","last_became_peered":"2024-09-17T05:25:16.836576+0000","last_unstale":"2024-09-17T05:25:17.852768+0000","last_undegraded":"2024-09-17T05:25:17.852768+0000","last_fullsized":"2024-09-17T05:25:17.852768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:41.094836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-09-17T05:25:16.817331+0000","last_change":"2024-09-17T05:25:16.817331+0000","last_active":"2024-09-17T05:25:16.805809+0000","last_peered":"2024-09-17T05:25:16.805809+0000","last_clean":"2024-09-17T05:25:16.805809+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:16.817331+0000","last_undegraded":"2024-09-17T05:25:16.817331+0000","last_fullsized":"2024-09-17T05:25:16.817331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819191+0000","last_change":"2024-09-17T05:25:16.837411+0000","last_active":"2024-09-17T05:25:17.819191+0000","last_peered":"2024-09-17T05:25:17.819191+0000","last_clean":"2024-09-17T05:25:17.819191+0000","last_became_active":"2024-09-17T05:25:16.837058+0000","last_became_peered":"2024-09-17T05:25:16.837058+0000","last_unstale":"2024-09-17T05:25:17.819191+0000","last_undegraded":"2024-09-17T05:25:17.819191+0000","last_fullsized":"2024-09-17T05:25:17.819191+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:42.610461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859220+0000","last_change":"2024-09-17T05:25:17.850227+0000","last_active":"2024-09-17T05:25:17.859220+0000","last_peered":"2024-09-17T05:25:17.859220+0000","last_clean":"2024-09-17T05:25:17.859220+0000","last_became_active":"2024-09-17T05:25:17.849926+0000","last_became_peered":"2024-09-17T05:25:17.849926+0000","last_unstale":"2024-09-17T05:25:17.859220+0000","last_undegraded":"2024-09-17T05:25:17.859220+0000","last_fullsized":"2024-09-17T05:25:17.859220+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:35.791860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.821447+0000","last_change":"2024-09-17T05:25:16.843105+0000","last_active":"2024-09-17T05:25:17.821447+0000","last_peered":"2024-09-17T05:25:17.821447+0000","last_clean":"2024-09-17T05:25:17.821447+0000","last_became_active":"2024-09-17T05:25:16.842668+0000","last_became_peered":"2024-09-17T05:25:16.842668+0000","last_unstale":"2024-09-17T05:25:17.821447+0000","last_undegraded":"2024-09-17T05:25:17.821447+0000","last_fullsized":"2024-09-17T05:25:17.821447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859135+0000","last_change":"2024-09-17T05:25:17.847878+0000","last_active":"2024-09-17T05:25:17.859135+0000","last_peered":"2024-09-17T05:25:17.859135+0000","last_clean":"2024-09-17T05:25:17.859135+0000","last_became_active":"2024-09-17T05:25:17.844937+0000","last_became_peered":"2024-09-17T05:25:17.844937+0000","last_unstale":"2024-09-17T05:25:17.859135+0000","last_undegraded":"2024-09-17T05:25:17.859135+0000","last_fullsized":"2024-09-17T05:25:17.859135+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:11.788487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837181+0000","last_change":"2024-09-17T05:25:16.839427+0000","last_active":"2024-09-17T05:25:17.837181+0000","last_peered":"2024-09-17T05:25:17.837181+0000","last_clean":"2024-09-17T05:25:17.837181+0000","last_became_active":"2024-09-17T05:25:16.838642+0000","last_became_peered":"2024-09-17T05:25:16.838642+0000","last_unstale":"2024-09-17T05:25:17.837181+0000","last_undegraded":"2024-09-17T05:25:17.837181+0000","last_fullsized":"2024-09-17T05:25:17.837181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.129441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.859527+0000","last_change":"2024-09-17T05:25:17.854279+0000","last_active":"2024-09-17T05:25:17.859527+0000","last_peered":"2024-09-17T05:25:17.859527+0000","last_clean":"2024-09-17T05:25:17.859527+0000","last_became_active":"2024-09-17T05:25:17.853347+0000","last_became_peered":"2024-09-17T05:25:17.853347+0000","last_unstale":"2024-09-17T05:25:17.859527+0000","last_undegraded":"2024-09-17T05:25:17.859527+0000","last_fullsized":"2024-09-17T05:25:17.859527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:48.657810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.832834+0000","last_change":"2024-09-17T05:25:16.845000+0000","last_active":"2024-09-17T05:25:17.832834+0000","last_peered":"2024-09-17T05:25:17.832834+0000","last_clean":"2024-09-17T05:25:17.832834+0000","last_became_active":"2024-09-17T05:25:16.844264+0000","last_became_peered":"2024-09-17T05:25:16.844264+0000","last_unstale":"2024-09-17T05:25:17.832834+0000","last_undegraded":"2024-09-17T05:25:17.832834+0000","last_fullsized":"2024-09-17T05:25:17.832834+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:10.422465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.852661+0000","last_change":"2024-09-17T05:25:17.848021+0000","last_active":"2024-09-17T05:25:17.852661+0000","last_peered":"2024-09-17T05:25:17.852661+0000","last_clean":"2024-09-17T05:25:17.852661+0000","last_became_active":"2024-09-17T05:25:17.847264+0000","last_became_peered":"2024-09-17T05:25:17.847264+0000","last_unstale":"2024-09-17T05:25:17.852661+0000","last_undegraded":"2024-09-17T05:25:17.852661+0000","last_fullsized":"2024-09-17T05:25:17.852661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:48.742377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.820417+0000","last_change":"2024-09-17T05:25:16.838187+0000","last_active":"2024-09-17T05:25:17.820417+0000","last_peered":"2024-09-17T05:25:17.820417+0000","last_clean":"2024-09-17T05:25:17.820417+0000","last_became_active":"2024-09-17T05:25:16.837770+0000","last_became_peered":"2024-09-17T05:25:16.837770+0000","last_unstale":"2024-09-17T05:25:17.820417+0000","last_undegraded":"2024-09-17T05:25:17.820417+0000","last_fullsized":"2024-09-17T05:25:17.820417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864215+0000","last_change":"2024-09-17T05:25:17.860851+0000","last_active":"2024-09-17T05:25:17.864215+0000","last_peered":"2024-09-17T05:25:17.864215+0000","last_clean":"2024-09-17T05:25:17.864215+0000","last_became_active":"2024-09-17T05:25:17.860262+0000","last_became_peered":"2024-09-17T05:25:17.860262+0000","last_unstale":"2024-09-17T05:25:17.864215+0000","last_undegraded":"2024-09-17T05:25:17.864215+0000","last_fullsized":"2024-09-17T05:25:17.864215+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:15.173253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.827881+0000","last_change":"2024-09-17T05:25:16.841538+0000","last_active":"2024-09-17T05:25:17.827881+0000","last_peered":"2024-09-17T05:25:17.827881+0000","last_clean":"2024-09-17T05:25:17.827881+0000","last_became_active":"2024-09-17T05:25:16.840839+0000","last_became_peered":"2024-09-17T05:25:16.840839+0000","last_unstale":"2024-09-17T05:25:17.827881+0000","last_undegraded":"2024-09-17T05:25:17.827881+0000","last_fullsized":"2024-09-17T05:25:17.827881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:35.655115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.856628+0000","last_change":"2024-09-17T05:25:17.849451+0000","last_active":"2024-09-17T05:25:17.856628+0000","last_peered":"2024-09-17T05:25:17.856628+0000","last_clean":"2024-09-17T05:25:17.856628+0000","last_became_active":"2024-09-17T05:25:17.848905+0000","last_became_peered":"2024-09-17T05:25:17.848905+0000","last_unstale":"2024-09-17T05:25:17.856628+0000","last_undegraded":"2024-09-17T05:25:17.856628+0000","last_fullsized":"2024-09-17T05:25:17.856628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:55.535278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.819647+0000","last_change":"2024-09-17T05:25:16.841637+0000","last_active":"2024-09-17T05:25:17.819647+0000","last_peered":"2024-09-17T05:25:17.819647+0000","last_clean":"2024-09-17T05:25:17.819647+0000","last_became_active":"2024-09-17T05:25:16.840417+0000","last_became_peered":"2024-09-17T05:25:16.840417+0000","last_unstale":"2024-09-17T05:25:17.819647+0000","last_undegraded":"2024-09-17T05:25:17.819647+0000","last_fullsized":"2024-09-17T05:25:17.819647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:36.939696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.862665+0000","last_change":"2024-09-17T05:25:17.858282+0000","last_active":"2024-09-17T05:25:17.862665+0000","last_peered":"2024-09-17T05:25:17.862665+0000","last_clean":"2024-09-17T05:25:17.862665+0000","last_became_active":"2024-09-17T05:25:17.856926+0000","last_became_peered":"2024-09-17T05:25:17.856926+0000","last_unstale":"2024-09-17T05:25:17.862665+0000","last_undegraded":"2024-09-17T05:25:17.862665+0000","last_fullsized":"2024-09-17T05:25:17.862665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:10.392801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.837706+0000","last_change":"2024-09-17T05:25:16.843421+0000","last_active":"2024-09-17T05:25:17.837706+0000","last_peered":"2024-09-17T05:25:17.837706+0000","last_clean":"2024-09-17T05:25:17.837706+0000","last_became_active":"2024-09-17T05:25:16.843071+0000","last_became_peered":"2024-09-17T05:25:16.843071+0000","last_unstale":"2024-09-17T05:25:17.837706+0000","last_undegraded":"2024-09-17T05:25:17.837706+0000","last_fullsized":"2024-09-17T05:25:17.837706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.855103+0000","last_change":"2024-09-17T05:25:17.851994+0000","last_active":"2024-09-17T05:25:17.855103+0000","last_peered":"2024-09-17T05:25:17.855103+0000","last_clean":"2024-09-17T05:25:17.855103+0000","last_became_active":"2024-09-17T05:25:17.851553+0000","last_became_peered":"2024-09-17T05:25:17.851553+0000","last_unstale":"2024-09-17T05:25:17.855103+0000","last_undegraded":"2024-09-17T05:25:17.855103+0000","last_fullsized":"2024-09-17T05:25:17.855103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:57.581440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T05:25:16.839648+0000","last_change":"2024-09-17T05:25:16.838372+0000","last_active":"2024-09-17T05:25:16.839648+0000","last_peered":"2024-09-17T05:25:16.839648+0000","last_clean":"2024-09-17T05:25:16.839648+0000","last_became_active":"2024-09-17T05:25:16.837915+0000","last_became_peered":"2024-09-17T05:25:16.837915+0000","last_unstale":"2024-09-17T05:25:16.839648+0000","last_undegraded":"2024-09-17T05:25:16.839648+0000","last_fullsized":"2024-09-17T05:25:16.839648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:11.525857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.863332+0000","last_change":"2024-09-17T05:25:17.857155+0000","last_active":"2024-09-17T05:25:17.863332+0000","last_peered":"2024-09-17T05:25:17.863332+0000","last_clean":"2024-09-17T05:25:17.863332+0000","last_became_active":"2024-09-17T05:25:17.855676+0000","last_became_peered":"2024-09-17T05:25:17.855676+0000","last_unstale":"2024-09-17T05:25:17.863332+0000","last_undegraded":"2024-09-17T05:25:17.863332+0000","last_fullsized":"2024-09-17T05:25:17.863332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:37.942881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.826545+0000","last_change":"2024-09-17T05:25:16.844953+0000","last_active":"2024-09-17T05:25:17.826545+0000","last_peered":"2024-09-17T05:25:17.826545+0000","last_clean":"2024-09-17T05:25:17.826545+0000","last_became_active":"2024-09-17T05:25:16.843607+0000","last_became_peered":"2024-09-17T05:25:16.843607+0000","last_unstale":"2024-09-17T05:25:17.826545+0000","last_undegraded":"2024-09-17T05:25:17.826545+0000","last_fullsized":"2024-09-17T05:25:17.826545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.778959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.858722+0000","last_change":"2024-09-17T05:25:17.843412+0000","last_active":"2024-09-17T05:25:17.858722+0000","last_peered":"2024-09-17T05:25:17.858722+0000","last_clean":"2024-09-17T05:25:17.858722+0000","last_became_active":"2024-09-17T05:25:17.842545+0000","last_became_peered":"2024-09-17T05:25:17.842545+0000","last_unstale":"2024-09-17T05:25:17.858722+0000","last_undegraded":"2024-09-17T05:25:17.858722+0000","last_fullsized":"2024-09-17T05:25:17.858722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:20.366914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.864068+0000","last_change":"2024-09-17T05:25:16.842097+0000","last_active":"2024-09-17T05:25:17.864068+0000","last_peered":"2024-09-17T05:25:17.864068+0000","last_clean":"2024-09-17T05:25:17.864068+0000","last_became_active":"2024-09-17T05:25:16.841570+0000","last_became_peered":"2024-09-17T05:25:16.841570+0000","last_unstale":"2024-09-17T05:25:17.864068+0000","last_undegraded":"2024-09-17T05:25:17.864068+0000","last_fullsized":"2024-09-17T05:25:17.864068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":154,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:25:17.830646+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:17.830646+0000","last_peered":"2024-09-17T05:25:17.830646+0000","last_clean":"2024-09-17T05:25:17.830646+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:17.830646+0000","last_undegraded":"2024-09-17T05:25:17.830646+0000","last_fullsized":"2024-09-17T05:25:17.830646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":215170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":805526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":552960,"data_stored":216840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":216748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":215226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":805574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":215808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":208361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:16.817734+0000', 'last_change': '2024-09-17T05:25:16.817734+0000', 'last_active': '2024-09-17T05:25:16.805809+0000', 'last_peered': '2024-09-17T05:25:16.805809+0000', 'last_clean': '2024-09-17T05:25:16.805809+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:16.817734+0000', 'last_undegraded': '2024-09-17T05:25:16.817734+0000', 'last_fullsized': '2024-09-17T05:25:16.817734+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:16.818454+0000', 'last_change': '2024-09-17T05:25:16.818454+0000', 'last_active': '2024-09-17T05:25:16.805809+0000', 'last_peered': '2024-09-17T05:25:16.805809+0000', 'last_clean': '2024-09-17T05:25:16.805809+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:16.818454+0000', 'last_undegraded': '2024-09-17T05:25:16.818454+0000', 'last_fullsized': '2024-09-17T05:25:16.818454+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:16.819073+0000', 'last_change': '2024-09-17T05:25:16.819073+0000', 'last_active': '2024-09-17T05:25:16.805809+0000', 'last_peered': '2024-09-17T05:25:16.805809+0000', 'last_clean': '2024-09-17T05:25:16.805809+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:16.819073+0000', 'last_undegraded': '2024-09-17T05:25:16.819073+0000', 'last_fullsized': '2024-09-17T05:25:16.819073+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:16.817820+0000', 'last_change': '2024-09-17T05:25:16.817820+0000', 'last_active': '2024-09-17T05:25:16.805809+0000', 'last_peered': '2024-09-17T05:25:16.805809+0000', 'last_clean': '2024-09-17T05:25:16.805809+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:16.817820+0000', 'last_undegraded': '2024-09-17T05:25:16.817820+0000', 'last_fullsized': '2024-09-17T05:25:16.817820+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:22.925 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-17T05:25:22.926 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:16.817331+0000', 'last_change': '2024-09-17T05:25:16.817331+0000', 'last_active': '2024-09-17T05:25:16.805809+0000', 'last_peered': '2024-09-17T05:25:16.805809+0000', 'last_clean': '2024-09-17T05:25:16.805809+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:16.817331+0000', 'last_undegraded': '2024-09-17T05:25:16.817331+0000', 'last_fullsized': '2024-09-17T05:25:16.817331+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:16.805809+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:22.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:25:23.043 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-09-17T05:25:23.041+0000 7fbc96a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- 172.21.15.43:0/1537236571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5054155d70 msgr2=0x7f505415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 --2- 172.21.15.43:0/1537236571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5054155d70 0x7f505415c7e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f50540af0a0 tx=0x7f504401c620 comp rx=0 tx=0).stop 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- 172.21.15.43:0/1537236571 shutdown_connections 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 --2- 172.21.15.43:0/1537236571 >> v2:172.21.15.62:3300/0 conn(0x7f505406f110 0x7f50540b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 --2- 172.21.15.43:0/1537236571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5054155d70 0x7f505415c7e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- 172.21.15.43:0/1537236571 >> 172.21.15.43:0/1537236571 conn(0x7f50540b23d0 msgr2=0x7f50540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- 172.21.15.43:0/1537236571 shutdown_connections 2024-09-17T05:25:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- 172.21.15.43:0/1537236571 wait complete. 2024-09-17T05:25:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 Processor -- start 2024-09-17T05:25:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.116+0000 7f5059200640 1 -- start start 2024-09-17T05:25:23.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50540b3920 0x7f50540afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5054155d70 0x7f50540b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5054168a50 con 0x7f505406f110 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f50541688d0 con 0x7f50540b3920 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5054168bd0 con 0x7f5054155d70 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51330/0 (socket says 172.21.15.43:51330) 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 -- 172.21.15.43:0/2595845019 learned_addr learned my addr 172.21.15.43:0/2595845019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5052800640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5054155d70 0x7f50540b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f504be00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50540b3920 0x7f50540afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 -- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5054155d70 msgr2=0x7f50540b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5054155d70 0x7f50540b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 -- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50540b3920 msgr2=0x7f50540afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50540b3920 0x7f50540afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 -- 172.21.15.43:0/2595845019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50541f71d0 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5051e00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5044009850 tx=0x7f50440a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50440ac070 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5044004320 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50440053a0 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50541f74b0 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541f7a70 con 0x7f505406f110 2024-09-17T05:25:23.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50540b33e0 con 0x7f505406f110 2024-09-17T05:25:23.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.120+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50440af070 con 0x7f505406f110 2024-09-17T05:25:23.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.124+0000 7f504a000640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 0x7f5030075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.124+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f504412e2a0 con 0x7f505406f110 2024-09-17T05:25:23.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.124+0000 7f504be00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 0x7f5030075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.124+0000 7f504be00640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 0x7f5030075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f503c006fd0 tx=0x7f503c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.128+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50540b3d30 con 0x7f505406f110 2024-09-17T05:25:23.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f50540d1150 con 0x7f505406f110 2024-09-17T05:25:23.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f504a000640 1 -- 172.21.15.43:0/2595845019 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1302 (secure 0 0 0) 0x7f50540d1150 con 0x7f505406f110 2024-09-17T05:25:23.754 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:23.754 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550593,"num_in_osds":8,"osd_in_since":1726550582,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592196,"bytes_used":248246272,"bytes_avail":772845867008,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":30,"btime":"2024-09-17T05:25:20:838273+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5439}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T05:25:05.864738+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:25:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 msgr2=0x7f5030075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 0x7f5030075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f503c006fd0 tx=0x7f503c008040 comp rx=0 tx=0).stop 2024-09-17T05:25:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 msgr2=0x7f505409cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5044009850 tx=0x7f50440a6580 comp rx=0 tx=0).stop 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 shutdown_connections 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5030073200 0x7f5030075710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50540b3920 0x7f50540afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5054155d70 0x7f50540b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 --2- 172.21.15.43:0/2595845019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f505406f110 0x7f505409cac0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 >> 172.21.15.43:0/2595845019 conn(0x7f50540b23d0 msgr2=0x7f50540ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 shutdown_connections 2024-09-17T05:25:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.364+0000 7f5059200640 1 -- 172.21.15.43:0/2595845019 wait complete. 2024-09-17T05:25:23.756 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:25:23.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 -- 172.21.15.43:0/4177955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 msgr2=0x7efe041609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 --2- 172.21.15.43:0/4177955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe041609c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efe040d23a0 tx=0x7efdfc01cac0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 -- 172.21.15.43:0/4177955196 shutdown_connections 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 --2- 172.21.15.43:0/4177955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe041609c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 --2- 172.21.15.43:0/4177955196 >> v2:172.21.15.62:3300/0 conn(0x7efe040d0f50 0x7efe040cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.912+0000 7efe08c00640 1 -- 172.21.15.43:0/4177955196 >> 172.21.15.43:0/4177955196 conn(0x7efe040cee80 msgr2=0x7efe040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- 172.21.15.43:0/4177955196 shutdown_connections 2024-09-17T05:25:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- 172.21.15.43:0/4177955196 wait complete. 2024-09-17T05:25:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 Processor -- start 2024-09-17T05:25:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- start start 2024-09-17T05:25:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe040b7ce0 0x7efe040b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe040d0f50 0x7efe0415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe0416a020 con 0x7efe040cd3b0 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe04169ea0 con 0x7efe040d0f50 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.916+0000 7efe08c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efe0416a1a0 con 0x7efe040b7ce0 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54588/0 (socket says 172.21.15.43:54588) 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 -- 172.21.15.43:0/1981546157 learned_addr learned my addr 172.21.15.43:0/1981546157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe02800640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe040d0f50 0x7efe0415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01e00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe040b7ce0 0x7efe040b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 -- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe040b7ce0 msgr2=0x7efe040b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe040b7ce0 0x7efe040b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 -- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe040d0f50 msgr2=0x7efe0415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe040d0f50 0x7efe0415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 -- 172.21.15.43:0/1981546157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041fc970 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe01400640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c970 tx=0x7efdf000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0007bb0 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdf0007e40 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000f6b0 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041fcc50 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0415d5d0 con 0x7efe040cd3b0 2024-09-17T05:25:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe041fc6e0 con 0x7efe040cd3b0 2024-09-17T05:25:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.920+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efdf000f850 con 0x7efe040cd3b0 2024-09-17T05:25:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.924+0000 7efdfa000640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 0x7efde4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.924+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7efdf0094970 con 0x7efe040cd3b0 2024-09-17T05:25:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.924+0000 7efe01e00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 0x7efde4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.924+0000 7efe01e00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 0x7efde4075710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006fd0 tx=0x7efdf4009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:23.928+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efdf0014030 con 0x7efe040cd3b0 2024-09-17T05:25:24.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.104+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe041fd170 con 0x7efde4073200 2024-09-17T05:25:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.108+0000 7efdfa000640 1 -- 172.21.15.43:0/1981546157 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7efdf4009550 con 0x7efde4073200 2024-09-17T05:25:24.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 msgr2=0x7efde4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 0x7efde4075710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efdf4006fd0 tx=0x7efdf4009060 comp rx=0 tx=0).stop 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 msgr2=0x7efe0415ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c970 tx=0x7efdf000ce30 comp rx=0 tx=0).stop 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 shutdown_connections 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efde4073200 0x7efde4075710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe040d0f50 0x7efe0415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe040cd3b0 0x7efe0415ee30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 --2- 172.21.15.43:0/1981546157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe040b7ce0 0x7efe040b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 >> 172.21.15.43:0/1981546157 conn(0x7efe040cee80 msgr2=0x7efe040af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 shutdown_connections 2024-09-17T05:25:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:24.112+0000 7efe08c00640 1 -- 172.21.15.43:0/1981546157 wait complete. 2024-09-17T05:25:24.137 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-17T05:25:23.868855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242520,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732200,"statfs":{"total":773094113280,"available":772845772800,"internally_reserved":0,"allocated":5611520,"data_stored":2957044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.054391"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834551+0000","last_change":"2024-09-17T05:25:17.830654+0000","last_active":"2024-09-17T05:25:18.834551+0000","last_peered":"2024-09-17T05:25:18.834551+0000","last_clean":"2024-09-17T05:25:18.834551+0000","last_became_active":"2024-09-17T05:25:17.830321+0000","last_became_peered":"2024-09-17T05:25:17.830321+0000","last_unstale":"2024-09-17T05:25:18.834551+0000","last_undegraded":"2024-09-17T05:25:18.834551+0000","last_fullsized":"2024-09-17T05:25:18.834551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:24.011997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974636+0000","last_change":"2024-09-17T05:25:17.844139+0000","last_active":"2024-09-17T05:25:19.974636+0000","last_peered":"2024-09-17T05:25:19.974636+0000","last_clean":"2024-09-17T05:25:19.974636+0000","last_became_active":"2024-09-17T05:25:17.843821+0000","last_became_peered":"2024-09-17T05:25:17.843821+0000","last_unstale":"2024-09-17T05:25:19.974636+0000","last_undegraded":"2024-09-17T05:25:19.974636+0000","last_fullsized":"2024-09-17T05:25:19.974636+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835616+0000","last_change":"2024-09-17T05:25:17.847168+0000","last_active":"2024-09-17T05:25:18.835616+0000","last_peered":"2024-09-17T05:25:18.835616+0000","last_clean":"2024-09-17T05:25:18.835616+0000","last_became_active":"2024-09-17T05:25:17.846513+0000","last_became_peered":"2024-09-17T05:25:17.846513+0000","last_unstale":"2024-09-17T05:25:18.835616+0000","last_undegraded":"2024-09-17T05:25:18.835616+0000","last_fullsized":"2024-09-17T05:25:18.835616+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:23.951106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836253+0000","last_change":"2024-09-17T05:25:17.845500+0000","last_active":"2024-09-17T05:25:18.836253+0000","last_peered":"2024-09-17T05:25:18.836253+0000","last_clean":"2024-09-17T05:25:18.836253+0000","last_became_active":"2024-09-17T05:25:17.845221+0000","last_became_peered":"2024-09-17T05:25:17.845221+0000","last_unstale":"2024-09-17T05:25:18.836253+0000","last_undegraded":"2024-09-17T05:25:18.836253+0000","last_fullsized":"2024-09-17T05:25:18.836253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:49.677965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833774+0000","last_change":"2024-09-17T05:25:17.858007+0000","last_active":"2024-09-17T05:25:18.833774+0000","last_peered":"2024-09-17T05:25:18.833774+0000","last_clean":"2024-09-17T05:25:18.833774+0000","last_became_active":"2024-09-17T05:25:17.855860+0000","last_became_peered":"2024-09-17T05:25:17.855860+0000","last_unstale":"2024-09-17T05:25:18.833774+0000","last_undegraded":"2024-09-17T05:25:18.833774+0000","last_fullsized":"2024-09-17T05:25:18.833774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974201+0000","last_change":"2024-09-17T05:25:17.854210+0000","last_active":"2024-09-17T05:25:19.974201+0000","last_peered":"2024-09-17T05:25:19.974201+0000","last_clean":"2024-09-17T05:25:19.974201+0000","last_became_active":"2024-09-17T05:25:17.853018+0000","last_became_peered":"2024-09-17T05:25:17.853018+0000","last_unstale":"2024-09-17T05:25:19.974201+0000","last_undegraded":"2024-09-17T05:25:19.974201+0000","last_fullsized":"2024-09-17T05:25:19.974201+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:25.044106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973527+0000","last_change":"2024-09-17T05:25:17.863014+0000","last_active":"2024-09-17T05:25:19.973527+0000","last_peered":"2024-09-17T05:25:19.973527+0000","last_clean":"2024-09-17T05:25:19.973527+0000","last_became_active":"2024-09-17T05:25:17.862710+0000","last_became_peered":"2024-09-17T05:25:17.862710+0000","last_unstale":"2024-09-17T05:25:19.973527+0000","last_undegraded":"2024-09-17T05:25:19.973527+0000","last_fullsized":"2024-09-17T05:25:19.973527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:27.074967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835477+0000","last_change":"2024-09-17T05:25:17.843269+0000","last_active":"2024-09-17T05:25:18.835477+0000","last_peered":"2024-09-17T05:25:18.835477+0000","last_clean":"2024-09-17T05:25:18.835477+0000","last_became_active":"2024-09-17T05:25:17.842790+0000","last_became_peered":"2024-09-17T05:25:17.842790+0000","last_unstale":"2024-09-17T05:25:18.835477+0000","last_undegraded":"2024-09-17T05:25:18.835477+0000","last_fullsized":"2024-09-17T05:25:18.835477+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:49.922792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975288+0000","last_change":"2024-09-17T05:25:17.853479+0000","last_active":"2024-09-17T05:25:19.975288+0000","last_peered":"2024-09-17T05:25:19.975288+0000","last_clean":"2024-09-17T05:25:19.975288+0000","last_became_active":"2024-09-17T05:25:17.853163+0000","last_became_peered":"2024-09-17T05:25:17.853163+0000","last_unstale":"2024-09-17T05:25:19.975288+0000","last_undegraded":"2024-09-17T05:25:19.975288+0000","last_fullsized":"2024-09-17T05:25:19.975288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:48.306067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975778+0000","last_change":"2024-09-17T05:25:17.834657+0000","last_active":"2024-09-17T05:25:19.975778+0000","last_peered":"2024-09-17T05:25:19.975778+0000","last_clean":"2024-09-17T05:25:19.975778+0000","last_became_active":"2024-09-17T05:25:17.834339+0000","last_became_peered":"2024-09-17T05:25:17.834339+0000","last_unstale":"2024-09-17T05:25:19.975778+0000","last_undegraded":"2024-09-17T05:25:19.975778+0000","last_fullsized":"2024-09-17T05:25:19.975778+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.959803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836527+0000","last_change":"2024-09-17T05:25:17.857677+0000","last_active":"2024-09-17T05:25:18.836527+0000","last_peered":"2024-09-17T05:25:18.836527+0000","last_clean":"2024-09-17T05:25:18.836527+0000","last_became_active":"2024-09-17T05:25:17.857316+0000","last_became_peered":"2024-09-17T05:25:17.857316+0000","last_unstale":"2024-09-17T05:25:18.836527+0000","last_undegraded":"2024-09-17T05:25:18.836527+0000","last_fullsized":"2024-09-17T05:25:18.836527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:10.394231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974428+0000","last_change":"2024-09-17T05:25:17.828984+0000","last_active":"2024-09-17T05:25:19.974428+0000","last_peered":"2024-09-17T05:25:19.974428+0000","last_clean":"2024-09-17T05:25:19.974428+0000","last_became_active":"2024-09-17T05:25:17.828631+0000","last_became_peered":"2024-09-17T05:25:17.828631+0000","last_unstale":"2024-09-17T05:25:19.974428+0000","last_undegraded":"2024-09-17T05:25:19.974428+0000","last_fullsized":"2024-09-17T05:25:19.974428+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.055649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974327+0000","last_change":"2024-09-17T05:25:17.852290+0000","last_active":"2024-09-17T05:25:19.974327+0000","last_peered":"2024-09-17T05:25:19.974327+0000","last_clean":"2024-09-17T05:25:19.974327+0000","last_became_active":"2024-09-17T05:25:17.851997+0000","last_became_peered":"2024-09-17T05:25:17.851997+0000","last_unstale":"2024-09-17T05:25:19.974327+0000","last_undegraded":"2024-09-17T05:25:19.974327+0000","last_fullsized":"2024-09-17T05:25:19.974327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:02.499276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835677+0000","last_change":"2024-09-17T05:25:17.847391+0000","last_active":"2024-09-17T05:25:18.835677+0000","last_peered":"2024-09-17T05:25:18.835677+0000","last_clean":"2024-09-17T05:25:18.835677+0000","last_became_active":"2024-09-17T05:25:17.847099+0000","last_became_peered":"2024-09-17T05:25:17.847099+0000","last_unstale":"2024-09-17T05:25:18.835677+0000","last_undegraded":"2024-09-17T05:25:18.835677+0000","last_fullsized":"2024-09-17T05:25:18.835677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:50.815519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835197+0000","last_change":"2024-09-17T05:25:17.852627+0000","last_active":"2024-09-17T05:25:18.835197+0000","last_peered":"2024-09-17T05:25:18.835197+0000","last_clean":"2024-09-17T05:25:18.835197+0000","last_became_active":"2024-09-17T05:25:17.849911+0000","last_became_peered":"2024-09-17T05:25:17.849911+0000","last_unstale":"2024-09-17T05:25:18.835197+0000","last_undegraded":"2024-09-17T05:25:18.835197+0000","last_fullsized":"2024-09-17T05:25:18.835197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:24.011997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833509+0000","last_change":"2024-09-17T05:25:17.849702+0000","last_active":"2024-09-17T05:25:18.833509+0000","last_peered":"2024-09-17T05:25:18.833509+0000","last_clean":"2024-09-17T05:25:18.833509+0000","last_became_active":"2024-09-17T05:25:17.849304+0000","last_became_peered":"2024-09-17T05:25:17.849304+0000","last_unstale":"2024-09-17T05:25:18.833509+0000","last_undegraded":"2024-09-17T05:25:18.833509+0000","last_fullsized":"2024-09-17T05:25:18.833509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974714+0000","last_change":"2024-09-17T05:25:17.838654+0000","last_active":"2024-09-17T05:25:19.974714+0000","last_peered":"2024-09-17T05:25:19.974714+0000","last_clean":"2024-09-17T05:25:19.974714+0000","last_became_active":"2024-09-17T05:25:17.838259+0000","last_became_peered":"2024-09-17T05:25:17.838259+0000","last_unstale":"2024-09-17T05:25:19.974714+0000","last_undegraded":"2024-09-17T05:25:19.974714+0000","last_fullsized":"2024-09-17T05:25:19.974714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835757+0000","last_change":"2024-09-17T05:25:17.862226+0000","last_active":"2024-09-17T05:25:18.835757+0000","last_peered":"2024-09-17T05:25:18.835757+0000","last_clean":"2024-09-17T05:25:18.835757+0000","last_became_active":"2024-09-17T05:25:17.860613+0000","last_became_peered":"2024-09-17T05:25:17.860613+0000","last_unstale":"2024-09-17T05:25:18.835757+0000","last_undegraded":"2024-09-17T05:25:18.835757+0000","last_fullsized":"2024-09-17T05:25:18.835757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:03.283332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834234+0000","last_change":"2024-09-17T05:25:17.855133+0000","last_active":"2024-09-17T05:25:18.834234+0000","last_peered":"2024-09-17T05:25:18.834234+0000","last_clean":"2024-09-17T05:25:18.834234+0000","last_became_active":"2024-09-17T05:25:17.854806+0000","last_became_peered":"2024-09-17T05:25:17.854806+0000","last_unstale":"2024-09-17T05:25:18.834234+0000","last_undegraded":"2024-09-17T05:25:18.834234+0000","last_fullsized":"2024-09-17T05:25:18.834234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:10.392801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975163+0000","last_change":"2024-09-17T05:25:17.840899+0000","last_active":"2024-09-17T05:25:19.975163+0000","last_peered":"2024-09-17T05:25:19.975163+0000","last_clean":"2024-09-17T05:25:19.975163+0000","last_became_active":"2024-09-17T05:25:17.832241+0000","last_became_peered":"2024-09-17T05:25:17.832241+0000","last_unstale":"2024-09-17T05:25:19.975163+0000","last_undegraded":"2024-09-17T05:25:19.975163+0000","last_fullsized":"2024-09-17T05:25:19.975163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:42.840394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835960+0000","last_change":"2024-09-17T05:25:17.846375+0000","last_active":"2024-09-17T05:25:18.835960+0000","last_peered":"2024-09-17T05:25:18.835960+0000","last_clean":"2024-09-17T05:25:18.835960+0000","last_became_active":"2024-09-17T05:25:17.846001+0000","last_became_peered":"2024-09-17T05:25:17.846001+0000","last_unstale":"2024-09-17T05:25:18.835960+0000","last_undegraded":"2024-09-17T05:25:18.835960+0000","last_fullsized":"2024-09-17T05:25:18.835960+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.959803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973545+0000","last_change":"2024-09-17T05:25:17.849145+0000","last_active":"2024-09-17T05:25:19.973545+0000","last_peered":"2024-09-17T05:25:19.973545+0000","last_clean":"2024-09-17T05:25:19.973545+0000","last_became_active":"2024-09-17T05:25:17.848763+0000","last_became_peered":"2024-09-17T05:25:17.848763+0000","last_unstale":"2024-09-17T05:25:19.973545+0000","last_undegraded":"2024-09-17T05:25:19.973545+0000","last_fullsized":"2024-09-17T05:25:19.973545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:49.180399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973316+0000","last_change":"2024-09-17T05:25:17.839774+0000","last_active":"2024-09-17T05:25:19.973316+0000","last_peered":"2024-09-17T05:25:19.973316+0000","last_clean":"2024-09-17T05:25:19.973316+0000","last_became_active":"2024-09-17T05:25:17.839418+0000","last_became_peered":"2024-09-17T05:25:17.839418+0000","last_unstale":"2024-09-17T05:25:19.973316+0000","last_undegraded":"2024-09-17T05:25:19.973316+0000","last_fullsized":"2024-09-17T05:25:19.973316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:46.886790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973068+0000","last_change":"2024-09-17T05:25:17.860865+0000","last_active":"2024-09-17T05:25:19.973068+0000","last_peered":"2024-09-17T05:25:19.973068+0000","last_clean":"2024-09-17T05:25:19.973068+0000","last_became_active":"2024-09-17T05:25:17.857489+0000","last_became_peered":"2024-09-17T05:25:17.857489+0000","last_unstale":"2024-09-17T05:25:19.973068+0000","last_undegraded":"2024-09-17T05:25:19.973068+0000","last_fullsized":"2024-09-17T05:25:19.973068+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:18.658668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834591+0000","last_change":"2024-09-17T05:25:17.827002+0000","last_active":"2024-09-17T05:25:18.834591+0000","last_peered":"2024-09-17T05:25:18.834591+0000","last_clean":"2024-09-17T05:25:18.834591+0000","last_became_active":"2024-09-17T05:25:17.826648+0000","last_became_peered":"2024-09-17T05:25:17.826648+0000","last_unstale":"2024-09-17T05:25:18.834591+0000","last_undegraded":"2024-09-17T05:25:18.834591+0000","last_fullsized":"2024-09-17T05:25:18.834591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836456+0000","last_change":"2024-09-17T05:25:17.855977+0000","last_active":"2024-09-17T05:25:18.836456+0000","last_peered":"2024-09-17T05:25:18.836456+0000","last_clean":"2024-09-17T05:25:18.836456+0000","last_became_active":"2024-09-17T05:25:17.855650+0000","last_became_peered":"2024-09-17T05:25:17.855650+0000","last_unstale":"2024-09-17T05:25:18.836456+0000","last_undegraded":"2024-09-17T05:25:18.836456+0000","last_fullsized":"2024-09-17T05:25:18.836456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:21.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973589+0000","last_change":"2024-09-17T05:25:17.860605+0000","last_active":"2024-09-17T05:25:19.973589+0000","last_peered":"2024-09-17T05:25:19.973589+0000","last_clean":"2024-09-17T05:25:19.973589+0000","last_became_active":"2024-09-17T05:25:17.859963+0000","last_became_peered":"2024-09-17T05:25:17.859963+0000","last_unstale":"2024-09-17T05:25:19.973589+0000","last_undegraded":"2024-09-17T05:25:19.973589+0000","last_fullsized":"2024-09-17T05:25:19.973589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:31.673127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973771+0000","last_change":"2024-09-17T05:25:17.841362+0000","last_active":"2024-09-17T05:25:19.973771+0000","last_peered":"2024-09-17T05:25:19.973771+0000","last_clean":"2024-09-17T05:25:19.973771+0000","last_became_active":"2024-09-17T05:25:17.840961+0000","last_became_peered":"2024-09-17T05:25:17.840961+0000","last_unstale":"2024-09-17T05:25:19.973771+0000","last_undegraded":"2024-09-17T05:25:19.973771+0000","last_fullsized":"2024-09-17T05:25:19.973771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:38.352016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834121+0000","last_change":"2024-09-17T05:25:17.829627+0000","last_active":"2024-09-17T05:25:18.834121+0000","last_peered":"2024-09-17T05:25:18.834121+0000","last_clean":"2024-09-17T05:25:18.834121+0000","last_became_active":"2024-09-17T05:25:17.829328+0000","last_became_peered":"2024-09-17T05:25:17.829328+0000","last_unstale":"2024-09-17T05:25:18.834121+0000","last_undegraded":"2024-09-17T05:25:18.834121+0000","last_fullsized":"2024-09-17T05:25:18.834121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973692+0000","last_change":"2024-09-17T05:25:17.855557+0000","last_active":"2024-09-17T05:25:19.973692+0000","last_peered":"2024-09-17T05:25:19.973692+0000","last_clean":"2024-09-17T05:25:19.973692+0000","last_became_active":"2024-09-17T05:25:17.854842+0000","last_became_peered":"2024-09-17T05:25:17.854842+0000","last_unstale":"2024-09-17T05:25:19.973692+0000","last_undegraded":"2024-09-17T05:25:19.973692+0000","last_fullsized":"2024-09-17T05:25:19.973692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:35.791860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973401+0000","last_change":"2024-09-17T05:25:17.847426+0000","last_active":"2024-09-17T05:25:19.973401+0000","last_peered":"2024-09-17T05:25:19.973401+0000","last_clean":"2024-09-17T05:25:19.973401+0000","last_became_active":"2024-09-17T05:25:17.847102+0000","last_became_peered":"2024-09-17T05:25:17.847102+0000","last_unstale":"2024-09-17T05:25:19.973401+0000","last_undegraded":"2024-09-17T05:25:19.973401+0000","last_fullsized":"2024-09-17T05:25:19.973401+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:42.964986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835337+0000","last_change":"2024-09-17T05:25:17.851832+0000","last_active":"2024-09-17T05:25:18.835337+0000","last_peered":"2024-09-17T05:25:18.835337+0000","last_clean":"2024-09-17T05:25:18.835337+0000","last_became_active":"2024-09-17T05:25:17.851503+0000","last_became_peered":"2024-09-17T05:25:17.851503+0000","last_unstale":"2024-09-17T05:25:18.835337+0000","last_undegraded":"2024-09-17T05:25:18.835337+0000","last_fullsized":"2024-09-17T05:25:18.835337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974341+0000","last_change":"2024-09-17T05:25:17.845069+0000","last_active":"2024-09-17T05:25:19.974341+0000","last_peered":"2024-09-17T05:25:19.974341+0000","last_clean":"2024-09-17T05:25:19.974341+0000","last_became_active":"2024-09-17T05:25:17.844766+0000","last_became_peered":"2024-09-17T05:25:17.844766+0000","last_unstale":"2024-09-17T05:25:19.974341+0000","last_undegraded":"2024-09-17T05:25:19.974341+0000","last_fullsized":"2024-09-17T05:25:19.974341+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:57.821005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975277+0000","last_change":"2024-09-17T05:25:16.830007+0000","last_active":"2024-09-17T05:25:19.975277+0000","last_peered":"2024-09-17T05:25:19.975277+0000","last_clean":"2024-09-17T05:25:19.975277+0000","last_became_active":"2024-09-17T05:25:16.828838+0000","last_became_peered":"2024-09-17T05:25:16.828838+0000","last_unstale":"2024-09-17T05:25:19.975277+0000","last_undegraded":"2024-09-17T05:25:19.975277+0000","last_fullsized":"2024-09-17T05:25:19.975277+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:49.011723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834768+0000","last_change":"2024-09-17T05:25:17.855463+0000","last_active":"2024-09-17T05:25:18.834768+0000","last_peered":"2024-09-17T05:25:18.834768+0000","last_clean":"2024-09-17T05:25:18.834768+0000","last_became_active":"2024-09-17T05:25:17.855089+0000","last_became_peered":"2024-09-17T05:25:17.855089+0000","last_unstale":"2024-09-17T05:25:18.834768+0000","last_undegraded":"2024-09-17T05:25:18.834768+0000","last_fullsized":"2024-09-17T05:25:18.834768+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973524+0000","last_change":"2024-09-17T05:25:16.841386+0000","last_active":"2024-09-17T05:25:19.973524+0000","last_peered":"2024-09-17T05:25:19.973524+0000","last_clean":"2024-09-17T05:25:19.973524+0000","last_became_active":"2024-09-17T05:25:16.840942+0000","last_became_peered":"2024-09-17T05:25:16.840942+0000","last_unstale":"2024-09-17T05:25:19.973524+0000","last_undegraded":"2024-09-17T05:25:19.973524+0000","last_fullsized":"2024-09-17T05:25:19.973524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:09.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974471+0000","last_change":"2024-09-17T05:25:17.844192+0000","last_active":"2024-09-17T05:25:19.974471+0000","last_peered":"2024-09-17T05:25:19.974471+0000","last_clean":"2024-09-17T05:25:19.974471+0000","last_became_active":"2024-09-17T05:25:17.843753+0000","last_became_peered":"2024-09-17T05:25:17.843753+0000","last_unstale":"2024-09-17T05:25:19.974471+0000","last_undegraded":"2024-09-17T05:25:19.974471+0000","last_fullsized":"2024-09-17T05:25:19.974471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:20.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974558+0000","last_change":"2024-09-17T05:25:16.835803+0000","last_active":"2024-09-17T05:25:19.974558+0000","last_peered":"2024-09-17T05:25:19.974558+0000","last_clean":"2024-09-17T05:25:19.974558+0000","last_became_active":"2024-09-17T05:25:16.835332+0000","last_became_peered":"2024-09-17T05:25:16.835332+0000","last_unstale":"2024-09-17T05:25:19.974558+0000","last_undegraded":"2024-09-17T05:25:19.974558+0000","last_fullsized":"2024-09-17T05:25:19.974558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:35.908542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836561+0000","last_change":"2024-09-17T05:25:17.853747+0000","last_active":"2024-09-17T05:25:18.836561+0000","last_peered":"2024-09-17T05:25:18.836561+0000","last_clean":"2024-09-17T05:25:18.836561+0000","last_became_active":"2024-09-17T05:25:17.853420+0000","last_became_peered":"2024-09-17T05:25:17.853420+0000","last_unstale":"2024-09-17T05:25:18.836561+0000","last_undegraded":"2024-09-17T05:25:18.836561+0000","last_fullsized":"2024-09-17T05:25:18.836561+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.926288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973950+0000","last_change":"2024-09-17T05:25:16.839690+0000","last_active":"2024-09-17T05:25:19.973950+0000","last_peered":"2024-09-17T05:25:19.973950+0000","last_clean":"2024-09-17T05:25:19.973950+0000","last_became_active":"2024-09-17T05:25:16.839208+0000","last_became_peered":"2024-09-17T05:25:16.839208+0000","last_unstale":"2024-09-17T05:25:19.973950+0000","last_undegraded":"2024-09-17T05:25:19.973950+0000","last_fullsized":"2024-09-17T05:25:19.973950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:20.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973304+0000","last_change":"2024-09-17T05:25:17.860246+0000","last_active":"2024-09-17T05:25:19.973304+0000","last_peered":"2024-09-17T05:25:19.973304+0000","last_clean":"2024-09-17T05:25:19.973304+0000","last_became_active":"2024-09-17T05:25:17.859898+0000","last_became_peered":"2024-09-17T05:25:17.859898+0000","last_unstale":"2024-09-17T05:25:19.973304+0000","last_undegraded":"2024-09-17T05:25:19.973304+0000","last_fullsized":"2024-09-17T05:25:19.973304+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:51.080816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974564+0000","last_change":"2024-09-17T05:25:16.825193+0000","last_active":"2024-09-17T05:25:19.974564+0000","last_peered":"2024-09-17T05:25:19.974564+0000","last_clean":"2024-09-17T05:25:19.974564+0000","last_became_active":"2024-09-17T05:25:16.824705+0000","last_became_peered":"2024-09-17T05:25:16.824705+0000","last_unstale":"2024-09-17T05:25:19.974564+0000","last_undegraded":"2024-09-17T05:25:19.974564+0000","last_fullsized":"2024-09-17T05:25:19.974564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:46.615285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974970+0000","last_change":"2024-09-17T05:25:17.844440+0000","last_active":"2024-09-17T05:25:19.974970+0000","last_peered":"2024-09-17T05:25:19.974970+0000","last_clean":"2024-09-17T05:25:19.974970+0000","last_became_active":"2024-09-17T05:25:17.843574+0000","last_became_peered":"2024-09-17T05:25:17.843574+0000","last_unstale":"2024-09-17T05:25:19.974970+0000","last_undegraded":"2024-09-17T05:25:19.974970+0000","last_fullsized":"2024-09-17T05:25:19.974970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:49.922792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973523+0000","last_change":"2024-09-17T05:25:16.839269+0000","last_active":"2024-09-17T05:25:19.973523+0000","last_peered":"2024-09-17T05:25:19.973523+0000","last_clean":"2024-09-17T05:25:19.973523+0000","last_became_active":"2024-09-17T05:25:16.838891+0000","last_became_peered":"2024-09-17T05:25:16.838891+0000","last_unstale":"2024-09-17T05:25:19.973523+0000","last_undegraded":"2024-09-17T05:25:19.973523+0000","last_fullsized":"2024-09-17T05:25:19.973523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.956617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973855+0000","last_change":"2024-09-17T05:25:17.840191+0000","last_active":"2024-09-17T05:25:19.973855+0000","last_peered":"2024-09-17T05:25:19.973855+0000","last_clean":"2024-09-17T05:25:19.973855+0000","last_became_active":"2024-09-17T05:25:17.839875+0000","last_became_peered":"2024-09-17T05:25:17.839875+0000","last_unstale":"2024-09-17T05:25:19.973855+0000","last_undegraded":"2024-09-17T05:25:19.973855+0000","last_fullsized":"2024-09-17T05:25:19.973855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833474+0000","last_change":"2024-09-17T05:25:16.834402+0000","last_active":"2024-09-17T05:25:18.833474+0000","last_peered":"2024-09-17T05:25:18.833474+0000","last_clean":"2024-09-17T05:25:18.833474+0000","last_became_active":"2024-09-17T05:25:16.834036+0000","last_became_peered":"2024-09-17T05:25:16.834036+0000","last_unstale":"2024-09-17T05:25:18.833474+0000","last_undegraded":"2024-09-17T05:25:18.833474+0000","last_fullsized":"2024-09-17T05:25:18.833474+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:37.453048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973036+0000","last_change":"2024-09-17T05:25:17.855835+0000","last_active":"2024-09-17T05:25:19.973036+0000","last_peered":"2024-09-17T05:25:19.973036+0000","last_clean":"2024-09-17T05:25:19.973036+0000","last_became_active":"2024-09-17T05:25:17.853327+0000","last_became_peered":"2024-09-17T05:25:17.853327+0000","last_unstale":"2024-09-17T05:25:19.973036+0000","last_undegraded":"2024-09-17T05:25:19.973036+0000","last_fullsized":"2024-09-17T05:25:19.973036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:00.732817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975025+0000","last_change":"2024-09-17T05:25:16.839214+0000","last_active":"2024-09-17T05:25:19.975025+0000","last_peered":"2024-09-17T05:25:19.975025+0000","last_clean":"2024-09-17T05:25:19.975025+0000","last_became_active":"2024-09-17T05:25:16.838825+0000","last_became_peered":"2024-09-17T05:25:16.838825+0000","last_unstale":"2024-09-17T05:25:19.975025+0000","last_undegraded":"2024-09-17T05:25:19.975025+0000","last_fullsized":"2024-09-17T05:25:19.975025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:30.086751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835775+0000","last_change":"2024-09-17T05:25:17.850811+0000","last_active":"2024-09-17T05:25:18.835775+0000","last_peered":"2024-09-17T05:25:18.835775+0000","last_clean":"2024-09-17T05:25:18.835775+0000","last_became_active":"2024-09-17T05:25:17.850480+0000","last_became_peered":"2024-09-17T05:25:17.850480+0000","last_unstale":"2024-09-17T05:25:18.835775+0000","last_undegraded":"2024-09-17T05:25:18.835775+0000","last_fullsized":"2024-09-17T05:25:18.835775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:37.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835574+0000","last_change":"2024-09-17T05:25:16.841708+0000","last_active":"2024-09-17T05:25:18.835574+0000","last_peered":"2024-09-17T05:25:18.835574+0000","last_clean":"2024-09-17T05:25:18.835574+0000","last_became_active":"2024-09-17T05:25:16.841263+0000","last_became_peered":"2024-09-17T05:25:16.841263+0000","last_unstale":"2024-09-17T05:25:18.835574+0000","last_undegraded":"2024-09-17T05:25:18.835574+0000","last_fullsized":"2024-09-17T05:25:18.835574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:21.553815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835605+0000","last_change":"2024-09-17T05:25:17.851388+0000","last_active":"2024-09-17T05:25:18.835605+0000","last_peered":"2024-09-17T05:25:18.835605+0000","last_clean":"2024-09-17T05:25:18.835605+0000","last_became_active":"2024-09-17T05:25:17.851036+0000","last_became_peered":"2024-09-17T05:25:17.851036+0000","last_unstale":"2024-09-17T05:25:18.835605+0000","last_undegraded":"2024-09-17T05:25:18.835605+0000","last_fullsized":"2024-09-17T05:25:18.835605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:11.962536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975691+0000","last_change":"2024-09-17T05:25:16.843023+0000","last_active":"2024-09-17T05:25:19.975691+0000","last_peered":"2024-09-17T05:25:19.975691+0000","last_clean":"2024-09-17T05:25:19.975691+0000","last_became_active":"2024-09-17T05:25:16.842671+0000","last_became_peered":"2024-09-17T05:25:16.842671+0000","last_unstale":"2024-09-17T05:25:19.975691+0000","last_undegraded":"2024-09-17T05:25:19.975691+0000","last_fullsized":"2024-09-17T05:25:19.975691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:40.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836306+0000","last_change":"2024-09-17T05:25:17.844156+0000","last_active":"2024-09-17T05:25:18.836306+0000","last_peered":"2024-09-17T05:25:18.836306+0000","last_clean":"2024-09-17T05:25:18.836306+0000","last_became_active":"2024-09-17T05:25:17.843870+0000","last_became_peered":"2024-09-17T05:25:17.843870+0000","last_unstale":"2024-09-17T05:25:18.836306+0000","last_undegraded":"2024-09-17T05:25:18.836306+0000","last_fullsized":"2024-09-17T05:25:18.836306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:44.575485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974447+0000","last_change":"2024-09-17T05:25:16.825444+0000","last_active":"2024-09-17T05:25:19.974447+0000","last_peered":"2024-09-17T05:25:19.974447+0000","last_clean":"2024-09-17T05:25:19.974447+0000","last_became_active":"2024-09-17T05:25:16.825122+0000","last_became_peered":"2024-09-17T05:25:16.825122+0000","last_unstale":"2024-09-17T05:25:19.974447+0000","last_undegraded":"2024-09-17T05:25:19.974447+0000","last_fullsized":"2024-09-17T05:25:19.974447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:38.050632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973843+0000","last_change":"2024-09-17T05:25:17.860864+0000","last_active":"2024-09-17T05:25:19.973843+0000","last_peered":"2024-09-17T05:25:19.973843+0000","last_clean":"2024-09-17T05:25:19.973843+0000","last_became_active":"2024-09-17T05:25:17.860380+0000","last_became_peered":"2024-09-17T05:25:17.860380+0000","last_unstale":"2024-09-17T05:25:19.973843+0000","last_undegraded":"2024-09-17T05:25:19.973843+0000","last_fullsized":"2024-09-17T05:25:19.973843+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:13.785901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975390+0000","last_change":"2024-09-17T05:25:16.845121+0000","last_active":"2024-09-17T05:25:19.975390+0000","last_peered":"2024-09-17T05:25:19.975390+0000","last_clean":"2024-09-17T05:25:19.975390+0000","last_became_active":"2024-09-17T05:25:16.844739+0000","last_became_peered":"2024-09-17T05:25:16.844739+0000","last_unstale":"2024-09-17T05:25:19.975390+0000","last_undegraded":"2024-09-17T05:25:19.975390+0000","last_fullsized":"2024-09-17T05:25:19.975390+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:26.485416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835427+0000","last_change":"2024-09-17T05:25:17.854295+0000","last_active":"2024-09-17T05:25:18.835427+0000","last_peered":"2024-09-17T05:25:18.835427+0000","last_clean":"2024-09-17T05:25:18.835427+0000","last_became_active":"2024-09-17T05:25:17.853814+0000","last_became_peered":"2024-09-17T05:25:17.853814+0000","last_unstale":"2024-09-17T05:25:18.835427+0000","last_undegraded":"2024-09-17T05:25:18.835427+0000","last_fullsized":"2024-09-17T05:25:18.835427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:12.909789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974417+0000","last_change":"2024-09-17T05:25:16.839573+0000","last_active":"2024-09-17T05:25:19.974417+0000","last_peered":"2024-09-17T05:25:19.974417+0000","last_clean":"2024-09-17T05:25:19.974417+0000","last_became_active":"2024-09-17T05:25:16.838379+0000","last_became_peered":"2024-09-17T05:25:16.838379+0000","last_unstale":"2024-09-17T05:25:19.974417+0000","last_undegraded":"2024-09-17T05:25:19.974417+0000","last_fullsized":"2024-09-17T05:25:19.974417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:00.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973436+0000","last_change":"2024-09-17T05:25:17.836620+0000","last_active":"2024-09-17T05:25:19.973436+0000","last_peered":"2024-09-17T05:25:19.973436+0000","last_clean":"2024-09-17T05:25:19.973436+0000","last_became_active":"2024-09-17T05:25:17.836304+0000","last_became_peered":"2024-09-17T05:25:17.836304+0000","last_unstale":"2024-09-17T05:25:19.973436+0000","last_undegraded":"2024-09-17T05:25:19.973436+0000","last_fullsized":"2024-09-17T05:25:19.973436+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:12.909789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833521+0000","last_change":"2024-09-17T05:25:16.837793+0000","last_active":"2024-09-17T05:25:18.833521+0000","last_peered":"2024-09-17T05:25:18.833521+0000","last_clean":"2024-09-17T05:25:18.833521+0000","last_became_active":"2024-09-17T05:25:16.837076+0000","last_became_peered":"2024-09-17T05:25:16.837076+0000","last_unstale":"2024-09-17T05:25:18.833521+0000","last_undegraded":"2024-09-17T05:25:18.833521+0000","last_fullsized":"2024-09-17T05:25:18.833521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:44.279363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973435+0000","last_change":"2024-09-17T05:25:17.846642+0000","last_active":"2024-09-17T05:25:19.973435+0000","last_peered":"2024-09-17T05:25:19.973435+0000","last_clean":"2024-09-17T05:25:19.973435+0000","last_became_active":"2024-09-17T05:25:17.844111+0000","last_became_peered":"2024-09-17T05:25:17.844111+0000","last_unstale":"2024-09-17T05:25:19.973435+0000","last_undegraded":"2024-09-17T05:25:19.973435+0000","last_fullsized":"2024-09-17T05:25:19.973435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:41.914430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973739+0000","last_change":"2024-09-17T05:25:16.842966+0000","last_active":"2024-09-17T05:25:19.973739+0000","last_peered":"2024-09-17T05:25:19.973739+0000","last_clean":"2024-09-17T05:25:19.973739+0000","last_became_active":"2024-09-17T05:25:16.842650+0000","last_became_peered":"2024-09-17T05:25:16.842650+0000","last_unstale":"2024-09-17T05:25:19.973739+0000","last_undegraded":"2024-09-17T05:25:19.973739+0000","last_fullsized":"2024-09-17T05:25:19.973739+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:33.733304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973028+0000","last_change":"2024-09-17T05:25:17.827842+0000","last_active":"2024-09-17T05:25:19.973028+0000","last_peered":"2024-09-17T05:25:19.973028+0000","last_clean":"2024-09-17T05:25:19.973028+0000","last_became_active":"2024-09-17T05:25:17.827506+0000","last_became_peered":"2024-09-17T05:25:17.827506+0000","last_unstale":"2024-09-17T05:25:19.973028+0000","last_undegraded":"2024-09-17T05:25:19.973028+0000","last_fullsized":"2024-09-17T05:25:19.973028+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:34.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974092+0000","last_change":"2024-09-17T05:25:16.835808+0000","last_active":"2024-09-17T05:25:19.974092+0000","last_peered":"2024-09-17T05:25:19.974092+0000","last_clean":"2024-09-17T05:25:19.974092+0000","last_became_active":"2024-09-17T05:25:16.835452+0000","last_became_peered":"2024-09-17T05:25:16.835452+0000","last_unstale":"2024-09-17T05:25:19.974092+0000","last_undegraded":"2024-09-17T05:25:19.974092+0000","last_fullsized":"2024-09-17T05:25:19.974092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:12.812210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974814+0000","last_change":"2024-09-17T05:25:17.855358+0000","last_active":"2024-09-17T05:25:19.974814+0000","last_peered":"2024-09-17T05:25:19.974814+0000","last_clean":"2024-09-17T05:25:19.974814+0000","last_became_active":"2024-09-17T05:25:17.854829+0000","last_became_peered":"2024-09-17T05:25:17.854829+0000","last_unstale":"2024-09-17T05:25:19.974814+0000","last_undegraded":"2024-09-17T05:25:19.974814+0000","last_fullsized":"2024-09-17T05:25:19.974814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:03.283332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836556+0000","last_change":"2024-09-17T05:25:16.842074+0000","last_active":"2024-09-17T05:25:18.836556+0000","last_peered":"2024-09-17T05:25:18.836556+0000","last_clean":"2024-09-17T05:25:18.836556+0000","last_became_active":"2024-09-17T05:25:16.841700+0000","last_became_peered":"2024-09-17T05:25:16.841700+0000","last_unstale":"2024-09-17T05:25:18.836556+0000","last_undegraded":"2024-09-17T05:25:18.836556+0000","last_fullsized":"2024-09-17T05:25:18.836556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:38.831286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974505+0000","last_change":"2024-09-17T05:25:17.843043+0000","last_active":"2024-09-17T05:25:19.974505+0000","last_peered":"2024-09-17T05:25:19.974505+0000","last_clean":"2024-09-17T05:25:19.974505+0000","last_became_active":"2024-09-17T05:25:17.842110+0000","last_became_peered":"2024-09-17T05:25:17.842110+0000","last_unstale":"2024-09-17T05:25:19.974505+0000","last_undegraded":"2024-09-17T05:25:19.974505+0000","last_fullsized":"2024-09-17T05:25:19.974505+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:11.962536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835990+0000","last_change":"2024-09-17T05:25:16.843994+0000","last_active":"2024-09-17T05:25:18.835990+0000","last_peered":"2024-09-17T05:25:18.835990+0000","last_clean":"2024-09-17T05:25:18.835990+0000","last_became_active":"2024-09-17T05:25:16.843335+0000","last_became_peered":"2024-09-17T05:25:16.843335+0000","last_unstale":"2024-09-17T05:25:18.835990+0000","last_undegraded":"2024-09-17T05:25:18.835990+0000","last_fullsized":"2024-09-17T05:25:18.835990+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:10.828864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974672+0000","last_change":"2024-09-17T05:25:17.858165+0000","last_active":"2024-09-17T05:25:19.974672+0000","last_peered":"2024-09-17T05:25:19.974672+0000","last_clean":"2024-09-17T05:25:19.974672+0000","last_became_active":"2024-09-17T05:25:17.856741+0000","last_became_peered":"2024-09-17T05:25:17.856741+0000","last_unstale":"2024-09-17T05:25:19.974672+0000","last_undegraded":"2024-09-17T05:25:19.974672+0000","last_fullsized":"2024-09-17T05:25:19.974672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836058+0000","last_change":"2024-09-17T05:25:16.842760+0000","last_active":"2024-09-17T05:25:18.836058+0000","last_peered":"2024-09-17T05:25:18.836058+0000","last_clean":"2024-09-17T05:25:18.836058+0000","last_became_active":"2024-09-17T05:25:16.842399+0000","last_became_peered":"2024-09-17T05:25:16.842399+0000","last_unstale":"2024-09-17T05:25:18.836058+0000","last_undegraded":"2024-09-17T05:25:18.836058+0000","last_fullsized":"2024-09-17T05:25:18.836058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:04.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835672+0000","last_change":"2024-09-17T05:25:17.846811+0000","last_active":"2024-09-17T05:25:18.835672+0000","last_peered":"2024-09-17T05:25:18.835672+0000","last_clean":"2024-09-17T05:25:18.835672+0000","last_became_active":"2024-09-17T05:25:17.846191+0000","last_became_peered":"2024-09-17T05:25:17.846191+0000","last_unstale":"2024-09-17T05:25:18.835672+0000","last_undegraded":"2024-09-17T05:25:18.835672+0000","last_fullsized":"2024-09-17T05:25:18.835672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:21.507732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974614+0000","last_change":"2024-09-17T05:25:16.836689+0000","last_active":"2024-09-17T05:25:19.974614+0000","last_peered":"2024-09-17T05:25:19.974614+0000","last_clean":"2024-09-17T05:25:19.974614+0000","last_became_active":"2024-09-17T05:25:16.836345+0000","last_became_peered":"2024-09-17T05:25:16.836345+0000","last_unstale":"2024-09-17T05:25:19.974614+0000","last_undegraded":"2024-09-17T05:25:19.974614+0000","last_fullsized":"2024-09-17T05:25:19.974614+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:32.420753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974896+0000","last_change":"2024-09-17T05:25:17.849888+0000","last_active":"2024-09-17T05:25:19.974896+0000","last_peered":"2024-09-17T05:25:19.974896+0000","last_clean":"2024-09-17T05:25:19.974896+0000","last_became_active":"2024-09-17T05:25:17.848605+0000","last_became_peered":"2024-09-17T05:25:17.848605+0000","last_unstale":"2024-09-17T05:25:19.974896+0000","last_undegraded":"2024-09-17T05:25:19.974896+0000","last_fullsized":"2024-09-17T05:25:19.974896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974961+0000","last_change":"2024-09-17T05:25:16.836994+0000","last_active":"2024-09-17T05:25:19.974961+0000","last_peered":"2024-09-17T05:25:19.974961+0000","last_clean":"2024-09-17T05:25:19.974961+0000","last_became_active":"2024-09-17T05:25:16.836576+0000","last_became_peered":"2024-09-17T05:25:16.836576+0000","last_unstale":"2024-09-17T05:25:19.974961+0000","last_undegraded":"2024-09-17T05:25:19.974961+0000","last_fullsized":"2024-09-17T05:25:19.974961+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:41.094836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974351+0000","last_change":"2024-09-17T05:25:17.861155+0000","last_active":"2024-09-17T05:25:19.974351+0000","last_peered":"2024-09-17T05:25:19.974351+0000","last_clean":"2024-09-17T05:25:19.974351+0000","last_became_active":"2024-09-17T05:25:17.860868+0000","last_became_peered":"2024-09-17T05:25:17.860868+0000","last_unstale":"2024-09-17T05:25:19.974351+0000","last_undegraded":"2024-09-17T05:25:19.974351+0000","last_fullsized":"2024-09-17T05:25:19.974351+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:03.657042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833682+0000","last_change":"2024-09-17T05:25:16.837411+0000","last_active":"2024-09-17T05:25:18.833682+0000","last_peered":"2024-09-17T05:25:18.833682+0000","last_clean":"2024-09-17T05:25:18.833682+0000","last_became_active":"2024-09-17T05:25:16.837058+0000","last_became_peered":"2024-09-17T05:25:16.837058+0000","last_unstale":"2024-09-17T05:25:18.833682+0000","last_undegraded":"2024-09-17T05:25:18.833682+0000","last_fullsized":"2024-09-17T05:25:18.833682+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:42.610461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835654+0000","last_change":"2024-09-17T05:25:17.850227+0000","last_active":"2024-09-17T05:25:18.835654+0000","last_peered":"2024-09-17T05:25:18.835654+0000","last_clean":"2024-09-17T05:25:18.835654+0000","last_became_active":"2024-09-17T05:25:17.849926+0000","last_became_peered":"2024-09-17T05:25:17.849926+0000","last_unstale":"2024-09-17T05:25:18.835654+0000","last_undegraded":"2024-09-17T05:25:18.835654+0000","last_fullsized":"2024-09-17T05:25:18.835654+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:08.132627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973538+0000","last_change":"2024-09-17T05:25:16.843105+0000","last_active":"2024-09-17T05:25:19.973538+0000","last_peered":"2024-09-17T05:25:19.973538+0000","last_clean":"2024-09-17T05:25:19.973538+0000","last_became_active":"2024-09-17T05:25:16.842668+0000","last_became_peered":"2024-09-17T05:25:16.842668+0000","last_unstale":"2024-09-17T05:25:19.973538+0000","last_undegraded":"2024-09-17T05:25:19.973538+0000","last_fullsized":"2024-09-17T05:25:19.973538+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835465+0000","last_change":"2024-09-17T05:25:17.847878+0000","last_active":"2024-09-17T05:25:18.835465+0000","last_peered":"2024-09-17T05:25:18.835465+0000","last_clean":"2024-09-17T05:25:18.835465+0000","last_became_active":"2024-09-17T05:25:17.844937+0000","last_became_peered":"2024-09-17T05:25:17.844937+0000","last_unstale":"2024-09-17T05:25:18.835465+0000","last_undegraded":"2024-09-17T05:25:18.835465+0000","last_fullsized":"2024-09-17T05:25:18.835465+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.972889+0000","last_change":"2024-09-17T05:25:16.839427+0000","last_active":"2024-09-17T05:25:19.972889+0000","last_peered":"2024-09-17T05:25:19.972889+0000","last_clean":"2024-09-17T05:25:19.972889+0000","last_became_active":"2024-09-17T05:25:16.838642+0000","last_became_peered":"2024-09-17T05:25:16.838642+0000","last_unstale":"2024-09-17T05:25:19.972889+0000","last_undegraded":"2024-09-17T05:25:19.972889+0000","last_fullsized":"2024-09-17T05:25:19.972889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.129441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973455+0000","last_change":"2024-09-17T05:25:17.854279+0000","last_active":"2024-09-17T05:25:19.973455+0000","last_peered":"2024-09-17T05:25:19.973455+0000","last_clean":"2024-09-17T05:25:19.973455+0000","last_became_active":"2024-09-17T05:25:17.853347+0000","last_became_peered":"2024-09-17T05:25:17.853347+0000","last_unstale":"2024-09-17T05:25:19.973455+0000","last_undegraded":"2024-09-17T05:25:19.973455+0000","last_fullsized":"2024-09-17T05:25:19.973455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:52.026834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836677+0000","last_change":"2024-09-17T05:25:16.845000+0000","last_active":"2024-09-17T05:25:18.836677+0000","last_peered":"2024-09-17T05:25:18.836677+0000","last_clean":"2024-09-17T05:25:18.836677+0000","last_became_active":"2024-09-17T05:25:16.844264+0000","last_became_peered":"2024-09-17T05:25:16.844264+0000","last_unstale":"2024-09-17T05:25:18.836677+0000","last_undegraded":"2024-09-17T05:25:18.836677+0000","last_fullsized":"2024-09-17T05:25:18.836677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:10.422465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975224+0000","last_change":"2024-09-17T05:25:17.848021+0000","last_active":"2024-09-17T05:25:19.975224+0000","last_peered":"2024-09-17T05:25:19.975224+0000","last_clean":"2024-09-17T05:25:19.975224+0000","last_became_active":"2024-09-17T05:25:17.847264+0000","last_became_peered":"2024-09-17T05:25:17.847264+0000","last_unstale":"2024-09-17T05:25:19.975224+0000","last_undegraded":"2024-09-17T05:25:19.975224+0000","last_fullsized":"2024-09-17T05:25:19.975224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:08.132627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834237+0000","last_change":"2024-09-17T05:25:16.838187+0000","last_active":"2024-09-17T05:25:18.834237+0000","last_peered":"2024-09-17T05:25:18.834237+0000","last_clean":"2024-09-17T05:25:18.834237+0000","last_became_active":"2024-09-17T05:25:16.837770+0000","last_became_peered":"2024-09-17T05:25:16.837770+0000","last_unstale":"2024-09-17T05:25:18.834237+0000","last_undegraded":"2024-09-17T05:25:18.834237+0000","last_fullsized":"2024-09-17T05:25:18.834237+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973951+0000","last_change":"2024-09-17T05:25:17.860851+0000","last_active":"2024-09-17T05:25:19.973951+0000","last_peered":"2024-09-17T05:25:19.973951+0000","last_clean":"2024-09-17T05:25:19.973951+0000","last_became_active":"2024-09-17T05:25:17.860262+0000","last_became_peered":"2024-09-17T05:25:17.860262+0000","last_unstale":"2024-09-17T05:25:19.973951+0000","last_undegraded":"2024-09-17T05:25:19.973951+0000","last_fullsized":"2024-09-17T05:25:19.973951+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:20.085739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974300+0000","last_change":"2024-09-17T05:25:16.841538+0000","last_active":"2024-09-17T05:25:19.974300+0000","last_peered":"2024-09-17T05:25:19.974300+0000","last_clean":"2024-09-17T05:25:19.974300+0000","last_became_active":"2024-09-17T05:25:16.840839+0000","last_became_peered":"2024-09-17T05:25:16.840839+0000","last_unstale":"2024-09-17T05:25:19.974300+0000","last_undegraded":"2024-09-17T05:25:19.974300+0000","last_fullsized":"2024-09-17T05:25:19.974300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:35.655115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975174+0000","last_change":"2024-09-17T05:25:17.849451+0000","last_active":"2024-09-17T05:25:19.975174+0000","last_peered":"2024-09-17T05:25:19.975174+0000","last_clean":"2024-09-17T05:25:19.975174+0000","last_became_active":"2024-09-17T05:25:17.848905+0000","last_became_peered":"2024-09-17T05:25:17.848905+0000","last_unstale":"2024-09-17T05:25:19.975174+0000","last_undegraded":"2024-09-17T05:25:19.975174+0000","last_fullsized":"2024-09-17T05:25:19.975174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:30.495528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834011+0000","last_change":"2024-09-17T05:25:16.841637+0000","last_active":"2024-09-17T05:25:18.834011+0000","last_peered":"2024-09-17T05:25:18.834011+0000","last_clean":"2024-09-17T05:25:18.834011+0000","last_became_active":"2024-09-17T05:25:16.840417+0000","last_became_peered":"2024-09-17T05:25:16.840417+0000","last_unstale":"2024-09-17T05:25:18.834011+0000","last_undegraded":"2024-09-17T05:25:18.834011+0000","last_fullsized":"2024-09-17T05:25:18.834011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:36.939696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835591+0000","last_change":"2024-09-17T05:25:17.858282+0000","last_active":"2024-09-17T05:25:18.835591+0000","last_peered":"2024-09-17T05:25:18.835591+0000","last_clean":"2024-09-17T05:25:18.835591+0000","last_became_active":"2024-09-17T05:25:17.856926+0000","last_became_peered":"2024-09-17T05:25:17.856926+0000","last_unstale":"2024-09-17T05:25:18.835591+0000","last_undegraded":"2024-09-17T05:25:18.835591+0000","last_fullsized":"2024-09-17T05:25:18.835591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:30.495528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973627+0000","last_change":"2024-09-17T05:25:16.843421+0000","last_active":"2024-09-17T05:25:19.973627+0000","last_peered":"2024-09-17T05:25:19.973627+0000","last_clean":"2024-09-17T05:25:19.973627+0000","last_became_active":"2024-09-17T05:25:16.843071+0000","last_became_peered":"2024-09-17T05:25:16.843071+0000","last_unstale":"2024-09-17T05:25:19.973627+0000","last_undegraded":"2024-09-17T05:25:19.973627+0000","last_fullsized":"2024-09-17T05:25:19.973627+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834887+0000","last_change":"2024-09-17T05:25:17.851994+0000","last_active":"2024-09-17T05:25:18.834887+0000","last_peered":"2024-09-17T05:25:18.834887+0000","last_clean":"2024-09-17T05:25:18.834887+0000","last_became_active":"2024-09-17T05:25:17.851553+0000","last_became_peered":"2024-09-17T05:25:17.851553+0000","last_unstale":"2024-09-17T05:25:18.834887+0000","last_undegraded":"2024-09-17T05:25:18.834887+0000","last_fullsized":"2024-09-17T05:25:18.834887+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:49.180399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973876+0000","last_change":"2024-09-17T05:25:16.838372+0000","last_active":"2024-09-17T05:25:19.973876+0000","last_peered":"2024-09-17T05:25:19.973876+0000","last_clean":"2024-09-17T05:25:19.973876+0000","last_became_active":"2024-09-17T05:25:16.837915+0000","last_became_peered":"2024-09-17T05:25:16.837915+0000","last_unstale":"2024-09-17T05:25:19.973876+0000","last_undegraded":"2024-09-17T05:25:19.973876+0000","last_fullsized":"2024-09-17T05:25:19.973876+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:52.416005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973650+0000","last_change":"2024-09-17T05:25:17.857155+0000","last_active":"2024-09-17T05:25:19.973650+0000","last_peered":"2024-09-17T05:25:19.973650+0000","last_clean":"2024-09-17T05:25:19.973650+0000","last_became_active":"2024-09-17T05:25:17.855676+0000","last_became_peered":"2024-09-17T05:25:17.855676+0000","last_unstale":"2024-09-17T05:25:19.973650+0000","last_undegraded":"2024-09-17T05:25:19.973650+0000","last_fullsized":"2024-09-17T05:25:19.973650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:11.788487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973690+0000","last_change":"2024-09-17T05:25:16.844953+0000","last_active":"2024-09-17T05:25:19.973690+0000","last_peered":"2024-09-17T05:25:19.973690+0000","last_clean":"2024-09-17T05:25:19.973690+0000","last_became_active":"2024-09-17T05:25:16.843607+0000","last_became_peered":"2024-09-17T05:25:16.843607+0000","last_unstale":"2024-09-17T05:25:19.973690+0000","last_undegraded":"2024-09-17T05:25:19.973690+0000","last_fullsized":"2024-09-17T05:25:19.973690+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.778959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836561+0000","last_change":"2024-09-17T05:25:17.843412+0000","last_active":"2024-09-17T05:25:18.836561+0000","last_peered":"2024-09-17T05:25:18.836561+0000","last_clean":"2024-09-17T05:25:18.836561+0000","last_became_active":"2024-09-17T05:25:17.842545+0000","last_became_peered":"2024-09-17T05:25:17.842545+0000","last_unstale":"2024-09-17T05:25:18.836561+0000","last_undegraded":"2024-09-17T05:25:18.836561+0000","last_fullsized":"2024-09-17T05:25:18.836561+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:59.137139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973423+0000","last_change":"2024-09-17T05:25:16.842097+0000","last_active":"2024-09-17T05:25:19.973423+0000","last_peered":"2024-09-17T05:25:19.973423+0000","last_clean":"2024-09-17T05:25:19.973423+0000","last_became_active":"2024-09-17T05:25:16.841570+0000","last_became_peered":"2024-09-17T05:25:16.841570+0000","last_unstale":"2024-09-17T05:25:19.973423+0000","last_undegraded":"2024-09-17T05:25:19.973423+0000","last_fullsized":"2024-09-17T05:25:19.973423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834969+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:18.834969+0000","last_peered":"2024-09-17T05:25:18.834969+0000","last_clean":"2024-09-17T05:25:18.834969+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:18.834969+0000","last_undegraded":"2024-09-17T05:25:18.834969+0000","last_fullsized":"2024-09-17T05:25:18.834969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":221470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1142784,"data_stored":811826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":223140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":223048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":811874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":222108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":552960,"data_stored":222052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:27.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 -- 172.21.15.43:0/2245620386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 msgr2=0x7fc4d40cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2245620386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d40cddb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009c80 tx=0x7fc4cc01cd10 comp rx=0 tx=0).stop 2024-09-17T05:25:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 -- 172.21.15.43:0/2245620386 shutdown_connections 2024-09-17T05:25:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2245620386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d40cddb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2245620386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b2330 0x7fc4d40b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 -- 172.21.15.43:0/2245620386 >> 172.21.15.43:0/2245620386 conn(0x7fc4d40b2e80 msgr2=0x7fc4d40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 -- 172.21.15.43:0/2245620386 shutdown_connections 2024-09-17T05:25:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.308+0000 7fc4dc600640 1 -- 172.21.15.43:0/2245620386 wait complete. 2024-09-17T05:25:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 Processor -- start 2024-09-17T05:25:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 -- start start 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4d40b2330 0x7fc4d415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b6b10 0x7fc4d4154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d416a060 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4169ee0 con 0x7fc4d40b2330 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4dc600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d416a1e0 con 0x7fc4d40b6b10 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4da600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4da600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54606/0 (socket says 172.21.15.43:54606) 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4da600640 1 -- 172.21.15.43:0/2424650061 learned_addr learned my addr 172.21.15.43:0/2424650061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.312+0000 7fc4d9200640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b6b10 0x7fc4d4154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4d9c00640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4d40b2330 0x7fc4d415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 -- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b6b10 msgr2=0x7fc4d4154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b6b10 0x7fc4d4154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 -- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4d40b2330 msgr2=0x7fc4d415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4d40b2330 0x7fc4d415c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 -- 172.21.15.43:0/2424650061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d415cf60 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4da600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc098920 tx=0x7fc4cc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0b2040 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4cc0bb070 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc004530 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d415d240 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d415d800 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d415ccd0 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.316+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4cc0046d0 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.320+0000 7fc4d1600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 0x7fc4b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.320+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc4cc12eb80 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.320+0000 7fc4d9c00640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 0x7fc4b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.320+0000 7fc4d9c00640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 0x7fc4b4075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8000f70 tx=0x7fc4c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:27.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.324+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4cc016990 con 0x7fc4d40cd9a0 2024-09-17T05:25:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.496+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4d406dd50 con 0x7fc4b4073200 2024-09-17T05:25:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.500+0000 7fc4d1600640 1 -- 172.21.15.43:0/2424650061 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fc4d406dd50 con 0x7fc4b4073200 2024-09-17T05:25:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 msgr2=0x7fc4b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 0x7fc4b4075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8000f70 tx=0x7fc4c8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 msgr2=0x7fc4d41fa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc098920 tx=0x7fc4cc005ae0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 shutdown_connections 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4b4073200 0x7fc4b4075710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4d40cd9a0 0x7fc4d41fa3f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4d40b6b10 0x7fc4d4154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 --2- 172.21.15.43:0/2424650061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4d40b2330 0x7fc4d415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 >> 172.21.15.43:0/2424650061 conn(0x7fc4d40b2e80 msgr2=0x7fc4d40b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 shutdown_connections 2024-09-17T05:25:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.504+0000 7fc4dc600640 1 -- 172.21.15.43:0/2424650061 wait complete. 2024-09-17T05:25:27.530 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-17T05:25:25.869439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242520,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732200,"statfs":{"total":773094113280,"available":772845772800,"internally_reserved":0,"allocated":5611520,"data_stored":2957044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.052267"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834551+0000","last_change":"2024-09-17T05:25:17.830654+0000","last_active":"2024-09-17T05:25:18.834551+0000","last_peered":"2024-09-17T05:25:18.834551+0000","last_clean":"2024-09-17T05:25:18.834551+0000","last_became_active":"2024-09-17T05:25:17.830321+0000","last_became_peered":"2024-09-17T05:25:17.830321+0000","last_unstale":"2024-09-17T05:25:18.834551+0000","last_undegraded":"2024-09-17T05:25:18.834551+0000","last_fullsized":"2024-09-17T05:25:18.834551+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:24.011997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974636+0000","last_change":"2024-09-17T05:25:17.844139+0000","last_active":"2024-09-17T05:25:19.974636+0000","last_peered":"2024-09-17T05:25:19.974636+0000","last_clean":"2024-09-17T05:25:19.974636+0000","last_became_active":"2024-09-17T05:25:17.843821+0000","last_became_peered":"2024-09-17T05:25:17.843821+0000","last_unstale":"2024-09-17T05:25:19.974636+0000","last_undegraded":"2024-09-17T05:25:19.974636+0000","last_fullsized":"2024-09-17T05:25:19.974636+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835616+0000","last_change":"2024-09-17T05:25:17.847168+0000","last_active":"2024-09-17T05:25:18.835616+0000","last_peered":"2024-09-17T05:25:18.835616+0000","last_clean":"2024-09-17T05:25:18.835616+0000","last_became_active":"2024-09-17T05:25:17.846513+0000","last_became_peered":"2024-09-17T05:25:17.846513+0000","last_unstale":"2024-09-17T05:25:18.835616+0000","last_undegraded":"2024-09-17T05:25:18.835616+0000","last_fullsized":"2024-09-17T05:25:18.835616+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:23.951106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836253+0000","last_change":"2024-09-17T05:25:17.845500+0000","last_active":"2024-09-17T05:25:18.836253+0000","last_peered":"2024-09-17T05:25:18.836253+0000","last_clean":"2024-09-17T05:25:18.836253+0000","last_became_active":"2024-09-17T05:25:17.845221+0000","last_became_peered":"2024-09-17T05:25:17.845221+0000","last_unstale":"2024-09-17T05:25:18.836253+0000","last_undegraded":"2024-09-17T05:25:18.836253+0000","last_fullsized":"2024-09-17T05:25:18.836253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:49.677965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833774+0000","last_change":"2024-09-17T05:25:17.858007+0000","last_active":"2024-09-17T05:25:18.833774+0000","last_peered":"2024-09-17T05:25:18.833774+0000","last_clean":"2024-09-17T05:25:18.833774+0000","last_became_active":"2024-09-17T05:25:17.855860+0000","last_became_peered":"2024-09-17T05:25:17.855860+0000","last_unstale":"2024-09-17T05:25:18.833774+0000","last_undegraded":"2024-09-17T05:25:18.833774+0000","last_fullsized":"2024-09-17T05:25:18.833774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974201+0000","last_change":"2024-09-17T05:25:17.854210+0000","last_active":"2024-09-17T05:25:19.974201+0000","last_peered":"2024-09-17T05:25:19.974201+0000","last_clean":"2024-09-17T05:25:19.974201+0000","last_became_active":"2024-09-17T05:25:17.853018+0000","last_became_peered":"2024-09-17T05:25:17.853018+0000","last_unstale":"2024-09-17T05:25:19.974201+0000","last_undegraded":"2024-09-17T05:25:19.974201+0000","last_fullsized":"2024-09-17T05:25:19.974201+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:25.044106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973527+0000","last_change":"2024-09-17T05:25:17.863014+0000","last_active":"2024-09-17T05:25:19.973527+0000","last_peered":"2024-09-17T05:25:19.973527+0000","last_clean":"2024-09-17T05:25:19.973527+0000","last_became_active":"2024-09-17T05:25:17.862710+0000","last_became_peered":"2024-09-17T05:25:17.862710+0000","last_unstale":"2024-09-17T05:25:19.973527+0000","last_undegraded":"2024-09-17T05:25:19.973527+0000","last_fullsized":"2024-09-17T05:25:19.973527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:27.074967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835477+0000","last_change":"2024-09-17T05:25:17.843269+0000","last_active":"2024-09-17T05:25:18.835477+0000","last_peered":"2024-09-17T05:25:18.835477+0000","last_clean":"2024-09-17T05:25:18.835477+0000","last_became_active":"2024-09-17T05:25:17.842790+0000","last_became_peered":"2024-09-17T05:25:17.842790+0000","last_unstale":"2024-09-17T05:25:18.835477+0000","last_undegraded":"2024-09-17T05:25:18.835477+0000","last_fullsized":"2024-09-17T05:25:18.835477+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:49.922792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975288+0000","last_change":"2024-09-17T05:25:17.853479+0000","last_active":"2024-09-17T05:25:19.975288+0000","last_peered":"2024-09-17T05:25:19.975288+0000","last_clean":"2024-09-17T05:25:19.975288+0000","last_became_active":"2024-09-17T05:25:17.853163+0000","last_became_peered":"2024-09-17T05:25:17.853163+0000","last_unstale":"2024-09-17T05:25:19.975288+0000","last_undegraded":"2024-09-17T05:25:19.975288+0000","last_fullsized":"2024-09-17T05:25:19.975288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:48.306067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975778+0000","last_change":"2024-09-17T05:25:17.834657+0000","last_active":"2024-09-17T05:25:19.975778+0000","last_peered":"2024-09-17T05:25:19.975778+0000","last_clean":"2024-09-17T05:25:19.975778+0000","last_became_active":"2024-09-17T05:25:17.834339+0000","last_became_peered":"2024-09-17T05:25:17.834339+0000","last_unstale":"2024-09-17T05:25:19.975778+0000","last_undegraded":"2024-09-17T05:25:19.975778+0000","last_fullsized":"2024-09-17T05:25:19.975778+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.959803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836527+0000","last_change":"2024-09-17T05:25:17.857677+0000","last_active":"2024-09-17T05:25:18.836527+0000","last_peered":"2024-09-17T05:25:18.836527+0000","last_clean":"2024-09-17T05:25:18.836527+0000","last_became_active":"2024-09-17T05:25:17.857316+0000","last_became_peered":"2024-09-17T05:25:17.857316+0000","last_unstale":"2024-09-17T05:25:18.836527+0000","last_undegraded":"2024-09-17T05:25:18.836527+0000","last_fullsized":"2024-09-17T05:25:18.836527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:10.394231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974428+0000","last_change":"2024-09-17T05:25:17.828984+0000","last_active":"2024-09-17T05:25:19.974428+0000","last_peered":"2024-09-17T05:25:19.974428+0000","last_clean":"2024-09-17T05:25:19.974428+0000","last_became_active":"2024-09-17T05:25:17.828631+0000","last_became_peered":"2024-09-17T05:25:17.828631+0000","last_unstale":"2024-09-17T05:25:19.974428+0000","last_undegraded":"2024-09-17T05:25:19.974428+0000","last_fullsized":"2024-09-17T05:25:19.974428+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.055649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974327+0000","last_change":"2024-09-17T05:25:17.852290+0000","last_active":"2024-09-17T05:25:19.974327+0000","last_peered":"2024-09-17T05:25:19.974327+0000","last_clean":"2024-09-17T05:25:19.974327+0000","last_became_active":"2024-09-17T05:25:17.851997+0000","last_became_peered":"2024-09-17T05:25:17.851997+0000","last_unstale":"2024-09-17T05:25:19.974327+0000","last_undegraded":"2024-09-17T05:25:19.974327+0000","last_fullsized":"2024-09-17T05:25:19.974327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:02.499276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835677+0000","last_change":"2024-09-17T05:25:17.847391+0000","last_active":"2024-09-17T05:25:18.835677+0000","last_peered":"2024-09-17T05:25:18.835677+0000","last_clean":"2024-09-17T05:25:18.835677+0000","last_became_active":"2024-09-17T05:25:17.847099+0000","last_became_peered":"2024-09-17T05:25:17.847099+0000","last_unstale":"2024-09-17T05:25:18.835677+0000","last_undegraded":"2024-09-17T05:25:18.835677+0000","last_fullsized":"2024-09-17T05:25:18.835677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:50.815519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835197+0000","last_change":"2024-09-17T05:25:17.852627+0000","last_active":"2024-09-17T05:25:18.835197+0000","last_peered":"2024-09-17T05:25:18.835197+0000","last_clean":"2024-09-17T05:25:18.835197+0000","last_became_active":"2024-09-17T05:25:17.849911+0000","last_became_peered":"2024-09-17T05:25:17.849911+0000","last_unstale":"2024-09-17T05:25:18.835197+0000","last_undegraded":"2024-09-17T05:25:18.835197+0000","last_fullsized":"2024-09-17T05:25:18.835197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:24.011997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833509+0000","last_change":"2024-09-17T05:25:17.849702+0000","last_active":"2024-09-17T05:25:18.833509+0000","last_peered":"2024-09-17T05:25:18.833509+0000","last_clean":"2024-09-17T05:25:18.833509+0000","last_became_active":"2024-09-17T05:25:17.849304+0000","last_became_peered":"2024-09-17T05:25:17.849304+0000","last_unstale":"2024-09-17T05:25:18.833509+0000","last_undegraded":"2024-09-17T05:25:18.833509+0000","last_fullsized":"2024-09-17T05:25:18.833509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.848451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974714+0000","last_change":"2024-09-17T05:25:17.838654+0000","last_active":"2024-09-17T05:25:19.974714+0000","last_peered":"2024-09-17T05:25:19.974714+0000","last_clean":"2024-09-17T05:25:19.974714+0000","last_became_active":"2024-09-17T05:25:17.838259+0000","last_became_peered":"2024-09-17T05:25:17.838259+0000","last_unstale":"2024-09-17T05:25:19.974714+0000","last_undegraded":"2024-09-17T05:25:19.974714+0000","last_fullsized":"2024-09-17T05:25:19.974714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835757+0000","last_change":"2024-09-17T05:25:17.862226+0000","last_active":"2024-09-17T05:25:18.835757+0000","last_peered":"2024-09-17T05:25:18.835757+0000","last_clean":"2024-09-17T05:25:18.835757+0000","last_became_active":"2024-09-17T05:25:17.860613+0000","last_became_peered":"2024-09-17T05:25:17.860613+0000","last_unstale":"2024-09-17T05:25:18.835757+0000","last_undegraded":"2024-09-17T05:25:18.835757+0000","last_fullsized":"2024-09-17T05:25:18.835757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:03.283332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834234+0000","last_change":"2024-09-17T05:25:17.855133+0000","last_active":"2024-09-17T05:25:18.834234+0000","last_peered":"2024-09-17T05:25:18.834234+0000","last_clean":"2024-09-17T05:25:18.834234+0000","last_became_active":"2024-09-17T05:25:17.854806+0000","last_became_peered":"2024-09-17T05:25:17.854806+0000","last_unstale":"2024-09-17T05:25:18.834234+0000","last_undegraded":"2024-09-17T05:25:18.834234+0000","last_fullsized":"2024-09-17T05:25:18.834234+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:10.392801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975163+0000","last_change":"2024-09-17T05:25:17.840899+0000","last_active":"2024-09-17T05:25:19.975163+0000","last_peered":"2024-09-17T05:25:19.975163+0000","last_clean":"2024-09-17T05:25:19.975163+0000","last_became_active":"2024-09-17T05:25:17.832241+0000","last_became_peered":"2024-09-17T05:25:17.832241+0000","last_unstale":"2024-09-17T05:25:19.975163+0000","last_undegraded":"2024-09-17T05:25:19.975163+0000","last_fullsized":"2024-09-17T05:25:19.975163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:42.840394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835960+0000","last_change":"2024-09-17T05:25:17.846375+0000","last_active":"2024-09-17T05:25:18.835960+0000","last_peered":"2024-09-17T05:25:18.835960+0000","last_clean":"2024-09-17T05:25:18.835960+0000","last_became_active":"2024-09-17T05:25:17.846001+0000","last_became_peered":"2024-09-17T05:25:17.846001+0000","last_unstale":"2024-09-17T05:25:18.835960+0000","last_undegraded":"2024-09-17T05:25:18.835960+0000","last_fullsized":"2024-09-17T05:25:18.835960+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.959803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973545+0000","last_change":"2024-09-17T05:25:17.849145+0000","last_active":"2024-09-17T05:25:19.973545+0000","last_peered":"2024-09-17T05:25:19.973545+0000","last_clean":"2024-09-17T05:25:19.973545+0000","last_became_active":"2024-09-17T05:25:17.848763+0000","last_became_peered":"2024-09-17T05:25:17.848763+0000","last_unstale":"2024-09-17T05:25:19.973545+0000","last_undegraded":"2024-09-17T05:25:19.973545+0000","last_fullsized":"2024-09-17T05:25:19.973545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:49.180399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973316+0000","last_change":"2024-09-17T05:25:17.839774+0000","last_active":"2024-09-17T05:25:19.973316+0000","last_peered":"2024-09-17T05:25:19.973316+0000","last_clean":"2024-09-17T05:25:19.973316+0000","last_became_active":"2024-09-17T05:25:17.839418+0000","last_became_peered":"2024-09-17T05:25:17.839418+0000","last_unstale":"2024-09-17T05:25:19.973316+0000","last_undegraded":"2024-09-17T05:25:19.973316+0000","last_fullsized":"2024-09-17T05:25:19.973316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:46.886790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973068+0000","last_change":"2024-09-17T05:25:17.860865+0000","last_active":"2024-09-17T05:25:19.973068+0000","last_peered":"2024-09-17T05:25:19.973068+0000","last_clean":"2024-09-17T05:25:19.973068+0000","last_became_active":"2024-09-17T05:25:17.857489+0000","last_became_peered":"2024-09-17T05:25:17.857489+0000","last_unstale":"2024-09-17T05:25:19.973068+0000","last_undegraded":"2024-09-17T05:25:19.973068+0000","last_fullsized":"2024-09-17T05:25:19.973068+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:18.658668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834591+0000","last_change":"2024-09-17T05:25:17.827002+0000","last_active":"2024-09-17T05:25:18.834591+0000","last_peered":"2024-09-17T05:25:18.834591+0000","last_clean":"2024-09-17T05:25:18.834591+0000","last_became_active":"2024-09-17T05:25:17.826648+0000","last_became_peered":"2024-09-17T05:25:17.826648+0000","last_unstale":"2024-09-17T05:25:18.834591+0000","last_undegraded":"2024-09-17T05:25:18.834591+0000","last_fullsized":"2024-09-17T05:25:18.834591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836456+0000","last_change":"2024-09-17T05:25:17.855977+0000","last_active":"2024-09-17T05:25:18.836456+0000","last_peered":"2024-09-17T05:25:18.836456+0000","last_clean":"2024-09-17T05:25:18.836456+0000","last_became_active":"2024-09-17T05:25:17.855650+0000","last_became_peered":"2024-09-17T05:25:17.855650+0000","last_unstale":"2024-09-17T05:25:18.836456+0000","last_undegraded":"2024-09-17T05:25:18.836456+0000","last_fullsized":"2024-09-17T05:25:18.836456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:21.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973589+0000","last_change":"2024-09-17T05:25:17.860605+0000","last_active":"2024-09-17T05:25:19.973589+0000","last_peered":"2024-09-17T05:25:19.973589+0000","last_clean":"2024-09-17T05:25:19.973589+0000","last_became_active":"2024-09-17T05:25:17.859963+0000","last_became_peered":"2024-09-17T05:25:17.859963+0000","last_unstale":"2024-09-17T05:25:19.973589+0000","last_undegraded":"2024-09-17T05:25:19.973589+0000","last_fullsized":"2024-09-17T05:25:19.973589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:31.673127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973771+0000","last_change":"2024-09-17T05:25:17.841362+0000","last_active":"2024-09-17T05:25:19.973771+0000","last_peered":"2024-09-17T05:25:19.973771+0000","last_clean":"2024-09-17T05:25:19.973771+0000","last_became_active":"2024-09-17T05:25:17.840961+0000","last_became_peered":"2024-09-17T05:25:17.840961+0000","last_unstale":"2024-09-17T05:25:19.973771+0000","last_undegraded":"2024-09-17T05:25:19.973771+0000","last_fullsized":"2024-09-17T05:25:19.973771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:38.352016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834121+0000","last_change":"2024-09-17T05:25:17.829627+0000","last_active":"2024-09-17T05:25:18.834121+0000","last_peered":"2024-09-17T05:25:18.834121+0000","last_clean":"2024-09-17T05:25:18.834121+0000","last_became_active":"2024-09-17T05:25:17.829328+0000","last_became_peered":"2024-09-17T05:25:17.829328+0000","last_unstale":"2024-09-17T05:25:18.834121+0000","last_undegraded":"2024-09-17T05:25:18.834121+0000","last_fullsized":"2024-09-17T05:25:18.834121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:40.555763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973692+0000","last_change":"2024-09-17T05:25:17.855557+0000","last_active":"2024-09-17T05:25:19.973692+0000","last_peered":"2024-09-17T05:25:19.973692+0000","last_clean":"2024-09-17T05:25:19.973692+0000","last_became_active":"2024-09-17T05:25:17.854842+0000","last_became_peered":"2024-09-17T05:25:17.854842+0000","last_unstale":"2024-09-17T05:25:19.973692+0000","last_undegraded":"2024-09-17T05:25:19.973692+0000","last_fullsized":"2024-09-17T05:25:19.973692+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:35.791860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973401+0000","last_change":"2024-09-17T05:25:17.847426+0000","last_active":"2024-09-17T05:25:19.973401+0000","last_peered":"2024-09-17T05:25:19.973401+0000","last_clean":"2024-09-17T05:25:19.973401+0000","last_became_active":"2024-09-17T05:25:17.847102+0000","last_became_peered":"2024-09-17T05:25:17.847102+0000","last_unstale":"2024-09-17T05:25:19.973401+0000","last_undegraded":"2024-09-17T05:25:19.973401+0000","last_fullsized":"2024-09-17T05:25:19.973401+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:42.964986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835337+0000","last_change":"2024-09-17T05:25:17.851832+0000","last_active":"2024-09-17T05:25:18.835337+0000","last_peered":"2024-09-17T05:25:18.835337+0000","last_clean":"2024-09-17T05:25:18.835337+0000","last_became_active":"2024-09-17T05:25:17.851503+0000","last_became_peered":"2024-09-17T05:25:17.851503+0000","last_unstale":"2024-09-17T05:25:18.835337+0000","last_undegraded":"2024-09-17T05:25:18.835337+0000","last_fullsized":"2024-09-17T05:25:18.835337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974341+0000","last_change":"2024-09-17T05:25:17.845069+0000","last_active":"2024-09-17T05:25:19.974341+0000","last_peered":"2024-09-17T05:25:19.974341+0000","last_clean":"2024-09-17T05:25:19.974341+0000","last_became_active":"2024-09-17T05:25:17.844766+0000","last_became_peered":"2024-09-17T05:25:17.844766+0000","last_unstale":"2024-09-17T05:25:19.974341+0000","last_undegraded":"2024-09-17T05:25:19.974341+0000","last_fullsized":"2024-09-17T05:25:19.974341+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:57.821005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975277+0000","last_change":"2024-09-17T05:25:16.830007+0000","last_active":"2024-09-17T05:25:19.975277+0000","last_peered":"2024-09-17T05:25:19.975277+0000","last_clean":"2024-09-17T05:25:19.975277+0000","last_became_active":"2024-09-17T05:25:16.828838+0000","last_became_peered":"2024-09-17T05:25:16.828838+0000","last_unstale":"2024-09-17T05:25:19.975277+0000","last_undegraded":"2024-09-17T05:25:19.975277+0000","last_fullsized":"2024-09-17T05:25:19.975277+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:49.011723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834768+0000","last_change":"2024-09-17T05:25:17.855463+0000","last_active":"2024-09-17T05:25:18.834768+0000","last_peered":"2024-09-17T05:25:18.834768+0000","last_clean":"2024-09-17T05:25:18.834768+0000","last_became_active":"2024-09-17T05:25:17.855089+0000","last_became_peered":"2024-09-17T05:25:17.855089+0000","last_unstale":"2024-09-17T05:25:18.834768+0000","last_undegraded":"2024-09-17T05:25:18.834768+0000","last_fullsized":"2024-09-17T05:25:18.834768+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973524+0000","last_change":"2024-09-17T05:25:16.841386+0000","last_active":"2024-09-17T05:25:19.973524+0000","last_peered":"2024-09-17T05:25:19.973524+0000","last_clean":"2024-09-17T05:25:19.973524+0000","last_became_active":"2024-09-17T05:25:16.840942+0000","last_became_peered":"2024-09-17T05:25:16.840942+0000","last_unstale":"2024-09-17T05:25:19.973524+0000","last_undegraded":"2024-09-17T05:25:19.973524+0000","last_fullsized":"2024-09-17T05:25:19.973524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:09.391046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974471+0000","last_change":"2024-09-17T05:25:17.844192+0000","last_active":"2024-09-17T05:25:19.974471+0000","last_peered":"2024-09-17T05:25:19.974471+0000","last_clean":"2024-09-17T05:25:19.974471+0000","last_became_active":"2024-09-17T05:25:17.843753+0000","last_became_peered":"2024-09-17T05:25:17.843753+0000","last_unstale":"2024-09-17T05:25:19.974471+0000","last_undegraded":"2024-09-17T05:25:19.974471+0000","last_fullsized":"2024-09-17T05:25:19.974471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:20.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974558+0000","last_change":"2024-09-17T05:25:16.835803+0000","last_active":"2024-09-17T05:25:19.974558+0000","last_peered":"2024-09-17T05:25:19.974558+0000","last_clean":"2024-09-17T05:25:19.974558+0000","last_became_active":"2024-09-17T05:25:16.835332+0000","last_became_peered":"2024-09-17T05:25:16.835332+0000","last_unstale":"2024-09-17T05:25:19.974558+0000","last_undegraded":"2024-09-17T05:25:19.974558+0000","last_fullsized":"2024-09-17T05:25:19.974558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:35.908542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836561+0000","last_change":"2024-09-17T05:25:17.853747+0000","last_active":"2024-09-17T05:25:18.836561+0000","last_peered":"2024-09-17T05:25:18.836561+0000","last_clean":"2024-09-17T05:25:18.836561+0000","last_became_active":"2024-09-17T05:25:17.853420+0000","last_became_peered":"2024-09-17T05:25:17.853420+0000","last_unstale":"2024-09-17T05:25:18.836561+0000","last_undegraded":"2024-09-17T05:25:18.836561+0000","last_fullsized":"2024-09-17T05:25:18.836561+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.926288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973950+0000","last_change":"2024-09-17T05:25:16.839690+0000","last_active":"2024-09-17T05:25:19.973950+0000","last_peered":"2024-09-17T05:25:19.973950+0000","last_clean":"2024-09-17T05:25:19.973950+0000","last_became_active":"2024-09-17T05:25:16.839208+0000","last_became_peered":"2024-09-17T05:25:16.839208+0000","last_unstale":"2024-09-17T05:25:19.973950+0000","last_undegraded":"2024-09-17T05:25:19.973950+0000","last_fullsized":"2024-09-17T05:25:19.973950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:20.943894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973304+0000","last_change":"2024-09-17T05:25:17.860246+0000","last_active":"2024-09-17T05:25:19.973304+0000","last_peered":"2024-09-17T05:25:19.973304+0000","last_clean":"2024-09-17T05:25:19.973304+0000","last_became_active":"2024-09-17T05:25:17.859898+0000","last_became_peered":"2024-09-17T05:25:17.859898+0000","last_unstale":"2024-09-17T05:25:19.973304+0000","last_undegraded":"2024-09-17T05:25:19.973304+0000","last_fullsized":"2024-09-17T05:25:19.973304+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:51.080816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974564+0000","last_change":"2024-09-17T05:25:16.825193+0000","last_active":"2024-09-17T05:25:19.974564+0000","last_peered":"2024-09-17T05:25:19.974564+0000","last_clean":"2024-09-17T05:25:19.974564+0000","last_became_active":"2024-09-17T05:25:16.824705+0000","last_became_peered":"2024-09-17T05:25:16.824705+0000","last_unstale":"2024-09-17T05:25:19.974564+0000","last_undegraded":"2024-09-17T05:25:19.974564+0000","last_fullsized":"2024-09-17T05:25:19.974564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:46.615285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974970+0000","last_change":"2024-09-17T05:25:17.844440+0000","last_active":"2024-09-17T05:25:19.974970+0000","last_peered":"2024-09-17T05:25:19.974970+0000","last_clean":"2024-09-17T05:25:19.974970+0000","last_became_active":"2024-09-17T05:25:17.843574+0000","last_became_peered":"2024-09-17T05:25:17.843574+0000","last_unstale":"2024-09-17T05:25:19.974970+0000","last_undegraded":"2024-09-17T05:25:19.974970+0000","last_fullsized":"2024-09-17T05:25:19.974970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:49.922792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973523+0000","last_change":"2024-09-17T05:25:16.839269+0000","last_active":"2024-09-17T05:25:19.973523+0000","last_peered":"2024-09-17T05:25:19.973523+0000","last_clean":"2024-09-17T05:25:19.973523+0000","last_became_active":"2024-09-17T05:25:16.838891+0000","last_became_peered":"2024-09-17T05:25:16.838891+0000","last_unstale":"2024-09-17T05:25:19.973523+0000","last_undegraded":"2024-09-17T05:25:19.973523+0000","last_fullsized":"2024-09-17T05:25:19.973523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.956617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973855+0000","last_change":"2024-09-17T05:25:17.840191+0000","last_active":"2024-09-17T05:25:19.973855+0000","last_peered":"2024-09-17T05:25:19.973855+0000","last_clean":"2024-09-17T05:25:19.973855+0000","last_became_active":"2024-09-17T05:25:17.839875+0000","last_became_peered":"2024-09-17T05:25:17.839875+0000","last_unstale":"2024-09-17T05:25:19.973855+0000","last_undegraded":"2024-09-17T05:25:19.973855+0000","last_fullsized":"2024-09-17T05:25:19.973855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:01.983440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833474+0000","last_change":"2024-09-17T05:25:16.834402+0000","last_active":"2024-09-17T05:25:18.833474+0000","last_peered":"2024-09-17T05:25:18.833474+0000","last_clean":"2024-09-17T05:25:18.833474+0000","last_became_active":"2024-09-17T05:25:16.834036+0000","last_became_peered":"2024-09-17T05:25:16.834036+0000","last_unstale":"2024-09-17T05:25:18.833474+0000","last_undegraded":"2024-09-17T05:25:18.833474+0000","last_fullsized":"2024-09-17T05:25:18.833474+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:37.453048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973036+0000","last_change":"2024-09-17T05:25:17.855835+0000","last_active":"2024-09-17T05:25:19.973036+0000","last_peered":"2024-09-17T05:25:19.973036+0000","last_clean":"2024-09-17T05:25:19.973036+0000","last_became_active":"2024-09-17T05:25:17.853327+0000","last_became_peered":"2024-09-17T05:25:17.853327+0000","last_unstale":"2024-09-17T05:25:19.973036+0000","last_undegraded":"2024-09-17T05:25:19.973036+0000","last_fullsized":"2024-09-17T05:25:19.973036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:00.732817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975025+0000","last_change":"2024-09-17T05:25:16.839214+0000","last_active":"2024-09-17T05:25:19.975025+0000","last_peered":"2024-09-17T05:25:19.975025+0000","last_clean":"2024-09-17T05:25:19.975025+0000","last_became_active":"2024-09-17T05:25:16.838825+0000","last_became_peered":"2024-09-17T05:25:16.838825+0000","last_unstale":"2024-09-17T05:25:19.975025+0000","last_undegraded":"2024-09-17T05:25:19.975025+0000","last_fullsized":"2024-09-17T05:25:19.975025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:30.086751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835775+0000","last_change":"2024-09-17T05:25:17.850811+0000","last_active":"2024-09-17T05:25:18.835775+0000","last_peered":"2024-09-17T05:25:18.835775+0000","last_clean":"2024-09-17T05:25:18.835775+0000","last_became_active":"2024-09-17T05:25:17.850480+0000","last_became_peered":"2024-09-17T05:25:17.850480+0000","last_unstale":"2024-09-17T05:25:18.835775+0000","last_undegraded":"2024-09-17T05:25:18.835775+0000","last_fullsized":"2024-09-17T05:25:18.835775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:37.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835574+0000","last_change":"2024-09-17T05:25:16.841708+0000","last_active":"2024-09-17T05:25:18.835574+0000","last_peered":"2024-09-17T05:25:18.835574+0000","last_clean":"2024-09-17T05:25:18.835574+0000","last_became_active":"2024-09-17T05:25:16.841263+0000","last_became_peered":"2024-09-17T05:25:16.841263+0000","last_unstale":"2024-09-17T05:25:18.835574+0000","last_undegraded":"2024-09-17T05:25:18.835574+0000","last_fullsized":"2024-09-17T05:25:18.835574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:21.553815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835605+0000","last_change":"2024-09-17T05:25:17.851388+0000","last_active":"2024-09-17T05:25:18.835605+0000","last_peered":"2024-09-17T05:25:18.835605+0000","last_clean":"2024-09-17T05:25:18.835605+0000","last_became_active":"2024-09-17T05:25:17.851036+0000","last_became_peered":"2024-09-17T05:25:17.851036+0000","last_unstale":"2024-09-17T05:25:18.835605+0000","last_undegraded":"2024-09-17T05:25:18.835605+0000","last_fullsized":"2024-09-17T05:25:18.835605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:11.962536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975691+0000","last_change":"2024-09-17T05:25:16.843023+0000","last_active":"2024-09-17T05:25:19.975691+0000","last_peered":"2024-09-17T05:25:19.975691+0000","last_clean":"2024-09-17T05:25:19.975691+0000","last_became_active":"2024-09-17T05:25:16.842671+0000","last_became_peered":"2024-09-17T05:25:16.842671+0000","last_unstale":"2024-09-17T05:25:19.975691+0000","last_undegraded":"2024-09-17T05:25:19.975691+0000","last_fullsized":"2024-09-17T05:25:19.975691+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:40.753607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836306+0000","last_change":"2024-09-17T05:25:17.844156+0000","last_active":"2024-09-17T05:25:18.836306+0000","last_peered":"2024-09-17T05:25:18.836306+0000","last_clean":"2024-09-17T05:25:18.836306+0000","last_became_active":"2024-09-17T05:25:17.843870+0000","last_became_peered":"2024-09-17T05:25:17.843870+0000","last_unstale":"2024-09-17T05:25:18.836306+0000","last_undegraded":"2024-09-17T05:25:18.836306+0000","last_fullsized":"2024-09-17T05:25:18.836306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:44.575485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974447+0000","last_change":"2024-09-17T05:25:16.825444+0000","last_active":"2024-09-17T05:25:19.974447+0000","last_peered":"2024-09-17T05:25:19.974447+0000","last_clean":"2024-09-17T05:25:19.974447+0000","last_became_active":"2024-09-17T05:25:16.825122+0000","last_became_peered":"2024-09-17T05:25:16.825122+0000","last_unstale":"2024-09-17T05:25:19.974447+0000","last_undegraded":"2024-09-17T05:25:19.974447+0000","last_fullsized":"2024-09-17T05:25:19.974447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:38.050632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973843+0000","last_change":"2024-09-17T05:25:17.860864+0000","last_active":"2024-09-17T05:25:19.973843+0000","last_peered":"2024-09-17T05:25:19.973843+0000","last_clean":"2024-09-17T05:25:19.973843+0000","last_became_active":"2024-09-17T05:25:17.860380+0000","last_became_peered":"2024-09-17T05:25:17.860380+0000","last_unstale":"2024-09-17T05:25:19.973843+0000","last_undegraded":"2024-09-17T05:25:19.973843+0000","last_fullsized":"2024-09-17T05:25:19.973843+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:13.785901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975390+0000","last_change":"2024-09-17T05:25:16.845121+0000","last_active":"2024-09-17T05:25:19.975390+0000","last_peered":"2024-09-17T05:25:19.975390+0000","last_clean":"2024-09-17T05:25:19.975390+0000","last_became_active":"2024-09-17T05:25:16.844739+0000","last_became_peered":"2024-09-17T05:25:16.844739+0000","last_unstale":"2024-09-17T05:25:19.975390+0000","last_undegraded":"2024-09-17T05:25:19.975390+0000","last_fullsized":"2024-09-17T05:25:19.975390+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:26.485416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835427+0000","last_change":"2024-09-17T05:25:17.854295+0000","last_active":"2024-09-17T05:25:18.835427+0000","last_peered":"2024-09-17T05:25:18.835427+0000","last_clean":"2024-09-17T05:25:18.835427+0000","last_became_active":"2024-09-17T05:25:17.853814+0000","last_became_peered":"2024-09-17T05:25:17.853814+0000","last_unstale":"2024-09-17T05:25:18.835427+0000","last_undegraded":"2024-09-17T05:25:18.835427+0000","last_fullsized":"2024-09-17T05:25:18.835427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:12.909789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974417+0000","last_change":"2024-09-17T05:25:16.839573+0000","last_active":"2024-09-17T05:25:19.974417+0000","last_peered":"2024-09-17T05:25:19.974417+0000","last_clean":"2024-09-17T05:25:19.974417+0000","last_became_active":"2024-09-17T05:25:16.838379+0000","last_became_peered":"2024-09-17T05:25:16.838379+0000","last_unstale":"2024-09-17T05:25:19.974417+0000","last_undegraded":"2024-09-17T05:25:19.974417+0000","last_fullsized":"2024-09-17T05:25:19.974417+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:00.699933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973436+0000","last_change":"2024-09-17T05:25:17.836620+0000","last_active":"2024-09-17T05:25:19.973436+0000","last_peered":"2024-09-17T05:25:19.973436+0000","last_clean":"2024-09-17T05:25:19.973436+0000","last_became_active":"2024-09-17T05:25:17.836304+0000","last_became_peered":"2024-09-17T05:25:17.836304+0000","last_unstale":"2024-09-17T05:25:19.973436+0000","last_undegraded":"2024-09-17T05:25:19.973436+0000","last_fullsized":"2024-09-17T05:25:19.973436+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:12.909789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833521+0000","last_change":"2024-09-17T05:25:16.837793+0000","last_active":"2024-09-17T05:25:18.833521+0000","last_peered":"2024-09-17T05:25:18.833521+0000","last_clean":"2024-09-17T05:25:18.833521+0000","last_became_active":"2024-09-17T05:25:16.837076+0000","last_became_peered":"2024-09-17T05:25:16.837076+0000","last_unstale":"2024-09-17T05:25:18.833521+0000","last_undegraded":"2024-09-17T05:25:18.833521+0000","last_fullsized":"2024-09-17T05:25:18.833521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:44.279363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973435+0000","last_change":"2024-09-17T05:25:17.846642+0000","last_active":"2024-09-17T05:25:19.973435+0000","last_peered":"2024-09-17T05:25:19.973435+0000","last_clean":"2024-09-17T05:25:19.973435+0000","last_became_active":"2024-09-17T05:25:17.844111+0000","last_became_peered":"2024-09-17T05:25:17.844111+0000","last_unstale":"2024-09-17T05:25:19.973435+0000","last_undegraded":"2024-09-17T05:25:19.973435+0000","last_fullsized":"2024-09-17T05:25:19.973435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:41.914430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973739+0000","last_change":"2024-09-17T05:25:16.842966+0000","last_active":"2024-09-17T05:25:19.973739+0000","last_peered":"2024-09-17T05:25:19.973739+0000","last_clean":"2024-09-17T05:25:19.973739+0000","last_became_active":"2024-09-17T05:25:16.842650+0000","last_became_peered":"2024-09-17T05:25:16.842650+0000","last_unstale":"2024-09-17T05:25:19.973739+0000","last_undegraded":"2024-09-17T05:25:19.973739+0000","last_fullsized":"2024-09-17T05:25:19.973739+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:33.733304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973028+0000","last_change":"2024-09-17T05:25:17.827842+0000","last_active":"2024-09-17T05:25:19.973028+0000","last_peered":"2024-09-17T05:25:19.973028+0000","last_clean":"2024-09-17T05:25:19.973028+0000","last_became_active":"2024-09-17T05:25:17.827506+0000","last_became_peered":"2024-09-17T05:25:17.827506+0000","last_unstale":"2024-09-17T05:25:19.973028+0000","last_undegraded":"2024-09-17T05:25:19.973028+0000","last_fullsized":"2024-09-17T05:25:19.973028+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:34.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974092+0000","last_change":"2024-09-17T05:25:16.835808+0000","last_active":"2024-09-17T05:25:19.974092+0000","last_peered":"2024-09-17T05:25:19.974092+0000","last_clean":"2024-09-17T05:25:19.974092+0000","last_became_active":"2024-09-17T05:25:16.835452+0000","last_became_peered":"2024-09-17T05:25:16.835452+0000","last_unstale":"2024-09-17T05:25:19.974092+0000","last_undegraded":"2024-09-17T05:25:19.974092+0000","last_fullsized":"2024-09-17T05:25:19.974092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:12.812210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974814+0000","last_change":"2024-09-17T05:25:17.855358+0000","last_active":"2024-09-17T05:25:19.974814+0000","last_peered":"2024-09-17T05:25:19.974814+0000","last_clean":"2024-09-17T05:25:19.974814+0000","last_became_active":"2024-09-17T05:25:17.854829+0000","last_became_peered":"2024-09-17T05:25:17.854829+0000","last_unstale":"2024-09-17T05:25:19.974814+0000","last_undegraded":"2024-09-17T05:25:19.974814+0000","last_fullsized":"2024-09-17T05:25:19.974814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:03.283332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836556+0000","last_change":"2024-09-17T05:25:16.842074+0000","last_active":"2024-09-17T05:25:18.836556+0000","last_peered":"2024-09-17T05:25:18.836556+0000","last_clean":"2024-09-17T05:25:18.836556+0000","last_became_active":"2024-09-17T05:25:16.841700+0000","last_became_peered":"2024-09-17T05:25:16.841700+0000","last_unstale":"2024-09-17T05:25:18.836556+0000","last_undegraded":"2024-09-17T05:25:18.836556+0000","last_fullsized":"2024-09-17T05:25:18.836556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:38.831286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974505+0000","last_change":"2024-09-17T05:25:17.843043+0000","last_active":"2024-09-17T05:25:19.974505+0000","last_peered":"2024-09-17T05:25:19.974505+0000","last_clean":"2024-09-17T05:25:19.974505+0000","last_became_active":"2024-09-17T05:25:17.842110+0000","last_became_peered":"2024-09-17T05:25:17.842110+0000","last_unstale":"2024-09-17T05:25:19.974505+0000","last_undegraded":"2024-09-17T05:25:19.974505+0000","last_fullsized":"2024-09-17T05:25:19.974505+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:11.962536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835990+0000","last_change":"2024-09-17T05:25:16.843994+0000","last_active":"2024-09-17T05:25:18.835990+0000","last_peered":"2024-09-17T05:25:18.835990+0000","last_clean":"2024-09-17T05:25:18.835990+0000","last_became_active":"2024-09-17T05:25:16.843335+0000","last_became_peered":"2024-09-17T05:25:16.843335+0000","last_unstale":"2024-09-17T05:25:18.835990+0000","last_undegraded":"2024-09-17T05:25:18.835990+0000","last_fullsized":"2024-09-17T05:25:18.835990+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:10.828864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974672+0000","last_change":"2024-09-17T05:25:17.858165+0000","last_active":"2024-09-17T05:25:19.974672+0000","last_peered":"2024-09-17T05:25:19.974672+0000","last_clean":"2024-09-17T05:25:19.974672+0000","last_became_active":"2024-09-17T05:25:17.856741+0000","last_became_peered":"2024-09-17T05:25:17.856741+0000","last_unstale":"2024-09-17T05:25:19.974672+0000","last_undegraded":"2024-09-17T05:25:19.974672+0000","last_fullsized":"2024-09-17T05:25:19.974672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:16.339920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836058+0000","last_change":"2024-09-17T05:25:16.842760+0000","last_active":"2024-09-17T05:25:18.836058+0000","last_peered":"2024-09-17T05:25:18.836058+0000","last_clean":"2024-09-17T05:25:18.836058+0000","last_became_active":"2024-09-17T05:25:16.842399+0000","last_became_peered":"2024-09-17T05:25:16.842399+0000","last_unstale":"2024-09-17T05:25:18.836058+0000","last_undegraded":"2024-09-17T05:25:18.836058+0000","last_fullsized":"2024-09-17T05:25:18.836058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:04.653739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835672+0000","last_change":"2024-09-17T05:25:17.846811+0000","last_active":"2024-09-17T05:25:18.835672+0000","last_peered":"2024-09-17T05:25:18.835672+0000","last_clean":"2024-09-17T05:25:18.835672+0000","last_became_active":"2024-09-17T05:25:17.846191+0000","last_became_peered":"2024-09-17T05:25:17.846191+0000","last_unstale":"2024-09-17T05:25:18.835672+0000","last_undegraded":"2024-09-17T05:25:18.835672+0000","last_fullsized":"2024-09-17T05:25:18.835672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:21.507732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974614+0000","last_change":"2024-09-17T05:25:16.836689+0000","last_active":"2024-09-17T05:25:19.974614+0000","last_peered":"2024-09-17T05:25:19.974614+0000","last_clean":"2024-09-17T05:25:19.974614+0000","last_became_active":"2024-09-17T05:25:16.836345+0000","last_became_peered":"2024-09-17T05:25:16.836345+0000","last_unstale":"2024-09-17T05:25:19.974614+0000","last_undegraded":"2024-09-17T05:25:19.974614+0000","last_fullsized":"2024-09-17T05:25:19.974614+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:32.420753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974896+0000","last_change":"2024-09-17T05:25:17.849888+0000","last_active":"2024-09-17T05:25:19.974896+0000","last_peered":"2024-09-17T05:25:19.974896+0000","last_clean":"2024-09-17T05:25:19.974896+0000","last_became_active":"2024-09-17T05:25:17.848605+0000","last_became_peered":"2024-09-17T05:25:17.848605+0000","last_unstale":"2024-09-17T05:25:19.974896+0000","last_undegraded":"2024-09-17T05:25:19.974896+0000","last_fullsized":"2024-09-17T05:25:19.974896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974961+0000","last_change":"2024-09-17T05:25:16.836994+0000","last_active":"2024-09-17T05:25:19.974961+0000","last_peered":"2024-09-17T05:25:19.974961+0000","last_clean":"2024-09-17T05:25:19.974961+0000","last_became_active":"2024-09-17T05:25:16.836576+0000","last_became_peered":"2024-09-17T05:25:16.836576+0000","last_unstale":"2024-09-17T05:25:19.974961+0000","last_undegraded":"2024-09-17T05:25:19.974961+0000","last_fullsized":"2024-09-17T05:25:19.974961+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:41.094836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974351+0000","last_change":"2024-09-17T05:25:17.861155+0000","last_active":"2024-09-17T05:25:19.974351+0000","last_peered":"2024-09-17T05:25:19.974351+0000","last_clean":"2024-09-17T05:25:19.974351+0000","last_became_active":"2024-09-17T05:25:17.860868+0000","last_became_peered":"2024-09-17T05:25:17.860868+0000","last_unstale":"2024-09-17T05:25:19.974351+0000","last_undegraded":"2024-09-17T05:25:19.974351+0000","last_fullsized":"2024-09-17T05:25:19.974351+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:03.657042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.833682+0000","last_change":"2024-09-17T05:25:16.837411+0000","last_active":"2024-09-17T05:25:18.833682+0000","last_peered":"2024-09-17T05:25:18.833682+0000","last_clean":"2024-09-17T05:25:18.833682+0000","last_became_active":"2024-09-17T05:25:16.837058+0000","last_became_peered":"2024-09-17T05:25:16.837058+0000","last_unstale":"2024-09-17T05:25:18.833682+0000","last_undegraded":"2024-09-17T05:25:18.833682+0000","last_fullsized":"2024-09-17T05:25:18.833682+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:42.610461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835654+0000","last_change":"2024-09-17T05:25:17.850227+0000","last_active":"2024-09-17T05:25:18.835654+0000","last_peered":"2024-09-17T05:25:18.835654+0000","last_clean":"2024-09-17T05:25:18.835654+0000","last_became_active":"2024-09-17T05:25:17.849926+0000","last_became_peered":"2024-09-17T05:25:17.849926+0000","last_unstale":"2024-09-17T05:25:18.835654+0000","last_undegraded":"2024-09-17T05:25:18.835654+0000","last_fullsized":"2024-09-17T05:25:18.835654+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:08.132627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973538+0000","last_change":"2024-09-17T05:25:16.843105+0000","last_active":"2024-09-17T05:25:19.973538+0000","last_peered":"2024-09-17T05:25:19.973538+0000","last_clean":"2024-09-17T05:25:19.973538+0000","last_became_active":"2024-09-17T05:25:16.842668+0000","last_became_peered":"2024-09-17T05:25:16.842668+0000","last_unstale":"2024-09-17T05:25:19.973538+0000","last_undegraded":"2024-09-17T05:25:19.973538+0000","last_fullsized":"2024-09-17T05:25:19.973538+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835465+0000","last_change":"2024-09-17T05:25:17.847878+0000","last_active":"2024-09-17T05:25:18.835465+0000","last_peered":"2024-09-17T05:25:18.835465+0000","last_clean":"2024-09-17T05:25:18.835465+0000","last_became_active":"2024-09-17T05:25:17.844937+0000","last_became_peered":"2024-09-17T05:25:17.844937+0000","last_unstale":"2024-09-17T05:25:18.835465+0000","last_undegraded":"2024-09-17T05:25:18.835465+0000","last_fullsized":"2024-09-17T05:25:18.835465+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:54.558535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.972889+0000","last_change":"2024-09-17T05:25:16.839427+0000","last_active":"2024-09-17T05:25:19.972889+0000","last_peered":"2024-09-17T05:25:19.972889+0000","last_clean":"2024-09-17T05:25:19.972889+0000","last_became_active":"2024-09-17T05:25:16.838642+0000","last_became_peered":"2024-09-17T05:25:16.838642+0000","last_unstale":"2024-09-17T05:25:19.972889+0000","last_undegraded":"2024-09-17T05:25:19.972889+0000","last_fullsized":"2024-09-17T05:25:19.972889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.129441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973455+0000","last_change":"2024-09-17T05:25:17.854279+0000","last_active":"2024-09-17T05:25:19.973455+0000","last_peered":"2024-09-17T05:25:19.973455+0000","last_clean":"2024-09-17T05:25:19.973455+0000","last_became_active":"2024-09-17T05:25:17.853347+0000","last_became_peered":"2024-09-17T05:25:17.853347+0000","last_unstale":"2024-09-17T05:25:19.973455+0000","last_undegraded":"2024-09-17T05:25:19.973455+0000","last_fullsized":"2024-09-17T05:25:19.973455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:52.026834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836677+0000","last_change":"2024-09-17T05:25:16.845000+0000","last_active":"2024-09-17T05:25:18.836677+0000","last_peered":"2024-09-17T05:25:18.836677+0000","last_clean":"2024-09-17T05:25:18.836677+0000","last_became_active":"2024-09-17T05:25:16.844264+0000","last_became_peered":"2024-09-17T05:25:16.844264+0000","last_unstale":"2024-09-17T05:25:18.836677+0000","last_undegraded":"2024-09-17T05:25:18.836677+0000","last_fullsized":"2024-09-17T05:25:18.836677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:10.422465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975224+0000","last_change":"2024-09-17T05:25:17.848021+0000","last_active":"2024-09-17T05:25:19.975224+0000","last_peered":"2024-09-17T05:25:19.975224+0000","last_clean":"2024-09-17T05:25:19.975224+0000","last_became_active":"2024-09-17T05:25:17.847264+0000","last_became_peered":"2024-09-17T05:25:17.847264+0000","last_unstale":"2024-09-17T05:25:19.975224+0000","last_undegraded":"2024-09-17T05:25:19.975224+0000","last_fullsized":"2024-09-17T05:25:19.975224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:08.132627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834237+0000","last_change":"2024-09-17T05:25:16.838187+0000","last_active":"2024-09-17T05:25:18.834237+0000","last_peered":"2024-09-17T05:25:18.834237+0000","last_clean":"2024-09-17T05:25:18.834237+0000","last_became_active":"2024-09-17T05:25:16.837770+0000","last_became_peered":"2024-09-17T05:25:16.837770+0000","last_unstale":"2024-09-17T05:25:18.834237+0000","last_undegraded":"2024-09-17T05:25:18.834237+0000","last_fullsized":"2024-09-17T05:25:18.834237+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:49.145816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973951+0000","last_change":"2024-09-17T05:25:17.860851+0000","last_active":"2024-09-17T05:25:19.973951+0000","last_peered":"2024-09-17T05:25:19.973951+0000","last_clean":"2024-09-17T05:25:19.973951+0000","last_became_active":"2024-09-17T05:25:17.860262+0000","last_became_peered":"2024-09-17T05:25:17.860262+0000","last_unstale":"2024-09-17T05:25:19.973951+0000","last_undegraded":"2024-09-17T05:25:19.973951+0000","last_fullsized":"2024-09-17T05:25:19.973951+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:20.085739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.974300+0000","last_change":"2024-09-17T05:25:16.841538+0000","last_active":"2024-09-17T05:25:19.974300+0000","last_peered":"2024-09-17T05:25:19.974300+0000","last_clean":"2024-09-17T05:25:19.974300+0000","last_became_active":"2024-09-17T05:25:16.840839+0000","last_became_peered":"2024-09-17T05:25:16.840839+0000","last_unstale":"2024-09-17T05:25:19.974300+0000","last_undegraded":"2024-09-17T05:25:19.974300+0000","last_fullsized":"2024-09-17T05:25:19.974300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:35.655115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.975174+0000","last_change":"2024-09-17T05:25:17.849451+0000","last_active":"2024-09-17T05:25:19.975174+0000","last_peered":"2024-09-17T05:25:19.975174+0000","last_clean":"2024-09-17T05:25:19.975174+0000","last_became_active":"2024-09-17T05:25:17.848905+0000","last_became_peered":"2024-09-17T05:25:17.848905+0000","last_unstale":"2024-09-17T05:25:19.975174+0000","last_undegraded":"2024-09-17T05:25:19.975174+0000","last_fullsized":"2024-09-17T05:25:19.975174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:30.495528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834011+0000","last_change":"2024-09-17T05:25:16.841637+0000","last_active":"2024-09-17T05:25:18.834011+0000","last_peered":"2024-09-17T05:25:18.834011+0000","last_clean":"2024-09-17T05:25:18.834011+0000","last_became_active":"2024-09-17T05:25:16.840417+0000","last_became_peered":"2024-09-17T05:25:16.840417+0000","last_unstale":"2024-09-17T05:25:18.834011+0000","last_undegraded":"2024-09-17T05:25:18.834011+0000","last_fullsized":"2024-09-17T05:25:18.834011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:36.939696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.835591+0000","last_change":"2024-09-17T05:25:17.858282+0000","last_active":"2024-09-17T05:25:18.835591+0000","last_peered":"2024-09-17T05:25:18.835591+0000","last_clean":"2024-09-17T05:25:18.835591+0000","last_became_active":"2024-09-17T05:25:17.856926+0000","last_became_peered":"2024-09-17T05:25:17.856926+0000","last_unstale":"2024-09-17T05:25:18.835591+0000","last_undegraded":"2024-09-17T05:25:18.835591+0000","last_fullsized":"2024-09-17T05:25:18.835591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:30.495528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973627+0000","last_change":"2024-09-17T05:25:16.843421+0000","last_active":"2024-09-17T05:25:19.973627+0000","last_peered":"2024-09-17T05:25:19.973627+0000","last_clean":"2024-09-17T05:25:19.973627+0000","last_became_active":"2024-09-17T05:25:16.843071+0000","last_became_peered":"2024-09-17T05:25:16.843071+0000","last_unstale":"2024-09-17T05:25:19.973627+0000","last_undegraded":"2024-09-17T05:25:19.973627+0000","last_fullsized":"2024-09-17T05:25:19.973627+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:43.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834887+0000","last_change":"2024-09-17T05:25:17.851994+0000","last_active":"2024-09-17T05:25:18.834887+0000","last_peered":"2024-09-17T05:25:18.834887+0000","last_clean":"2024-09-17T05:25:18.834887+0000","last_became_active":"2024-09-17T05:25:17.851553+0000","last_became_peered":"2024-09-17T05:25:17.851553+0000","last_unstale":"2024-09-17T05:25:18.834887+0000","last_undegraded":"2024-09-17T05:25:18.834887+0000","last_fullsized":"2024-09-17T05:25:18.834887+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:49.180399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973876+0000","last_change":"2024-09-17T05:25:16.838372+0000","last_active":"2024-09-17T05:25:19.973876+0000","last_peered":"2024-09-17T05:25:19.973876+0000","last_clean":"2024-09-17T05:25:19.973876+0000","last_became_active":"2024-09-17T05:25:16.837915+0000","last_became_peered":"2024-09-17T05:25:16.837915+0000","last_unstale":"2024-09-17T05:25:19.973876+0000","last_undegraded":"2024-09-17T05:25:19.973876+0000","last_fullsized":"2024-09-17T05:25:19.973876+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:52.416005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973650+0000","last_change":"2024-09-17T05:25:17.857155+0000","last_active":"2024-09-17T05:25:19.973650+0000","last_peered":"2024-09-17T05:25:19.973650+0000","last_clean":"2024-09-17T05:25:19.973650+0000","last_became_active":"2024-09-17T05:25:17.855676+0000","last_became_peered":"2024-09-17T05:25:17.855676+0000","last_unstale":"2024-09-17T05:25:19.973650+0000","last_undegraded":"2024-09-17T05:25:19.973650+0000","last_fullsized":"2024-09-17T05:25:19.973650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:11.788487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973690+0000","last_change":"2024-09-17T05:25:16.844953+0000","last_active":"2024-09-17T05:25:19.973690+0000","last_peered":"2024-09-17T05:25:19.973690+0000","last_clean":"2024-09-17T05:25:19.973690+0000","last_became_active":"2024-09-17T05:25:16.843607+0000","last_became_peered":"2024-09-17T05:25:16.843607+0000","last_unstale":"2024-09-17T05:25:19.973690+0000","last_undegraded":"2024-09-17T05:25:19.973690+0000","last_fullsized":"2024-09-17T05:25:19.973690+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.778959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.836561+0000","last_change":"2024-09-17T05:25:17.843412+0000","last_active":"2024-09-17T05:25:18.836561+0000","last_peered":"2024-09-17T05:25:18.836561+0000","last_clean":"2024-09-17T05:25:18.836561+0000","last_became_active":"2024-09-17T05:25:17.842545+0000","last_became_peered":"2024-09-17T05:25:17.842545+0000","last_unstale":"2024-09-17T05:25:18.836561+0000","last_undegraded":"2024-09-17T05:25:18.836561+0000","last_fullsized":"2024-09-17T05:25:18.836561+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:16.805809+0000","last_clean_scrub_stamp":"2024-09-17T05:25:16.805809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:59.137139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:19.973423+0000","last_change":"2024-09-17T05:25:16.842097+0000","last_active":"2024-09-17T05:25:19.973423+0000","last_peered":"2024-09-17T05:25:19.973423+0000","last_clean":"2024-09-17T05:25:19.973423+0000","last_became_active":"2024-09-17T05:25:16.841570+0000","last_became_peered":"2024-09-17T05:25:16.841570+0000","last_unstale":"2024-09-17T05:25:19.973423+0000","last_undegraded":"2024-09-17T05:25:19.973423+0000","last_fullsized":"2024-09-17T05:25:19.973423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:15.802624+0000","last_clean_scrub_stamp":"2024-09-17T05:25:15.802624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.674779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T05:25:18.834969+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:18.834969+0000","last_peered":"2024-09-17T05:25:18.834969+0000","last_clean":"2024-09-17T05:25:18.834969+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:18.834969+0000","last_undegraded":"2024-09-17T05:25:18.834969+0000","last_fullsized":"2024-09-17T05:25:18.834969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":221470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1142784,"data_stored":811826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":223140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":223048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1142784,"data_stored":811874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":222108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":552960,"data_stored":222052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:27.537 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:25:27.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:25:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- 172.21.15.43:0/487963735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea6806f110 msgr2=0x7fea680ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 --2- 172.21.15.43:0/487963735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea6806f110 0x7fea680ad0b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fea680af0a0 tx=0x7fea6001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- 172.21.15.43:0/487963735 shutdown_connections 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 --2- 172.21.15.43:0/487963735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea6806f110 0x7fea680ad0b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 --2- 172.21.15.43:0/487963735 >> v2:172.21.15.62:3300/0 conn(0x7fea680b3af0 0x7fea6815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- 172.21.15.43:0/487963735 >> 172.21.15.43:0/487963735 conn(0x7fea680b23d0 msgr2=0x7fea680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- 172.21.15.43:0/487963735 shutdown_connections 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- 172.21.15.43:0/487963735 wait complete. 2024-09-17T05:25:27.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 Processor -- start 2024-09-17T05:25:27.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.732+0000 7fea70000640 1 -- start start 2024-09-17T05:25:27.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806f110 0x7fea6809d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea6809ccd0 0x7fea680cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea68168b90 con 0x7fea68155d70 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fea68168a10 con 0x7fea6806f110 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fea68168d10 con 0x7fea6809ccd0 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54632/0 (socket says 172.21.15.43:54632) 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 -- 172.21.15.43:0/414108340 learned_addr learned my addr 172.21.15.43:0/414108340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6d600640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806f110 0x7fea6809d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6e000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea6809ccd0 0x7fea680cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 -- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea6809ccd0 msgr2=0x7fea680cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea6809ccd0 0x7fea680cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 -- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806f110 msgr2=0x7fea6809d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806f110 0x7fea6809d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 -- 172.21.15.43:0/414108340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea680cb540 con 0x7fea68155d70 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea6cc00640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e7f0 tx=0x7fea5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5400cc60 con 0x7fea68155d70 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea5400ce00 con 0x7fea68155d70 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54005280 con 0x7fea68155d70 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea680cb820 con 0x7fea68155d70 2024-09-17T05:25:27.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea680c9920 con 0x7fea68155d70 2024-09-17T05:25:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.736+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea680ad0b0 con 0x7fea68155d70 2024-09-17T05:25:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.740+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea54005420 con 0x7fea68155d70 2024-09-17T05:25:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.740+0000 7fea65600640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 0x7fea48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.740+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fea54094cc0 con 0x7fea68155d70 2024-09-17T05:25:27.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.740+0000 7fea6d600640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 0x7fea48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:27.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.740+0000 7fea6d600640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 0x7fea48075710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000f70 tx=0x7fea5c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:27.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.744+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea5409b080 con 0x7fea68155d70 2024-09-17T05:25:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.980+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fea680b0300 con 0x7fea68155d70 2024-09-17T05:25:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea65600640 1 -- 172.21.15.43:0/414108340 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fea5405fc40 con 0x7fea68155d70 2024-09-17T05:25:27.988 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:25:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:25:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:25:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:25:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:25:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 msgr2=0x7fea48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 0x7fea48075710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000f70 tx=0x7fea5c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 msgr2=0x7fea6809be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e7f0 tx=0x7fea5400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 shutdown_connections 2024-09-17T05:25:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fea48073200 0x7fea48075710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fea6809ccd0 0x7fea680cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea68155d70 0x7fea6809be00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 --2- 172.21.15.43:0/414108340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806f110 0x7fea6809d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 >> 172.21.15.43:0/414108340 conn(0x7fea680b23d0 msgr2=0x7fea680b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 shutdown_connections 2024-09-17T05:25:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:27.984+0000 7fea70000640 1 -- 172.21.15.43:0/414108340 wait complete. 2024-09-17T05:25:28.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 -- 172.21.15.43:0/2820853122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c0737c0 msgr2=0x7f470c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 --2- 172.21.15.43:0/2820853122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c0737c0 0x7f470c0700a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4704009bc0 tx=0x7f470401ca80 comp rx=0 tx=0).stop 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 -- 172.21.15.43:0/2820853122 shutdown_connections 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 --2- 172.21.15.43:0/2820853122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c0737c0 0x7f470c0700a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 --2- 172.21.15.43:0/2820853122 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c15fe10 0x7f470c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 -- 172.21.15.43:0/2820853122 >> 172.21.15.43:0/2820853122 conn(0x7f470c0b23d0 msgr2=0x7f470c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 -- 172.21.15.43:0/2820853122 shutdown_connections 2024-09-17T05:25:28.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.164+0000 7f4710c00640 1 -- 172.21.15.43:0/2820853122 wait complete. 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 Processor -- start 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 -- start start 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c15fe10 0x7f470c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c157ee0 0x7f470c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f470c169fd0 con 0x7f470c06fc90 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f470c169e50 con 0x7f470c15fe10 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4710c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f470c16a150 con 0x7f470c157ee0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4709e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4709e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54642/0 (socket says 172.21.15.43:54642) 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4709e00640 1 -- 172.21.15.43:0/3917951031 learned_addr learned my addr 172.21.15.43:0/3917951031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f470a800640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c157ee0 0x7f470c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.168+0000 7f4701400640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c15fe10 0x7f470c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 -- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c157ee0 msgr2=0x7f470c1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c157ee0 0x7f470c1582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 -- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c15fe10 msgr2=0x7f470c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c15fe10 0x7f470c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 -- 172.21.15.43:0/3917951031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c158830 con 0x7f470c06fc90 2024-09-17T05:25:28.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4709e00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e7a0 tx=0x7f46fc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00ccc0 con 0x7f470c06fc90 2024-09-17T05:25:28.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46fc010430 con 0x7f470c06fc90 2024-09-17T05:25:28.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c15ddf0 con 0x7f470c06fc90 2024-09-17T05:25:28.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc010860 con 0x7f470c06fc90 2024-09-17T05:25:28.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c15c600 con 0x7f470c06fc90 2024-09-17T05:25:28.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c0700a0 con 0x7f470c06fc90 2024-09-17T05:25:28.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.172+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46fc010a00 con 0x7f470c06fc90 2024-09-17T05:25:28.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.176+0000 7f4702a00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 0x7f46e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.176+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f46fc094d50 con 0x7f470c06fc90 2024-09-17T05:25:28.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.176+0000 7f4701400640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 0x7f46e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.176+0000 7f4701400640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 0x7f46e8075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f470c162b20 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.180+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f470c158830 con 0x7f470c06fc90 2024-09-17T05:25:28.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.392+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f470c0cd9d0 con 0x7f470c06fc90 2024-09-17T05:25:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.392+0000 7f4702a00640 1 -- 172.21.15.43:0/3917951031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5217 (secure 0 0 0) 0x7f46fc05fcd0 con 0x7f470c06fc90 2024-09-17T05:25:28.397 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:28.397 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-17T05:25:20:838273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:20.251677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/980890216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":980890216},{"type":"v1","addr":"172.21.15.62:6835","nonce":980890216}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5439,"qdb_cluster":[5439]},"id":3}]} 2024-09-17T05:25:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-17T05:25:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 msgr2=0x7f46e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 0x7f46e8075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f470c162b20 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2024-09-17T05:25:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 msgr2=0x7f470c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e7a0 tx=0x7f46fc00ec60 comp rx=0 tx=0).stop 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 shutdown_connections 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f46e8073200 0x7f46e8075710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f470c15fe10 0x7f470c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f470c157ee0 0x7f470c1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 --2- 172.21.15.43:0/3917951031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f470c06fc90 0x7f470c160380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 >> 172.21.15.43:0/3917951031 conn(0x7f470c0b23d0 msgr2=0x7f470c0ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 shutdown_connections 2024-09-17T05:25:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.396+0000 7f4710c00640 1 -- 172.21.15.43:0/3917951031 wait complete. 2024-09-17T05:25:28.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:17.258682+0000', 'modified': '2024-09-17T05:25:20.251677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5439': {'gid': 5439, 'name': 'b', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/980890216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 980890216}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 980890216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5439, 'qdb_cluster': [5439]} 2024-09-17T05:25:28.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:25:28.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 -- 172.21.15.43:0/449821398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c0cb860 msgr2=0x7f960c0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 --2- 172.21.15.43:0/449821398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c0cb860 0x7f960c0cbc70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f960c0d12a0 tx=0x7f960401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 -- 172.21.15.43:0/449821398 shutdown_connections 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 --2- 172.21.15.43:0/449821398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c0cb860 0x7f960c0cbc70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 --2- 172.21.15.43:0/449821398 >> v2:172.21.15.62:3300/0 conn(0x7f960c070620 0x7f960c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 -- 172.21.15.43:0/449821398 >> 172.21.15.43:0/449821398 conn(0x7f960c0b23d0 msgr2=0x7f960c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 -- 172.21.15.43:0/449821398 shutdown_connections 2024-09-17T05:25:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.980+0000 7f9611e00640 1 -- 172.21.15.43:0/449821398 wait complete. 2024-09-17T05:25:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 Processor -- start 2024-09-17T05:25:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 -- start start 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f960c070620 0x7f960c1f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f960c0cb860 0x7f960c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0cedb0 con 0x7f960c06ce60 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0cec30 con 0x7f960c070620 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f9611e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f960c0d0910 con 0x7f960c0cb860 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54662/0 (socket says 172.21.15.43:54662) 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f960aa00640 1 -- 172.21.15.43:0/3935002527 learned_addr learned my addr 172.21.15.43:0/3935002527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f960a000640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f960c070620 0x7f960c1f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.984+0000 7f960b400640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f960c0cb860 0x7f960c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 -- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f960c0cb860 msgr2=0x7f960c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f960c0cb860 0x7f960c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 -- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f960c070620 msgr2=0x7f960c1f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f960c070620 0x7f960c1f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 -- 172.21.15.43:0/3935002527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c0ae350 con 0x7f960c06ce60 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f960aa00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d9b0 tx=0x7f95f400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9603400640 1 -- 172.21.15.43:0/3935002527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x7f960c06ce60 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9603400640 1 -- 172.21.15.43:0/3935002527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f4004310 con 0x7f960c06ce60 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9603400640 1 -- 172.21.15.43:0/3935002527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4009c30 con 0x7f960c06ce60 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c0ae630 con 0x7f960c06ce60 2024-09-17T05:25:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c0aebf0 con 0x7f960c06ce60 2024-09-17T05:25:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.988+0000 7f9603400640 1 -- 172.21.15.43:0/3935002527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f95f4005020 con 0x7f960c06ce60 2024-09-17T05:25:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9603400640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 0x7f95e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9603400640 1 -- 172.21.15.43:0/3935002527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f95f4094680 con 0x7f960c06ce60 2024-09-17T05:25:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 Processor -- start 2024-09-17T05:25:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960a000640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 0x7f95e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 -- start start 2024-09-17T05:25:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960a000640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 0x7f95e8075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f960c1660b0 tx=0x7f95fc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 0x56089c1edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56089c0ba410 con 0x56089c07dca0 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56089bff7b40 con 0x56089c094950 2024-09-17T05:25:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56089bfe6f70 con 0x56089bca4460 2024-09-17T05:25:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54668/0 (socket says 172.21.15.43:54668) 2024-09-17T05:25:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960aa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 0x56089c1edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 -- 172.21.15.43:0/3048592009 learned_addr learned my addr 172.21.15.43:0/3048592009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960a000640 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 -- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 msgr2=0x56089c1edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 0x56089c1edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 -- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 msgr2=0x56089c1ee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1ee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 -- 172.21.15.43:0/3048592009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089bfae5b0 con 0x56089c07dca0 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960aa00640 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 0x56089c1edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f960b400640 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96040b0040 tx=0x7f9604019a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3f5767e80c4c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f95fb400640 1 -- 172.21.15.43:0/3048592009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96040bc030 con 0x56089c07dca0 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f95fb400640 1 -- 172.21.15.43:0/3048592009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96040092c0 con 0x56089c07dca0 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.992+0000 7f95fb400640 1 -- 172.21.15.43:0/3048592009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96040b2040 con 0x56089c07dca0 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- 172.21.15.43:0/3048592009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 msgr2=0x56089c1ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96040b0040 tx=0x7f9604019a90 comp rx=0 tx=0).stop 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- 172.21.15.43:0/3048592009 shutdown_connections 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1ee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089c07dca0 0x56089c1ee930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- 172.21.15.43:0/3048592009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089bca4460 0x56089c1edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- 172.21.15.43:0/3048592009 >> 172.21.15.43:0/3048592009 conn(0x56089c16ad90 msgr2=0x56089c19a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- 172.21.15.43:0/3048592009 shutdown_connections 2024-09-17T05:25:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- 172.21.15.43:0/3048592009 wait complete. 2024-09-17T05:25:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- start start 2024-09-17T05:25:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 Processor -- start 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56089bf815f0 con 0x56089bca4460 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56089bfead80 con 0x56089c094950 2024-09-17T05:25:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56089bf8d520 con 0x56089c07dca0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f9619500000 10 client.0 _ll_register_callbacks cb 0x56089bf91830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54680/0 (socket says 172.21.15.43:54680) 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 -- 172.21.15.43:0/3485739897 learned_addr learned my addr 172.21.15.43:0/3485739897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960a000640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960b400640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 -- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 msgr2=0x56089c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 -- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 msgr2=0x56089c1d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1d9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 -- 172.21.15.43:0/3485739897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089bf85190 con 0x56089bca4460 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f960aa00640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c10 tx=0x7f95f4005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14d1d3896410e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:28.996+0000 7f95efe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f405e410 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f95f405a900 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56089bf7f110 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95f4068920 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 10 client.5586 fetch_fsmap learned FSMap version 30 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x56089c06bf10 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56089bf7f110 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f95f405ab00 con 0x56089bca4460 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 10 client.5586 fetch_fsmap finished waiting for FSMap version 30 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 10 client.5586 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 20 client.5586 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc897d0 0x56089c1f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 4 client.5586 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 --> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089bff9a90 con 0x56089bc897d0 2024-09-17T05:25:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f960a000640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc897d0 0x56089c1f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f960a000640 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc897d0 0x56089c1f5130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb44b8c94c6f5407 server_cookie=29fd3814b6e561cb in_seq=0 out_seq=0 2024-09-17T05:25:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 10 client.5586 ms_handle_connect on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 1 -- 172.21.15.43:0/3485739897 <== mds.0 v2:172.21.15.62:6834/980890216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56089bff9a90 con 0x56089bc897d0 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 10 client.5586 handle_command_reply: tid=0 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 client.5586 shutdown 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 msgr2=0x56089c1a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c10 tx=0x7f95f4005f20 comp rx=0 tx=0).stop 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 shutdown_connections 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc897d0 0x56089c1f5130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c1d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 --2- 172.21.15.43:0/3485739897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c1a0f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f95efe00640 0 client.5586 ms_handle_reset on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 >> 172.21.15.43:0/3485739897 conn(0x56089c16ad90 msgr2=0x56089c19a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.000+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 shutdown_connections 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.004+0000 7f9619500000 1 -- 172.21.15.43:0/3485739897 wait complete. 2024-09-17T05:25:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.004+0000 7f9619500000 20 client.5586 trim_cache size 0 max 16384 2024-09-17T05:25:29.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.020+0000 7f9619500000 1 Processor -- start 2024-09-17T05:25:29.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.020+0000 7f9619500000 1 -- start start 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 0x56089c2605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 0x56089c260b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f960c15ed20 con 0x56089bca4460 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f960c15dd40 con 0x56089c07dca0 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f960c0c6040 con 0x56089c094950 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54686/0 (socket says 172.21.15.43:54686) 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 -- 172.21.15.43:0/2896042957 learned_addr learned my addr 172.21.15.43:0/2896042957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960b400640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 0x56089c260b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 -- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 msgr2=0x56089c260b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 0x56089c260b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960a000640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 0x56089c2605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 -- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 msgr2=0x56089c2605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 0x56089c2605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 -- 172.21.15.43:0/2896042957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089c042db0 con 0x56089bca4460 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960b400640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 0x56089c260b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960a000640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 0x56089c2605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f960aa00640 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95f405ec60 tx=0x7f95f40929e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe0ddf9489738f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f95faa00640 1 -- 172.21.15.43:0/2896042957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f405e210 con 0x56089bca4460 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f95faa00640 1 -- 172.21.15.43:0/2896042957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f405e4a0 con 0x56089bca4460 2024-09-17T05:25:29.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f95faa00640 1 -- 172.21.15.43:0/2896042957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4068d90 con 0x56089bca4460 2024-09-17T05:25:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- 172.21.15.43:0/2896042957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 msgr2=0x56089c260090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f95f405ec60 tx=0x7f95f40929e0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- 172.21.15.43:0/2896042957 shutdown_connections 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c07dca0 0x56089c2605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c094950 0x56089c260b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 --2- 172.21.15.43:0/2896042957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c260090 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- 172.21.15.43:0/2896042957 >> 172.21.15.43:0/2896042957 conn(0x56089c16ad90 msgr2=0x56089c205b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- 172.21.15.43:0/2896042957 shutdown_connections 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- 172.21.15.43:0/2896042957 wait complete. 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 -- start start 2024-09-17T05:25:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.024+0000 7f9619500000 1 Processor -- start 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c220670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0c6690 con 0x56089bca4460 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56089c06ba80 con 0x56089c094950 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56089c0a6750 con 0x56089c07dca0 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 10 client.0 _ll_register_callbacks cb 0x7f95f405aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54702/0 (socket says 172.21.15.43:54702) 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 -- 172.21.15.43:0/602502147 learned_addr learned my addr 172.21.15.43:0/602502147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960a000640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960b400640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c220670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 -- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 msgr2=0x56089c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 -- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 msgr2=0x56089c220670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c220670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 -- 172.21.15.43:0/602502147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089bfadab0 con 0x56089bca4460 2024-09-17T05:25:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960aa00640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009610 tx=0x7f95f4063b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78b34e04a4ef3db server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f405ab00 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f95f4005310 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95f405e4a0 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95f4005510 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 10 client.5592 fetch_fsmap learned FSMap version 30 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x56089bf8c070 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f405e4a0 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f95f405dc80 con 0x56089bca4460 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 10 client.5592 fetch_fsmap finished waiting for FSMap version 30 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 10 client.5592 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 20 client.5592 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc77830 0x56089c220fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 4 client.5592 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:25:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 --> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] -- command(tid 0: {"prefix": "status"}) -- 0x56089bff5540 con 0x56089bc77830 2024-09-17T05:25:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.028+0000 7f960a000640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc77830 0x56089c220fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f960a000640 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc77830 0x56089c220fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d17bed78f90124 server_cookie=806ba6ff359511fb in_seq=0 out_seq=0 2024-09-17T05:25:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f95efe00640 10 client.5592 ms_handle_connect on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f95efe00640 1 -- 172.21.15.43:0/602502147 <== mds.0 v2:172.21.15.62:6834/980890216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56089bff5540 con 0x56089bc77830 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f95efe00640 10 client.5592 handle_command_reply: tid=0 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 client.5592 shutdown 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 msgr2=0x56089c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009610 tx=0x7f95f4063b60 comp rx=0 tx=0).stop 2024-09-17T05:25:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 shutdown_connections 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x56089bc77830 0x56089c220fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56089c094950 0x56089c220670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56089c07dca0 0x56089c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 --2- 172.21.15.43:0/602502147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56089bca4460 0x56089c19cfb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f95efe00640 0 client.5592 ms_handle_reset on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 >> 172.21.15.43:0/602502147 conn(0x56089c16ad90 msgr2=0x56089c197cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 shutdown_connections 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 1 -- 172.21.15.43:0/602502147 wait complete. 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9619500000 20 client.5592 trim_cache size 0 max 16384 2024-09-17T05:25:29.040 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5439, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 11.220129744999999, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 30, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 47, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 47, 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 16.390998530000001 2024-09-17T05:25:29.041 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:25:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 msgr2=0x7f95e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 0x7f95e8075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f960c1660b0 tx=0x7f95fc009040 comp rx=0 tx=0).stop 2024-09-17T05:25:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 msgr2=0x7f960c157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d9b0 tx=0x7f95f400de70 comp rx=0 tx=0).stop 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 shutdown_connections 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f95e8073200 0x7f95e8075710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f960c070620 0x7f960c1f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f960c0cb860 0x7f960c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 --2- 172.21.15.43:0/3935002527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f960c06ce60 0x7f960c157b60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 >> 172.21.15.43:0/3935002527 conn(0x7f960c0b23d0 msgr2=0x7f960c169450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 shutdown_connections 2024-09-17T05:25:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.032+0000 7f9611e00640 1 -- 172.21.15.43:0/3935002527 wait complete. 2024-09-17T05:25:29.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- 172.21.15.43:0/2919666084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b806f110 msgr2=0x7fe3b80b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 --2- 172.21.15.43:0/2919666084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b806f110 0x7fe3b80b33e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8072b60 tx=0x7fe3a001c620 comp rx=0 tx=0).stop 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- 172.21.15.43:0/2919666084 shutdown_connections 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 --2- 172.21.15.43:0/2919666084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b806f110 0x7fe3b80b33e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 --2- 172.21.15.43:0/2919666084 >> v2:172.21.15.62:3300/0 conn(0x7fe3b8155d70 0x7fe3b815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- 172.21.15.43:0/2919666084 >> 172.21.15.43:0/2919666084 conn(0x7fe3b80b23d0 msgr2=0x7fe3b80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- 172.21.15.43:0/2919666084 shutdown_connections 2024-09-17T05:25:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- 172.21.15.43:0/2919666084 wait complete. 2024-09-17T05:25:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 Processor -- start 2024-09-17T05:25:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.212+0000 7fe3bf200640 1 -- start start 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 0x7fe3b81fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b8155d70 0x7fe3b8160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b8168a50 con 0x7fe3b80b3920 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81688d0 con 0x7fe3b806f110 2024-09-17T05:25:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b8168bd0 con 0x7fe3b8155d70 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54726/0 (socket says 172.21.15.43:54726) 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 -- 172.21.15.43:0/1746964400 learned_addr learned my addr 172.21.15.43:0/1746964400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bd200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b8155d70 0x7fe3b8160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3b7e00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 0x7fe3b81fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 -- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b8155d70 msgr2=0x7fe3b8160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b8155d70 0x7fe3b8160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 -- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 msgr2=0x7fe3b81fccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 0x7fe3b81fccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 -- 172.21.15.43:0/1746964400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b81fd1f0 con 0x7fe3b80b3920 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3b7e00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 0x7fe3b81fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3afe00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009850 tx=0x7fe3a00a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00ac070 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3a0004320 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b80ce1b0 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00053a0 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b81f8e80 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.216+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b81f8cb0 con 0x7fe3b80b3920 2024-09-17T05:25:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.220+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3a00af070 con 0x7fe3b80b3920 2024-09-17T05:25:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.220+0000 7fe3b4c00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 0x7fe394075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.220+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fe3a012e360 con 0x7fe3b80b3920 2024-09-17T05:25:29.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.220+0000 7fe3b7e00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 0x7fe394075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:29.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.220+0000 7fe3b7e00640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 0x7fe394075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005fd0 tx=0x7fe3a800a560 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.224+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3b80b3d30 con 0x7fe3b80b3920 2024-09-17T05:25:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3b80ce3e0 con 0x7fe3b80b3920 2024-09-17T05:25:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3b4c00640 1 -- 172.21.15.43:0/1746964400 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7fe3a00f9370 con 0x7fe3b80b3920 2024-09-17T05:25:29.440 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 msgr2=0x7fe394075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 0x7fe394075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005fd0 tx=0x7fe3a800a560 comp rx=0 tx=0).stop 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 msgr2=0x7fe3b8164b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009850 tx=0x7fe3a00a6580 comp rx=0 tx=0).stop 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 shutdown_connections 2024-09-17T05:25:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe394073200 0x7fe394075710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe3b8155d70 0x7fe3b8160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b80b3920 0x7fe3b8164b20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 --2- 172.21.15.43:0/1746964400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b806f110 0x7fe3b81fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 >> 172.21.15.43:0/1746964400 conn(0x7fe3b80b23d0 msgr2=0x7fe3b8165940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 shutdown_connections 2024-09-17T05:25:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29.436+0000 7fe3bf200640 1 -- 172.21.15.43:0/1746964400 wait complete. 2024-09-17T05:25:29.462 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:25:29.462 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:29.463 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:29.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.463 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:29.473 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42314sec preferred_lft 42314sec 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.474 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:25:29.475 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:25:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:29.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.477 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:29.533 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:25:29.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.535 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:25:29.588 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:25:29.588 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:29.588 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:25:29.588 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:29.589 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:25:29.589 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:25:29.589 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:25:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.728 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:25:29.736 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:25:29.736 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:25:29.737 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:29.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:25:29.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:29.800 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:29.855 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:29.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:29.855 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:29.906 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:29.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:30.012 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:30.013 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:30.013 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:30.031 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:30.031 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:30.031 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:30.031 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:25:30.031 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:25:30.042 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:25:30.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:30.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:30.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:30.067 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:30.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:30.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:30.179 INFO:teuthology.orchestra.run.smithi043.stdout:client5601 2024-09-17T05:25:30.179 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:30.179 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:30.188 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-09-17T05:25:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:30.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 -- 172.21.15.43:0/1153909719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f710006f110 msgr2=0x7f71000b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 --2- 172.21.15.43:0/1153909719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f710006f110 0x7f71000b33e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009900 tx=0x7f70e801c820 comp rx=0 tx=0).stop 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 -- 172.21.15.43:0/1153909719 shutdown_connections 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 --2- 172.21.15.43:0/1153909719 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f710006f110 0x7f71000b33e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 --2- 172.21.15.43:0/1153909719 >> v2:172.21.15.62:3300/0 conn(0x7f7100155d70 0x7f710015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 -- 172.21.15.43:0/1153909719 >> 172.21.15.43:0/1153909719 conn(0x7f71000b23d0 msgr2=0x7f71000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 -- 172.21.15.43:0/1153909719 shutdown_connections 2024-09-17T05:25:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.456+0000 7f7106400640 1 -- 172.21.15.43:0/1153909719 wait complete. 2024-09-17T05:25:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 Processor -- start 2024-09-17T05:25:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 -- start start 2024-09-17T05:25:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71000b3920 0x7f71000cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 0x7f71000cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7100168b10 con 0x7f7100155d70 2024-09-17T05:25:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7100168990 con 0x7f710006f110 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f7106400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7100168c90 con 0x7f71000b3920 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f70ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 0x7f71000cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f70ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:42380/0 (socket says 172.21.15.43:42380) 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f70ff400640 1 -- 172.21.15.43:0/3120126317 learned_addr learned my addr 172.21.15.43:0/3120126317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.460+0000 7f70f6a00640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71000b3920 0x7f71000cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 -- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71000b3920 msgr2=0x7f71000cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71000b3920 0x7f71000cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 -- 172.21.15.43:0/3120126317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 msgr2=0x7f71000cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 0x7f71000cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 -- 172.21.15.43:0/3120126317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71000cd700 con 0x7f710006f110 2024-09-17T05:25:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ffe00640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 0x7f71000cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70ff400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e770 tx=0x7f70f000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70f7e00640 1 -- 172.21.15.43:0/3120126317 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000ccc0 con 0x7f710006f110 2024-09-17T05:25:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70f7e00640 1 -- 172.21.15.43:0/3120126317 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f000ee80 con 0x7f710006f110 2024-09-17T05:25:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71000cd9e0 con 0x7f710006f110 2024-09-17T05:25:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70f7e00640 1 -- 172.21.15.43:0/3120126317 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000ce60 con 0x7f710006f110 2024-09-17T05:25:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710009b880 con 0x7f710006f110 2024-09-17T05:25:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.464+0000 7f70f7e00640 1 -- 172.21.15.43:0/3120126317 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70f0004260 con 0x7f710006f110 2024-09-17T05:25:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70f7e00640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 0x7f70dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70f7e00640 1 -- 172.21.15.43:0/3120126317 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f70f0094b90 con 0x7f710006f110 2024-09-17T05:25:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 Processor -- start 2024-09-17T05:25:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 -- start start 2024-09-17T05:25:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70f6a00640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 0x7f70dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70f6a00640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 0x7f70dc075740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70e80af000 tx=0x7f70e801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878cad76d0 0x55878cc439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c72e890 0x55878cc43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55878cae2cd0 con 0x55878c7f2aa0 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55878c9f8330 con 0x55878c72e890 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55878ca55bb0 con 0x55878cad76d0 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ff400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878cad76d0 0x55878cc439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70f6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c72e890 0x55878cc43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54750/0 (socket says 172.21.15.43:54750) 2024-09-17T05:25:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 -- 172.21.15.43:0/1820905602 learned_addr learned my addr 172.21.15.43:0/1820905602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 -- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878cad76d0 msgr2=0x55878cc439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878cad76d0 0x55878cc439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 -- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c72e890 msgr2=0x55878cc43f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c72e890 0x55878cc43f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 -- 172.21.15.43:0/1820905602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878c9f9c80 con 0x55878c7f2aa0 2024-09-17T05:25:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f70f800f1d0 tx=0x7f70f8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bad8a126dfb5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70efe00640 1 -- 172.21.15.43:0/1820905602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f800fe50 con 0x55878c7f2aa0 2024-09-17T05:25:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70efe00640 1 -- 172.21.15.43:0/1820905602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f8018070 con 0x55878c7f2aa0 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.468+0000 7f70efe00640 1 -- 172.21.15.43:0/1820905602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80050d0 con 0x55878c7f2aa0 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- 172.21.15.43:0/1820905602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 msgr2=0x55878cc44470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f70f800f1d0 tx=0x7f70f8001640 comp rx=0 tx=0).stop 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- 172.21.15.43:0/1820905602 shutdown_connections 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c72e890 0x55878cc43f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c7f2aa0 0x55878cc44470 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- 172.21.15.43:0/1820905602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878cad76d0 0x55878cc439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- 172.21.15.43:0/1820905602 >> 172.21.15.43:0/1820905602 conn(0x55878c7025c0 msgr2=0x55878cc00ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- 172.21.15.43:0/1820905602 shutdown_connections 2024-09-17T05:25:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- 172.21.15.43:0/1820905602 wait complete. 2024-09-17T05:25:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- start start 2024-09-17T05:25:30.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 Processor -- start 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878cbff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878cbffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55878ca48df0 con 0x55878cad76d0 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55878ca44fe0 con 0x55878c7f2aa0 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55878c9ea4d0 con 0x55878c72e890 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f710db4d000 10 client.0 _ll_register_callbacks cb 0x55878c9c87f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54762/0 (socket says 172.21.15.43:54762) 2024-09-17T05:25:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 -- 172.21.15.43:0/1089797336 learned_addr learned my addr 172.21.15.43:0/1089797336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70f6a00640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878cbffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 -- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 msgr2=0x55878cbff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ff400640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878cbff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878cbff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 -- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 msgr2=0x55878cbffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878cbffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ffe00640 1 -- 172.21.15.43:0/1089797336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878c9e3210 con 0x55878cad76d0 2024-09-17T05:25:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70ff400640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878cbff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.472+0000 7f70f6a00640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878cbffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ffe00640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f90 tx=0x7f70f8005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f559309d90e3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8021070 con 0x55878cad76d0 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f70f8026460 con 0x55878cad76d0 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55878c9dd350 con 0x55878cad76d0 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70f802e050 con 0x55878cad76d0 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 10 client.5610 fetch_fsmap learned FSMap version 30 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55878c9c9fc0 con 0x55878cad76d0 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55878c9dd350 con 0x55878cad76d0 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f70f8031020 con 0x55878cad76d0 2024-09-17T05:25:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 10 client.5610 fetch_fsmap finished waiting for FSMap version 30 2024-09-17T05:25:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 10 client.5610 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T05:25:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 20 client.5610 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6e7930 0x55878cc4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 4 client.5610 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 --> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55878ca57b00 con 0x55878c6e7930 2024-09-17T05:25:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ff400640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6e7930 0x55878cc4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ff400640 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6e7930 0x55878cc4ae80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab463ea87463e54 server_cookie=bc1cc93f9b50baa6 in_seq=0 out_seq=0 2024-09-17T05:25:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 10 client.5610 ms_handle_connect on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 1 -- 172.21.15.43:0/1089797336 <== mds.0 v2:172.21.15.62:6834/980890216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55878ca57b00 con 0x55878c6e7930 2024-09-17T05:25:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 10 client.5610 handle_command_reply: tid=0 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 client.5610 shutdown 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 msgr2=0x55878cc065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f90 tx=0x7f70f8005300 comp rx=0 tx=0).stop 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 shutdown_connections 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6e7930 0x55878cc4ae80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878cbffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878cc065f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 --2- 172.21.15.43:0/1089797336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878cbff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f70ecc00640 0 client.5610 ms_handle_reset on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 >> 172.21.15.43:0/1089797336 conn(0x55878c7025c0 msgr2=0x55878cbf8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 shutdown_connections 2024-09-17T05:25:30.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 1 -- 172.21.15.43:0/1089797336 wait complete. 2024-09-17T05:25:30.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.476+0000 7f710db4d000 20 client.5610 trim_cache size 0 max 16384 2024-09-17T05:25:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.492+0000 7f710db4d000 1 Processor -- start 2024-09-17T05:25:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.492+0000 7f710db4d000 1 -- start start 2024-09-17T05:25:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c7f2aa0 0x55878ccb5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878cad76d0 0x55878ccb5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7100205430 con 0x55878c72e890 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7100204470 con 0x55878cad76d0 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f71000cc920 con 0x55878c7f2aa0 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54768/0 (socket says 172.21.15.43:54768) 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 -- 172.21.15.43:0/4141965360 learned_addr learned my addr 172.21.15.43:0/4141965360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ffe00640 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878cad76d0 0x55878ccb5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70f6a00640 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c7f2aa0 0x55878ccb5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 -- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c7f2aa0 msgr2=0x55878ccb5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c7f2aa0 0x55878ccb5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 -- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878cad76d0 msgr2=0x55878ccb5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878cad76d0 0x55878ccb5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 -- 172.21.15.43:0/4141965360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878caa2680 con 0x55878c72e890 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ff400640 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f70f00058d0 tx=0x7f70f0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd66219002ced7f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ef400640 1 -- 172.21.15.43:0/4141965360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0063e70 con 0x55878c72e890 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ef400640 1 -- 172.21.15.43:0/4141965360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f0068db0 con 0x55878c72e890 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f70ef400640 1 -- 172.21.15.43:0/4141965360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f007f2c0 con 0x55878c72e890 2024-09-17T05:25:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- 172.21.15.43:0/4141965360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 msgr2=0x55878ccb4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f70f00058d0 tx=0x7f70f0009640 comp rx=0 tx=0).stop 2024-09-17T05:25:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- 172.21.15.43:0/4141965360 shutdown_connections 2024-09-17T05:25:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878cad76d0 0x55878ccb5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c7f2aa0 0x55878ccb5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 --2- 172.21.15.43:0/4141965360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878c72e890 0x55878ccb4fe0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- 172.21.15.43:0/4141965360 >> 172.21.15.43:0/4141965360 conn(0x55878c7025c0 msgr2=0x55878cc55fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- 172.21.15.43:0/4141965360 shutdown_connections 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- 172.21.15.43:0/4141965360 wait complete. 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.496+0000 7f710db4d000 1 -- start start 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 Processor -- start 2024-09-17T05:25:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878ccb9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878ccb9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55878ca3fee0 con 0x55878cad76d0 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55878ca0d610 con 0x55878c7f2aa0 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55878ca48100 con 0x55878c72e890 2024-09-17T05:25:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 10 client.0 _ll_register_callbacks cb 0x7f70f0005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54778/0 (socket says 172.21.15.43:54778) 2024-09-17T05:25:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 -- 172.21.15.43:0/2783574595 learned_addr learned my addr 172.21.15.43:0/2783574595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ff400640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878ccb9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70f6a00640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878ccb9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 -- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 msgr2=0x55878ccb9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878ccb9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 -- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 msgr2=0x55878ccb9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878ccb9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 -- 172.21.15.43:0/2783574595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878ca0f0b0 con 0x55878cad76d0 2024-09-17T05:25:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70f6a00640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878ccb9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ffe00640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f70f80012f0 tx=0x7f70f8026b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2142e433581cb528 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8021040 con 0x55878cad76d0 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f70f8006b60 con 0x55878cad76d0 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f0068db0 con 0x55878cad76d0 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70f8004eb0 con 0x55878cad76d0 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 10 client.5616 fetch_fsmap learned FSMap version 30 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55878c9db610 con 0x55878cad76d0 2024-09-17T05:25:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f0068db0 con 0x55878cad76d0 2024-09-17T05:25:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f70f8031040 con 0x55878cad76d0 2024-09-17T05:25:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 10 client.5616 fetch_fsmap finished waiting for FSMap version 30 2024-09-17T05:25:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 10 client.5616 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T05:25:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 20 client.5616 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6d5990 0x55878cbfa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 4 client.5616 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:25:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.500+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 --> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55878ca535b0 con 0x55878c6d5990 2024-09-17T05:25:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.504+0000 7f70ff400640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6d5990 0x55878cbfa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.504+0000 7f70ecc00640 10 client.5616 ms_handle_connect on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.504+0000 7f70ff400640 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6d5990 0x55878cbfa400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d1c1be8cca4ef4 server_cookie=14ba2c29426a5769 in_seq=0 out_seq=0 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f70ecc00640 1 -- 172.21.15.43:0/2783574595 <== mds.0 v2:172.21.15.62:6834/980890216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55878ca535b0 con 0x55878c6d5990 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f70ecc00640 10 client.5616 handle_command_reply: tid=0 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 client.5616 shutdown 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 msgr2=0x55878ccb9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f70f80012f0 tx=0x7f70f8026b10 comp rx=0 tx=0).stop 2024-09-17T05:25:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 shutdown_connections 2024-09-17T05:25:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:6834/980890216,v1:172.21.15.62:6835/980890216] conn(0x55878c6d5990 0x55878cbfa400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55878c7f2aa0 0x55878ccb9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55878cad76d0 0x55878ccb9aa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 --2- 172.21.15.43:0/2783574595 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55878c72e890 0x55878ccb9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f70ecc00640 0 client.5616 ms_handle_reset on v2:172.21.15.62:6834/980890216 2024-09-17T05:25:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 >> 172.21.15.43:0/2783574595 conn(0x55878c7025c0 msgr2=0x55878cc055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 shutdown_connections 2024-09-17T05:25:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 1 -- 172.21.15.43:0/2783574595 wait complete. 2024-09-17T05:25:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f710db4d000 20 client.5616 trim_cache size 0 max 16384 2024-09-17T05:25:30.529 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:25:30.529 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:25:30.529 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:25:30.529 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:25:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 msgr2=0x7f70dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 0x7f70dc075740 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70e80af000 tx=0x7f70e801b060 comp rx=0 tx=0).stop 2024-09-17T05:25:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 msgr2=0x7f710009b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e770 tx=0x7f70f000ec30 comp rx=0 tx=0).stop 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 shutdown_connections 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70dc073230 0x7f70dc075740 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7100155d70 0x7f71000cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71000b3920 0x7f71000cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 --2- 172.21.15.43:0/3120126317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f710006f110 0x7f710009b340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 >> 172.21.15.43:0/3120126317 conn(0x7f71000b23d0 msgr2=0x7f710009da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 shutdown_connections 2024-09-17T05:25:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.516+0000 7f7106400640 1 -- 172.21.15.43:0/3120126317 wait complete. 2024-09-17T05:25:30.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-17T05:25:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 -- 172.21.15.43:0/2403001872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a980048d0 msgr2=0x7f7a98004da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2403001872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a980048d0 0x7f7a98004da0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009d10 tx=0x7f7aa001d3f0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 -- 172.21.15.43:0/2403001872 shutdown_connections 2024-09-17T05:25:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2403001872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a980048d0 0x7f7a98004da0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 -- 172.21.15.43:0/2403001872 >> 172.21.15.43:0/2403001872 conn(0x7f7ab00b2810 msgr2=0x7f7ab00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 -- 172.21.15.43:0/2403001872 shutdown_connections 2024-09-17T05:25:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.700+0000 7f7ab8800640 1 -- 172.21.15.43:0/2403001872 wait complete. 2024-09-17T05:25:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 Processor -- start 2024-09-17T05:25:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- start start 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a980048d0 0x7f7ab015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ab0155ca0 0x7f7ab015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab00b4140 con 0x7f7ab0159ad0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab00b3fc0 con 0x7f7a980048d0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab00b42c0 con 0x7f7ab0155ca0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a980048d0 0x7f7ab015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ab0155ca0 0x7f7ab015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54794/0 (socket says 172.21.15.43:54794) 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 -- 172.21.15.43:0/2612665873 learned_addr learned my addr 172.21.15.43:0/2612665873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 -- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ab0155ca0 msgr2=0x7f7ab015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ab0155ca0 0x7f7ab015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 -- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a980048d0 msgr2=0x7f7ab015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a980048d0 0x7f7ab015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 -- 172.21.15.43:0/2612665873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab015dcd0 con 0x7f7ab0159ad0 2024-09-17T05:25:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab6800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0074d50 tx=0x7f7aac006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac011900 con 0x7f7ab0159ad0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7aac018070 con 0x7f7ab0159ad0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab015dfb0 con 0x7f7ab0159ad0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac011d60 con 0x7f7ab0159ad0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.704+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab015e570 con 0x7f7ab0159ad0 2024-09-17T05:25:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab00b4790 con 0x7f7ab0159ad0 2024-09-17T05:25:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7aac01d460 con 0x7f7ab0159ad0 2024-09-17T05:25:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7aaa000640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 0x7f7a94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f7aac095240 con 0x7f7ab0159ad0 2024-09-17T05:25:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7ab5e00640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 0x7f7a94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.708+0000 7f7ab5e00640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 0x7f7a94075710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ab006a400 tx=0x7f7aa4000fd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.712+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7aac05fd10 con 0x7f7ab0159ad0 2024-09-17T05:25:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.888+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f7ab00cd250 con 0x7f7ab0159ad0 2024-09-17T05:25:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7aaa000640 1 -- 172.21.15.43:0/2612665873 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v35)=0 v35) ==== 135+0+0 (secure 0 0 0) 0x7f7aac064740 con 0x7f7ab0159ad0 2024-09-17T05:25:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 msgr2=0x7f7a94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 0x7f7a94075710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7ab006a400 tx=0x7f7aa4000fd0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 msgr2=0x7f7ab0159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0074d50 tx=0x7f7aac006430 comp rx=0 tx=0).stop 2024-09-17T05:25:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 shutdown_connections 2024-09-17T05:25:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a94073200 0x7f7a94075710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab0159ad0 0x7f7ab0159ee0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ab0155ca0 0x7f7ab015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 --2- 172.21.15.43:0/2612665873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a980048d0 0x7f7ab015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 >> 172.21.15.43:0/2612665873 conn(0x7f7ab00b2810 msgr2=0x7f7ab00700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 shutdown_connections 2024-09-17T05:25:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:30.892+0000 7f7ab8800640 1 -- 172.21.15.43:0/2612665873 wait complete. 2024-09-17T05:25:30.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-17T05:25:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 -- 172.21.15.43:0/50491340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf58004810 msgr2=0x7faf58004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 --2- 172.21.15.43:0/50491340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf58004810 0x7faf58004c20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00c310 tx=0x7faf5c01cf20 comp rx=0 tx=0).stop 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 -- 172.21.15.43:0/50491340 shutdown_connections 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 --2- 172.21.15.43:0/50491340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf58004810 0x7faf58004c20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 -- 172.21.15.43:0/50491340 >> 172.21.15.43:0/50491340 conn(0x7faf740b2810 msgr2=0x7faf740b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 -- 172.21.15.43:0/50491340 shutdown_connections 2024-09-17T05:25:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.076+0000 7faf7a400640 1 -- 172.21.15.43:0/50491340 wait complete. 2024-09-17T05:25:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 Processor -- start 2024-09-17T05:25:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- start start 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf58004810 0x7faf741fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf740d0f70 0x7faf740ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf740b41b0 con 0x7faf741f7d80 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf740b4030 con 0x7faf58004810 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faf740b4330 con 0x7faf740d0f70 2024-09-17T05:25:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54822/0 (socket says 172.21.15.43:54822) 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 -- 172.21.15.43:0/1492773957 learned_addr learned my addr 172.21.15.43:0/1492773957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf73400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf58004810 0x7faf741fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf73e00640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf740d0f70 0x7faf740ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 -- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf740d0f70 msgr2=0x7faf740ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf740d0f70 0x7faf740ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 -- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf58004810 msgr2=0x7faf741fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf58004810 0x7faf741fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 -- 172.21.15.43:0/1492773957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740d1380 con 0x7faf741f7d80 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6be00640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01c9e0 tx=0x7faf5c0aea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c001d00 con 0x7faf741f7d80 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf5c006420 con 0x7faf741f7d80 2024-09-17T05:25:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf74144e10 con 0x7faf741f7d80 2024-09-17T05:25:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c006c80 con 0x7faf741f7d80 2024-09-17T05:25:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.080+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf74145310 con 0x7faf741f7d80 2024-09-17T05:25:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.084+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf74145080 con 0x7faf741f7d80 2024-09-17T05:25:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.084+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf5c0bf060 con 0x7faf741f7d80 2024-09-17T05:25:31.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.084+0000 7faf6b400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 0x7faf50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.084+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7faf5c12e750 con 0x7faf741f7d80 2024-09-17T05:25:31.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.084+0000 7faf73400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 0x7faf50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.088+0000 7faf73400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 0x7faf50075740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf64000c10 tx=0x7faf64006b80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.088+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf740d1380 con 0x7faf741f7d80 2024-09-17T05:25:31.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.264+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7faf740b4800 con 0x7faf741f7d80 2024-09-17T05:25:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.268+0000 7faf6b400640 1 -- 172.21.15.43:0/1492773957 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v36)=0 v36) ==== 138+0+0 (secure 0 0 0) 0x7faf5c016cd0 con 0x7faf741f7d80 2024-09-17T05:25:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 msgr2=0x7faf50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 0x7faf50075740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf64000c10 tx=0x7faf64006b80 comp rx=0 tx=0).stop 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 msgr2=0x7faf740d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01c9e0 tx=0x7faf5c0aea80 comp rx=0 tx=0).stop 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 shutdown_connections 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf50073230 0x7faf50075740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf740d0f70 0x7faf740ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf741f7d80 0x7faf740d0a30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 --2- 172.21.15.43:0/1492773957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf58004810 0x7faf741fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 >> 172.21.15.43:0/1492773957 conn(0x7faf740b2810 msgr2=0x7faf7415f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 shutdown_connections 2024-09-17T05:25:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:31.272+0000 7faf7a400640 1 -- 172.21.15.43:0/1492773957 wait complete. 2024-09-17T05:25:36.297 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-09-17T05:25:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:36.360 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-09-17T05:25:36.357+0000 7fbc99c00640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5601:44 nref=7 cr=0x562c0cd1d180)) (dversion lock w=1 last_client=5601) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562c0cf89680] with next_snap:2 2024-09-17T05:25:36.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:36.380 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:36.380 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:36.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- 172.21.15.43:0/1619158267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c84155d70 msgr2=0x7f7c8415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1619158267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c84155d70 0x7f7c8415c7e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009900 tx=0x7f7c7801c640 comp rx=0 tx=0).stop 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- 172.21.15.43:0/1619158267 shutdown_connections 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1619158267 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8406f110 0x7f7c840b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1619158267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c84155d70 0x7f7c8415c7e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- 172.21.15.43:0/1619158267 >> 172.21.15.43:0/1619158267 conn(0x7f7c840b23d0 msgr2=0x7f7c840d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- 172.21.15.43:0/1619158267 shutdown_connections 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- 172.21.15.43:0/1619158267 wait complete. 2024-09-17T05:25:36.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 Processor -- start 2024-09-17T05:25:36.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.596+0000 7f7c8b800640 1 -- start start 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c840b3920 0x7f7c840cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c84155d70 0x7f7c841fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84168a60 con 0x7f7c8406f110 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841688e0 con 0x7f7c840b3920 2024-09-17T05:25:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7c84168be0 con 0x7f7c84155d70 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52706/0 (socket says 172.21.15.43:52706) 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 -- 172.21.15.43:0/1105939282 learned_addr learned my addr 172.21.15.43:0/1105939282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c83e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c840b3920 0x7f7c840cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c89800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c84155d70 0x7f7c841fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 -- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c84155d70 msgr2=0x7f7c841fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c84155d70 0x7f7c841fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 -- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c840b3920 msgr2=0x7f7c840cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c840b3920 0x7f7c840cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 -- 172.21.15.43:0/1105939282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c840ca3c0 con 0x7f7c8406f110 2024-09-17T05:25:36.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c88e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7c7801cb20 tx=0x7f7c780a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780ac070 con 0x7f7c8406f110 2024-09-17T05:25:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c780054b0 con 0x7f7c8406f110 2024-09-17T05:25:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004290 con 0x7f7c8406f110 2024-09-17T05:25:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c840c5310 con 0x7f7c8406f110 2024-09-17T05:25:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c840c5af0 con 0x7f7c8406f110 2024-09-17T05:25:36.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.600+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c840b33e0 con 0x7f7c8406f110 2024-09-17T05:25:36.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.604+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c780af070 con 0x7f7c8406f110 2024-09-17T05:25:36.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.604+0000 7f7c80c00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 0x7f7c68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.604+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f7c7812e270 con 0x7f7c8406f110 2024-09-17T05:25:36.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.604+0000 7f7c83e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 0x7f7c68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.604+0000 7f7c83e00640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 0x7f7c68075710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c8415d0c0 tx=0x7f7c7400a250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.608+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c840ca3c0 con 0x7f7c8406f110 2024-09-17T05:25:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.792+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c8409df50 con 0x7f7c8406f110 2024-09-17T05:25:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.792+0000 7f7c80c00640 1 -- 172.21.15.43:0/1105939282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f7c840b3d30 con 0x7f7c8406f110 2024-09-17T05:25:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 msgr2=0x7f7c68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 0x7f7c68075710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c8415d0c0 tx=0x7f7c7400a250 comp rx=0 tx=0).stop 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 msgr2=0x7f7c840cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7c7801cb20 tx=0x7f7c780a6580 comp rx=0 tx=0).stop 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 shutdown_connections 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c68073200 0x7f7c68075710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c840b3920 0x7f7c840cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c84155d70 0x7f7c841fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 --2- 172.21.15.43:0/1105939282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8406f110 0x7f7c840cd0b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 >> 172.21.15.43:0/1105939282 conn(0x7f7c840b23d0 msgr2=0x7f7c8406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 shutdown_connections 2024-09-17T05:25:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:36.796+0000 7f7c8b800640 1 -- 172.21.15.43:0/1105939282 wait complete. 2024-09-17T05:25:36.821 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:25:36.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:36.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:36.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:25:36.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:36.851 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:25:36.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- 172.21.15.43:0/4102690490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa31015c7e0 msgr2=0x7fa31006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 --2- 172.21.15.43:0/4102690490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa31015c7e0 0x7fa31006f110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa308009b70 tx=0x7fa30801c980 comp rx=0 tx=0).stop 2024-09-17T05:25:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- 172.21.15.43:0/4102690490 shutdown_connections 2024-09-17T05:25:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 --2- 172.21.15.43:0/4102690490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa31015c7e0 0x7fa31006f110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- 172.21.15.43:0/4102690490 >> 172.21.15.43:0/4102690490 conn(0x7fa3100b23d0 msgr2=0x7fa3100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- 172.21.15.43:0/4102690490 shutdown_connections 2024-09-17T05:25:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- 172.21.15.43:0/4102690490 wait complete. 2024-09-17T05:25:37.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 Processor -- start 2024-09-17T05:25:37.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.108+0000 7fa316c00640 1 -- start start 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa310155d70 0x7fa3100cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa31015c7e0 0x7fa3100ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa310169e60 con 0x7fa3100cd620 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa310169ce0 con 0x7fa31015c7e0 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa310169fe0 con 0x7fa310155d70 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52710/0 (socket says 172.21.15.43:52710) 2024-09-17T05:25:37.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 -- 172.21.15.43:0/3023094736 learned_addr learned my addr 172.21.15.43:0/3023094736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:37.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa307e00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa31015c7e0 0x7fa3100ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa30fe00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa310155d70 0x7fa3100cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 -- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa310155d70 msgr2=0x7fa3100cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa310155d70 0x7fa3100cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 -- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa31015c7e0 msgr2=0x7fa3100ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa31015c7e0 0x7fa3100ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 -- 172.21.15.43:0/3023094736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3100b3d50 con 0x7fa3100cd620 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa314c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa30801cef0 tx=0x7fa308009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080b2040 con 0x7fa3100cd620 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3080bb070 con 0x7fa3100cd620 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3100af960 con 0x7fa3100cd620 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308004040 con 0x7fa3100cd620 2024-09-17T05:25:37.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3100b1bd0 con 0x7fa3100cd620 2024-09-17T05:25:37.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.112+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3100c85f0 con 0x7fa3100cd620 2024-09-17T05:25:37.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.116+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3080041e0 con 0x7fa3100cd620 2024-09-17T05:25:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.116+0000 7fa30cc00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 0x7fa2ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.116+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa30812e670 con 0x7fa3100cd620 2024-09-17T05:25:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.116+0000 7fa30fe00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 0x7fa2ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.116+0000 7fa30fe00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 0x7fa2ec075740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa300007fd0 tx=0x7fa300009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.120+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa308016690 con 0x7fa3100cd620 2024-09-17T05:25:37.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.328+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa31006f110 con 0x7fa3100cd620 2024-09-17T05:25:37.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.328+0000 7fa30cc00640 1 -- 172.21.15.43:0/3023094736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5217 (secure 0 0 0) 0x7fa3080f8fb0 con 0x7fa3100cd620 2024-09-17T05:25:37.334 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:37.334 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-17T05:25:20:838273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:20.251677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/980890216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":980890216},{"type":"v1","addr":"172.21.15.62:6835","nonce":980890216}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5439,"qdb_cluster":[5439]},"id":3}]} 2024-09-17T05:25:37.334 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-17T05:25:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 msgr2=0x7fa2ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 0x7fa2ec075740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa300007fd0 tx=0x7fa300009040 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 msgr2=0x7fa3100cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa30801cef0 tx=0x7fa308009200 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 shutdown_connections 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2ec073230 0x7fa2ec075740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa31015c7e0 0x7fa3100ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3100cd620 0x7fa3100cae20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 --2- 172.21.15.43:0/3023094736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa310155d70 0x7fa3100cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 >> 172.21.15.43:0/3023094736 conn(0x7fa3100b23d0 msgr2=0x7fa3100b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 shutdown_connections 2024-09-17T05:25:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.332+0000 7fa316c00640 1 -- 172.21.15.43:0/3023094736 wait complete. 2024-09-17T05:25:37.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 -- 172.21.15.43:0/547154703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef00cccb0 msgr2=0x7fcef00b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 --2- 172.21.15.43:0/547154703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef00cccb0 0x7fcef00b43f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcef00cf8a0 tx=0x7fcedc01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 -- 172.21.15.43:0/547154703 shutdown_connections 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 --2- 172.21.15.43:0/547154703 >> v2:172.21.15.62:3300/0 conn(0x7fcef00cd0c0 0x7fcef00b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 --2- 172.21.15.43:0/547154703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef00cccb0 0x7fcef00b43f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 -- 172.21.15.43:0/547154703 >> 172.21.15.43:0/547154703 conn(0x7fcef00691b0 msgr2=0x7fcef00695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 -- 172.21.15.43:0/547154703 shutdown_connections 2024-09-17T05:25:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.508+0000 7fcef6800640 1 -- 172.21.15.43:0/547154703 wait complete. 2024-09-17T05:25:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 Processor -- start 2024-09-17T05:25:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 -- start start 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 0x7fcef0149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 0x7fcef014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01532f0 con 0x7fcef015cea0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0153170 con 0x7fcef00cd0c0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef6800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcef0153470 con 0x7fcef00cccb0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52728/0 (socket says 172.21.15.43:52728) 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef5e00640 1 -- 172.21.15.43:0/2875876119 learned_addr learned my addr 172.21.15.43:0/2875876119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef5400640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 0x7fcef0149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.512+0000 7fcef4a00640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 0x7fcef014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 -- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 msgr2=0x7fcef0149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 0x7fcef0149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 -- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 msgr2=0x7fcef014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 0x7fcef014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 -- 172.21.15.43:0/2875876119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef01507f0 con 0x7fcef015cea0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5400640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 0x7fcef0149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef4a00640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 0x7fcef014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef5e00640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9a0 tx=0x7fcee800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8004d30 con 0x7fcef015cea0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcee800cac0 con 0x7fcef015cea0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8005020 con 0x7fcef015cea0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef00d1370 con 0x7fcef015cea0 2024-09-17T05:25:37.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef00b7b60 con 0x7fcef015cea0 2024-09-17T05:25:37.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef006a1e0 con 0x7fcef015cea0 2024-09-17T05:25:37.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcee80051c0 con 0x7fcef015cea0 2024-09-17T05:25:37.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.516+0000 7fceed600640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 0x7fced0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.520+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fcee8094a70 con 0x7fcef015cea0 2024-09-17T05:25:37.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.520+0000 7fcef5400640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 0x7fced0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.520+0000 7fcef5400640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 0x7fced0075740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcee40098b0 tx=0x7fcee4008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.524+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcee8016030 con 0x7fcef015cea0 2024-09-17T05:25:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.728+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcef00b2f50 con 0x7fcef015cea0 2024-09-17T05:25:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fceed600640 1 -- 172.21.15.43:0/2875876119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5217 (secure 0 0 0) 0x7fcee805f9f0 con 0x7fcef015cea0 2024-09-17T05:25:37.736 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-17T05:25:20:838273+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:20.251677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/980890216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":980890216},{"type":"v1","addr":"172.21.15.62:6835","nonce":980890216}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5439,"qdb_cluster":[5439]},"id":3}]} 2024-09-17T05:25:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-17T05:25:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 msgr2=0x7fced0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 0x7fced0075740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcee40098b0 tx=0x7fcee4008040 comp rx=0 tx=0).stop 2024-09-17T05:25:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 msgr2=0x7fcef014edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9a0 tx=0x7fcee800ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 shutdown_connections 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fced0073230 0x7fced0075740 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcef00cd0c0 0x7fcef014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef015cea0 0x7fcef014edf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 --2- 172.21.15.43:0/2875876119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcef00cccb0 0x7fcef0149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 >> 172.21.15.43:0/2875876119 conn(0x7fcef00691b0 msgr2=0x7fcef00b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 shutdown_connections 2024-09-17T05:25:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:37.732+0000 7fcef6800640 1 -- 172.21.15.43:0/2875876119 wait complete. 2024-09-17T05:25:37.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump tree /a 0 2024-09-17T05:25:37.944 INFO:teuthology.orchestra.run.smithi062.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-17T05:25:36.352630+0000","btime":"2024-09-17T05:25:30.284694+0000","mode":16893,"uid":1000,"gid":1271,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-17T05:25:36.352630+0000","atime":"2024-09-17T05:25:30.284694+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-17T05:25:36.352630+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-17T05:25:36.352630+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-17T05:25:36.352630+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-17T05:25:37.946 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-17T05:25:36.352630+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-17T05:25:30.284694+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-17T05:25:30.284694+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-17T05:25:36.352630+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-17T05:25:36.352630+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1271, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2024-09-17T05:25:36.352630+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-17T05:25:36.352630+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-09-17T05:25:37.946 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-17T05:25:36.352630+0000', 'btime': '2024-09-17T05:25:30.284694+0000', 'mode': 16893, 'uid': 1000, 'gid': 1271, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-17T05:25:36.352630+0000', 'atime': '2024-09-17T05:25:30.284694+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-17T05:25:36.352630+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-17T05:25:36.352630+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-17T05:25:36.352630+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-17T05:25:37.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 -- 172.21.15.43:0/2161722185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43340cb860 msgr2=0x7f43340cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 --2- 172.21.15.43:0/2161722185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43340cb860 0x7f43340cbc70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f432c009bc0 tx=0x7f432c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 -- 172.21.15.43:0/2161722185 shutdown_connections 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 --2- 172.21.15.43:0/2161722185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43340cb860 0x7f43340cbc70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 --2- 172.21.15.43:0/2161722185 >> v2:172.21.15.62:3300/0 conn(0x7f4334070620 0x7f43340b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 -- 172.21.15.43:0/2161722185 >> 172.21.15.43:0/2161722185 conn(0x7f43340b23d0 msgr2=0x7f43340d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 -- 172.21.15.43:0/2161722185 shutdown_connections 2024-09-17T05:25:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.104+0000 7f433a200640 1 -- 172.21.15.43:0/2161722185 wait complete. 2024-09-17T05:25:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 Processor -- start 2024-09-17T05:25:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- start start 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 0x7f4334153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43340cb860 0x7f43340c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43340ce5a0 con 0x7f4334070620 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43340ce420 con 0x7f43340cb860 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43340ce720 con 0x7f433406ce60 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52732/0 (socket says 172.21.15.43:52732) 2024-09-17T05:25:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 -- 172.21.15.43:0/3921864049 learned_addr learned my addr 172.21.15.43:0/3921864049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f4333400640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 0x7f4334153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f4333e00640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43340cb860 0x7f43340c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 -- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 msgr2=0x7f4334153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 0x7f4334153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 -- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43340cb860 msgr2=0x7f43340c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43340cb860 0x7f43340c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 -- 172.21.15.43:0/3921864049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433415a670 con 0x7f4334070620 2024-09-17T05:25:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f4333400640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 0x7f4334153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432be00640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c011790 con 0x7f4334070620 2024-09-17T05:25:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f4334070620 2024-09-17T05:25:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433415a8e0 con 0x7f4334070620 2024-09-17T05:25:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c011ce0 con 0x7f4334070620 2024-09-17T05:25:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.108+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341fa700 con 0x7f4334070620 2024-09-17T05:25:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43340cecb0 con 0x7f4334070620 2024-09-17T05:25:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f431c01c460 con 0x7f4334070620 2024-09-17T05:25:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f432b400640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 0x7f4310075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f431c093f80 con 0x7f4334070620 2024-09-17T05:25:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f4333400640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 0x7f4310075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.112+0000 7f4333400640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 0x7f4310075710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4324009f30 tx=0x7f43240091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.116+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f431c05e970 con 0x7f4334070620 2024-09-17T05:25:38.287 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:25:38.284+0000 7fcc80800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-17T05:25:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.332+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4334075d30 con 0x7f4334070620 2024-09-17T05:25:38.337 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:25:38.332+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:25:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f432b400640 1 -- 172.21.15.43:0/3921864049 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7f431c0633a0 con 0x7f4334070620 2024-09-17T05:25:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 msgr2=0x7f4310075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 0x7f4310075710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4324009f30 tx=0x7f43240091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 msgr2=0x7f43340c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c430 tx=0x7f431c00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 shutdown_connections 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4310073200 0x7f4310075710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43340cb860 0x7f43340c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4334070620 0x7f43340c9110 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 --2- 172.21.15.43:0/3921864049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433406ce60 0x7f4334153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 >> 172.21.15.43:0/3921864049 conn(0x7f43340b23d0 msgr2=0x7f43340d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.336+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 shutdown_connections 2024-09-17T05:25:38.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.340+0000 7f433a200640 1 -- 172.21.15.43:0/3921864049 wait complete. 2024-09-17T05:25:38.352 INFO:tasks.ceph.mds.b.smithi062.stderr: -4354> 2024-09-17T05:25:23.041+0000 7fbc96a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:38.359 INFO:tasks.ceph.mds.b.smithi062.stderr: -391> 2024-09-17T05:25:36.357+0000 7fbc99c00640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5601:44 nref=7 cr=0x562c0cd1d180)) (dversion lock w=1 last_client=5601) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562c0cf89680] with next_snap:2 2024-09-17T05:25:38.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-17T05:25:38.472 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 -- 172.21.15.43:0/2533364321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 msgr2=0x7f7c8c0ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 --2- 172.21.15.43:0/2533364321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c0ae0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009840 tx=0x7f7c7c01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 -- 172.21.15.43:0/2533364321 shutdown_connections 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 --2- 172.21.15.43:0/2533364321 >> v2:172.21.15.62:3300/0 conn(0x7f7c8c0ae5f0 0x7f7c8c0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 --2- 172.21.15.43:0/2533364321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c0ae0b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 -- 172.21.15.43:0/2533364321 >> 172.21.15.43:0/2533364321 conn(0x7f7c8c06d9f0 msgr2=0x7f7c8c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 -- 172.21.15.43:0/2533364321 shutdown_connections 2024-09-17T05:25:38.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 -- 172.21.15.43:0/2533364321 wait complete. 2024-09-17T05:25:38.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.520+0000 7f7c93200640 1 Processor -- start 2024-09-17T05:25:38.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- start start 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 0x7f7c8c14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c168dc0 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c168c40 con 0x7f7c8c15b9f0 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8c168f40 con 0x7f7c8c15c510 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52744/0 (socket says 172.21.15.43:52744) 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 -- 172.21.15.43:0/3639873204 learned_addr learned my addr 172.21.15.43:0/3639873204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 -- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 msgr2=0x7f7c8c14fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c92800640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91400640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 0x7f7c8c14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c14fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 -- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 msgr2=0x7f7c8c14f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 0x7f7c8c14f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 -- 172.21.15.43:0/3639873204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c14b340 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91400640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 0x7f7c8c14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c92800640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c91e00640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80011790 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c80017070 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c15e9e0 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80011ce0 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c15cfc0 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.524+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c0aea00 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.528+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c8001c460 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.528+0000 7f7c86000640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 0x7f7c70075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.528+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f7c80093fe0 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.528+0000 7f7c91400640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 0x7f7c70075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.528+0000 7f7c91400640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 0x7f7c70075740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0af000 tx=0x7f7c7c01b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.532+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c8005f120 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.712+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f7c8c0b2840 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c86000640 1 -- 172.21.15.43:0/3639873204 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7f7c8005ea20 con 0x7f7c8c0ae5f0 2024-09-17T05:25:38.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 msgr2=0x7f7c70075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 0x7f7c70075740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0af000 tx=0x7f7c7c01b060 comp rx=0 tx=0).stop 2024-09-17T05:25:38.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 msgr2=0x7f7c8c14ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 shutdown_connections 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c70073230 0x7f7c70075740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15b9f0 0x7f7c8c14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c8c15c510 0x7f7c8c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 --2- 172.21.15.43:0/3639873204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c8c0ae5f0 0x7f7c8c14ae00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 >> 172.21.15.43:0/3639873204 conn(0x7f7c8c06d9f0 msgr2=0x7f7c8c0cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 shutdown_connections 2024-09-17T05:25:38.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.716+0000 7f7c93200640 1 -- 172.21.15.43:0/3639873204 wait complete. 2024-09-17T05:25:38.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- 172.21.15.43:0/673454199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5406f110 msgr2=0x7f8d540b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 --2- 172.21.15.43:0/673454199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5406f110 0x7f8d540b33e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009900 tx=0x7f8d4001c6e0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- 172.21.15.43:0/673454199 shutdown_connections 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 --2- 172.21.15.43:0/673454199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5406f110 0x7f8d540b33e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 --2- 172.21.15.43:0/673454199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 0x7f8d5415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- 172.21.15.43:0/673454199 >> 172.21.15.43:0/673454199 conn(0x7f8d540b23d0 msgr2=0x7f8d540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- 172.21.15.43:0/673454199 shutdown_connections 2024-09-17T05:25:38.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- 172.21.15.43:0/673454199 wait complete. 2024-09-17T05:25:38.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 Processor -- start 2024-09-17T05:25:38.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.900+0000 7f8d5be00640 1 -- start start 2024-09-17T05:25:38.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 0x7f8d5409d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 0x7f8d540b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540adb50 con 0x7f8d5406f110 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540ad9d0 con 0x7f8d540b3920 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8d540adcd0 con 0x7f8d54155d70 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52760/0 (socket says 172.21.15.43:52760) 2024-09-17T05:25:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 -- 172.21.15.43:0/1430468789 learned_addr learned my addr 172.21.15.43:0/1430468789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:38.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 -- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 msgr2=0x7f8d540b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59e00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 0x7f8d540b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 0x7f8d540b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 -- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 msgr2=0x7f8d5409d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d58a00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 0x7f8d5409d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 0x7f8d5409d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 -- 172.21.15.43:0/1430468789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541f7450 con 0x7f8d5406f110 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d58a00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 0x7f8d5409d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d59400640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e9c0 tx=0x7f8d4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800cd80 con 0x7f8d5406f110 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d48010520 con 0x7f8d5406f110 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48010970 con 0x7f8d5406f110 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541f7730 con 0x7f8d5406f110 2024-09-17T05:25:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.904+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d540cd370 con 0x7f8d5406f110 2024-09-17T05:25:38.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d540b33e0 con 0x7f8d5406f110 2024-09-17T05:25:38.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8d48010b10 con 0x7f8d5406f110 2024-09-17T05:25:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d51600640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 0x7f8d34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f8d48094e30 con 0x7f8d5406f110 2024-09-17T05:25:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d58a00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 0x7f8d34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.908+0000 7f8d58a00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 0x7f8d34075710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af000 tx=0x7f8d4001af20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:38.912+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d4805fe00 con 0x7f8d5406f110 2024-09-17T05:25:39.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f8d5406dd50 con 0x7f8d5406f110 2024-09-17T05:25:39.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d51600640 1 -- 172.21.15.43:0/1430468789 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v37)=0 v37) ==== 135+0+0 (secure 0 0 0) 0x7f8d4805f700 con 0x7f8d5406f110 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 msgr2=0x7f8d34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 0x7f8d34075710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d400af000 tx=0x7f8d4001af20 comp rx=0 tx=0).stop 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 msgr2=0x7f8d540c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800e9c0 tx=0x7f8d4800ee80 comp rx=0 tx=0).stop 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 shutdown_connections 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8d34073200 0x7f8d34075710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d540b3920 0x7f8d5409d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8d54155d70 0x7f8d540b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 --2- 172.21.15.43:0/1430468789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d5406f110 0x7f8d540c68e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.088+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 >> 172.21.15.43:0/1430468789 conn(0x7f8d540b23d0 msgr2=0x7f8d5415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:39.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.092+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 shutdown_connections 2024-09-17T05:25:39.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.092+0000 7f8d5be00640 1 -- 172.21.15.43:0/1430468789 wait complete. 2024-09-17T05:25:39.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:25:39.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 -- 172.21.15.43:0/2066066789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e18155d70 msgr2=0x7f3e1815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 --2- 172.21.15.43:0/2066066789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e18155d70 0x7f3e1815c7e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009900 tx=0x7f3e0801c820 comp rx=0 tx=0).stop 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 -- 172.21.15.43:0/2066066789 shutdown_connections 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 --2- 172.21.15.43:0/2066066789 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e180b3af0 0x7f3e180ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 --2- 172.21.15.43:0/2066066789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e18155d70 0x7f3e1815c7e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 -- 172.21.15.43:0/2066066789 >> 172.21.15.43:0/2066066789 conn(0x7f3e180b23d0 msgr2=0x7f3e180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 -- 172.21.15.43:0/2066066789 shutdown_connections 2024-09-17T05:25:39.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.272+0000 7f3e1d000640 1 -- 172.21.15.43:0/2066066789 wait complete. 2024-09-17T05:25:39.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 Processor -- start 2024-09-17T05:25:39.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- start start 2024-09-17T05:25:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1806f110 0x7f3e1809ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e18155d70 0x7f3e1809a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18168c60 con 0x7f3e180ccab0 2024-09-17T05:25:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18168ae0 con 0x7f3e18155d70 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18168de0 con 0x7f3e1806f110 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52786/0 (socket says 172.21.15.43:52786) 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 -- 172.21.15.43:0/3198817505 learned_addr learned my addr 172.21.15.43:0/3198817505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e15e00640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1806f110 0x7f3e1809ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e0fe00640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e18155d70 0x7f3e1809a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 -- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1806f110 msgr2=0x7f3e1809ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1806f110 0x7f3e1809ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 -- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e18155d70 msgr2=0x7f3e1809a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e18155d70 0x7f3e1809a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 -- 172.21.15.43:0/3198817505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e180cd400 con 0x7f3e180ccab0 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e16800640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e9a0 tx=0x7f3e1000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:39.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10004d30 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e1000cb20 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10005020 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e181fa1a0 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.276+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e181fa760 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.280+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1809b1c0 con 0x7f3e180ccab0 2024-09-17T05:25:39.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.280+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e100051c0 con 0x7f3e180ccab0 2024-09-17T05:25:39.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.280+0000 7f3e0e000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.280+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f3e10094ff0 con 0x7f3e180ccab0 2024-09-17T05:25:39.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.280+0000 7f3e15e00640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.284+0000 7f3e15e00640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 0x7f3df4075710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e080af000 tx=0x7f3e0801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:39.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.284+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e180cd400 con 0x7f3e180ccab0 2024-09-17T05:25:39.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:39.492+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3e180ad0b0 con 0x7f3e180ccab0 2024-09-17T05:25:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e0e000640 1 -- 172.21.15.43:0/3198817505 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v34) ==== 162+0+0 (secure 0 0 0) 0x7f3e10016030 con 0x7f3e180ccab0 2024-09-17T05:25:40.348 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:25:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 msgr2=0x7f3df4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 0x7f3df4075710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e080af000 tx=0x7f3e0801b060 comp rx=0 tx=0).stop 2024-09-17T05:25:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 msgr2=0x7f3e180ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e9a0 tx=0x7f3e1000ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 shutdown_connections 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e18155d70 0x7f3e1809a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ccab0 0x7f3e180ccec0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 --2- 172.21.15.43:0/3198817505 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3e1806f110 0x7f3e1809ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 >> 172.21.15.43:0/3198817505 conn(0x7f3e180b23d0 msgr2=0x7f3e1815da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 shutdown_connections 2024-09-17T05:25:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.344+0000 7f3e1d000640 1 -- 172.21.15.43:0/3198817505 wait complete. 2024-09-17T05:25:40.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 -- 172.21.15.43:0/3125081069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 msgr2=0x7fab640cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 --2- 172.21.15.43:0/3125081069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640cbc70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009c80 tx=0x7fab5c01ccb0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 -- 172.21.15.43:0/3125081069 shutdown_connections 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 --2- 172.21.15.43:0/3125081069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640cbc70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 --2- 172.21.15.43:0/3125081069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab6406ce60 0x7fab6406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 -- 172.21.15.43:0/3125081069 >> 172.21.15.43:0/3125081069 conn(0x7fab640b23d0 msgr2=0x7fab640d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 -- 172.21.15.43:0/3125081069 shutdown_connections 2024-09-17T05:25:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.536+0000 7fab69e00640 1 -- 172.21.15.43:0/3125081069 wait complete. 2024-09-17T05:25:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 Processor -- start 2024-09-17T05:25:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- start start 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6406ce60 0x7fab6409a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 0x7fab640c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab640ce690 con 0x7fab640cb860 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab640ce510 con 0x7fab6406ce60 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fab640ce810 con 0x7fab64070620 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52804/0 (socket says 172.21.15.43:52804) 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 -- 172.21.15.43:0/1633221233 learned_addr learned my addr 172.21.15.43:0/1633221233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab62a00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6406ce60 0x7fab6409a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab5be00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 0x7fab640c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 -- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 msgr2=0x7fab640c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 0x7fab640c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 -- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6406ce60 msgr2=0x7fab6409a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6406ce60 0x7fab6409a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 -- 172.21.15.43:0/1633221233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab640c6c80 con 0x7fab640cb860 2024-09-17T05:25:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab5be00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 0x7fab640c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:40.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab63400640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01a940 tx=0x7fab5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0bb070 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab5c004a50 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0b2040 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab640c4a40 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.540+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab640c8610 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.544+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab640cedc0 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.544+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fab5c0be070 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.544+0000 7fab5b400640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 0x7fab40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.544+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fab5c12e1f0 con 0x7fab640cb860 2024-09-17T05:25:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.544+0000 7fab62a00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 0x7fab40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.548+0000 7fab62a00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 0x7fab40075710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab54007fd0 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.548+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fab5c016990 con 0x7fab640cb860 2024-09-17T05:25:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.760+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab640c8840 con 0x7fab640cb860 2024-09-17T05:25:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.760+0000 7fab5b400640 1 -- 172.21.15.43:0/1633221233 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5214 (secure 0 0 0) 0x7fab5c0f8b50 con 0x7fab640cb860 2024-09-17T05:25:40.766 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:40.766 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"btime":"2024-09-17T05:25:40:352360+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:40.352344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T05:25:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 msgr2=0x7fab40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 0x7fab40075710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fab54007fd0 tx=0x7fab54009040 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 msgr2=0x7fab640c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01a940 tx=0x7fab5c004270 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 shutdown_connections 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab40073200 0x7fab40075710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab640cb860 0x7fab640c7c90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab64070620 0x7fab640c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 --2- 172.21.15.43:0/1633221233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab6406ce60 0x7fab6409a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 >> 172.21.15.43:0/1633221233 conn(0x7fab640b23d0 msgr2=0x7fab6415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 shutdown_connections 2024-09-17T05:25:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.764+0000 7fab69e00640 1 -- 172.21.15.43:0/1633221233 wait complete. 2024-09-17T05:25:40.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-17T05:25:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- 172.21.15.43:0/1540833098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c0b5fe0 msgr2=0x7fa88c15f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 --2- 172.21.15.43:0/1540833098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c0b5fe0 0x7fa88c15f530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009900 tx=0x7fa87c01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- 172.21.15.43:0/1540833098 shutdown_connections 2024-09-17T05:25:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 --2- 172.21.15.43:0/1540833098 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b43c0 0x7fa88c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 --2- 172.21.15.43:0/1540833098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c0b5fe0 0x7fa88c15f530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- 172.21.15.43:0/1540833098 >> 172.21.15.43:0/1540833098 conn(0x7fa88c0b2e80 msgr2=0x7fa88c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- 172.21.15.43:0/1540833098 shutdown_connections 2024-09-17T05:25:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- 172.21.15.43:0/1540833098 wait complete. 2024-09-17T05:25:40.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 Processor -- start 2024-09-17T05:25:40.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.948+0000 7fa892200640 1 -- start start 2024-09-17T05:25:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c0b43c0 0x7fa88c0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b5fe0 0x7fa88c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c169130 con 0x7fa88c15fa70 2024-09-17T05:25:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c168fb0 con 0x7fa88c0b43c0 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c1692b0 con 0x7fa88c0b5fe0 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52824/0 (socket says 172.21.15.43:52824) 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 -- 172.21.15.43:0/4047119128 learned_addr learned my addr 172.21.15.43:0/4047119128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88b400640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c0b43c0 0x7fa88c0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa882a00640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b5fe0 0x7fa88c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 -- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b5fe0 msgr2=0x7fa88c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b5fe0 0x7fa88c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 -- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c0b43c0 msgr2=0x7fa88c0afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c0b43c0 0x7fa88c0afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 -- 172.21.15.43:0/4047119128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c0cc140 con 0x7fa88c15fa70 2024-09-17T05:25:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa88be00640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa88400eab0 tx=0x7fa884009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa883e00640 1 -- 172.21.15.43:0/4047119128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88400ce40 con 0x7fa88c15fa70 2024-09-17T05:25:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa883e00640 1 -- 172.21.15.43:0/4047119128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa884004c20 con 0x7fa88c15fa70 2024-09-17T05:25:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa883e00640 1 -- 172.21.15.43:0/4047119128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884005020 con 0x7fa88c15fa70 2024-09-17T05:25:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c0af190 con 0x7fa88c15fa70 2024-09-17T05:25:40.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.952+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c164bd0 con 0x7fa88c15fa70 2024-09-17T05:25:40.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa883e00640 1 -- 172.21.15.43:0/4047119128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8840051e0 con 0x7fa88c15fa70 2024-09-17T05:25:40.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa883e00640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 0x7fa868075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa883e00640 1 -- 172.21.15.43:0/4047119128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fa884094fe0 con 0x7fa88c15fa70 2024-09-17T05:25:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa899a6b000 1 Processor -- start 2024-09-17T05:25:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa899a6b000 1 -- start start 2024-09-17T05:25:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa88b400640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 0x7fa868075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.956+0000 7fa88b400640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 0x7fa868075710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0af000 tx=0x7fa87c01b0f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 0x562ccd0439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd043f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cccee2cd0 con 0x562cccbf2aa0 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562cccdf8330 con 0x562ccced76d0 2024-09-17T05:25:40.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562ccce55bb0 con 0x562cccb2e890 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52832/0 (socket says 172.21.15.43:52832) 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 -- 172.21.15.43:0/4162537521 learned_addr learned my addr 172.21.15.43:0/4162537521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa882a00640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd043f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88b400640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 0x562ccd0439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 -- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 msgr2=0x562ccd043f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd043f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 -- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 msgr2=0x562ccd0439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 0x562ccd0439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 -- 172.21.15.43:0/4162537521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cccdf9c80 con 0x562cccbf2aa0 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa882a00640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd043f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88b400640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 0x562ccd0439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa88be00640 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa8840934d0 tx=0x7fa88405f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44c0a8d6764f011 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa87b400640 1 -- 172.21.15.43:0/4162537521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88401d070 con 0x562cccbf2aa0 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa87b400640 1 -- 172.21.15.43:0/4162537521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa884064150 con 0x562cccbf2aa0 2024-09-17T05:25:40.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa87b400640 1 -- 172.21.15.43:0/4162537521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88405fd60 con 0x562cccbf2aa0 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- 172.21.15.43:0/4162537521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 msgr2=0x562ccd044470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa8840934d0 tx=0x7fa88405f7a0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- 172.21.15.43:0/4162537521 shutdown_connections 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccbf2aa0 0x562ccd044470 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd043f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 --2- 172.21.15.43:0/4162537521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ccced76d0 0x562ccd0439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- 172.21.15.43:0/4162537521 >> 172.21.15.43:0/4162537521 conn(0x562cccb025c0 msgr2=0x562ccd000ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- 172.21.15.43:0/4162537521 shutdown_connections 2024-09-17T05:25:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- 172.21.15.43:0/4162537521 wait complete. 2024-09-17T05:25:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 -- start start 2024-09-17T05:25:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.960+0000 7fa899a6b000 1 Processor -- start 2024-09-17T05:25:40.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562cccffff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562ccce48df0 con 0x562cccb2e890 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562ccce44fe0 con 0x562cccbf2aa0 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562cccdea4d0 con 0x562ccced76d0 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 10 client.0 _ll_register_callbacks cb 0x562cccdc87f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52836/0 (socket says 172.21.15.43:52836) 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 -- 172.21.15.43:0/3038437332 learned_addr learned my addr 172.21.15.43:0/3038437332 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa882a00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562cccffff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88be00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 -- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 msgr2=0x562ccd0065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 -- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 msgr2=0x562cccffff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562cccffff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 -- 172.21.15.43:0/3038437332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cccde3210 con 0x562cccb2e890 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa882a00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562cccffff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88be00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa88b400640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0b0040 tx=0x7fa87c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e797127f40261566 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0b2040 con 0x562cccb2e890 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fa87c005240 con 0x562cccb2e890 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cccddd350 con 0x562cccb2e890 2024-09-17T05:25:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa87c005440 con 0x562cccb2e890 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 10 client.5691 fetch_fsmap learned FSMap version 35 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x562cccdc9fc0 con 0x562cccb2e890 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562cccddd350 con 0x562cccb2e890 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa87c0b5030 con 0x562cccb2e890 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 10 client.5691 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 10 client.5691 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 20 client.5691 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccae7930 0x562ccd04ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 4 client.5691 mds_command: new command op to 5512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.964+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 --> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ccce57b00 con 0x562cccae7930 2024-09-17T05:25:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa882a00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccae7930 0x562ccd04ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa882a00640 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccae7930 0x562ccd04ae80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3449c691ddf0d523 server_cookie=259eb653972e1886 in_seq=0 out_seq=0 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa86fe00640 10 client.5691 ms_handle_connect on v2:172.21.15.62:6834/2098700023 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa86fe00640 1 -- 172.21.15.43:0/3038437332 <== mds.0 v2:172.21.15.62:6834/2098700023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ccce57b00 con 0x562cccae7930 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa86fe00640 10 client.5691 handle_command_reply: tid=0 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 client.5691 shutdown 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 msgr2=0x562cccfffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0b0040 tx=0x7fa87c009260 comp rx=0 tx=0).stop 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 shutdown_connections 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccae7930 0x562ccd04ae80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562cccffff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa86fe00640 0 client.5691 ms_handle_reset on v2:172.21.15.62:6834/2098700023 2024-09-17T05:25:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3038437332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562cccfffa00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 >> 172.21.15.43:0/3038437332 conn(0x562cccb025c0 msgr2=0x562cccff8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 shutdown_connections 2024-09-17T05:25:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 1 -- 172.21.15.43:0/3038437332 wait complete. 2024-09-17T05:25:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.968+0000 7fa899a6b000 20 client.5691 trim_cache size 0 max 16384 2024-09-17T05:25:40.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.984+0000 7fa899a6b000 1 Processor -- start 2024-09-17T05:25:40.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.984+0000 7fa899a6b000 1 -- start start 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cccffeb20 con 0x562ccced76d0 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c2058b0 con 0x562cccbf2aa0 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c204a60 con 0x562cccb2e890 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52848/0 (socket says 172.21.15.43:52848) 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 -- 172.21.15.43:0/269706749 learned_addr learned my addr 172.21.15.43:0/269706749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa882a00640 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88b400640 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 -- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 msgr2=0x562ccd0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 -- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 msgr2=0x562ccd0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 -- 172.21.15.43:0/269706749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cccea2680 con 0x562ccced76d0 2024-09-17T05:25:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa88be00640 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa884095300 tx=0x7fa88405f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc6b2fd3d088c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa87aa00640 1 -- 172.21.15.43:0/269706749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88407f070 con 0x562ccced76d0 2024-09-17T05:25:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa87aa00640 1 -- 172.21.15.43:0/269706749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa884064380 con 0x562ccced76d0 2024-09-17T05:25:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa87aa00640 1 -- 172.21.15.43:0/269706749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88401d040 con 0x562ccced76d0 2024-09-17T05:25:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- 172.21.15.43:0/269706749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 msgr2=0x562ccd0b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa884095300 tx=0x7fa88405f140 comp rx=0 tx=0).stop 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- 172.21.15.43:0/269706749 shutdown_connections 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562ccced76d0 0x562ccd0b5e80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 --2- 172.21.15.43:0/269706749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562cccb2e890 0x562ccd0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- 172.21.15.43:0/269706749 >> 172.21.15.43:0/269706749 conn(0x562cccb025c0 msgr2=0x562ccd055fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- 172.21.15.43:0/269706749 shutdown_connections 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- 172.21.15.43:0/269706749 wait complete. 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 -- start start 2024-09-17T05:25:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.988+0000 7fa899a6b000 1 Processor -- start 2024-09-17T05:25:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c0c5cb0 con 0x562cccb2e890 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c0c5660 con 0x562cccbf2aa0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562cccb0a680 con 0x562ccced76d0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 10 client.0 _ll_register_callbacks cb 0x7fa87c0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52850/0 (socket says 172.21.15.43:52850) 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 -- 172.21.15.43:0/3049999592 learned_addr learned my addr 172.21.15.43:0/3049999592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa882a00640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88be00640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 -- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 msgr2=0x562ccd0b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 -- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 msgr2=0x562ccd0b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 -- 172.21.15.43:0/3049999592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccce0f0b0 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa88b400640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0056f0 tx=0x7fa87c005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072aeeef0b051a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0b2040 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fa87c003600 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa884064380 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa87c0d0020 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 10 client.5697 fetch_fsmap learned FSMap version 35 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x562cccddb610 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa87c0e1070 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa87c0b5040 con 0x562cccb2e890 2024-09-17T05:25:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 10 client.5697 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T05:25:41.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 10 client.5697 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T05:25:41.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 20 client.5697 populate_metadata read hostname 'smithi043' 2024-09-17T05:25:41.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccad5990 0x562cccffac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 4 client.5697 mds_command: new command op to 5512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:25:41.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 --> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ccce535b0 con 0x562cccad5990 2024-09-17T05:25:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa882a00640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccad5990 0x562cccffac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa882a00640 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccad5990 0x562cccffac10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02c52527af27330 server_cookie=68ddd812d5b3a0a8 in_seq=0 out_seq=0 2024-09-17T05:25:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.992+0000 7fa86fe00640 10 client.5697 ms_handle_connect on v2:172.21.15.62:6834/2098700023 2024-09-17T05:25:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa86fe00640 1 -- 172.21.15.43:0/3049999592 <== mds.0 v2:172.21.15.62:6834/2098700023 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ccce535b0 con 0x562cccad5990 2024-09-17T05:25:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa86fe00640 10 client.5697 handle_command_reply: tid=0 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 client.5697 shutdown 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 msgr2=0x562ccd0b9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0056f0 tx=0x7fa87c005090 comp rx=0 tx=0).stop 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 shutdown_connections 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:6834/2098700023,v1:172.21.15.62:6835/2098700023] conn(0x562cccad5990 0x562cccffac10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cccbf2aa0 0x562ccd0b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562ccced76d0 0x562ccd0b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 --2- 172.21.15.43:0/3049999592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cccb2e890 0x562ccd0b9440 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa86fe00640 0 client.5697 ms_handle_reset on v2:172.21.15.62:6834/2098700023 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 >> 172.21.15.43:0/3049999592 conn(0x562cccb025c0 msgr2=0x562ccd0055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 shutdown_connections 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 1 -- 172.21.15.43:0/3049999592 wait complete. 2024-09-17T05:25:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa899a6b000 20 client.5697 trim_cache size 0 max 16384 2024-09-17T05:25:41.003 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:25:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:25:41.003 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:25:41.003 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 msgr2=0x7fa868075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 0x7fa868075710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0af000 tx=0x7fa87c01b0f0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 msgr2=0x7fa88c1fc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa88400eab0 tx=0x7fa884009f70 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 shutdown_connections 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa868073200 0x7fa868075710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c15fa70 0x7fa88c1fc6f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa88c0b5fe0 0x7fa88c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 --2- 172.21.15.43:0/4047119128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa88c0b43c0 0x7fa88c0afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 >> 172.21.15.43:0/4047119128 conn(0x7fa88c0b2e80 msgr2=0x7fa88c15e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 shutdown_connections 2024-09-17T05:25:41.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:40.996+0000 7fa892200640 1 -- 172.21.15.43:0/4047119128 wait complete. 2024-09-17T05:25:41.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.172+0000 7f142ac00640 1 -- 172.21.15.43:0/3967773130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 msgr2=0x7f14240734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.172+0000 7f142ac00640 1 --2- 172.21.15.43:0/3967773130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14240734c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f141c009bc0 tx=0x7f141c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- 172.21.15.43:0/3967773130 shutdown_connections 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 --2- 172.21.15.43:0/3967773130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14240734c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 --2- 172.21.15.43:0/3967773130 >> v2:172.21.15.62:3300/0 conn(0x7f142406fc90 0x7f14240700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- 172.21.15.43:0/3967773130 >> 172.21.15.43:0/3967773130 conn(0x7f14240b23d0 msgr2=0x7f14240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- 172.21.15.43:0/3967773130 shutdown_connections 2024-09-17T05:25:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- 172.21.15.43:0/3967773130 wait complete. 2024-09-17T05:25:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 Processor -- start 2024-09-17T05:25:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- start start 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f142406fc90 0x7f14241559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14241fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1424169fc0 con 0x7f142415fe10 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1424169e40 con 0x7f142406fc90 2024-09-17T05:25:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.176+0000 7f142ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f142416a140 con 0x7f14240730b0 2024-09-17T05:25:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52862/0 (socket says 172.21.15.43:52862) 2024-09-17T05:25:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 -- 172.21.15.43:0/2594610 learned_addr learned my addr 172.21.15.43:0/2594610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1423e00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f142406fc90 0x7f14241559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f141be00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14241fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 -- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 msgr2=0x7f14241fa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14241fa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 -- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f142406fc90 msgr2=0x7f14241559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f142406fc90 0x7f14241559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 -- 172.21.15.43:0/2594610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14241537f0 con 0x7f142415fe10 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1428c00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b0040 tx=0x7f141c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c0b2040 con 0x7f142415fe10 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f141c004040 con 0x7f142415fe10 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1424153c60 con 0x7f142415fe10 2024-09-17T05:25:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c004430 con 0x7f142415fe10 2024-09-17T05:25:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1424157f90 con 0x7f142415fe10 2024-09-17T05:25:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14240700a0 con 0x7f142415fe10 2024-09-17T05:25:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.180+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f141c0bd030 con 0x7f142415fe10 2024-09-17T05:25:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.184+0000 7f1420c00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 0x7f1400075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.184+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f141c12e180 con 0x7f142415fe10 2024-09-17T05:25:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.184+0000 7f1423e00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 0x7f1400075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.184+0000 7f1423e00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 0x7f1400075710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1414007d70 tx=0x7f1414007d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.188+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f141c0c4060 con 0x7f142415fe10 2024-09-17T05:25:41.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14241539f0 con 0x7f142415fe10 2024-09-17T05:25:41.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f1420c00640 1 -- 172.21.15.43:0/2594610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7f141c016790 con 0x7f142415fe10 2024-09-17T05:25:41.408 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:41.408 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":36,"btime":"2024-09-17T05:25:41:353859+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:40.371855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T05:25:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 36 2024-09-17T05:25:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 msgr2=0x7f1400075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 0x7f1400075710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1414007d70 tx=0x7f1414007d00 comp rx=0 tx=0).stop 2024-09-17T05:25:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 msgr2=0x7f14241fa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b0040 tx=0x7f141c003f40 comp rx=0 tx=0).stop 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 shutdown_connections 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1400073200 0x7f1400075710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f142415fe10 0x7f14241fa8f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14240730b0 0x7f14241fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 --2- 172.21.15.43:0/2594610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f142406fc90 0x7f14241559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 >> 172.21.15.43:0/2594610 conn(0x7f14240b23d0 msgr2=0x7f14240702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 shutdown_connections 2024-09-17T05:25:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.404+0000 7f142ac00640 1 -- 172.21.15.43:0/2594610 wait complete. 2024-09-17T05:25:41.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- 172.21.15.43:0/1195943515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 msgr2=0x7fe2180cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 --2- 172.21.15.43:0/1195943515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2180cce70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe218075050 tx=0x7fe20401c600 comp rx=0 tx=0).stop 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- 172.21.15.43:0/1195943515 shutdown_connections 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 --2- 172.21.15.43:0/1195943515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2180cce70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 --2- 172.21.15.43:0/1195943515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180cd3b0 0x7fe2180b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 --2- 172.21.15.43:0/1195943515 >> v2:172.21.15.62:3300/0 conn(0x7fe2180b7ce0 0x7fe2180d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- 172.21.15.43:0/1195943515 >> 172.21.15.43:0/1195943515 conn(0x7fe2180cee80 msgr2=0x7fe2180cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- 172.21.15.43:0/1195943515 shutdown_connections 2024-09-17T05:25:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- 172.21.15.43:0/1195943515 wait complete. 2024-09-17T05:25:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 Processor -- start 2024-09-17T05:25:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.584+0000 7fe21fc00640 1 -- start start 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 0x7fe2180b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2180cd3b0 0x7fe218157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe218160b40 con 0x7fe2180d0d80 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181609c0 con 0x7fe2180cd3b0 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe218160cc0 con 0x7fe2180b7ce0 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52880/0 (socket says 172.21.15.43:52880) 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 -- 172.21.15.43:0/859339308 learned_addr learned my addr 172.21.15.43:0/859339308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe217e00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2180cd3b0 0x7fe218157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21d200640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 0x7fe2180b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 -- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 msgr2=0x7fe2180b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 0x7fe2180b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 -- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2180cd3b0 msgr2=0x7fe218157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2180cd3b0 0x7fe218157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 -- 172.21.15.43:0/859339308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2180b59f0 con 0x7fe2180d0d80 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21d200640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 0x7fe2180b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21dc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe21000da40 tx=0x7fe21000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210017070 con 0x7fe2180d0d80 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe210004b30 con 0x7fe2180d0d80 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2180b5cd0 con 0x7fe2180d0d80 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210005020 con 0x7fe2180d0d80 2024-09-17T05:25:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2180b6290 con 0x7fe2180d0d80 2024-09-17T05:25:41.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.588+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2180a03c0 con 0x7fe2180d0d80 2024-09-17T05:25:41.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.592+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2100051c0 con 0x7fe2180d0d80 2024-09-17T05:25:41.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.592+0000 7fe214c00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 0x7fe1f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.592+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fe210094720 con 0x7fe2180d0d80 2024-09-17T05:25:41.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.592+0000 7fe21d200640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 0x7fe1f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.592+0000 7fe21d200640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 0x7fe1f8075710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe218071220 tx=0x7fe20c005e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.596+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe210020070 con 0x7fe2180d0d80 2024-09-17T05:25:41.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.804+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2180b5780 con 0x7fe2180d0d80 2024-09-17T05:25:41.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.804+0000 7fe214c00640 1 -- 172.21.15.43:0/859339308 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7fe21005f6d0 con 0x7fe2180d0d80 2024-09-17T05:25:41.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:41.810 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":36,"btime":"2024-09-17T05:25:41:353859+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:40.371855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T05:25:41.810 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 36 2024-09-17T05:25:41.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 msgr2=0x7fe1f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 0x7fe1f8075710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe218071220 tx=0x7fe20c005e90 comp rx=0 tx=0).stop 2024-09-17T05:25:41.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 msgr2=0x7fe2181578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe21000da40 tx=0x7fe21000df00 comp rx=0 tx=0).stop 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 shutdown_connections 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1f8073200 0x7fe1f8075710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2180cd3b0 0x7fe218157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2180d0d80 0x7fe2181578a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 --2- 172.21.15.43:0/859339308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe2180b7ce0 0x7fe2180b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 >> 172.21.15.43:0/859339308 conn(0x7fe2180cee80 msgr2=0x7fe2180cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 shutdown_connections 2024-09-17T05:25:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.808+0000 7fe21fc00640 1 -- 172.21.15.43:0/859339308 wait complete. 2024-09-17T05:25:41.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 -- 172.21.15.43:0/1890142573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8868070620 msgr2=0x7f88680b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 --2- 172.21.15.43:0/1890142573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8868070620 0x7f88680b30d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8858009840 tx=0x7f885801c790 comp rx=0 tx=0).stop 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 -- 172.21.15.43:0/1890142573 shutdown_connections 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 --2- 172.21.15.43:0/1890142573 >> v2:172.21.15.62:3300/0 conn(0x7f88680cb860 0x7f88680cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 --2- 172.21.15.43:0/1890142573 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8868070620 0x7f88680b30d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 -- 172.21.15.43:0/1890142573 >> 172.21.15.43:0/1890142573 conn(0x7f88680b23d0 msgr2=0x7f88680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 -- 172.21.15.43:0/1890142573 shutdown_connections 2024-09-17T05:25:41.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.988+0000 7f886f200640 1 -- 172.21.15.43:0/1890142573 wait complete. 2024-09-17T05:25:41.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 Processor -- start 2024-09-17T05:25:41.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 -- start start 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 0x7f886815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88680cb860 0x7f88680cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88680ce680 con 0x7f8868070620 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88680ce500 con 0x7f88680cb860 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f88680ce800 con 0x7f886806ce60 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52890/0 (socket says 172.21.15.43:52890) 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 -- 172.21.15.43:0/2442621474 learned_addr learned my addr 172.21.15.43:0/2442621474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f8867e00640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 0x7f886815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886d200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88680cb860 0x7f88680cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 -- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 msgr2=0x7f886815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 0x7f886815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 -- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88680cb860 msgr2=0x7f88680cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88680cb860 0x7f88680cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 -- 172.21.15.43:0/2442621474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88681fc600 con 0x7f8868070620 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f8867e00640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 0x7f886815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f885f400640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8850001ec0 tx=0x7f885000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850011790 con 0x7f8868070620 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8850017070 con 0x7f8868070620 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850011d00 con 0x7f8868070620 2024-09-17T05:25:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.992+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88681fcb30 con 0x7f8868070620 2024-09-17T05:25:42.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88681fd0f0 con 0x7f8868070620 2024-09-17T05:25:42.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88681fc8a0 con 0x7f8868070620 2024-09-17T05:25:42.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f885001c460 con 0x7f8868070620 2024-09-17T05:25:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f8864c00640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 0x7f8844075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f8850093ed0 con 0x7f8868070620 2024-09-17T05:25:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:41.996+0000 7f8867e00640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 0x7f8844075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.000+0000 7f8867e00640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 0x7f8844075770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88580af000 tx=0x7f88580b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.004+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f885005e7b0 con 0x7f8868070620 2024-09-17T05:25:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.208+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88680cedd0 con 0x7f8868070620 2024-09-17T05:25:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f8864c00640 1 -- 172.21.15.43:0/2442621474 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7f88500631e0 con 0x7f8868070620 2024-09-17T05:25:42.216 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:42.216 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":36,"btime":"2024-09-17T05:25:41:353859+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:40.371855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T05:25:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 36 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 msgr2=0x7f8844075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 0x7f8844075770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88580af000 tx=0x7f88580b2040 comp rx=0 tx=0).stop 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 msgr2=0x7f88680ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8850001ec0 tx=0x7f885000c820 comp rx=0 tx=0).stop 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 shutdown_connections 2024-09-17T05:25:42.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8844073260 0x7f8844075770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88680cb860 0x7f88680cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8868070620 0x7f88680ccc80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 --2- 172.21.15.43:0/2442621474 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f886806ce60 0x7f886815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 >> 172.21.15.43:0/2442621474 conn(0x7f88680b23d0 msgr2=0x7f886815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 shutdown_connections 2024-09-17T05:25:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.212+0000 7f886f200640 1 -- 172.21.15.43:0/2442621474 wait complete. 2024-09-17T05:25:42.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump tree /a 0 2024-09-17T05:25:42.428 INFO:teuthology.orchestra.run.smithi062.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-17T05:25:36.352630+0000","btime":"2024-09-17T05:25:30.284694+0000","mode":16893,"uid":1000,"gid":1271,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-17T05:25:36.352630+0000","atime":"2024-09-17T05:25:30.284694+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-17T05:25:36.352630+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-17T05:25:36.352630+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-17T05:25:36.352630+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":1,"dirfrag":1,"dirty":1},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-17T05:25:42.429 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-17T05:25:36.352630+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-17T05:25:30.284694+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-17T05:25:30.284694+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-17T05:25:36.352630+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-17T05:25:36.352630+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1271, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2024-09-17T05:25:36.352630+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1, "dirtyscattered": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-17T05:25:36.352630+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-09-17T05:25:42.429 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-17T05:25:36.352630+0000', 'btime': '2024-09-17T05:25:30.284694+0000', 'mode': 16893, 'uid': 1000, 'gid': 1271, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-17T05:25:36.352630+0000', 'atime': '2024-09-17T05:25:30.284694+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-17T05:25:36.352630+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-17T05:25:36.352630+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-17T05:25:36.352630+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirtyscattered': 1, 'dirfrag': 1, 'dirty': 1}, 'nref': 3, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-17T05:25:42.430 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:25:42.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:42.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:42.668 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:25:42.668 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:42.668 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:42.668 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:42.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:42.677 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:42.677 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:42.678 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:42.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:42.686 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:42.687 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:42.687 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:42.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:42.727 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:25:42.727 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:42.727 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:42.741 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:42.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 -- 172.21.15.43:0/1322036070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc0b5d20 msgr2=0x7fe1dc0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 --2- 172.21.15.43:0/1322036070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc0b5d20 0x7fe1dc0d0390 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009840 tx=0x7fe1c801c300 comp rx=0 tx=0).stop 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 -- 172.21.15.43:0/1322036070 shutdown_connections 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 --2- 172.21.15.43:0/1322036070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc0b5d20 0x7fe1dc0d0390 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 -- 172.21.15.43:0/1322036070 >> 172.21.15.43:0/1322036070 conn(0x7fe1dc0b2810 msgr2=0x7fe1dc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 -- 172.21.15.43:0/1322036070 shutdown_connections 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.924+0000 7fe1e3600640 1 -- 172.21.15.43:0/1322036070 wait complete. 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 Processor -- start 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- start start 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 0x7fe1dc1fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 0x7fe1dc075d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc16ad90 con 0x7fe1dc15f470 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc16ac10 con 0x7fe1dc06fce0 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe1dc16af10 con 0x7fe1dc1f7b40 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52920/0 (socket says 172.21.15.43:52920) 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 -- 172.21.15.43:0/101642530 learned_addr learned my addr 172.21.15.43:0/101642530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e0c00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 0x7fe1dc1fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e1600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 0x7fe1dc075d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 -- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 msgr2=0x7fe1dc075d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 0x7fe1dc075d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 -- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 msgr2=0x7fe1dc1fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 0x7fe1dc1fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 -- 172.21.15.43:0/101642530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc1fac90 con 0x7fe1dc15f470 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e0c00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 0x7fe1dc1fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e1600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 0x7fe1dc075d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1dbe00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1c801c7e0 tx=0x7fe1c80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80b1040 con 0x7fe1dc15f470 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c8003600 con 0x7fe1dc15f470 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80039b0 con 0x7fe1dc15f470 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc158fe0 con 0x7fe1dc15f470 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.928+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc1fb1e0 con 0x7fe1dc15f470 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1dc158d50 con 0x7fe1dc15f470 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe1c80bc030 con 0x7fe1dc15f470 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1d8c00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 0x7fe1bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fe1c812d2e0 con 0x7fe1dc15f470 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1e0c00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 0x7fe1bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.932+0000 7fe1e0c00640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 0x7fe1bc075710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0000f70 tx=0x7fe1d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:42.936+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe1c80c3060 con 0x7fe1dc15f470 2024-09-17T05:25:43.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.144+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1dc0d0390 con 0x7fe1dc15f470 2024-09-17T05:25:43.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.144+0000 7fe1d8c00640 1 -- 172.21.15.43:0/101642530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5214 (secure 0 0 0) 0x7fe1c8016330 con 0x7fe1dc15f470 2024-09-17T05:25:43.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:43.150 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":37,"btime":"2024-09-17T05:25:42:355771+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:41.358562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T05:25:43.151 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 37 2024-09-17T05:25:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 msgr2=0x7fe1bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 0x7fe1bc075710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0000f70 tx=0x7fe1d0009040 comp rx=0 tx=0).stop 2024-09-17T05:25:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 msgr2=0x7fe1dc1f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe1c801c7e0 tx=0x7fe1c80043b0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 shutdown_connections 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1bc073200 0x7fe1bc075710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1dc1f7b40 0x7fe1dc075d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1dc15f470 0x7fe1dc1f7600 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 --2- 172.21.15.43:0/101642530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1dc06fce0 0x7fe1dc1fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 >> 172.21.15.43:0/101642530 conn(0x7fe1dc0b2810 msgr2=0x7fe1dc14edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 shutdown_connections 2024-09-17T05:25:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.148+0000 7fe1e3600640 1 -- 172.21.15.43:0/101642530 wait complete. 2024-09-17T05:25:43.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.324+0000 7fee9d600640 1 -- 172.21.15.43:0/3422729372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee98165f20 msgr2=0x7fee98168340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.324+0000 7fee9d600640 1 --2- 172.21.15.43:0/3422729372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee98165f20 0x7fee98168340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fee90009bc0 tx=0x7fee9001cba0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 -- 172.21.15.43:0/3422729372 shutdown_connections 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 --2- 172.21.15.43:0/3422729372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee98165f20 0x7fee98168340 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 -- 172.21.15.43:0/3422729372 >> 172.21.15.43:0/3422729372 conn(0x7fee980cee70 msgr2=0x7fee980cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 -- 172.21.15.43:0/3422729372 shutdown_connections 2024-09-17T05:25:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 -- 172.21.15.43:0/3422729372 wait complete. 2024-09-17T05:25:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 Processor -- start 2024-09-17T05:25:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 -- start start 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.328+0000 7fee9d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 0x7fee98152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee98165f20 0x7fee980cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fee9816b630 con 0x7fee980b1ca0 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee9816b4b0 con 0x7fee980d0950 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fee9816b7b0 con 0x7fee98165f20 2024-09-17T05:25:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52946/0 (socket says 172.21.15.43:52946) 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 -- 172.21.15.43:0/1395107978 learned_addr learned my addr 172.21.15.43:0/1395107978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee97400640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee98165f20 0x7fee980cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8fe00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 0x7fee98152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 -- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee98165f20 msgr2=0x7fee980cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee98165f20 0x7fee980cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 -- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 msgr2=0x7fee98152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 0x7fee98152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 -- 172.21.15.43:0/1395107978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee98160020 con 0x7fee980b1ca0 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8fe00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 0x7fee98152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee96a00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c3d0 tx=0x7fee8800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88017070 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee880053c0 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee98153520 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880038b0 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee980cb910 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee980b20b0 con 0x7fee980b1ca0 2024-09-17T05:25:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.332+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fee8801a070 con 0x7fee980b1ca0 2024-09-17T05:25:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.336+0000 7fee8ea00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.336+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fee88093620 con 0x7fee980b1ca0 2024-09-17T05:25:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.336+0000 7fee8fe00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.336+0000 7fee8fe00640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 0x7fee74075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee981687c0 tx=0x7fee80009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.340+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fee8805df90 con 0x7fee980b1ca0 2024-09-17T05:25:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.548+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee980cfa90 con 0x7fee980b1ca0 2024-09-17T05:25:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee8ea00640 1 -- 172.21.15.43:0/1395107978 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5221 (secure 0 0 0) 0x7fee880629c0 con 0x7fee980b1ca0 2024-09-17T05:25:43.555 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:43.555 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"btime":"2024-09-17T05:25:43:361440+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:43.361434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5512,"qdb_cluster":[5512]},"id":3}]} 2024-09-17T05:25:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 38 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 msgr2=0x7fee74075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 0x7fee74075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fee981687c0 tx=0x7fee80009040 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 msgr2=0x7fee98152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c3d0 tx=0x7fee8800c890 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 shutdown_connections 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee980d0950 0x7fee98152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fee98165f20 0x7fee980cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 --2- 172.21.15.43:0/1395107978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee980b1ca0 0x7fee98152aa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 >> 172.21.15.43:0/1395107978 conn(0x7fee980cee70 msgr2=0x7fee980b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 shutdown_connections 2024-09-17T05:25:43.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.552+0000 7fee9d600640 1 -- 172.21.15.43:0/1395107978 wait complete. 2024-09-17T05:25:43.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- 172.21.15.43:0/730592400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 msgr2=0x7fd48415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 --2- 172.21.15.43:0/730592400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd48415c7e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd4840af0a0 tx=0x7fd47401cae0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- 172.21.15.43:0/730592400 shutdown_connections 2024-09-17T05:25:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 --2- 172.21.15.43:0/730592400 >> v2:172.21.15.62:3300/0 conn(0x7fd48406f110 0x7fd4840b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 --2- 172.21.15.43:0/730592400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd48415c7e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- 172.21.15.43:0/730592400 >> 172.21.15.43:0/730592400 conn(0x7fd4840b23d0 msgr2=0x7fd4840d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- 172.21.15.43:0/730592400 shutdown_connections 2024-09-17T05:25:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- 172.21.15.43:0/730592400 wait complete. 2024-09-17T05:25:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 Processor -- start 2024-09-17T05:25:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.732+0000 7fd48ac00640 1 -- start start 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 0x7fd48409a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd4840ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd484168a60 con 0x7fd484155d70 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841688e0 con 0x7fd48406f110 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd484168be0 con 0x7fd4840b3920 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd483e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd47b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 0x7fd48409a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd4840ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd483e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:35812/0 (socket says 172.21.15.43:35812) 2024-09-17T05:25:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd483e00640 1 -- 172.21.15.43:0/2780248566 learned_addr learned my addr 172.21.15.43:0/2780248566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd47b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 0x7fd48409a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:44828/0 (socket says 172.21.15.43:44828) 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 -- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 msgr2=0x7fd48409a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 0x7fd48409a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 -- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 msgr2=0x7fd484074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 -- 172.21.15.43:0/2780248566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48409e1e0 con 0x7fd484155d70 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd483e00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd488c00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd4840ccab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd48415bc80 tx=0x7fd47c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c015030 con 0x7fd484155d70 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd47c004020 con 0x7fd484155d70 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c004470 con 0x7fd484155d70 2024-09-17T05:25:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4841f9120 con 0x7fd484155d70 2024-09-17T05:25:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841f96e0 con 0x7fd484155d70 2024-09-17T05:25:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.736+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48409b0a0 con 0x7fd484155d70 2024-09-17T05:25:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.740+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd47c018070 con 0x7fd484155d70 2024-09-17T05:25:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.740+0000 7fd480c00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 0x7fd460075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.740+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fd47c093bf0 con 0x7fd484155d70 2024-09-17T05:25:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.740+0000 7fd483e00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 0x7fd460075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.740+0000 7fd483e00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 0x7fd460075710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4740b0040 tx=0x7fd47401b320 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.744+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd47c05e4c0 con 0x7fd484155d70 2024-09-17T05:25:43.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.920+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4840b33e0 con 0x7fd484155d70 2024-09-17T05:25:43.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.920+0000 7fd480c00640 1 -- 172.21.15.43:0/2780248566 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16736 (secure 0 0 0) 0x7fd47c062ef0 con 0x7fd484155d70 2024-09-17T05:25:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:43.926 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:38.338103+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:15.006899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:16.208447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/980890216":"2024-09-18T05:25:38.338069+0000","172.21.15.62:6834/980890216":"2024-09-18T05:25:38.338069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 msgr2=0x7fd460075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 0x7fd460075710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4740b0040 tx=0x7fd47401b320 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 msgr2=0x7fd4840ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd4840ccab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd48415bc80 tx=0x7fd47c00b870 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 shutdown_connections 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd460073200 0x7fd460075710 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd484155d70 0x7fd4840ccab0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4840b3920 0x7fd48409a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 --2- 172.21.15.43:0/2780248566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd48406f110 0x7fd484074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 >> 172.21.15.43:0/2780248566 conn(0x7fd4840b23d0 msgr2=0x7fd4840d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 shutdown_connections 2024-09-17T05:25:43.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:43.924+0000 7fd48ac00640 1 -- 172.21.15.43:0/2780248566 wait complete. 2024-09-17T05:25:43.948 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:25:43.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- 172.21.15.43:0/1516160202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd415fc40 msgr2=0x7fbbd4160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 --2- 172.21.15.43:0/1516160202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd415fc40 0x7fbbd4160050 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc001c750 comp rx=0 tx=0).stop 2024-09-17T05:25:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- 172.21.15.43:0/1516160202 shutdown_connections 2024-09-17T05:25:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 --2- 172.21.15.43:0/1516160202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd415fc40 0x7fbbd4160050 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- 172.21.15.43:0/1516160202 >> 172.21.15.43:0/1516160202 conn(0x7fbbd40b2e80 msgr2=0x7fbbd40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- 172.21.15.43:0/1516160202 shutdown_connections 2024-09-17T05:25:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- 172.21.15.43:0/1516160202 wait complete. 2024-09-17T05:25:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 Processor -- start 2024-09-17T05:25:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.104+0000 7fbbdc200640 1 -- start start 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd40b43c0 0x7fbbd414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbd415fc40 0x7fbbd40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd416a170 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4169ff0 con 0x7fbbd415fc40 2024-09-17T05:25:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd416a2f0 con 0x7fbbd40b43c0 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37322/0 (socket says 172.21.15.43:37322) 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 -- 172.21.15.43:0/208763380 learned_addr learned my addr 172.21.15.43:0/208763380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbda200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbd415fc40 0x7fbbd40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd9800640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd40b43c0 0x7fbbd414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 -- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd40b43c0 msgr2=0x7fbbd414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd40b43c0 0x7fbbd414b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 -- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbd415fc40 msgr2=0x7fbbd40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbd415fc40 0x7fbbd40cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 -- 172.21.15.43:0/208763380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4158730 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd8e00640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c10 tx=0x7fbbc0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00bb070 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc00053e0 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd41fc450 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00b2040 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd41fca10 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.108+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd4160050 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.112+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbc00be070 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.112+0000 7fbbd1600640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 0x7fbbb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.112+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7fbbc012d820 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.112+0000 7fbbd9800640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 0x7fbbb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.112+0000 7fbbd9800640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 0x7fbbb4075710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8001250 tx=0x7fbbc8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.116+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbbc0016370 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.324+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbd40cf380 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.324+0000 7fbbd1600640 1 -- 172.21.15.43:0/208763380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5221 (secure 0 0 0) 0x7fbbc00f87e0 con 0x7fbbd40b5fe0 2024-09-17T05:25:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:44.331 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"btime":"2024-09-17T05:25:43:361440+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:43.361434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5512,"qdb_cluster":[5512]},"id":3}]} 2024-09-17T05:25:44.334 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 38 2024-09-17T05:25:44.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 msgr2=0x7fbbb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 0x7fbbb4075710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8001250 tx=0x7fbbc8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 msgr2=0x7fbbd40c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c10 tx=0x7fbbc0004600 comp rx=0 tx=0).stop 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 shutdown_connections 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbb4073200 0x7fbbb4075710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbd415fc40 0x7fbbd40cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbd40b5fe0 0x7fbbd40c8ec0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 --2- 172.21.15.43:0/208763380 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbd40b43c0 0x7fbbd414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 >> 172.21.15.43:0/208763380 conn(0x7fbbd40b2e80 msgr2=0x7fbbd40b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 shutdown_connections 2024-09-17T05:25:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.328+0000 7fbbdc200640 1 -- 172.21.15.43:0/208763380 wait complete. 2024-09-17T05:25:44.349 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-09-17T05:25:44.349+0000 7f2390200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:44.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:44.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 -- 172.21.15.43:0/2893363405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87180b3af0 msgr2=0x7f871815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 --2- 172.21.15.43:0/2893363405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87180b3af0 0x7f871815c7e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8700009900 tx=0x7f870001c820 comp rx=0 tx=0).stop 2024-09-17T05:25:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 -- 172.21.15.43:0/2893363405 shutdown_connections 2024-09-17T05:25:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 --2- 172.21.15.43:0/2893363405 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f871806f110 0x7f87180ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 --2- 172.21.15.43:0/2893363405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87180b3af0 0x7f871815c7e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 -- 172.21.15.43:0/2893363405 >> 172.21.15.43:0/2893363405 conn(0x7f87180b23d0 msgr2=0x7f87180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 -- 172.21.15.43:0/2893363405 shutdown_connections 2024-09-17T05:25:44.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.508+0000 7f871e600640 1 -- 172.21.15.43:0/2893363405 wait complete. 2024-09-17T05:25:44.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 Processor -- start 2024-09-17T05:25:44.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- start start 2024-09-17T05:25:44.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f871806f110 0x7f8718154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87180ae850 0x7f87180c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8718168c60 con 0x7f8718155d70 2024-09-17T05:25:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8718168ae0 con 0x7f871806f110 2024-09-17T05:25:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8718168de0 con 0x7f87180ae850 2024-09-17T05:25:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37348/0 (socket says 172.21.15.43:37348) 2024-09-17T05:25:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 -- 172.21.15.43:0/3854903972 learned_addr learned my addr 172.21.15.43:0/3854903972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f8717e00640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87180ae850 0x7f87180c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f8717400640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f871806f110 0x7f8718154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 -- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87180ae850 msgr2=0x7f87180c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87180ae850 0x7f87180c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 -- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f871806f110 msgr2=0x7f8718154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f871806f110 0x7f8718154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 -- 172.21.15.43:0/3854903972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871815a960 con 0x7f8718155d70 2024-09-17T05:25:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870ea00640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f87000b0040 tx=0x7f8700004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000b2040 con 0x7f8718155d70 2024-09-17T05:25:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8700005030 con 0x7f8718155d70 2024-09-17T05:25:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700005440 con 0x7f8718155d70 2024-09-17T05:25:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87180c7500 con 0x7f8718155d70 2024-09-17T05:25:44.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87180c77e0 con 0x7f8718155d70 2024-09-17T05:25:44.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.512+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180ad0b0 con 0x7f8718155d70 2024-09-17T05:25:44.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.516+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87000046d0 con 0x7f8718155d70 2024-09-17T05:25:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.516+0000 7f870fe00640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 0x7f86f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.516+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f870012db80 con 0x7f8718155d70 2024-09-17T05:25:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.516+0000 7f8717400640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 0x7f86f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.516+0000 7f8717400640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 0x7f86f4075740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87180715c0 tx=0x7f8708009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.520+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87000bc060 con 0x7f8718155d70 2024-09-17T05:25:44.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.732+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8718155310 con 0x7f8718155d70 2024-09-17T05:25:44.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.732+0000 7f870fe00640 1 -- 172.21.15.43:0/3854903972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5221 (secure 0 0 0) 0x7f87000164d0 con 0x7f8718155d70 2024-09-17T05:25:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:44.738 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"btime":"2024-09-17T05:25:43:361440+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:17.258682+0000","modified":"2024-09-17T05:25:43.361434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5512":{"gid":5512,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2098700023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2098700023},{"type":"v1","addr":"172.21.15.62:6835","nonce":2098700023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5512,"qdb_cluster":[5512]},"id":3}]} 2024-09-17T05:25:44.738 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 38 2024-09-17T05:25:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 msgr2=0x7f86f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 0x7f86f4075740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87180715c0 tx=0x7f8708009040 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 msgr2=0x7f8718072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f87000b0040 tx=0x7f8700004040 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 shutdown_connections 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f86f4073230 0x7f86f4075740 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87180ae850 0x7f87180c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8718155d70 0x7f8718072f00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 --2- 172.21.15.43:0/3854903972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f871806f110 0x7f8718154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 >> 172.21.15.43:0/3854903972 conn(0x7f87180b23d0 msgr2=0x7f871815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 shutdown_connections 2024-09-17T05:25:44.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.736+0000 7f871e600640 1 -- 172.21.15.43:0/3854903972 wait complete. 2024-09-17T05:25:44.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- 172.21.15.43:0/3609540769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 msgr2=0x7f5b4006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 --2- 172.21.15.43:0/3609540769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b4006d270 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5b40073040 tx=0x7f5b2801c830 comp rx=0 tx=0).stop 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- 172.21.15.43:0/3609540769 shutdown_connections 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 --2- 172.21.15.43:0/3609540769 >> v2:172.21.15.62:3300/0 conn(0x7f5b400cb860 0x7f5b400cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 --2- 172.21.15.43:0/3609540769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b4006d270 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- 172.21.15.43:0/3609540769 >> 172.21.15.43:0/3609540769 conn(0x7f5b400b23d0 msgr2=0x7f5b400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- 172.21.15.43:0/3609540769 shutdown_connections 2024-09-17T05:25:44.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- 172.21.15.43:0/3609540769 wait complete. 2024-09-17T05:25:44.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 Processor -- start 2024-09-17T05:25:44.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.916+0000 7f5b45400640 1 -- start start 2024-09-17T05:25:44.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b40070620 0x7f5b40159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b400cb860 0x7f5b400c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400ce5a0 con 0x7f5b4006ce60 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400ce420 con 0x7f5b400cb860 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5b400ce720 con 0x7f5b40070620 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37374/0 (socket says 172.21.15.43:37374) 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 -- 172.21.15.43:0/3584341641 learned_addr learned my addr 172.21.15.43:0/3584341641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b37e00640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b40070620 0x7f5b40159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3f200640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b400cb860 0x7f5b400c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 -- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b40070620 msgr2=0x7f5b40159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b40070620 0x7f5b40159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 -- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b400cb860 msgr2=0x7f5b400c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b400cb860 0x7f5b400c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 -- 172.21.15.43:0/3584341641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b401fa780 con 0x7f5b4006ce60 2024-09-17T05:25:44.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b3e800640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c3c0 tx=0x7f5b3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b4006ce60 2024-09-17T05:25:44.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b300053c0 con 0x7f5b4006ce60 2024-09-17T05:25:44.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b400c6a60 con 0x7f5b4006ce60 2024-09-17T05:25:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30003860 con 0x7f5b4006ce60 2024-09-17T05:25:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b400c84a0 con 0x7f5b4006ce60 2024-09-17T05:25:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.920+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b400cecb0 con 0x7f5b4006ce60 2024-09-17T05:25:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.924+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5b3001a070 con 0x7f5b4006ce60 2024-09-17T05:25:44.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.924+0000 7f5b36a00640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 0x7f5b1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.924+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7f5b30093560 con 0x7f5b4006ce60 2024-09-17T05:25:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.924+0000 7f5b37e00640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 0x7f5b1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.924+0000 7f5b37e00640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 0x7f5b1c075740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b280af000 tx=0x7f5b2801b070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:44.928+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5b4006d7b0 con 0x7f5b4006ce60 2024-09-17T05:25:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.100+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b40074fd0 con 0x7f5b4006ce60 2024-09-17T05:25:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b36a00640 1 -- 172.21.15.43:0/3584341641 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16736 (secure 0 0 0) 0x7f5b3005dec0 con 0x7f5b4006ce60 2024-09-17T05:25:45.108 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:45.108 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:38.338103+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:15.006899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:16.208447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/980890216":"2024-09-18T05:25:38.338069+0000","172.21.15.62:6834/980890216":"2024-09-18T05:25:38.338069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 msgr2=0x7f5b1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 0x7f5b1c075740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b280af000 tx=0x7f5b2801b070 comp rx=0 tx=0).stop 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 msgr2=0x7f5b400cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c3c0 tx=0x7f5b3000c880 comp rx=0 tx=0).stop 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 shutdown_connections 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b1c073230 0x7f5b1c075740 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b400cb860 0x7f5b400c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b40070620 0x7f5b40159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 --2- 172.21.15.43:0/3584341641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4006ce60 0x7f5b400cfc50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 >> 172.21.15.43:0/3584341641 conn(0x7f5b400b23d0 msgr2=0x7f5b400af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 shutdown_connections 2024-09-17T05:25:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.104+0000 7f5b45400640 1 -- 172.21.15.43:0/3584341641 wait complete. 2024-09-17T05:25:45.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:25:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- 172.21.15.43:0/245853342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee80cb860 msgr2=0x7efee80cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 --2- 172.21.15.43:0/245853342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee80cb860 0x7efee80cbc70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efee0009bf0 tx=0x7efee001cb50 comp rx=0 tx=0).stop 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- 172.21.15.43:0/245853342 shutdown_connections 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 --2- 172.21.15.43:0/245853342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee80cb860 0x7efee80cbc70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- 172.21.15.43:0/245853342 >> 172.21.15.43:0/245853342 conn(0x7efee80b23d0 msgr2=0x7efee80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- 172.21.15.43:0/245853342 shutdown_connections 2024-09-17T05:25:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- 172.21.15.43:0/245853342 wait complete. 2024-09-17T05:25:45.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 Processor -- start 2024-09-17T05:25:45.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 -- start start 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.288+0000 7efeef200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee806ce60 0x7efee8074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee8070620 0x7efee81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efee80ce850 con 0x7efee80cb860 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efee80ce6d0 con 0x7efee8070620 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efee80ce9d0 con 0x7efee806ce60 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37382/0 (socket says 172.21.15.43:37382) 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 -- 172.21.15.43:0/345752951 learned_addr learned my addr 172.21.15.43:0/345752951 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efee7e00640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee806ce60 0x7efee8074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efedfe00640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee8070620 0x7efee81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 -- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee806ce60 msgr2=0x7efee8074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee806ce60 0x7efee8074200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 -- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee8070620 msgr2=0x7efee81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee8070620 0x7efee81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 -- 172.21.15.43:0/345752951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee80c9e90 con 0x7efee80cb860 2024-09-17T05:25:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeed200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efee001a780 tx=0x7efee0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00b2040 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee0004d40 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee8074740 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00042a0 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee80c98d0 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee80cbc70 con 0x7efee80cb860 2024-09-17T05:25:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.292+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efee00be060 con 0x7efee80cb860 2024-09-17T05:25:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.296+0000 7efee4c00640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 0x7efec4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.296+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5795+0+0 (secure 0 0 0) 0x7efee012e1a0 con 0x7efee80cb860 2024-09-17T05:25:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.296+0000 7efee7e00640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 0x7efec4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.296+0000 7efee7e00640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 0x7efec4075710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efed8001640 tx=0x7efed8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.300+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efee00167d0 con 0x7efee80cb860 2024-09-17T05:25:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.516+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efee80af0c0 con 0x7efee80cb860 2024-09-17T05:25:45.521 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:25:45.516+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:25:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efee4c00640 1 -- 172.21.15.43:0/345752951 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7efee00f9150 con 0x7efee80cb860 2024-09-17T05:25:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 msgr2=0x7efec4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 0x7efec4075710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efed8001640 tx=0x7efed8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 msgr2=0x7efee8155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efee001a780 tx=0x7efee0009200 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 shutdown_connections 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efec4073200 0x7efec4075710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efee8070620 0x7efee81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee80cb860 0x7efee8155da0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 --2- 172.21.15.43:0/345752951 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efee806ce60 0x7efee8074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 >> 172.21.15.43:0/345752951 conn(0x7efee80b23d0 msgr2=0x7efee80b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 shutdown_connections 2024-09-17T05:25:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.520+0000 7efeef200640 1 -- 172.21.15.43:0/345752951 wait complete. 2024-09-17T05:25:45.533 INFO:tasks.ceph.mds.b.smithi062.stderr: -68> 2024-09-17T05:25:44.349+0000 7f2390200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:45.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:25:45.655 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2237856882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f4155d70 msgr2=0x7fa6f415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2237856882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f4155d70 0x7fa6f415c7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009840 tx=0x7fa6e401c760 comp rx=0 tx=0).stop 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2237856882 shutdown_connections 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2237856882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f40b3af0 0x7fa6f40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2237856882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f4155d70 0x7fa6f415c7e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2237856882 >> 172.21.15.43:0/2237856882 conn(0x7fa6f40b23d0 msgr2=0x7fa6f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2237856882 shutdown_connections 2024-09-17T05:25:45.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.704+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2237856882 wait complete. 2024-09-17T05:25:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 Processor -- start 2024-09-17T05:25:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 -- start start 2024-09-17T05:25:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 0x7fa6f409c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 0x7fa6f415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f41fc920 0x7fa6f415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4168c60 con 0x7fa6f4155d70 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4168ae0 con 0x7fa6f406f110 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f9a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f4168de0 con 0x7fa6f41fc920 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 0x7fa6f415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 0x7fa6f415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37396/0 (socket says 172.21.15.43:37396) 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6ebe00640 1 -- 172.21.15.43:0/2466705059 learned_addr learned my addr 172.21.15.43:0/2466705059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f3400640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f41fc920 0x7fa6f415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f2a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 0x7fa6f409c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f2a00640 1 -- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f41fc920 msgr2=0x7fa6f415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f2a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f41fc920 0x7fa6f415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f2a00640 1 -- 172.21.15.43:0/2466705059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 msgr2=0x7fa6f415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.708+0000 7fa6f2a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 0x7fa6f415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6f2a00640 1 -- 172.21.15.43:0/2466705059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f40cda00 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6f2a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 0x7fa6f409c2b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40d0ca0 tx=0x7fa6e40a56a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40bb070 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e40043b0 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b2040 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41fbe40 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f409c7f0 con 0x7fa6f406f110 2024-09-17T05:25:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f41fbc30 con 0x7fa6f406f110 2024-09-17T05:25:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa6e4005290 con 0x7fa6f406f110 2024-09-17T05:25:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 0x7fa6d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.712+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5881+0+0 (secure 0 0 0) 0x7fa6e412df00 con 0x7fa6f406f110 2024-09-17T05:25:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.716+0000 7fa6ebe00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 0x7fa6d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.716+0000 7fa6ebe00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 0x7fa6d0075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40c9a90 tx=0x7fa6dc007f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.716+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6e40c4020 con 0x7fa6f406f110 2024-09-17T05:25:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:45.928+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa6f40ad0b0 con 0x7fa6f406f110 2024-09-17T05:25:46.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.528+0000 7fa6eaa00640 1 -- 172.21.15.43:0/2466705059 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7fa6e4016410 con 0x7fa6f406f110 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 msgr2=0x7fa6d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 0x7fa6d0075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40c9a90 tx=0x7fa6dc007f30 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 msgr2=0x7fa6f409c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 0x7fa6f409c2b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40d0ca0 tx=0x7fa6e40a56a0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 shutdown_connections 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa6d0073200 0x7fa6d0075710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6f41fc920 0x7fa6f415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6f4155d70 0x7fa6f415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 --2- 172.21.15.43:0/2466705059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f406f110 0x7fa6f409c2b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 >> 172.21.15.43:0/2466705059 conn(0x7fa6f40b23d0 msgr2=0x7fa6f415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 shutdown_connections 2024-09-17T05:25:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.532+0000 7fa6f9a00640 1 -- 172.21.15.43:0/2466705059 wait complete. 2024-09-17T05:25:46.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 -- 172.21.15.43:0/4156937683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e815b4d0 msgr2=0x7f24e815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 --2- 172.21.15.43:0/4156937683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e815b4d0 0x7f24e815b8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009c80 tx=0x7f24e001cd70 comp rx=0 tx=0).stop 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 -- 172.21.15.43:0/4156937683 shutdown_connections 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 --2- 172.21.15.43:0/4156937683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e815b4d0 0x7f24e815b8e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 --2- 172.21.15.43:0/4156937683 >> v2:172.21.15.62:3300/0 conn(0x7f24e80ce160 0x7f24e80705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 -- 172.21.15.43:0/4156937683 >> 172.21.15.43:0/4156937683 conn(0x7f24e806ce60 msgr2=0x7f24e806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:46.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 -- 172.21.15.43:0/4156937683 shutdown_connections 2024-09-17T05:25:46.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.716+0000 7f24eea00640 1 -- 172.21.15.43:0/4156937683 wait complete. 2024-09-17T05:25:46.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 Processor -- start 2024-09-17T05:25:46.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 -- start start 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 0x7f24e814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e80ce160 0x7f24e80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8167fe0 con 0x7f24e815b4d0 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8167e60 con 0x7f24e80cd9a0 2024-09-17T05:25:46.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eea00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f24e8168160 con 0x7f24e80ce160 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37404/0 (socket says 172.21.15.43:37404) 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 -- 172.21.15.43:0/516740477 learned_addr learned my addr 172.21.15.43:0/516740477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24dfe00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e80ce160 0x7f24e80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24e7e00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 0x7f24e814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 -- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e80ce160 msgr2=0x7f24e80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e80ce160 0x7f24e80cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 -- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 msgr2=0x7f24e814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 0x7f24e814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 -- 172.21.15.43:0/516740477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e80ada60 con 0x7f24e815b4d0 2024-09-17T05:25:46.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24e7e00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 0x7f24e814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.720+0000 7f24eca00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f24e001b420 tx=0x7f24e0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00bb070 con 0x7f24e815b4d0 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24e0004b40 con 0x7f24e815b4d0 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e80adf60 con 0x7f24e815b4d0 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00b2040 con 0x7f24e815b4d0 2024-09-17T05:25:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e8153210 con 0x7f24e815b4d0 2024-09-17T05:25:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e8152d50 con 0x7f24e815b4d0 2024-09-17T05:25:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24e0004000 con 0x7f24e815b4d0 2024-09-17T05:25:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.724+0000 7f24e4c00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 0x7f24c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.728+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5881+0+0 (secure 0 0 0) 0x7f24e012e3b0 con 0x7f24e815b4d0 2024-09-17T05:25:46.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.728+0000 7f24e7e00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 0x7f24c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.728+0000 7f24e7e00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 0x7f24c4075740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24d8001630 tx=0x7f24d8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:46.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.732+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24e8152d50 con 0x7f24e815b4d0 2024-09-17T05:25:46.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:46.904+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24e80adcf0 con 0x7f24e815b4d0 2024-09-17T05:25:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24e4c00640 1 -- 172.21.15.43:0/516740477 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f24e0016990 con 0x7f24e815b4d0 2024-09-17T05:25:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:25:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 msgr2=0x7f24c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 0x7f24c4075740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24d8001630 tx=0x7f24d8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:47.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 msgr2=0x7f24e80cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f24e001b420 tx=0x7f24e0005b00 comp rx=0 tx=0).stop 2024-09-17T05:25:47.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 shutdown_connections 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f24c4073230 0x7f24c4075740 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24d8001630 tx=0x7f24d8009040 comp rx=0 tx=0).stop 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24e815b4d0 0x7f24e80cbfa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f24e80ce160 0x7f24e80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 --2- 172.21.15.43:0/516740477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e80cd9a0 0x7f24e814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 >> 172.21.15.43:0/516740477 conn(0x7f24e806ce60 msgr2=0x7f24e806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.528+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 shutdown_connections 2024-09-17T05:25:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.532+0000 7f24eea00640 1 -- 172.21.15.43:0/516740477 wait complete. 2024-09-17T05:25:47.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 -- 172.21.15.43:0/3946534387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8930160470 msgr2=0x7f8930160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 --2- 172.21.15.43:0/3946534387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8930160470 0x7f8930160880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8920009840 tx=0x7f892001c790 comp rx=0 tx=0).stop 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 -- 172.21.15.43:0/3946534387 shutdown_connections 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 --2- 172.21.15.43:0/3946534387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89301410d0 0x7f89301414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 --2- 172.21.15.43:0/3946534387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8930160470 0x7f8930160880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 -- 172.21.15.43:0/3946534387 >> 172.21.15.43:0/3946534387 conn(0x7f893006ce60 msgr2=0x7f893006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:47.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 -- 172.21.15.43:0/3946534387 shutdown_connections 2024-09-17T05:25:47.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.720+0000 7f8935200640 1 -- 172.21.15.43:0/3946534387 wait complete. 2024-09-17T05:25:47.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 Processor -- start 2024-09-17T05:25:47.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- start start 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 0x7f893014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 0x7f89300ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f893016a610 con 0x7f89301410d0 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89300b4e10 con 0x7f893015d120 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f893016a790 con 0x7f8930160470 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37418/0 (socket says 172.21.15.43:37418) 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 -- 172.21.15.43:0/1115828288 learned_addr learned my addr 172.21.15.43:0/1115828288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892e800640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 0x7f89300ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892d400640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 0x7f893014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 -- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 msgr2=0x7f89300ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 0x7f89300ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 -- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 msgr2=0x7f893014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 0x7f893014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 -- 172.21.15.43:0/1115828288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930074d90 con 0x7f89301410d0 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892d400640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 0x7f893014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892e800640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 0x7f89300ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f892de00640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8920000c10 tx=0x7f89200a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200ac030 con 0x7f89301410d0 2024-09-17T05:25:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89200053b0 con 0x7f89301410d0 2024-09-17T05:25:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200046d0 con 0x7f89301410d0 2024-09-17T05:25:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893014dc90 con 0x7f89301410d0 2024-09-17T05:25:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.724+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89300cf410 con 0x7f89301410d0 2024-09-17T05:25:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.728+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89301414e0 con 0x7f89301410d0 2024-09-17T05:25:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.728+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89200af070 con 0x7f89301410d0 2024-09-17T05:25:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.728+0000 7f8926000640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.728+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5443+0+0 (secure 0 0 0) 0x7f892012e650 con 0x7f89301410d0 2024-09-17T05:25:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.728+0000 7f892d400640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.732+0000 7f892d400640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8930070900 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:47.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.732+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8920016410 con 0x7f89301410d0 2024-09-17T05:25:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:47.908+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89300cf640 con 0x7f89301410d0 2024-09-17T05:25:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.528+0000 7f8926000640 1 -- 172.21.15.43:0/1115828288 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7f89200f9770 con 0x7f89301410d0 2024-09-17T05:25:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:25:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.532+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 msgr2=0x7f8910075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.532+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8930070900 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.532+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 msgr2=0x7f8930099540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.532+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8920000c10 tx=0x7f89200a64e0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 shutdown_connections 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8910073200 0x7f8910075710 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893015d120 0x7f893014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8930160470 0x7f89300ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 --2- 172.21.15.43:0/1115828288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89301410d0 0x7f8930099540 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 >> 172.21.15.43:0/1115828288 conn(0x7f893006ce60 msgr2=0x7f893015c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.536+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 shutdown_connections 2024-09-17T05:25:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.540+0000 7f8935200640 1 -- 172.21.15.43:0/1115828288 wait complete. 2024-09-17T05:25:48.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:25:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- 172.21.15.43:0/569193216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b0155d70 msgr2=0x7f43b015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 --2- 172.21.15.43:0/569193216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b0155d70 0x7f43b015c7e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43b00d0c40 tx=0x7f43a001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- 172.21.15.43:0/569193216 shutdown_connections 2024-09-17T05:25:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 --2- 172.21.15.43:0/569193216 >> v2:172.21.15.62:3300/0 conn(0x7f43b00b3af0 0x7f43b00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 --2- 172.21.15.43:0/569193216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b0155d70 0x7f43b015c7e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- 172.21.15.43:0/569193216 >> 172.21.15.43:0/569193216 conn(0x7f43b00b23d0 msgr2=0x7f43b00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- 172.21.15.43:0/569193216 shutdown_connections 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- 172.21.15.43:0/569193216 wait complete. 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 Processor -- start 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.728+0000 7f43b5e00640 1 -- start start 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b006f110 0x7f43b015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0155d70 0x7f43b015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0168b60 con 0x7f43b01f89f0 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01689e0 con 0x7f43b0155d70 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43b0168ce0 con 0x7f43b006f110 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37428/0 (socket says 172.21.15.43:37428) 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 -- 172.21.15.43:0/1168312614 learned_addr learned my addr 172.21.15.43:0/1168312614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43a6000640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0155d70 0x7f43b015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43aea00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b006f110 0x7f43b015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 -- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b006f110 msgr2=0x7f43b015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b006f110 0x7f43b015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 -- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0155d70 msgr2=0x7f43b015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0155d70 0x7f43b015b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 -- 172.21.15.43:0/1168312614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b00cfa00 con 0x7f43b01f89f0 2024-09-17T05:25:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43af400640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d660 tx=0x7f43a800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8017070 con 0x7f43b01f89f0 2024-09-17T05:25:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a8005310 con 0x7f43b01f89f0 2024-09-17T05:25:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b00ce5f0 con 0x7f43b01f89f0 2024-09-17T05:25:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8009820 con 0x7f43b01f89f0 2024-09-17T05:25:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01f8f10 con 0x7f43b01f89f0 2024-09-17T05:25:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.732+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b0074430 con 0x7f43b01f89f0 2024-09-17T05:25:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.736+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43a80099c0 con 0x7f43b01f89f0 2024-09-17T05:25:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.736+0000 7f43a7e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.736+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7f43a8093f30 con 0x7f43b01f89f0 2024-09-17T05:25:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.736+0000 7f43aea00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.736+0000 7f43aea00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 0x7f438c075740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43a00af000 tx=0x7f43a001b500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:48.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.740+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43b0074430 con 0x7f43b01f89f0 2024-09-17T05:25:48.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.916+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f43b01f9170 con 0x7f43b01f89f0 2024-09-17T05:25:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.920+0000 7f43a7e00640 1 -- 172.21.15.43:0/1168312614 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v38) ==== 71+0+0 (secure 0 0 0) 0x7f43a805ec20 con 0x7f43b01f89f0 2024-09-17T05:25:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.920+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 msgr2=0x7f438c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.920+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 0x7f438c075740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43a00af000 tx=0x7f43a001b500 comp rx=0 tx=0).stop 2024-09-17T05:25:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.920+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 msgr2=0x7f43b00cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.920+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d660 tx=0x7f43a800db20 comp rx=0 tx=0).stop 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 shutdown_connections 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073230 0x7f438c075740 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0155d70 0x7f43b015b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b01f89f0 0x7f43b00cea10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 --2- 172.21.15.43:0/1168312614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b006f110 0x7f43b015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 >> 172.21.15.43:0/1168312614 conn(0x7f43b00b23d0 msgr2=0x7f43b00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 shutdown_connections 2024-09-17T05:25:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:48.924+0000 7f43b5e00640 1 -- 172.21.15.43:0/1168312614 wait complete. 2024-09-17T05:25:48.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-17T05:25:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 -- 172.21.15.43:0/2949173064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7840cb860 msgr2=0x7fd7840cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 --2- 172.21.15.43:0/2949173064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7840cb860 0x7fd7840cbc70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd7840d12a0 tx=0x7fd77c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 -- 172.21.15.43:0/2949173064 shutdown_connections 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 --2- 172.21.15.43:0/2949173064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7840cb860 0x7fd7840cbc70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 --2- 172.21.15.43:0/2949173064 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd78406ce60 0x7fd78406d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 --2- 172.21.15.43:0/2949173064 >> v2:172.21.15.62:3300/0 conn(0x7fd784070620 0x7fd7840b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 -- 172.21.15.43:0/2949173064 >> 172.21.15.43:0/2949173064 conn(0x7fd7840b23d0 msgr2=0x7fd7840d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 -- 172.21.15.43:0/2949173064 shutdown_connections 2024-09-17T05:25:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.104+0000 7fd78b800640 1 -- 172.21.15.43:0/2949173064 wait complete. 2024-09-17T05:25:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 Processor -- start 2024-09-17T05:25:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- start start 2024-09-17T05:25:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd784070620 0x7fd784159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840cb860 0x7fd78415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7840ceca0 con 0x7fd78406ce60 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7840ceb20 con 0x7fd784070620 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd7840cee20 con 0x7fd7840cb860 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37454/0 (socket says 172.21.15.43:37454) 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 -- 172.21.15.43:0/451426974 learned_addr learned my addr 172.21.15.43:0/451426974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd783e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd784070620 0x7fd784159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd789800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840cb860 0x7fd78415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 -- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840cb860 msgr2=0x7fd78415a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840cb860 0x7fd78415a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 -- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd784070620 msgr2=0x7fd784159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd784070620 0x7fd784159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 -- 172.21.15.43:0/451426974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd784159620 con 0x7fd78406ce60 2024-09-17T05:25:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd788e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c9f0 tx=0x7fd77800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778007b50 con 0x7fd78406ce60 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd778007de0 con 0x7fd78406ce60 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800f6b0 con 0x7fd78406ce60 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd784159340 con 0x7fd78406ce60 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.108+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7840c69e0 con 0x7fd78406ce60 2024-09-17T05:25:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.112+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7840b30d0 con 0x7fd78406ce60 2024-09-17T05:25:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.112+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd77800f850 con 0x7fd78406ce60 2024-09-17T05:25:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.112+0000 7fd780c00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 0x7fd764075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.112+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7fd7780946b0 con 0x7fd78406ce60 2024-09-17T05:25:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.112+0000 7fd783e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 0x7fd764075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.116+0000 7fd783e00640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 0x7fd764075740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd770005fd0 tx=0x7fd770005d10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.120+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd778014030 con 0x7fd78406ce60 2024-09-17T05:25:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.360+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fd7840cd880 con 0x7fd78406ce60 2024-09-17T05:25:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd780c00640 1 -- 172.21.15.43:0/451426974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v148) ==== 141+0+0 (secure 0 0 0) 0x7fd77805f280 con 0x7fd78406ce60 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 msgr2=0x7fd764075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 0x7fd764075740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd770005fd0 tx=0x7fd770005d10 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 msgr2=0x7fd7840c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c9f0 tx=0x7fd77800ceb0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 shutdown_connections 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd764073230 0x7fd764075740 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd784070620 0x7fd784159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd7840cb860 0x7fd78415a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 --2- 172.21.15.43:0/451426974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd78406ce60 0x7fd7840c3da0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 >> 172.21.15.43:0/451426974 conn(0x7fd7840b23d0 msgr2=0x7fd784166320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 shutdown_connections 2024-09-17T05:25:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.540+0000 7fd78b800640 1 -- 172.21.15.43:0/451426974 wait complete. 2024-09-17T05:25:49.567 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-09-17T05:25:49.567 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-17T05:25:49.567 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:25:49.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 -- 172.21.15.43:0/602087008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 msgr2=0x7f09780b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 --2- 172.21.15.43:0/602087008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f09780b33e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0964009840 tx=0x7f096401c760 comp rx=0 tx=0).stop 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 -- 172.21.15.43:0/602087008 shutdown_connections 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 --2- 172.21.15.43:0/602087008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f09780b33e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 --2- 172.21.15.43:0/602087008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0978155d70 0x7f097815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 -- 172.21.15.43:0/602087008 >> 172.21.15.43:0/602087008 conn(0x7f09780b23d0 msgr2=0x7f09780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 -- 172.21.15.43:0/602087008 shutdown_connections 2024-09-17T05:25:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.720+0000 7f097fe00640 1 -- 172.21.15.43:0/602087008 wait complete. 2024-09-17T05:25:49.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 Processor -- start 2024-09-17T05:25:49.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 -- start start 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09780b3920 0x7f0978074360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 0x7f097815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0978168a60 con 0x7f0978155d70 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09781688e0 con 0x7f097806f110 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097fe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0978168be0 con 0x7f09780b3920 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 0x7f097815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:35978/0 (socket says 172.21.15.43:35978) 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097d400640 1 -- 172.21.15.43:0/1962109051 learned_addr learned my addr 172.21.15.43:0/1962109051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:49.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.724+0000 7f097ca00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09780b3920 0x7f0978074360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 -- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09780b3920 msgr2=0x7f0978074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09780b3920 0x7f0978074360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 -- 172.21.15.43:0/1962109051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 msgr2=0x7f097815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 0x7f097815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 -- 172.21.15.43:0/1962109051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09780c6c30 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097de00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 0x7f097815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097d400640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e9f0 tx=0x7f096c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00cd80 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f096c010430 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c010860 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0978074f40 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0978073370 con 0x7f097806f110 2024-09-17T05:25:49.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09780b33e0 con 0x7f097806f110 2024-09-17T05:25:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.728+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f096c010a00 con 0x7f097806f110 2024-09-17T05:25:49.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.732+0000 7f0975600640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 0x7f0958075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.732+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7f096c094e50 con 0x7f097806f110 2024-09-17T05:25:49.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.732+0000 7f097ca00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 0x7f0958075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.732+0000 7f097ca00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 0x7f0958075710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f09640af000 tx=0x7f096401afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.736+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f096c026020 con 0x7f097806f110 2024-09-17T05:25:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.908+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f097815e9d0 con 0x7f097806f110 2024-09-17T05:25:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f0975600640 1 -- 172.21.15.43:0/1962109051 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13100 (secure 0 0 0) 0x7f096c05fb60 con 0x7f097806f110 2024-09-17T05:25:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:49.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":51,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:48.532210+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2098700023":"2024-09-18T05:25:45.522308+0000","172.21.15.62:6834/2098700023":"2024-09-18T05:25:45.522308+0000","172.21.15.62:6835/980890216":"2024-09-18T05:25:38.338069+0000","172.21.15.62:6834/980890216":"2024-09-18T05:25:38.338069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:49.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 msgr2=0x7f0958075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 0x7f0958075710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f09640af000 tx=0x7f096401afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 msgr2=0x7f097809dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e9f0 tx=0x7f096c00eeb0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 shutdown_connections 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0958073200 0x7f0958075710 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0978155d70 0x7f097815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f09780b3920 0x7f0978074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 --2- 172.21.15.43:0/1962109051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f097806f110 0x7f097809dad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 >> 172.21.15.43:0/1962109051 conn(0x7f09780b23d0 msgr2=0x7f09781551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 shutdown_connections 2024-09-17T05:25:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:49.912+0000 7f097fe00640 1 -- 172.21.15.43:0/1962109051 wait complete. 2024-09-17T05:25:49.938 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:25:49.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:25:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- 172.21.15.43:0/4091045971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd79406f110 msgr2=0x7fd7940b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 --2- 172.21.15.43:0/4091045971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd79406f110 0x7fd7940b33e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd794072b60 tx=0x7fd77c01c800 comp rx=0 tx=0).stop 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- 172.21.15.43:0/4091045971 shutdown_connections 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 --2- 172.21.15.43:0/4091045971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd79406f110 0x7fd7940b33e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 --2- 172.21.15.43:0/4091045971 >> v2:172.21.15.62:3300/0 conn(0x7fd794155d70 0x7fd79415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- 172.21.15.43:0/4091045971 >> 172.21.15.43:0/4091045971 conn(0x7fd7940b23d0 msgr2=0x7fd7940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- 172.21.15.43:0/4091045971 shutdown_connections 2024-09-17T05:25:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- 172.21.15.43:0/4091045971 wait complete. 2024-09-17T05:25:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 Processor -- start 2024-09-17T05:25:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.096+0000 7fd79a200640 1 -- start start 2024-09-17T05:25:50.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 0x7fd7940cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 0x7fd79415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd794168a60 con 0x7fd7940b3920 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941688e0 con 0x7fd79406f110 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd794168be0 con 0x7fd794155d70 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37496/0 (socket says 172.21.15.43:37496) 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 -- 172.21.15.43:0/3215971964 learned_addr learned my addr 172.21.15.43:0/3215971964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd793e00640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 0x7fd79415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd793400640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 0x7fd7940cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 -- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 msgr2=0x7fd79415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 0x7fd79415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 -- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 msgr2=0x7fd7940cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 0x7fd7940cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 -- 172.21.15.43:0/3215971964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd794161bf0 con 0x7fd7940b3920 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd793e00640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 0x7fd79415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd793400640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 0x7fd7940cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78be00640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0b0040 tx=0x7fd77c0a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0b2040 con 0x7fd7940b3920 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd77c0043c0 con 0x7fd7940b3920 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd794161910 con 0x7fd7940b3920 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0052d0 con 0x7fd7940b3920 2024-09-17T05:25:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79409d000 con 0x7fd7940b3920 2024-09-17T05:25:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd794161680 con 0x7fd7940b3920 2024-09-17T05:25:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.100+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd77c005470 con 0x7fd7940b3920 2024-09-17T05:25:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.104+0000 7fd78b400640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 0x7fd770075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.104+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7fd77c12dac0 con 0x7fd7940b3920 2024-09-17T05:25:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.104+0000 7fd793400640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 0x7fd770075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.104+0000 7fd793400640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 0x7fd770075740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7940715c0 tx=0x7fd78400b5b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.112+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd77c0164b0 con 0x7fd7940b3920 2024-09-17T05:25:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.288+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7940b33e0 con 0x7fd7940b3920 2024-09-17T05:25:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.288+0000 7fd78b400640 1 -- 172.21.15.43:0/3215971964 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7fd77c0f8e10 con 0x7fd7940b3920 2024-09-17T05:25:50.294 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 msgr2=0x7fd770075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 0x7fd770075740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7940715c0 tx=0x7fd78400b5b0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 msgr2=0x7fd79409d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0b0040 tx=0x7fd77c0a5820 comp rx=0 tx=0).stop 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 shutdown_connections 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd770073230 0x7fd770075740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd794155d70 0x7fd79415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7940b3920 0x7fd79409d800 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 --2- 172.21.15.43:0/3215971964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd79406f110 0x7fd7940cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 >> 172.21.15.43:0/3215971964 conn(0x7fd7940b23d0 msgr2=0x7fd7940c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 shutdown_connections 2024-09-17T05:25:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.292+0000 7fd79a200640 1 -- 172.21.15.43:0/3215971964 wait complete. 2024-09-17T05:25:50.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-17T05:25:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 -- 172.21.15.43:0/3775834648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae415fe10 msgr2=0x7f3ae40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 --2- 172.21.15.43:0/3775834648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae415fe10 0x7f3ae40b4b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009900 tx=0x7f3ad401c780 comp rx=0 tx=0).stop 2024-09-17T05:25:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 -- 172.21.15.43:0/3775834648 shutdown_connections 2024-09-17T05:25:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 --2- 172.21.15.43:0/3775834648 >> v2:172.21.15.62:3300/0 conn(0x7f3ae40737c0 0x7f3ae40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 --2- 172.21.15.43:0/3775834648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae415fe10 0x7f3ae40b4b80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 -- 172.21.15.43:0/3775834648 >> 172.21.15.43:0/3775834648 conn(0x7f3ae40b23d0 msgr2=0x7f3ae40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 -- 172.21.15.43:0/3775834648 shutdown_connections 2024-09-17T05:25:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.472+0000 7f3aea400640 1 -- 172.21.15.43:0/3775834648 wait complete. 2024-09-17T05:25:50.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 Processor -- start 2024-09-17T05:25:50.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- start start 2024-09-17T05:25:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae415fe10 0x7f3ae40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 0x7f3ae41fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae416a050 con 0x7f3ae406fc90 2024-09-17T05:25:50.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4169ed0 con 0x7f3ae415fe10 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae416a1d0 con 0x7f3ae41fc500 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37514/0 (socket says 172.21.15.43:37514) 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 -- 172.21.15.43:0/1577946478 learned_addr learned my addr 172.21.15.43:0/1577946478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3e00640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 0x7f3ae41fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3adbe00640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae415fe10 0x7f3ae40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 -- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 msgr2=0x7f3ae41fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 0x7f3ae41fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 -- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae415fe10 msgr2=0x7f3ae40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae415fe10 0x7f3ae40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 -- 172.21.15.43:0/1577946478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae41fce50 con 0x7f3ae406fc90 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3e00640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 0x7f3ae41fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3ae3400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009850 tx=0x7f3ad40a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40bb070 con 0x7f3ae406fc90 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad40043b0 con 0x7f3ae406fc90 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae415e500 con 0x7f3ae406fc90 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40b2040 con 0x7f3ae406fc90 2024-09-17T05:25:50.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.476+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae415eac0 con 0x7f3ae406fc90 2024-09-17T05:25:50.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.480+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae40700a0 con 0x7f3ae406fc90 2024-09-17T05:25:50.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.480+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ad40be070 con 0x7f3ae406fc90 2024-09-17T05:25:50.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.480+0000 7f3adb400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 0x7f3ac0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.480+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7f3ad412d9e0 con 0x7f3ae406fc90 2024-09-17T05:25:50.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.480+0000 7f3adbe00640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 0x7f3ac0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.484+0000 7f3adbe00640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 0x7f3ac0075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ae41520a0 tx=0x7f3acc008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.484+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ae41fce50 con 0x7f3ae406fc90 2024-09-17T05:25:50.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:50.724+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7f3ae40b4ad0 con 0x7f3ae406fc90 2024-09-17T05:25:50.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:50.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:51.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3adb400640 1 -- 172.21.15.43:0/1577946478 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v150) ==== 145+0+0 (secure 0 0 0) 0x7f3ad4016490 con 0x7f3ae406fc90 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 msgr2=0x7f3ac0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 0x7f3ac0075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ae41520a0 tx=0x7f3acc008040 comp rx=0 tx=0).stop 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 msgr2=0x7f3ae40994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009850 tx=0x7f3ad40a5510 comp rx=0 tx=0).stop 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 shutdown_connections 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ac0073200 0x7f3ac0075710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ae415fe10 0x7f3ae40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ae41fc500 0x7f3ae41fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 --2- 172.21.15.43:0/1577946478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ae406fc90 0x7f3ae40994b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 >> 172.21.15.43:0/1577946478 conn(0x7f3ae40b23d0 msgr2=0x7f3ae4075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 shutdown_connections 2024-09-17T05:25:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.548+0000 7f3aea400640 1 -- 172.21.15.43:0/1577946478 wait complete. 2024-09-17T05:25:51.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 -- 172.21.15.43:0/3006552797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0b4b00 msgr2=0x7fb88c1603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 --2- 172.21.15.43:0/3006552797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0b4b00 0x7fb88c1603e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb884009c80 tx=0x7fb88401cd10 comp rx=0 tx=0).stop 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 -- 172.21.15.43:0/3006552797 shutdown_connections 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 --2- 172.21.15.43:0/3006552797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0b4b00 0x7fb88c1603e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 --2- 172.21.15.43:0/3006552797 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 -- 172.21.15.43:0/3006552797 >> 172.21.15.43:0/3006552797 conn(0x7fb88c0691b0 msgr2=0x7fb88c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 -- 172.21.15.43:0/3006552797 shutdown_connections 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.732+0000 7fb891a00640 1 -- 172.21.15.43:0/3006552797 wait complete. 2024-09-17T05:25:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 Processor -- start 2024-09-17T05:25:51.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 -- start start 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c1fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 0x7fb88c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c16a390 con 0x7fb88c0cccb0 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c16a210 con 0x7fb88c0cbf60 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb891a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb88c16a510 con 0x7fb88c15cea0 2024-09-17T05:25:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb88be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37522/0 (socket says 172.21.15.43:37522) 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb88be00640 1 -- 172.21.15.43:0/3402314336 learned_addr learned my addr 172.21.15.43:0/3402314336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb88be00640 1 -- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 msgr2=0x7fb88c1fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.736+0000 7fb883e00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c1fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb891000640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 0x7fb88c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb88be00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c1fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb88be00640 1 -- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 msgr2=0x7fb88c15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb88be00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 0x7fb88c15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb88be00640 1 -- 172.21.15.43:0/3402314336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c1fd0b0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb883e00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c1fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb891000640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 0x7fb88c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb88be00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00e7c0 tx=0x7fb87c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c00ccc0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb87c00ce80 con 0x7fb88c0cccb0 2024-09-17T05:25:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c1f8d10 con 0x7fb88c0cccb0 2024-09-17T05:25:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c005280 con 0x7fb88c0cccb0 2024-09-17T05:25:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c1f9290 con 0x7fb88c0cccb0 2024-09-17T05:25:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88c1603e0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb87c005420 con 0x7fb88c0cccb0 2024-09-17T05:25:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.740+0000 7fb888c00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 0x7fb868075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.744+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7fb87c094980 con 0x7fb88c0cccb0 2024-09-17T05:25:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.744+0000 7fb883e00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 0x7fb868075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.744+0000 7fb883e00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 0x7fb868075710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0cc7d0 tx=0x7fb874009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.748+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb88c1fd0b0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.920+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb88c06a1e0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.920+0000 7fb888c00640 1 -- 172.21.15.43:0/3402314336 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) ==== 59+0+2294 (secure 0 0 0) 0x7fb87c05fcc0 con 0x7fb88c0cccb0 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:25:51.927 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:25:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:25:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:25:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:25:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:25:51.929 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:25:51.930 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:25:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 msgr2=0x7fb868075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 0x7fb868075710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0cc7d0 tx=0x7fb874009040 comp rx=0 tx=0).stop 2024-09-17T05:25:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 msgr2=0x7fb88c0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00e7c0 tx=0x7fb87c00ec80 comp rx=0 tx=0).stop 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 shutdown_connections 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb868073200 0x7fb868075710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb88c0cbf60 0x7fb88c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb88c15cea0 0x7fb88c1fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 --2- 172.21.15.43:0/3402314336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c0cccb0 0x7fb88c0cb950 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 >> 172.21.15.43:0/3402314336 conn(0x7fb88c0691b0 msgr2=0x7fb88c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 shutdown_connections 2024-09-17T05:25:51.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:51.924+0000 7fb891a00640 1 -- 172.21.15.43:0/3402314336 wait complete. 2024-09-17T05:25:51.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- 172.21.15.43:0/18592273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c155d70 msgr2=0x7fa08c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 --2- 172.21.15.43:0/18592273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c155d70 0x7fa08c15c7e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- 172.21.15.43:0/18592273 shutdown_connections 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 --2- 172.21.15.43:0/18592273 >> v2:172.21.15.62:3300/0 conn(0x7fa08c0b3af0 0x7fa08c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 --2- 172.21.15.43:0/18592273 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c155d70 0x7fa08c15c7e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- 172.21.15.43:0/18592273 >> 172.21.15.43:0/18592273 conn(0x7fa08c0b23d0 msgr2=0x7fa08c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- 172.21.15.43:0/18592273 shutdown_connections 2024-09-17T05:25:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- 172.21.15.43:0/18592273 wait complete. 2024-09-17T05:25:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 Processor -- start 2024-09-17T05:25:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.108+0000 7fa091000640 1 -- start start 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 0x7fa08c1fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c168c80 con 0x7fa08c06f110 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c168b00 con 0x7fa08c155d70 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c168e00 con 0x7fa08c154df0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37526/0 (socket says 172.21.15.43:37526) 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 -- 172.21.15.43:0/3548666071 learned_addr learned my addr 172.21.15.43:0/3548666071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa083e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa08a800640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 0x7fa08c1fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 -- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 msgr2=0x7fa08c1fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 0x7fa08c1fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 -- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 msgr2=0x7fa08c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 -- 172.21.15.43:0/3548666071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0cea70 con 0x7fa08c06f110 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa08a800640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 0x7fa08c1fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa089e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009850 tx=0x7fa07c0a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0b2040 con 0x7fa08c06f110 2024-09-17T05:25:52.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa07c004340 con 0x7fa08c06f110 2024-09-17T05:25:52.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0044f0 con 0x7fa08c06f110 2024-09-17T05:25:52.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c0ced50 con 0x7fa08c06f110 2024-09-17T05:25:52.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c0cf310 con 0x7fa08c06f110 2024-09-17T05:25:52.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c0ad0b0 con 0x7fa08c06f110 2024-09-17T05:25:52.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.112+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa07c005290 con 0x7fa08c06f110 2024-09-17T05:25:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.116+0000 7fa082000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.116+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7fa07c12db40 con 0x7fa08c06f110 2024-09-17T05:25:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.116+0000 7fa083e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.116+0000 7fa083e00640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa08c09dac0 tx=0x7fa074009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.120+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa07c0164d0 con 0x7fa08c06f110 2024-09-17T05:25:52.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.296+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa08c15c650 con 0x7fa08c06f110 2024-09-17T05:25:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.300+0000 7fa082000640 1 -- 172.21.15.43:0/3548666071 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7fa07c005580 con 0x7fa08c06f110 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 msgr2=0x7fa068075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa08c09dac0 tx=0x7fa074009040 comp rx=0 tx=0).stop 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 msgr2=0x7fa08c0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009850 tx=0x7fa07c0a57f0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 shutdown_connections 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa068073200 0x7fa068075710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa08c155d70 0x7fa08c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa08c154df0 0x7fa08c1fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 --2- 172.21.15.43:0/3548666071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa08c06f110 0x7fa08c0cfb10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 >> 172.21.15.43:0/3548666071 conn(0x7fa08c0b23d0 msgr2=0x7fa08c0c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 shutdown_connections 2024-09-17T05:25:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.304+0000 7fa091000640 1 -- 172.21.15.43:0/3548666071 wait complete. 2024-09-17T05:25:52.328 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:25:52.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:52.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:52.561 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:25:52.561 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:52.561 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:52.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:52.570 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:52.570 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:52.579 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:52.580 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:25:52.580 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:52.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:25:52.618 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:25:52.618 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:52.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:25:52.633 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:25:52.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- 172.21.15.43:0/2675104008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c9406f110 msgr2=0x7f2c940b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 --2- 172.21.15.43:0/2675104008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c9406f110 0x7f2c940b33e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2c94072b60 tx=0x7f2c8001c800 comp rx=0 tx=0).stop 2024-09-17T05:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- 172.21.15.43:0/2675104008 shutdown_connections 2024-09-17T05:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 --2- 172.21.15.43:0/2675104008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c9406f110 0x7f2c940b33e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 --2- 172.21.15.43:0/2675104008 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c940b3920 0x7f2c940ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 --2- 172.21.15.43:0/2675104008 >> v2:172.21.15.62:3300/0 conn(0x7f2c94155d70 0x7f2c9415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- 172.21.15.43:0/2675104008 >> 172.21.15.43:0/2675104008 conn(0x7f2c940b23d0 msgr2=0x7f2c940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- 172.21.15.43:0/2675104008 shutdown_connections 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- 172.21.15.43:0/2675104008 wait complete. 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 Processor -- start 2024-09-17T05:25:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.816+0000 7f2c9a000640 1 -- start start 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9406f110 0x7f2c940ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c94155d70 0x7f2c940cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415f7e0 con 0x7f2c940b3920 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415f660 con 0x7f2c94155d70 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9415f960 con 0x7f2c9406f110 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37540/0 (socket says 172.21.15.43:37540) 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 -- 172.21.15.43:0/1247317538 learned_addr learned my addr 172.21.15.43:0/1247317538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c93400640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9406f110 0x7f2c940ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c93e00640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c94155d70 0x7f2c940cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 -- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9406f110 msgr2=0x7f2c940ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9406f110 0x7f2c940ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 -- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c94155d70 msgr2=0x7f2c940cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c94155d70 0x7f2c940cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 -- 172.21.15.43:0/1247317538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9415de60 con 0x7f2c940b3920 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c92a00640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c800b0040 tx=0x7f2c80005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800b2040 con 0x7f2c940b3920 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c80004240 con 0x7f2c940b3920 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9415e140 con 0x7f2c940b3920 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80005020 con 0x7f2c940b3920 2024-09-17T05:25:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9415e700 con 0x7f2c940b3920 2024-09-17T05:25:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.820+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9415eae0 con 0x7f2c940b3920 2024-09-17T05:25:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.824+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c800051c0 con 0x7f2c940b3920 2024-09-17T05:25:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.824+0000 7f2c8b400640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 0x7f2c74075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.824+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7f2c8012db20 con 0x7f2c940b3920 2024-09-17T05:25:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.824+0000 7f2c93400640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 0x7f2c74075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.824+0000 7f2c93400640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 0x7f2c74075710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c940715c0 tx=0x7f2c840091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:52.828+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c9415eae0 con 0x7f2c940b3920 2024-09-17T05:25:53.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.036+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c9415fe10 con 0x7f2c940b3920 2024-09-17T05:25:53.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.036+0000 7f2c8b400640 1 -- 172.21.15.43:0/1247317538 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3908 (secure 0 0 0) 0x7f2c800164b0 con 0x7f2c940b3920 2024-09-17T05:25:53.043 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:53.043 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"btime":"2024-09-17T05:25:46:524749+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/614863803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":614863803},{"type":"v1","addr":"172.21.15.43:6834","nonce":614863803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1845260253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1845260253},{"type":"v1","addr":"172.21.15.62:6837","nonce":1845260253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5336,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3512346985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3512346985},{"type":"v1","addr":"172.21.15.62:6835","nonce":3512346985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2835043378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2835043378},{"type":"v1","addr":"172.21.15.43:6835","nonce":2835043378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-09-17T05:25:53.043 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 40 2024-09-17T05:25:53.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 msgr2=0x7f2c74075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 0x7f2c74075710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c940715c0 tx=0x7f2c840091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 msgr2=0x7f2c9415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2c800b0040 tx=0x7f2c80005e70 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 shutdown_connections 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c74073200 0x7f2c74075710 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c94155d70 0x7f2c940cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c940b3920 0x7f2c9415d1e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 --2- 172.21.15.43:0/1247317538 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2c9406f110 0x7f2c940ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 >> 172.21.15.43:0/1247317538 conn(0x7f2c940b23d0 msgr2=0x7f2c9406d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 shutdown_connections 2024-09-17T05:25:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.040+0000 7f2c9a000640 1 -- 172.21.15.43:0/1247317538 wait complete. 2024-09-17T05:25:53.066 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:25:53.066 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:25:53.066 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:25:53.066 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:53.067 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:25:53.067 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:25:53.067 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:25:53.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:53.067 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:25:53.067 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:25:53.068 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:25:53.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:53.068 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:25:53.068 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:25:53.068 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:25:53.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:53.079 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:25:53.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:25:53.082 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:25:53.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:25:53.083 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:25:53.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:25:53.090 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:25:53.090 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:25:53.125 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:25:53.126 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:53.128 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:25:53.128 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:53.135 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:25:53.135 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:53.137 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:25:53.138 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:53.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:25:53.224 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:25:53.228 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:25:53.246 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:25:53.246 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.296+0000 7fe0f8800640 1 -- 172.21.15.43:0/1586197056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0155d70 msgr2=0x7fe0f015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.296+0000 7fe0f8800640 1 --2- 172.21.15.43:0/1586197056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0155d70 0x7fe0f015c7e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009840 tx=0x7fe0e401c760 comp rx=0 tx=0).stop 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- 172.21.15.43:0/1586197056 shutdown_connections 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 --2- 172.21.15.43:0/1586197056 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 0x7fe0f00b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 --2- 172.21.15.43:0/1586197056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0155d70 0x7fe0f015c7e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- 172.21.15.43:0/1586197056 >> 172.21.15.43:0/1586197056 conn(0x7fe0f00b23d0 msgr2=0x7fe0f00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- 172.21.15.43:0/1586197056 shutdown_connections 2024-09-17T05:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- 172.21.15.43:0/1586197056 wait complete. 2024-09-17T05:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 Processor -- start 2024-09-17T05:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- start start 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 0x7fe0f00cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0f0155d70 0x7fe0f009d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0168a00 con 0x7fe0f00b3920 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0168880 con 0x7fe0f0155d70 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.300+0000 7fe0f8800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f0168b80 con 0x7fe0f006f110 2024-09-17T05:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37606/0 (socket says 172.21.15.43:37606) 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 -- 172.21.15.43:0/3115787048 learned_addr learned my addr 172.21.15.43:0/3115787048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5e00640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 0x7fe0f00cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f6800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0f0155d70 0x7fe0f009d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 -- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 msgr2=0x7fe0f00cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 0x7fe0f00cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 -- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0f0155d70 msgr2=0x7fe0f009d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0f0155d70 0x7fe0f009d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 -- 172.21.15.43:0/3115787048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01f9090 con 0x7fe0f00b3920 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f5400640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000cd80 con 0x7fe0f00b3920 2024-09-17T05:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0e0004020 con 0x7fe0f00b3920 2024-09-17T05:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0004450 con 0x7fe0f00b3920 2024-09-17T05:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01f9370 con 0x7fe0f00b3920 2024-09-17T05:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01f9930 con 0x7fe0f00b3920 2024-09-17T05:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f00b33e0 con 0x7fe0f00b3920 2024-09-17T05:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.304+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe0e0003600 con 0x7fe0f00b3920 2024-09-17T05:25:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.308+0000 7fe0ea000640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 0x7fe0d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.308+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5029+0+0 (secure 0 0 0) 0x7fe0e00947f0 con 0x7fe0f00b3920 2024-09-17T05:25:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.308+0000 7fe0f5e00640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 0x7fe0d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.308+0000 7fe0f5e00640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 0x7fe0d4075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40af000 tx=0x7fe0e401afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.312+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0e005f450 con 0x7fe0f00b3920 2024-09-17T05:25:53.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.496+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe0f01fd170 con 0x7fe0f00b3920 2024-09-17T05:25:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.564+0000 7fe0ea000640 1 -- 172.21.15.43:0/3115787048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7fe0e0063e80 con 0x7fe0f00b3920 2024-09-17T05:25:53.569 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 msgr2=0x7fe0d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 0x7fe0d4075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40af000 tx=0x7fe0e401afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 msgr2=0x7fe0f01fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 shutdown_connections 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe0d4073200 0x7fe0d4075710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0f0155d70 0x7fe0f009d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f00b3920 0x7fe0f01fc6d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 --2- 172.21.15.43:0/3115787048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe0f006f110 0x7fe0f00cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 >> 172.21.15.43:0/3115787048 conn(0x7fe0f00b23d0 msgr2=0x7fe0f015d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 shutdown_connections 2024-09-17T05:25:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.568+0000 7fe0f8800640 1 -- 172.21.15.43:0/3115787048 wait complete. 2024-09-17T05:25:53.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:25:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- 172.21.15.43:0/3649689562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc06f110 msgr2=0x7f48cc0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 --2- 172.21.15.43:0/3649689562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc06f110 0x7f48cc0b33e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b401c820 comp rx=0 tx=0).stop 2024-09-17T05:25:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- 172.21.15.43:0/3649689562 shutdown_connections 2024-09-17T05:25:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 --2- 172.21.15.43:0/3649689562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc06f110 0x7f48cc0b33e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 --2- 172.21.15.43:0/3649689562 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc155d70 0x7f48cc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- 172.21.15.43:0/3649689562 >> 172.21.15.43:0/3649689562 conn(0x7f48cc0b23d0 msgr2=0x7f48cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- 172.21.15.43:0/3649689562 shutdown_connections 2024-09-17T05:25:53.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- 172.21.15.43:0/3649689562 wait complete. 2024-09-17T05:25:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 Processor -- start 2024-09-17T05:25:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.748+0000 7f48d0e00640 1 -- start start 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 0x7f48cc09cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc155d70 0x7f48cc074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc168b10 con 0x7f48cc0b3920 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc168990 con 0x7f48cc155d70 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc168c90 con 0x7f48cc06f110 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39774/0 (socket says 172.21.15.43:39774) 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 -- 172.21.15.43:0/825575209 learned_addr learned my addr 172.21.15.43:0/825575209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c9e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 0x7f48cc09cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48ca800640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc155d70 0x7f48cc074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 -- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 msgr2=0x7f48cc09cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 0x7f48cc09cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 -- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc155d70 msgr2=0x7f48cc074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc155d70 0x7f48cc074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 -- 172.21.15.43:0/825575209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc09d190 con 0x7f48cc0b3920 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c9e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 0x7f48cc09cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c3e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48b401aed0 tx=0x7f48b40a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40b2040 con 0x7f48cc0b3920 2024-09-17T05:25:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b40043c0 con 0x7f48cc0b3920 2024-09-17T05:25:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc0c88e0 con 0x7f48cc0b3920 2024-09-17T05:25:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4005270 con 0x7f48cc0b3920 2024-09-17T05:25:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc0c6a80 con 0x7f48cc0b3920 2024-09-17T05:25:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.752+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0c6590 con 0x7f48cc0b3920 2024-09-17T05:25:53.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.756+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48b4005410 con 0x7f48cc0b3920 2024-09-17T05:25:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.756+0000 7f48c2000640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 0x7f48a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.756+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f48b412dce0 con 0x7f48cc0b3920 2024-09-17T05:25:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.756+0000 7f48c9e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 0x7f48a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.756+0000 7f48c9e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 0x7f48a8075740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0715c0 tx=0x7f48bc00b5b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.760+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48cc09d190 con 0x7f48cc0b3920 2024-09-17T05:25:54.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f48cc0b33e0 con 0x7f48cc0b3920 2024-09-17T05:25:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48c2000640 1 -- 172.21.15.43:0/825575209 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f48b40164d0 con 0x7f48cc0b3920 2024-09-17T05:25:54.626 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:54.626 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:25:54.626 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:25:54.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:25:54.627 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:54.629 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.630 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.632 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-09-17T05:25:54.636 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.637 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:25:54.638 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:54.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:25:54.641 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 msgr2=0x7f48a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 0x7f48a8075740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0715c0 tx=0x7f48bc00b5b0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 msgr2=0x7f48cc0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48b401aed0 tx=0x7f48b40a5800 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 shutdown_connections 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a8073230 0x7f48a8075740 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48cc155d70 0x7f48cc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48cc0b3920 0x7f48cc0740e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 --2- 172.21.15.43:0/825575209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48cc06f110 0x7f48cc09cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:53.996+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 >> 172.21.15.43:0/825575209 conn(0x7f48cc0b23d0 msgr2=0x7f48cc15e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.000+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 shutdown_connections 2024-09-17T05:25:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.000+0000 7f48d0e00640 1 -- 172.21.15.43:0/825575209 wait complete. 2024-09-17T05:25:54.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:25:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 -- 172.21.15.43:0/3836955872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d000b3af0 msgr2=0x7f6d0015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 --2- 172.21.15.43:0/3836955872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d000b3af0 0x7f6d0015c7e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6d000d0c40 tx=0x7f6cec01c740 comp rx=0 tx=0).stop 2024-09-17T05:25:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 -- 172.21.15.43:0/3836955872 shutdown_connections 2024-09-17T05:25:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 --2- 172.21.15.43:0/3836955872 >> v2:172.21.15.62:3300/0 conn(0x7f6d0006f110 0x7f6d000ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 --2- 172.21.15.43:0/3836955872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d000b3af0 0x7f6d0015c7e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 -- 172.21.15.43:0/3836955872 >> 172.21.15.43:0/3836955872 conn(0x7f6d000b23d0 msgr2=0x7f6d000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 -- 172.21.15.43:0/3836955872 shutdown_connections 2024-09-17T05:25:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.804+0000 7f6d07c00640 1 -- 172.21.15.43:0/3836955872 wait complete. 2024-09-17T05:25:54.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 Processor -- start 2024-09-17T05:25:54.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- start start 2024-09-17T05:25:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d0006f110 0x7f6d00074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0009d9f0 0x7f6d001f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00168c30 con 0x7f6d00155d70 2024-09-17T05:25:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00168ab0 con 0x7f6d0009d9f0 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6d00168db0 con 0x7f6d0006f110 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39790/0 (socket says 172.21.15.43:39790) 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 -- 172.21.15.43:0/1861378335 learned_addr learned my addr 172.21.15.43:0/1861378335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d05c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0009d9f0 0x7f6d001f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d05200640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d0006f110 0x7f6d00074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 -- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d0006f110 msgr2=0x7f6d00074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d0006f110 0x7f6d00074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 -- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0009d9f0 msgr2=0x7f6d001f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0009d9f0 0x7f6d001f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 -- 172.21.15.43:0/1861378335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000d05a0 con 0x7f6d00155d70 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cffe00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009790 tx=0x7f6cec0a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0bb070 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cec004470 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0b2040 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d00074160 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.808+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d00074720 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.812+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d000ad0b0 con 0x7f6d00155d70 2024-09-17T05:25:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.812+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6cec005230 con 0x7f6d00155d70 2024-09-17T05:25:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.812+0000 7f6cfcc00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 0x7f6ce0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.812+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f6cec12d800 con 0x7f6d00155d70 2024-09-17T05:25:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.812+0000 7f6d05200640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 0x7f6ce0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.816+0000 7f6d05200640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 0x7f6ce0075740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000f70 tx=0x7f6cf4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:54.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.816+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6cec0163f0 con 0x7f6d00155d70 2024-09-17T05:25:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6d000d0330 con 0x7f6d00155d70 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6cfcc00640 1 -- 172.21.15.43:0/1861378335 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7f6cec0f8520 con 0x7f6d00155d70 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:25:55.001 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 msgr2=0x7f6ce0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 0x7f6ce0075740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000f70 tx=0x7f6cf4009040 comp rx=0 tx=0).stop 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 msgr2=0x7f6d00155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:54.996+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009790 tx=0x7f6cec0a5680 comp rx=0 tx=0).stop 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 shutdown_connections 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ce0073230 0x7f6ce0075740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d0009d9f0 0x7f6d001f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d00155d70 0x7f6d00155140 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 --2- 172.21.15.43:0/1861378335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6d0006f110 0x7f6d00074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 >> 172.21.15.43:0/1861378335 conn(0x7f6d000b23d0 msgr2=0x7f6d000ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 shutdown_connections 2024-09-17T05:25:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.000+0000 7f6d07c00640 1 -- 172.21.15.43:0/1861378335 wait complete. 2024-09-17T05:25:55.023 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:25:55.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 -- 172.21.15.43:0/642054771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815d120 msgr2=0x7f13e815d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 --2- 172.21.15.43:0/642054771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815d120 0x7f13e815d530 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009900 tx=0x7f13d401c850 comp rx=0 tx=0).stop 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 -- 172.21.15.43:0/642054771 shutdown_connections 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 --2- 172.21.15.43:0/642054771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e81410d0 0x7f13e81414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 --2- 172.21.15.43:0/642054771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815d120 0x7f13e815d530 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 -- 172.21.15.43:0/642054771 >> 172.21.15.43:0/642054771 conn(0x7f13e806ce60 msgr2=0x7f13e806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 -- 172.21.15.43:0/642054771 shutdown_connections 2024-09-17T05:25:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.180+0000 7f13f0200640 1 -- 172.21.15.43:0/642054771 wait complete. 2024-09-17T05:25:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 Processor -- start 2024-09-17T05:25:55.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- start start 2024-09-17T05:25:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e815d120 0x7f13e8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e8160470 0x7f13e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13e816a610 con 0x7f13e81410d0 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13e80b4e10 con 0x7f13e8160470 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f13e816a790 con 0x7f13e815d120 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39806/0 (socket says 172.21.15.43:39806) 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 -- 172.21.15.43:0/3906436010 learned_addr learned my addr 172.21.15.43:0/3906436010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ece00640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e815d120 0x7f13e8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ee200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e8160470 0x7f13e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 -- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e815d120 msgr2=0x7f13e8074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e815d120 0x7f13e8074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 -- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e8160470 msgr2=0x7f13e8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e8160470 0x7f13e8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 -- 172.21.15.43:0/3906436010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e8074fe0 con 0x7f13e81410d0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13ed800640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c3c0 tx=0x7f13dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc017070 con 0x7f13e81410d0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13dc0053c0 con 0x7f13e81410d0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc003860 con 0x7f13e81410d0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e80734e0 con 0x7f13e81410d0 2024-09-17T05:25:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.184+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e8073aa0 con 0x7f13e81410d0 2024-09-17T05:25:55.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.188+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e81414e0 con 0x7f13e81410d0 2024-09-17T05:25:55.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.188+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13dc01a070 con 0x7f13e81410d0 2024-09-17T05:25:55.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.188+0000 7f13e5600640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 0x7f13c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.188+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f13dc093220 con 0x7f13e81410d0 2024-09-17T05:25:55.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.188+0000 7f13ece00640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 0x7f13c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.192+0000 7f13ece00640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 0x7f13c8075710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13d40af000 tx=0x7f13d401b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:55.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.192+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13dc05dea0 con 0x7f13e81410d0 2024-09-17T05:25:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.368+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f13e8151630 con 0x7f13e81410d0 2024-09-17T05:25:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.576+0000 7f13e5600640 1 -- 172.21.15.43:0/3906436010 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7f13dc091e20 con 0x7f13e81410d0 2024-09-17T05:25:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:25:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 msgr2=0x7f13c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 0x7f13c8075710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13d40af000 tx=0x7f13d401b090 comp rx=0 tx=0).stop 2024-09-17T05:25:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 msgr2=0x7f13e8152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c3c0 tx=0x7f13dc00c880 comp rx=0 tx=0).stop 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 shutdown_connections 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13c8073200 0x7f13c8075710 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e8160470 0x7f13e8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e815d120 0x7f13e8074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 --2- 172.21.15.43:0/3906436010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81410d0 0x7f13e8152e90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 >> 172.21.15.43:0/3906436010 conn(0x7f13e806ce60 msgr2=0x7f13e8099690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 shutdown_connections 2024-09-17T05:25:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.580+0000 7f13f0200640 1 -- 172.21.15.43:0/3906436010 wait complete. 2024-09-17T05:25:55.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 -- 172.21.15.43:0/3617077398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cd580 msgr2=0x7f82c41609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 --2- 172.21.15.43:0/3617077398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cd580 0x7f82c41609c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82c409bbe0 tx=0x7f82bc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 -- 172.21.15.43:0/3617077398 shutdown_connections 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 --2- 172.21.15.43:0/3617077398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40cd580 0x7f82c41609c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 --2- 172.21.15.43:0/3617077398 >> v2:172.21.15.62:3300/0 conn(0x7f82c40b7ce0 0x7f82c40d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 -- 172.21.15.43:0/3617077398 >> 172.21.15.43:0/3617077398 conn(0x7f82c40cee80 msgr2=0x7f82c40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 -- 172.21.15.43:0/3617077398 shutdown_connections 2024-09-17T05:25:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 -- 172.21.15.43:0/3617077398 wait complete. 2024-09-17T05:25:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.772+0000 7f82cba00640 1 Processor -- start 2024-09-17T05:25:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- start start 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40b7ce0 0x7f82c406f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 0x7f82c4154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4169ea0 con 0x7f82c40cd580 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c40b3800 con 0x7f82c40b7ce0 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f82c416a020 con 0x7f82c40d0d80 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39808/0 (socket says 172.21.15.43:39808) 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 -- 172.21.15.43:0/515730202 learned_addr learned my addr 172.21.15.43:0/515730202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c9a00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 0x7f82c4154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c9000640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40b7ce0 0x7f82c406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 -- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 msgr2=0x7f82c4154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 0x7f82c4154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 -- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40b7ce0 msgr2=0x7f82c406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40b7ce0 0x7f82c406f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 -- 172.21.15.43:0/515730202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c4158cf0 con 0x7f82c40cd580 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c9a00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 0x7f82c4154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82bbe00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c390 tx=0x7f82ac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac017070 con 0x7f82c40cd580 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82ac0053c0 con 0x7f82c40cd580 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c40cb940 con 0x7f82c40cd580 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac003840 con 0x7f82c40cd580 2024-09-17T05:25:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.776+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c406e980 con 0x7f82c40cd580 2024-09-17T05:25:55.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.780+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c40d0840 con 0x7f82c40cd580 2024-09-17T05:25:55.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.780+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82ac01a070 con 0x7f82c40cd580 2024-09-17T05:25:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.780+0000 7f82c1600640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 0x7f82a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.780+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7f82ac093660 con 0x7f82c40cd580 2024-09-17T05:25:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.784+0000 7f82c9000640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 0x7f82a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:55.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.784+0000 7f82c9000640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 0x7f82a0075710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009660 tx=0x7f82b40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:55.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.784+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82ac021020 con 0x7f82c40cd580 2024-09-17T05:25:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:55.960+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f82c40b7240 con 0x7f82c40cd580 2024-09-17T05:25:56.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.584+0000 7f82c1600640 1 -- 172.21.15.43:0/515730202 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f82ac05e280 con 0x7f82c40cd580 2024-09-17T05:25:56.590 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:25:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 msgr2=0x7f82a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 0x7f82a0075710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009660 tx=0x7f82b40091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 msgr2=0x7f82c41587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:56.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c390 tx=0x7f82ac00c850 comp rx=0 tx=0).stop 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 shutdown_connections 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a0073200 0x7f82a0075710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c40d0d80 0x7f82c4154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40cd580 0x7f82c41587b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 --2- 172.21.15.43:0/515730202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40b7ce0 0x7f82c406f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 >> 172.21.15.43:0/515730202 conn(0x7f82c40cee80 msgr2=0x7f82c40af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 shutdown_connections 2024-09-17T05:25:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.588+0000 7f82cba00640 1 -- 172.21.15.43:0/515730202 wait complete. 2024-09-17T05:25:56.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:25:56.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- 172.21.15.43:0/2898136029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 msgr2=0x7f46b415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:56.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 --2- 172.21.15.43:0/2898136029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b415c7e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009840 tx=0x7f46a401c760 comp rx=0 tx=0).stop 2024-09-17T05:25:56.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- 172.21.15.43:0/2898136029 shutdown_connections 2024-09-17T05:25:56.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 --2- 172.21.15.43:0/2898136029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b406f110 0x7f46b40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 --2- 172.21.15.43:0/2898136029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b415c7e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- 172.21.15.43:0/2898136029 >> 172.21.15.43:0/2898136029 conn(0x7f46b40b23d0 msgr2=0x7f46b40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- 172.21.15.43:0/2898136029 shutdown_connections 2024-09-17T05:25:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- 172.21.15.43:0/2898136029 wait complete. 2024-09-17T05:25:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 Processor -- start 2024-09-17T05:25:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.792+0000 7f46bac00640 1 -- start start 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 0x7f46b40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b40b3920 0x7f46b40cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4168a00 con 0x7f46b4155d70 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4168880 con 0x7f46b406f110 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f46b4168b80 con 0x7f46b40b3920 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39828/0 (socket says 172.21.15.43:39828) 2024-09-17T05:25:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 -- 172.21.15.43:0/3194021080 learned_addr learned my addr 172.21.15.43:0/3194021080 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46ab400640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b40b3920 0x7f46b40cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b3e00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 0x7f46b40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 -- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b40b3920 msgr2=0x7f46b40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b40b3920 0x7f46b40cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 -- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 msgr2=0x7f46b40c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 0x7f46b40c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 -- 172.21.15.43:0/3194021080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41fc980 con 0x7f46b4155d70 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b3e00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 0x7f46b40c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b8c00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c760 tx=0x7f46ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003db0 con 0x7f46b4155d70 2024-09-17T05:25:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ac00ce10 con 0x7f46b4155d70 2024-09-17T05:25:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0052d0 con 0x7f46b4155d70 2024-09-17T05:25:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41fcc60 con 0x7f46b4155d70 2024-09-17T05:25:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41fd220 con 0x7f46b4155d70 2024-09-17T05:25:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.796+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b40b33e0 con 0x7f46b4155d70 2024-09-17T05:25:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.800+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46ac005470 con 0x7f46b4155d70 2024-09-17T05:25:56.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.800+0000 7f46b0c00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 0x7f4690075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:56.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.800+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7f46ac094a00 con 0x7f46b4155d70 2024-09-17T05:25:56.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.800+0000 7f46b3e00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 0x7f4690075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:56.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.804+0000 7f46b3e00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 0x7f4690075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46a40af000 tx=0x7f46a401afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:56.804+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f46b40b3d30 con 0x7f46b4155d70 2024-09-17T05:25:57.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.012+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f46b41fc710 con 0x7f46b4155d70 2024-09-17T05:25:57.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.588+0000 7f46b0c00640 1 -- 172.21.15.43:0/3194021080 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v43) ==== 172+0+0 (secure 0 0 0) 0x7f46ac05fa60 con 0x7f46b4155d70 2024-09-17T05:25:57.592 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 8 and data pool 9 2024-09-17T05:25:57.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 msgr2=0x7f4690075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 0x7f4690075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46a40af000 tx=0x7f46a401afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 msgr2=0x7f46b4074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c760 tx=0x7f46ac00cc20 comp rx=0 tx=0).stop 2024-09-17T05:25:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 shutdown_connections 2024-09-17T05:25:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4690073200 0x7f4690075710 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46a40af000 tx=0x7f46a401afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b4155d70 0x7f46b4074210 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46b40b3920 0x7f46b40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 --2- 172.21.15.43:0/3194021080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46b406f110 0x7f46b40c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 >> 172.21.15.43:0/3194021080 conn(0x7f46b40b23d0 msgr2=0x7f46b415e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 shutdown_connections 2024-09-17T05:25:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.592+0000 7f46bac00640 1 -- 172.21.15.43:0/3194021080 wait complete. 2024-09-17T05:25:57.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- 172.21.15.43:0/3302587948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080b3af0 msgr2=0x7f2f080ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3302587948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080b3af0 0x7f2f080ad0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009c80 tx=0x7f2f0001cce0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- 172.21.15.43:0/3302587948 shutdown_connections 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3302587948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f080b3af0 0x7f2f080ad0b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3302587948 >> v2:172.21.15.62:3300/0 conn(0x7f2f08155d70 0x7f2f0815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- 172.21.15.43:0/3302587948 >> 172.21.15.43:0/3302587948 conn(0x7f2f080b23d0 msgr2=0x7f2f080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:57.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- 172.21.15.43:0/3302587948 shutdown_connections 2024-09-17T05:25:57.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- 172.21.15.43:0/3302587948 wait complete. 2024-09-17T05:25:57.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 Processor -- start 2024-09-17T05:25:57.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.808+0000 7f2f0e400640 1 -- start start 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f0806f110 0x7f2f080cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f0809d830 0x7f2f08155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08168cb0 con 0x7f2f08155d70 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08168b30 con 0x7f2f0806f110 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08168e30 con 0x7f2f0809d830 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39832/0 (socket says 172.21.15.43:39832) 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 -- 172.21.15.43:0/3132555396 learned_addr learned my addr 172.21.15.43:0/3132555396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f07400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f0806f110 0x7f2f080cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f07e00640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f0809d830 0x7f2f08155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 -- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f0809d830 msgr2=0x7f2f08155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f0809d830 0x7f2f08155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 -- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f0806f110 msgr2=0x7f2f080cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f0806f110 0x7f2f080cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 -- 172.21.15.43:0/3132555396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0815f9c0 con 0x7f2f08155d70 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2efea00640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c3f0 tx=0x7f2ef000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0017070 con 0x7f2f08155d70 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef00053c0 con 0x7f2f08155d70 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0815fca0 con 0x7f2f08155d70 2024-09-17T05:25:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0003860 con 0x7f2f08155d70 2024-09-17T05:25:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081f93d0 con 0x7f2f08155d70 2024-09-17T05:25:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.812+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f081f84a0 con 0x7f2f08155d70 2024-09-17T05:25:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.816+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ef001a070 con 0x7f2f08155d70 2024-09-17T05:25:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.816+0000 7f2effe00640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 0x7f2ee4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.816+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef00936f0 con 0x7f2f08155d70 2024-09-17T05:25:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.816+0000 7f2f07400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 0x7f2ee4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.820+0000 7f2f07400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 0x7f2ee4075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009600 tx=0x7f2ef80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.820+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ef0021080 con 0x7f2f08155d70 2024-09-17T05:25:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.996+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f080cf930 con 0x7f2f08155d70 2024-09-17T05:25:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:57.996+0000 7f2effe00640 1 -- 172.21.15.43:0/3132555396 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7f2ef005e130 con 0x7f2f08155d70 2024-09-17T05:25:58.002 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:58.003 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":58,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:57.018717+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:55.383804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:55.976123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 msgr2=0x7f2ee4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 0x7f2ee4075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009600 tx=0x7f2ef80091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 msgr2=0x7f2f081f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c3f0 tx=0x7f2ef000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 shutdown_connections 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ee4073200 0x7f2ee4075710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f0809d830 0x7f2f08155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f08155d70 0x7f2f081f8e90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 --2- 172.21.15.43:0/3132555396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f0806f110 0x7f2f080cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 >> 172.21.15.43:0/3132555396 conn(0x7f2f080b23d0 msgr2=0x7f2f0815e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 shutdown_connections 2024-09-17T05:25:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.000+0000 7f2f0e400640 1 -- 172.21.15.43:0/3132555396 wait complete. 2024-09-17T05:25:58.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- 172.21.15.43:0/3253024567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0650 msgr2=0x7f74380cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 --2- 172.21.15.43:0/3253024567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0650 0x7f74380cd750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7430009c80 tx=0x7f743001cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- 172.21.15.43:0/3253024567 shutdown_connections 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 --2- 172.21.15.43:0/3253024567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0650 0x7f74380cd750 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 --2- 172.21.15.43:0/3253024567 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 0x7f74380b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- 172.21.15.43:0/3253024567 >> 172.21.15.43:0/3253024567 conn(0x7f74380cee60 msgr2=0x7f74380cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- 172.21.15.43:0/3253024567 shutdown_connections 2024-09-17T05:25:58.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- 172.21.15.43:0/3253024567 wait complete. 2024-09-17T05:25:58.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 Processor -- start 2024-09-17T05:25:58.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.180+0000 7f743ec00640 1 -- start start 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 0x7f74381581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74380d0650 0x7f743806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74380b2b80 con 0x7f74380d0240 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74380b2a00 con 0x7f74380d0650 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7438169e30 con 0x7f743809ba60 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39850/0 (socket says 172.21.15.43:39850) 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 -- 172.21.15.43:0/2172288979 learned_addr learned my addr 172.21.15.43:0/2172288979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f7437e00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 0x7f74381581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743cc00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74380d0650 0x7f743806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 -- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 msgr2=0x7f74381581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 0x7f74381581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 -- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74380d0650 msgr2=0x7f743806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74380d0650 0x7f743806e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 -- 172.21.15.43:0/2172288979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74381fbfd0 con 0x7f74380d0240 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f742f400640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f742000b410 tx=0x7f742000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420015020 con 0x7f74380d0240 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7420011920 con 0x7f74380d0240 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74381fc2b0 con 0x7f74380d0240 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011d70 con 0x7f74380d0240 2024-09-17T05:25:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381fc870 con 0x7f74380d0240 2024-09-17T05:25:58.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.184+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74380b3130 con 0x7f74380d0240 2024-09-17T05:25:58.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.188+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7420018020 con 0x7f74380d0240 2024-09-17T05:25:58.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.188+0000 7f7434c00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 0x7f7414075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.188+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f74200941a0 con 0x7f74380d0240 2024-09-17T05:25:58.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.188+0000 7f7437e00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 0x7f7414075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.188+0000 7f7437e00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 0x7f7414075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428006f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.192+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f742001f020 con 0x7f74380d0240 2024-09-17T05:25:58.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74380cea00 con 0x7f74380d0240 2024-09-17T05:25:58.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f7434c00640 1 -- 172.21.15.43:0/2172288979 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7f742005ea50 con 0x7f74380d0240 2024-09-17T05:25:58.373 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:25:58.373 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":58,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:57.018717+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:55.383804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:55.976123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 msgr2=0x7f7414075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 0x7f7414075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428006f90 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 msgr2=0x7f74380b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f742000b410 tx=0x7f742000b8d0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 shutdown_connections 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7414073200 0x7f7414075710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74380d0650 0x7f743806e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74380d0240 0x7f74380b0270 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 --2- 172.21.15.43:0/2172288979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f743809ba60 0x7f74381581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 >> 172.21.15.43:0/2172288979 conn(0x7f74380cee60 msgr2=0x7f74380cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 shutdown_connections 2024-09-17T05:25:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.368+0000 7f743ec00640 1 -- 172.21.15.43:0/2172288979 wait complete. 2024-09-17T05:25:58.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- 172.21.15.43:0/661069083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef80b3af0 msgr2=0x7feef815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 --2- 172.21.15.43:0/661069083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef80b3af0 0x7feef815c7e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feee4009840 tx=0x7feee401c790 comp rx=0 tx=0).stop 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- 172.21.15.43:0/661069083 shutdown_connections 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 --2- 172.21.15.43:0/661069083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef80b3af0 0x7feef815c7e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- 172.21.15.43:0/661069083 >> 172.21.15.43:0/661069083 conn(0x7feef80b23d0 msgr2=0x7feef80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- 172.21.15.43:0/661069083 shutdown_connections 2024-09-17T05:25:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- 172.21.15.43:0/661069083 wait complete. 2024-09-17T05:25:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 Processor -- start 2024-09-17T05:25:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.556+0000 7fef00000640 1 -- start start 2024-09-17T05:25:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 0x7feef809dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef8074da0 0x7feef815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feef8169d70 con 0x7feef8155d70 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feef8169bf0 con 0x7feef8074da0 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7feef8169ef0 con 0x7feef806f110 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39872/0 (socket says 172.21.15.43:39872) 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 -- 172.21.15.43:0/208300427 learned_addr learned my addr 172.21.15.43:0/208300427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefe000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef8074da0 0x7feef815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefd600640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 0x7feef809dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 -- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 msgr2=0x7feef809dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 0x7feef809dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 -- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef8074da0 msgr2=0x7feef815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef8074da0 0x7feef815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 -- 172.21.15.43:0/208300427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80c3d30 con 0x7feef8155d70 2024-09-17T05:25:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefd600640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 0x7feef809dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feefcc00640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feee4009790 tx=0x7feee4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40b2040 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee40043e0 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40052d0 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef80cb9d0 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef81fc750 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.560+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef80c3f00 con 0x7feef8155d70 2024-09-17T05:25:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.564+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feee4005470 con 0x7feef8155d70 2024-09-17T05:25:58.568 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:25:58.564+0000 7fa5ec600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:58.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.564+0000 7feef5600640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 0x7feed8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.564+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7feee412dce0 con 0x7feef8155d70 2024-09-17T05:25:58.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.564+0000 7feefd600640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 0x7feed8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.564+0000 7feefd600640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 0x7feed8075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f70 tx=0x7feeec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.568+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feef80c3f00 con 0x7feef8155d70 2024-09-17T05:25:58.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:58.788+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feef815c7e0 con 0x7feef8155d70 2024-09-17T05:25:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.608+0000 7feef5600640 1 -- 172.21.15.43:0/208300427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7feee4016410 con 0x7feef8155d70 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 msgr2=0x7feed8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 0x7feed8075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f70 tx=0x7feeec009040 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 msgr2=0x7feef815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feee4009790 tx=0x7feee4005db0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 shutdown_connections 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feed8073200 0x7feed8075710 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feef8074da0 0x7feef815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef8155d70 0x7feef815a710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 --2- 172.21.15.43:0/208300427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feef806f110 0x7feef809dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 >> 172.21.15.43:0/208300427 conn(0x7feef80b23d0 msgr2=0x7feef80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 shutdown_connections 2024-09-17T05:25:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.612+0000 7fef00000640 1 -- 172.21.15.43:0/208300427 wait complete. 2024-09-17T05:25:59.641 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:25:59.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 -- 172.21.15.43:0/1024233920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 msgr2=0x7f93d0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 --2- 172.21.15.43:0/1024233920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93d0004c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f93ec073060 tx=0x7f93d401cf00 comp rx=0 tx=0).stop 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 -- 172.21.15.43:0/1024233920 shutdown_connections 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 --2- 172.21.15.43:0/1024233920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93d0004c20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 -- 172.21.15.43:0/1024233920 >> 172.21.15.43:0/1024233920 conn(0x7f93ec0b23d0 msgr2=0x7f93ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 -- 172.21.15.43:0/1024233920 shutdown_connections 2024-09-17T05:25:59.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 -- 172.21.15.43:0/1024233920 wait complete. 2024-09-17T05:25:59.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.804+0000 7f93f1c00640 1 Processor -- start 2024-09-17T05:25:59.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- start start 2024-09-17T05:25:59.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93ec0c7620 0x7f93ec0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93ec0cef70 0x7f93ec073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0ad6c0 con 0x7f93d0004810 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0ad540 con 0x7f93ec0cef70 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec0ad840 con 0x7f93ec0c7620 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39882/0 (socket says 172.21.15.43:39882) 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 -- 172.21.15.43:0/3989736685 learned_addr learned my addr 172.21.15.43:0/3989736685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93e2000640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93ec0c7620 0x7f93ec0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eb400640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93ec0cef70 0x7f93ec073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 -- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93ec0c7620 msgr2=0x7f93ec0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93ec0c7620 0x7f93ec0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 -- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93ec0cef70 msgr2=0x7f93ec073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93ec0cef70 0x7f93ec073bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 -- 172.21.15.43:0/3989736685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec0b0370 con 0x7f93d0004810 2024-09-17T05:25:59.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93eaa00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f93dc003ed0 tx=0x7f93dc001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc015070 con 0x7f93d0004810 2024-09-17T05:25:59.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93dc011920 con 0x7f93d0004810 2024-09-17T05:25:59.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec0cc5a0 con 0x7f93d0004810 2024-09-17T05:25:59.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc011d70 con 0x7f93d0004810 2024-09-17T05:25:59.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.808+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec153cc0 con 0x7f93d0004810 2024-09-17T05:25:59.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec0c7a90 con 0x7f93d0004810 2024-09-17T05:25:59.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93dc018070 con 0x7f93d0004810 2024-09-17T05:25:59.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93e3400640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 0x7f93c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f93dc094f70 con 0x7f93d0004810 2024-09-17T05:25:59.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93e2000640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 0x7f93c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.812+0000 7f93e2000640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 0x7f93c8075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93d40ab000 tx=0x7f93d4019d40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:25:59.816+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93dc05ff90 con 0x7f93d0004810 2024-09-17T05:26:00.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.028+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f93ec0adda0 con 0x7f93d0004810 2024-09-17T05:26:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.616+0000 7f93e3400640 1 -- 172.21.15.43:0/3989736685 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f93dc05f890 con 0x7f93d0004810 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 msgr2=0x7f93c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 0x7f93c8075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93d40ab000 tx=0x7f93d4019d40 comp rx=0 tx=0).stop 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 msgr2=0x7f93ec09ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f93dc003ed0 tx=0x7f93dc001720 comp rx=0 tx=0).stop 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 shutdown_connections 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f93c8073200 0x7f93c8075710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93ec0cef70 0x7f93ec073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93ec0c7620 0x7f93ec0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 --2- 172.21.15.43:0/3989736685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004810 0x7f93ec09ba40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 >> 172.21.15.43:0/3989736685 conn(0x7f93ec0b23d0 msgr2=0x7f93ec06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 shutdown_connections 2024-09-17T05:26:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.620+0000 7f93f1c00640 1 -- 172.21.15.43:0/3989736685 wait complete. 2024-09-17T05:26:00.646 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:26:00.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 -- 172.21.15.43:0/2174211515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c0ce3c0 msgr2=0x7fd76c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 --2- 172.21.15.43:0/2174211515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0cd8b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0750e0 tx=0x7fd75401c770 comp rx=0 tx=0).stop 2024-09-17T05:26:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 -- 172.21.15.43:0/2174211515 shutdown_connections 2024-09-17T05:26:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 --2- 172.21.15.43:0/2174211515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0cd8b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 --2- 172.21.15.43:0/2174211515 >> v2:172.21.15.62:3300/0 conn(0x7fd76c0cdfb0 0x7fd76c0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 -- 172.21.15.43:0/2174211515 >> 172.21.15.43:0/2174211515 conn(0x7fd76c0b2810 msgr2=0x7fd76c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 -- 172.21.15.43:0/2174211515 shutdown_connections 2024-09-17T05:26:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.800+0000 7fd770c00640 1 -- 172.21.15.43:0/2174211515 wait complete. 2024-09-17T05:26:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 Processor -- start 2024-09-17T05:26:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- start start 2024-09-17T05:26:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 0x7fd76c14b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c1603e0 0x7fd76c0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c16a150 con 0x7fd76c1603e0 2024-09-17T05:26:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c169fd0 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c16a2d0 con 0x7fd76c0cdfb0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd76a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c1603e0 0x7fd76c0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd769e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 0x7fd76c14b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:59850/0 (socket says 172.21.15.43:59850) 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 -- 172.21.15.43:0/1622795631 learned_addr learned my addr 172.21.15.43:0/1622795631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 -- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 msgr2=0x7fd76c14b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 0x7fd76c14b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 -- 172.21.15.43:0/1622795631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c1603e0 msgr2=0x7fd76c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c1603e0 0x7fd76c0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 -- 172.21.15.43:0/1622795631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c15a350 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd769e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 0x7fd76c14b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd763e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd754000c10 tx=0x7fd7540a6b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7540a6d00 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7540a6ea0 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c152da0 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754004290 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.804+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c153360 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c15a5c0 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd754005070 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd762000640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd75412dbc0 con 0x7fd76c0ce3c0 2024-09-17T05:26:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd769e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.808+0000 7fd769e00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 0x7fd748075710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00bf40 tx=0x7fd75c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:00.812+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd7540bf070 con 0x7fd76c0ce3c0 2024-09-17T05:26:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.024+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd76c1607f0 con 0x7fd76c0ce3c0 2024-09-17T05:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.024+0000 7fd762000640 1 -- 172.21.15.43:0/1622795631 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5214 (secure 0 0 0) 0x7fd7540ac030 con 0x7fd76c0ce3c0 2024-09-17T05:26:01.030 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:01.031 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"btime":"2024-09-17T05:26:00:616105+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:00.035083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2376863639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2376863639},{"type":"v1","addr":"172.21.15.43:6834","nonce":2376863639}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5838,"qdb_cluster":[5838]},"id":4}]} 2024-09-17T05:26:01.031 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 46 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 msgr2=0x7fd748075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 0x7fd748075710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00bf40 tx=0x7fd75c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 msgr2=0x7fd76c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd754000c10 tx=0x7fd7540a6b10 comp rx=0 tx=0).stop 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 shutdown_connections 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd748073200 0x7fd748075710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76c0ce3c0 0x7fd76c0b1900 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76c1603e0 0x7fd76c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 --2- 172.21.15.43:0/1622795631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76c0cdfb0 0x7fd76c14b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 >> 172.21.15.43:0/1622795631 conn(0x7fd76c0b2810 msgr2=0x7fd76c150300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 shutdown_connections 2024-09-17T05:26:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.028+0000 7fd770c00640 1 -- 172.21.15.43:0/1622795631 wait complete. 2024-09-17T05:26:01.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:26:01.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 -- 172.21.15.43:0/2942962709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 msgr2=0x7f38e40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 --2- 172.21.15.43:0/2942962709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e40b4b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009900 tx=0x7f38d401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 -- 172.21.15.43:0/2942962709 shutdown_connections 2024-09-17T05:26:01.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 --2- 172.21.15.43:0/2942962709 >> v2:172.21.15.62:3300/0 conn(0x7f38e406fc90 0x7f38e40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 --2- 172.21.15.43:0/2942962709 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e40b4b80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.204+0000 7f38e9c00640 1 -- 172.21.15.43:0/2942962709 >> 172.21.15.43:0/2942962709 conn(0x7f38e40b23d0 msgr2=0x7f38e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- 172.21.15.43:0/2942962709 shutdown_connections 2024-09-17T05:26:01.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- 172.21.15.43:0/2942962709 wait complete. 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 Processor -- start 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- start start 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 0x7f38e4151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e41fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4169ff0 con 0x7f38e40730b0 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4169e70 con 0x7f38e406fc90 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.208+0000 7f38e9c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f38e416a170 con 0x7f38e415fe10 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39900/0 (socket says 172.21.15.43:39900) 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 -- 172.21.15.43:0/3024260165 learned_addr learned my addr 172.21.15.43:0/3024260165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e3400640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e41fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e2a00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 0x7f38e4151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 -- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 msgr2=0x7f38e41fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e41fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 -- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 msgr2=0x7f38e4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 0x7f38e4151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 -- 172.21.15.43:0/3024260165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4158f60 con 0x7f38e40730b0 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e2a00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 0x7f38e4151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38dbe00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e7c0 tx=0x7f38cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00ccc0 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38cc00ce80 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc005280 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e4157da0 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e4158360 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e4157b10 con 0x7f38e40730b0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.212+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f38cc005420 con 0x7f38e40730b0 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.216+0000 7f38daa00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 0x7f38c0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.216+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f38cc094bd0 con 0x7f38e40730b0 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.216+0000 7f38e2a00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 0x7f38c0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.216+0000 7f38e2a00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 0x7f38c0075740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af000 tx=0x7f38d401b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.220+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f38cc05fb50 con 0x7f38e40730b0 2024-09-17T05:26:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.392+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38e40700a0 con 0x7f38e40730b0 2024-09-17T05:26:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38daa00640 1 -- 172.21.15.43:0/3024260165 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f38cc05f450 con 0x7f38e40730b0 2024-09-17T05:26:01.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:01.400 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":59,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:25:58.601682+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:55.383804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:55.976123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:26:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 msgr2=0x7f38c0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 0x7f38c0075740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af000 tx=0x7f38d401b000 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 msgr2=0x7f38e40c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e7c0 tx=0x7f38cc00ec80 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 shutdown_connections 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38c0073230 0x7f38c0075740 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38e415fe10 0x7f38e41fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e40730b0 0x7f38e40c2e70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 --2- 172.21.15.43:0/3024260165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38e406fc90 0x7f38e4151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 >> 172.21.15.43:0/3024260165 conn(0x7f38e40b23d0 msgr2=0x7f38e4074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 shutdown_connections 2024-09-17T05:26:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.396+0000 7f38e9c00640 1 -- 172.21.15.43:0/3024260165 wait complete. 2024-09-17T05:26:01.422 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:26:01.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:26:01.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 -- 172.21.15.43:0/3480193627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f892c004d50 msgr2=0x7f892c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 --2- 172.21.15.43:0/3480193627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f892c004d50 0x7f892c005160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c640 tx=0x7f893c01d300 comp rx=0 tx=0).stop 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894a600640 1 -- 172.21.15.43:0/3480193627 reap_dead start 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 -- 172.21.15.43:0/3480193627 shutdown_connections 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 -- 172.21.15.43:0/3480193627 >> 172.21.15.43:0/3480193627 conn(0x7f89440cee80 msgr2=0x7f89440cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 -- 172.21.15.43:0/3480193627 shutdown_connections 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.576+0000 7f894c600640 1 -- 172.21.15.43:0/3480193627 wait complete. 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 Processor -- start 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 -- start start 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 0x7f894415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 0x7f894409fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8944070be0 con 0x7f89441f6b60 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8944070a60 con 0x7f894409f680 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894c600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89440cd7c0 con 0x7f892c004d50 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f8949200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f8949200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39930/0 (socket says 172.21.15.43:39930) 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f8949200640 1 -- 172.21.15.43:0/1638900299 learned_addr learned my addr 172.21.15.43:0/1638900299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f8949200640 1 -- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 msgr2=0x7f894415ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f894a600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 0x7f894409fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.580+0000 7f8949c00640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 0x7f894415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949200640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 0x7f894415ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949200640 1 -- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 msgr2=0x7f894409fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949200640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 0x7f894409fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949200640 1 -- 172.21.15.43:0/1638900299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441e6770 con 0x7f89441f6b60 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f894a600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 0x7f894409fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949c00640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 0x7f894415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:01.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8949200640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8930011b10 tx=0x7f8930009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893001e070 con 0x7f89441f6b60 2024-09-17T05:26:01.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f893000fe20 con 0x7f89441f6b60 2024-09-17T05:26:01.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441e6a50 con 0x7f89441f6b60 2024-09-17T05:26:01.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89300042b0 con 0x7f89441f6b60 2024-09-17T05:26:01.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441e7010 con 0x7f89441f6b60 2024-09-17T05:26:01.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89440ccf50 con 0x7f89441f6b60 2024-09-17T05:26:01.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.584+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8930021070 con 0x7f89441f6b60 2024-09-17T05:26:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.588+0000 7f8941600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 0x7f8924075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.588+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f8930094d40 con 0x7f89441f6b60 2024-09-17T05:26:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.588+0000 7f8949c00640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 0x7f8924075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.588+0000 7f8949c00640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 0x7f8924075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8944071220 tx=0x7f8938005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.592+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f893005f700 con 0x7f89441f6b60 2024-09-17T05:26:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.768+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8944148f90 con 0x7f8924073200 2024-09-17T05:26:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.772+0000 7f8941600640 1 -- 172.21.15.43:0/1638900299 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f8938013bd0 con 0x7f8924073200 2024-09-17T05:26:01.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:01.781 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 msgr2=0x7f8924075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 0x7f8924075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8944071220 tx=0x7f8938005f40 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 msgr2=0x7f89441f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8930011b10 tx=0x7f8930009e50 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 shutdown_connections 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8924073200 0x7f8924075710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894409f680 0x7f894409fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89441f6b60 0x7f89441f6f70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 --2- 172.21.15.43:0/1638900299 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f892c004d50 0x7f894415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 >> 172.21.15.43:0/1638900299 conn(0x7f89440cee80 msgr2=0x7f89441e9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 shutdown_connections 2024-09-17T05:26:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:01.776+0000 7f894c600640 1 -- 172.21.15.43:0/1638900299 wait complete. 2024-09-17T05:26:01.802 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-17T05:25:59.878900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247776,"kb_used_data":6584,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726944,"statfs":{"total":773094113280,"available":772840390656,"internally_reserved":0,"allocated":6742016,"data_stored":3457407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.295408"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635604+0000","last_change":"2024-09-17T05:25:57.631016+0000","last_active":"2024-09-17T05:25:57.635604+0000","last_peered":"2024-09-17T05:25:57.635604+0000","last_clean":"2024-09-17T05:25:57.635604+0000","last_became_active":"2024-09-17T05:25:57.627295+0000","last_became_peered":"2024-09-17T05:25:57.627295+0000","last_unstale":"2024-09-17T05:25:57.635604+0000","last_undegraded":"2024-09-17T05:25:57.635604+0000","last_fullsized":"2024-09-17T05:25:57.635604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:22.011741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633627+0000","last_change":"2024-09-17T05:25:57.623604+0000","last_active":"2024-09-17T05:25:57.633627+0000","last_peered":"2024-09-17T05:25:57.633627+0000","last_clean":"2024-09-17T05:25:57.633627+0000","last_became_active":"2024-09-17T05:25:57.623264+0000","last_became_peered":"2024-09-17T05:25:57.623264+0000","last_unstale":"2024-09-17T05:25:57.633627+0000","last_undegraded":"2024-09-17T05:25:57.633627+0000","last_fullsized":"2024-09-17T05:25:57.633627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:00.291409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611147+0000","last_change":"2024-09-17T05:25:57.631707+0000","last_active":"2024-09-17T05:25:58.611147+0000","last_peered":"2024-09-17T05:25:58.611147+0000","last_clean":"2024-09-17T05:25:58.611147+0000","last_became_active":"2024-09-17T05:25:57.631066+0000","last_became_peered":"2024-09-17T05:25:57.631066+0000","last_unstale":"2024-09-17T05:25:58.611147+0000","last_undegraded":"2024-09-17T05:25:58.611147+0000","last_fullsized":"2024-09-17T05:25:58.611147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:11.450539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638976+0000","last_change":"2024-09-17T05:25:57.625950+0000","last_active":"2024-09-17T05:25:57.638976+0000","last_peered":"2024-09-17T05:25:57.638976+0000","last_clean":"2024-09-17T05:25:57.638976+0000","last_became_active":"2024-09-17T05:25:57.625620+0000","last_became_peered":"2024-09-17T05:25:57.625620+0000","last_unstale":"2024-09-17T05:25:57.638976+0000","last_undegraded":"2024-09-17T05:25:57.638976+0000","last_fullsized":"2024-09-17T05:25:57.638976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:44.634325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639525+0000","last_change":"2024-09-17T05:25:57.636677+0000","last_active":"2024-09-17T05:25:57.639525+0000","last_peered":"2024-09-17T05:25:57.639525+0000","last_clean":"2024-09-17T05:25:57.639525+0000","last_became_active":"2024-09-17T05:25:57.636018+0000","last_became_peered":"2024-09-17T05:25:57.636018+0000","last_unstale":"2024-09-17T05:25:57.639525+0000","last_undegraded":"2024-09-17T05:25:57.639525+0000","last_fullsized":"2024-09-17T05:25:57.639525+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:55.616900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632365+0000","last_change":"2024-09-17T05:25:57.626965+0000","last_active":"2024-09-17T05:25:57.632365+0000","last_peered":"2024-09-17T05:25:57.632365+0000","last_clean":"2024-09-17T05:25:57.632365+0000","last_became_active":"2024-09-17T05:25:57.626268+0000","last_became_peered":"2024-09-17T05:25:57.626268+0000","last_unstale":"2024-09-17T05:25:57.632365+0000","last_undegraded":"2024-09-17T05:25:57.632365+0000","last_fullsized":"2024-09-17T05:25:57.632365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:31.263493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638603+0000","last_change":"2024-09-17T05:25:57.633222+0000","last_active":"2024-09-17T05:25:57.638603+0000","last_peered":"2024-09-17T05:25:57.638603+0000","last_clean":"2024-09-17T05:25:57.638603+0000","last_became_active":"2024-09-17T05:25:57.632854+0000","last_became_peered":"2024-09-17T05:25:57.632854+0000","last_unstale":"2024-09-17T05:25:57.638603+0000","last_undegraded":"2024-09-17T05:25:57.638603+0000","last_fullsized":"2024-09-17T05:25:57.638603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:57.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625974+0000","last_change":"2024-09-17T05:25:57.620215+0000","last_active":"2024-09-17T05:25:57.625974+0000","last_peered":"2024-09-17T05:25:57.625974+0000","last_clean":"2024-09-17T05:25:57.625974+0000","last_became_active":"2024-09-17T05:25:57.616145+0000","last_became_peered":"2024-09-17T05:25:57.616145+0000","last_unstale":"2024-09-17T05:25:57.625974+0000","last_undegraded":"2024-09-17T05:25:57.625974+0000","last_fullsized":"2024-09-17T05:25:57.625974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:25.171418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630959+0000","last_change":"2024-09-17T05:25:57.622118+0000","last_active":"2024-09-17T05:25:57.630959+0000","last_peered":"2024-09-17T05:25:57.630959+0000","last_clean":"2024-09-17T05:25:57.630959+0000","last_became_active":"2024-09-17T05:25:57.621310+0000","last_became_peered":"2024-09-17T05:25:57.621310+0000","last_unstale":"2024-09-17T05:25:57.630959+0000","last_undegraded":"2024-09-17T05:25:57.630959+0000","last_fullsized":"2024-09-17T05:25:57.630959+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:14.953015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620581+0000","last_change":"2024-09-17T05:25:57.615977+0000","last_active":"2024-09-17T05:25:57.620581+0000","last_peered":"2024-09-17T05:25:57.620581+0000","last_clean":"2024-09-17T05:25:57.620581+0000","last_became_active":"2024-09-17T05:25:57.615669+0000","last_became_peered":"2024-09-17T05:25:57.615669+0000","last_unstale":"2024-09-17T05:25:57.620581+0000","last_undegraded":"2024-09-17T05:25:57.620581+0000","last_fullsized":"2024-09-17T05:25:57.620581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:59.852702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626925+0000","last_change":"2024-09-17T05:25:57.621394+0000","last_active":"2024-09-17T05:25:57.626925+0000","last_peered":"2024-09-17T05:25:57.626925+0000","last_clean":"2024-09-17T05:25:57.626925+0000","last_became_active":"2024-09-17T05:25:57.620144+0000","last_became_peered":"2024-09-17T05:25:57.620144+0000","last_unstale":"2024-09-17T05:25:57.626925+0000","last_undegraded":"2024-09-17T05:25:57.626925+0000","last_fullsized":"2024-09-17T05:25:57.626925+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:25.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620544+0000","last_change":"2024-09-17T05:25:57.611565+0000","last_active":"2024-09-17T05:25:57.620544+0000","last_peered":"2024-09-17T05:25:57.620544+0000","last_clean":"2024-09-17T05:25:57.620544+0000","last_became_active":"2024-09-17T05:25:57.611202+0000","last_became_peered":"2024-09-17T05:25:57.611202+0000","last_unstale":"2024-09-17T05:25:57.620544+0000","last_undegraded":"2024-09-17T05:25:57.620544+0000","last_fullsized":"2024-09-17T05:25:57.620544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:07.436706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609911+0000","last_change":"2024-09-17T05:25:57.614032+0000","last_active":"2024-09-17T05:25:58.609911+0000","last_peered":"2024-09-17T05:25:58.609911+0000","last_clean":"2024-09-17T05:25:58.609911+0000","last_became_active":"2024-09-17T05:25:57.612857+0000","last_became_peered":"2024-09-17T05:25:57.612857+0000","last_unstale":"2024-09-17T05:25:58.609911+0000","last_undegraded":"2024-09-17T05:25:58.609911+0000","last_fullsized":"2024-09-17T05:25:58.609911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:30.858228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609971+0000","last_change":"2024-09-17T05:25:57.619743+0000","last_active":"2024-09-17T05:25:58.609971+0000","last_peered":"2024-09-17T05:25:58.609971+0000","last_clean":"2024-09-17T05:25:58.609971+0000","last_became_active":"2024-09-17T05:25:57.619444+0000","last_became_peered":"2024-09-17T05:25:57.619444+0000","last_unstale":"2024-09-17T05:25:58.609971+0000","last_undegraded":"2024-09-17T05:25:58.609971+0000","last_fullsized":"2024-09-17T05:25:58.609971+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.664202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609353+0000","last_change":"2024-09-17T05:25:57.618322+0000","last_active":"2024-09-17T05:25:58.609353+0000","last_peered":"2024-09-17T05:25:58.609353+0000","last_clean":"2024-09-17T05:25:58.609353+0000","last_became_active":"2024-09-17T05:25:57.618005+0000","last_became_peered":"2024-09-17T05:25:57.618005+0000","last_unstale":"2024-09-17T05:25:58.609353+0000","last_undegraded":"2024-09-17T05:25:58.609353+0000","last_fullsized":"2024-09-17T05:25:58.609353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:40.510229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.629375+0000","last_change":"2024-09-17T05:25:57.621062+0000","last_active":"2024-09-17T05:25:57.629375+0000","last_peered":"2024-09-17T05:25:57.629375+0000","last_clean":"2024-09-17T05:25:57.629375+0000","last_became_active":"2024-09-17T05:25:57.620738+0000","last_became_peered":"2024-09-17T05:25:57.620738+0000","last_unstale":"2024-09-17T05:25:57.629375+0000","last_undegraded":"2024-09-17T05:25:57.629375+0000","last_fullsized":"2024-09-17T05:25:57.629375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:21.005563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636295+0000","last_change":"2024-09-17T05:25:57.630743+0000","last_active":"2024-09-17T05:25:57.636295+0000","last_peered":"2024-09-17T05:25:57.636295+0000","last_clean":"2024-09-17T05:25:57.636295+0000","last_became_active":"2024-09-17T05:25:57.630402+0000","last_became_peered":"2024-09-17T05:25:57.630402+0000","last_unstale":"2024-09-17T05:25:57.636295+0000","last_undegraded":"2024-09-17T05:25:57.636295+0000","last_fullsized":"2024-09-17T05:25:57.636295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.191803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611142+0000","last_change":"2024-09-17T05:25:57.628918+0000","last_active":"2024-09-17T05:25:58.611142+0000","last_peered":"2024-09-17T05:25:58.611142+0000","last_clean":"2024-09-17T05:25:58.611142+0000","last_became_active":"2024-09-17T05:25:57.627711+0000","last_became_peered":"2024-09-17T05:25:57.627711+0000","last_unstale":"2024-09-17T05:25:58.611142+0000","last_undegraded":"2024-09-17T05:25:58.611142+0000","last_fullsized":"2024-09-17T05:25:58.611142+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:36.945516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633911+0000","last_change":"2024-09-17T05:25:57.625475+0000","last_active":"2024-09-17T05:25:57.633911+0000","last_peered":"2024-09-17T05:25:57.633911+0000","last_clean":"2024-09-17T05:25:57.633911+0000","last_became_active":"2024-09-17T05:25:57.625068+0000","last_became_peered":"2024-09-17T05:25:57.625068+0000","last_unstale":"2024-09-17T05:25:57.633911+0000","last_undegraded":"2024-09-17T05:25:57.633911+0000","last_fullsized":"2024-09-17T05:25:57.633911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:56.117332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626881+0000","last_change":"2024-09-17T05:25:57.619473+0000","last_active":"2024-09-17T05:25:57.626881+0000","last_peered":"2024-09-17T05:25:57.626881+0000","last_clean":"2024-09-17T05:25:57.626881+0000","last_became_active":"2024-09-17T05:25:57.619214+0000","last_became_peered":"2024-09-17T05:25:57.619214+0000","last_unstale":"2024-09-17T05:25:57.626881+0000","last_undegraded":"2024-09-17T05:25:57.626881+0000","last_fullsized":"2024-09-17T05:25:57.626881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:54.995659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610026+0000","last_change":"2024-09-17T05:25:57.609437+0000","last_active":"2024-09-17T05:25:58.610026+0000","last_peered":"2024-09-17T05:25:58.610026+0000","last_clean":"2024-09-17T05:25:58.610026+0000","last_became_active":"2024-09-17T05:25:57.609065+0000","last_became_peered":"2024-09-17T05:25:57.609065+0000","last_unstale":"2024-09-17T05:25:58.610026+0000","last_undegraded":"2024-09-17T05:25:58.610026+0000","last_fullsized":"2024-09-17T05:25:58.610026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:12.203019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626675+0000","last_change":"2024-09-17T05:25:57.614217+0000","last_active":"2024-09-17T05:25:57.626675+0000","last_peered":"2024-09-17T05:25:57.626675+0000","last_clean":"2024-09-17T05:25:57.626675+0000","last_became_active":"2024-09-17T05:25:57.613860+0000","last_became_peered":"2024-09-17T05:25:57.613860+0000","last_unstale":"2024-09-17T05:25:57.626675+0000","last_undegraded":"2024-09-17T05:25:57.626675+0000","last_fullsized":"2024-09-17T05:25:57.626675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:57.659052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.641900+0000","last_change":"2024-09-17T05:25:57.638376+0000","last_active":"2024-09-17T05:25:57.641900+0000","last_peered":"2024-09-17T05:25:57.641900+0000","last_clean":"2024-09-17T05:25:57.641900+0000","last_became_active":"2024-09-17T05:25:57.638064+0000","last_became_peered":"2024-09-17T05:25:57.638064+0000","last_unstale":"2024-09-17T05:25:57.641900+0000","last_undegraded":"2024-09-17T05:25:57.641900+0000","last_fullsized":"2024-09-17T05:25:57.641900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:16.470698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630529+0000","last_change":"2024-09-17T05:25:57.618378+0000","last_active":"2024-09-17T05:25:57.630529+0000","last_peered":"2024-09-17T05:25:57.630529+0000","last_clean":"2024-09-17T05:25:57.630529+0000","last_became_active":"2024-09-17T05:25:57.617935+0000","last_became_peered":"2024-09-17T05:25:57.617935+0000","last_unstale":"2024-09-17T05:25:57.630529+0000","last_undegraded":"2024-09-17T05:25:57.630529+0000","last_fullsized":"2024-09-17T05:25:57.630529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633216+0000","last_change":"2024-09-17T05:25:57.622847+0000","last_active":"2024-09-17T05:25:57.633216+0000","last_peered":"2024-09-17T05:25:57.633216+0000","last_clean":"2024-09-17T05:25:57.633216+0000","last_became_active":"2024-09-17T05:25:57.622508+0000","last_became_peered":"2024-09-17T05:25:57.622508+0000","last_unstale":"2024-09-17T05:25:57.633216+0000","last_undegraded":"2024-09-17T05:25:57.633216+0000","last_fullsized":"2024-09-17T05:25:57.633216+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:26.833060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610442+0000","last_change":"2024-09-17T05:25:57.622610+0000","last_active":"2024-09-17T05:25:58.610442+0000","last_peered":"2024-09-17T05:25:58.610442+0000","last_clean":"2024-09-17T05:25:58.610442+0000","last_became_active":"2024-09-17T05:25:57.622274+0000","last_became_peered":"2024-09-17T05:25:57.622274+0000","last_unstale":"2024-09-17T05:25:58.610442+0000","last_undegraded":"2024-09-17T05:25:58.610442+0000","last_fullsized":"2024-09-17T05:25:58.610442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:06.852378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633822+0000","last_change":"2024-09-17T05:25:57.624501+0000","last_active":"2024-09-17T05:25:57.633822+0000","last_peered":"2024-09-17T05:25:57.633822+0000","last_clean":"2024-09-17T05:25:57.633822+0000","last_became_active":"2024-09-17T05:25:57.624158+0000","last_became_peered":"2024-09-17T05:25:57.624158+0000","last_unstale":"2024-09-17T05:25:57.633822+0000","last_undegraded":"2024-09-17T05:25:57.633822+0000","last_fullsized":"2024-09-17T05:25:57.633822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:32.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625249+0000","last_change":"2024-09-17T05:25:57.616712+0000","last_active":"2024-09-17T05:25:57.625249+0000","last_peered":"2024-09-17T05:25:57.625249+0000","last_clean":"2024-09-17T05:25:57.625249+0000","last_became_active":"2024-09-17T05:25:57.616385+0000","last_became_peered":"2024-09-17T05:25:57.616385+0000","last_unstale":"2024-09-17T05:25:57.625249+0000","last_undegraded":"2024-09-17T05:25:57.625249+0000","last_fullsized":"2024-09-17T05:25:57.625249+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:33.942936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.627648+0000","last_change":"2024-09-17T05:25:57.622667+0000","last_active":"2024-09-17T05:25:57.627648+0000","last_peered":"2024-09-17T05:25:57.627648+0000","last_clean":"2024-09-17T05:25:57.627648+0000","last_became_active":"2024-09-17T05:25:57.622137+0000","last_became_peered":"2024-09-17T05:25:57.622137+0000","last_unstale":"2024-09-17T05:25:57.627648+0000","last_undegraded":"2024-09-17T05:25:57.627648+0000","last_fullsized":"2024-09-17T05:25:57.627648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:02.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.640261+0000","last_change":"2024-09-17T05:25:57.632398+0000","last_active":"2024-09-17T05:25:57.640261+0000","last_peered":"2024-09-17T05:25:57.640261+0000","last_clean":"2024-09-17T05:25:57.640261+0000","last_became_active":"2024-09-17T05:25:57.631992+0000","last_became_peered":"2024-09-17T05:25:57.631992+0000","last_unstale":"2024-09-17T05:25:57.640261+0000","last_undegraded":"2024-09-17T05:25:57.640261+0000","last_fullsized":"2024-09-17T05:25:57.640261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:41.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634331+0000","last_change":"2024-09-17T05:25:57.629493+0000","last_active":"2024-09-17T05:25:57.634331+0000","last_peered":"2024-09-17T05:25:57.634331+0000","last_clean":"2024-09-17T05:25:57.634331+0000","last_became_active":"2024-09-17T05:25:57.624157+0000","last_became_peered":"2024-09-17T05:25:57.624157+0000","last_unstale":"2024-09-17T05:25:57.634331+0000","last_undegraded":"2024-09-17T05:25:57.634331+0000","last_fullsized":"2024-09-17T05:25:57.634331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:50.170213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626934+0000","last_change":"2024-09-17T05:25:57.618377+0000","last_active":"2024-09-17T05:25:57.626934+0000","last_peered":"2024-09-17T05:25:57.626934+0000","last_clean":"2024-09-17T05:25:57.626934+0000","last_became_active":"2024-09-17T05:25:57.615243+0000","last_became_peered":"2024-09-17T05:25:57.615243+0000","last_unstale":"2024-09-17T05:25:57.626934+0000","last_undegraded":"2024-09-17T05:25:57.626934+0000","last_fullsized":"2024-09-17T05:25:57.626934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:08.640363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620614+0000","last_change":"2024-09-17T05:25:57.617672+0000","last_active":"2024-09-17T05:25:57.620614+0000","last_peered":"2024-09-17T05:25:57.620614+0000","last_clean":"2024-09-17T05:25:57.620614+0000","last_became_active":"2024-09-17T05:25:57.617221+0000","last_became_peered":"2024-09-17T05:25:57.617221+0000","last_unstale":"2024-09-17T05:25:57.620614+0000","last_undegraded":"2024-09-17T05:25:57.620614+0000","last_fullsized":"2024-09-17T05:25:57.620614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:13.140113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.637380+0000","last_change":"2024-09-17T05:25:56.602138+0000","last_active":"2024-09-17T05:25:57.637380+0000","last_peered":"2024-09-17T05:25:57.637380+0000","last_clean":"2024-09-17T05:25:57.637380+0000","last_became_active":"2024-09-17T05:25:56.601770+0000","last_became_peered":"2024-09-17T05:25:56.601770+0000","last_unstale":"2024-09-17T05:25:57.637380+0000","last_undegraded":"2024-09-17T05:25:57.637380+0000","last_fullsized":"2024-09-17T05:25:57.637380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:34.638935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638893+0000","last_change":"2024-09-17T05:25:57.636046+0000","last_active":"2024-09-17T05:25:57.638893+0000","last_peered":"2024-09-17T05:25:57.638893+0000","last_clean":"2024-09-17T05:25:57.638893+0000","last_became_active":"2024-09-17T05:25:57.635614+0000","last_became_peered":"2024-09-17T05:25:57.635614+0000","last_unstale":"2024-09-17T05:25:57.638893+0000","last_undegraded":"2024-09-17T05:25:57.638893+0000","last_fullsized":"2024-09-17T05:25:57.638893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:42.351638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632973+0000","last_change":"2024-09-17T05:25:56.620724+0000","last_active":"2024-09-17T05:25:57.632973+0000","last_peered":"2024-09-17T05:25:57.632973+0000","last_clean":"2024-09-17T05:25:57.632973+0000","last_became_active":"2024-09-17T05:25:56.619901+0000","last_became_peered":"2024-09-17T05:25:56.619901+0000","last_unstale":"2024-09-17T05:25:57.632973+0000","last_undegraded":"2024-09-17T05:25:57.632973+0000","last_fullsized":"2024-09-17T05:25:57.632973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:50.563424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625308+0000","last_change":"2024-09-17T05:25:57.618668+0000","last_active":"2024-09-17T05:25:57.625308+0000","last_peered":"2024-09-17T05:25:57.625308+0000","last_clean":"2024-09-17T05:25:57.625308+0000","last_became_active":"2024-09-17T05:25:57.617954+0000","last_became_peered":"2024-09-17T05:25:57.617954+0000","last_unstale":"2024-09-17T05:25:57.625308+0000","last_undegraded":"2024-09-17T05:25:57.625308+0000","last_fullsized":"2024-09-17T05:25:57.625308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:55.254342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636015+0000","last_change":"2024-09-17T05:25:56.620360+0000","last_active":"2024-09-17T05:25:57.636015+0000","last_peered":"2024-09-17T05:25:57.636015+0000","last_clean":"2024-09-17T05:25:57.636015+0000","last_became_active":"2024-09-17T05:25:56.619964+0000","last_became_peered":"2024-09-17T05:25:56.619964+0000","last_unstale":"2024-09-17T05:25:57.636015+0000","last_undegraded":"2024-09-17T05:25:57.636015+0000","last_fullsized":"2024-09-17T05:25:57.636015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:20.504533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.629902+0000","last_change":"2024-09-17T05:25:57.624345+0000","last_active":"2024-09-17T05:25:57.629902+0000","last_peered":"2024-09-17T05:25:57.629902+0000","last_clean":"2024-09-17T05:25:57.629902+0000","last_became_active":"2024-09-17T05:25:57.623583+0000","last_became_peered":"2024-09-17T05:25:57.623583+0000","last_unstale":"2024-09-17T05:25:57.629902+0000","last_undegraded":"2024-09-17T05:25:57.629902+0000","last_fullsized":"2024-09-17T05:25:57.629902+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:31.160498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642788+0000","last_change":"2024-09-17T05:25:56.625871+0000","last_active":"2024-09-17T05:25:57.642788+0000","last_peered":"2024-09-17T05:25:57.642788+0000","last_clean":"2024-09-17T05:25:57.642788+0000","last_became_active":"2024-09-17T05:25:56.625569+0000","last_became_peered":"2024-09-17T05:25:56.625569+0000","last_unstale":"2024-09-17T05:25:57.642788+0000","last_undegraded":"2024-09-17T05:25:57.642788+0000","last_fullsized":"2024-09-17T05:25:57.642788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:12.956313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633671+0000","last_change":"2024-09-17T05:25:57.627051+0000","last_active":"2024-09-17T05:25:57.633671+0000","last_peered":"2024-09-17T05:25:57.633671+0000","last_clean":"2024-09-17T05:25:57.633671+0000","last_became_active":"2024-09-17T05:25:57.626651+0000","last_became_peered":"2024-09-17T05:25:57.626651+0000","last_unstale":"2024-09-17T05:25:57.633671+0000","last_undegraded":"2024-09-17T05:25:57.633671+0000","last_fullsized":"2024-09-17T05:25:57.633671+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:01.557423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610766+0000","last_change":"2024-09-17T05:25:56.615735+0000","last_active":"2024-09-17T05:25:58.610766+0000","last_peered":"2024-09-17T05:25:58.610766+0000","last_clean":"2024-09-17T05:25:58.610766+0000","last_became_active":"2024-09-17T05:25:56.615440+0000","last_became_peered":"2024-09-17T05:25:56.615440+0000","last_unstale":"2024-09-17T05:25:58.610766+0000","last_undegraded":"2024-09-17T05:25:58.610766+0000","last_fullsized":"2024-09-17T05:25:58.610766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.890221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626217+0000","last_change":"2024-09-17T05:25:57.614450+0000","last_active":"2024-09-17T05:25:57.626217+0000","last_peered":"2024-09-17T05:25:57.626217+0000","last_clean":"2024-09-17T05:25:57.626217+0000","last_became_active":"2024-09-17T05:25:57.614111+0000","last_became_peered":"2024-09-17T05:25:57.614111+0000","last_unstale":"2024-09-17T05:25:57.626217+0000","last_undegraded":"2024-09-17T05:25:57.626217+0000","last_fullsized":"2024-09-17T05:25:57.626217+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:08.532142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642238+0000","last_change":"2024-09-17T05:25:56.620062+0000","last_active":"2024-09-17T05:25:57.642238+0000","last_peered":"2024-09-17T05:25:57.642238+0000","last_clean":"2024-09-17T05:25:57.642238+0000","last_became_active":"2024-09-17T05:25:56.619711+0000","last_became_peered":"2024-09-17T05:25:56.619711+0000","last_unstale":"2024-09-17T05:25:57.642238+0000","last_undegraded":"2024-09-17T05:25:57.642238+0000","last_fullsized":"2024-09-17T05:25:57.642238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:40.597798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609807+0000","last_change":"2024-09-17T05:25:57.636411+0000","last_active":"2024-09-17T05:25:58.609807+0000","last_peered":"2024-09-17T05:25:58.609807+0000","last_clean":"2024-09-17T05:25:58.609807+0000","last_became_active":"2024-09-17T05:25:57.636080+0000","last_became_peered":"2024-09-17T05:25:57.636080+0000","last_unstale":"2024-09-17T05:25:58.609807+0000","last_undegraded":"2024-09-17T05:25:58.609807+0000","last_fullsized":"2024-09-17T05:25:58.609807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:25.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.600496+0000","last_change":"2024-09-17T05:25:56.600998+0000","last_active":"2024-09-17T05:25:57.600496+0000","last_peered":"2024-09-17T05:25:57.600496+0000","last_clean":"2024-09-17T05:25:57.600496+0000","last_became_active":"2024-09-17T05:25:56.600504+0000","last_became_peered":"2024-09-17T05:25:56.600504+0000","last_unstale":"2024-09-17T05:25:57.600496+0000","last_undegraded":"2024-09-17T05:25:57.600496+0000","last_fullsized":"2024-09-17T05:25:57.600496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:52.590333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635426+0000","last_change":"2024-09-17T05:25:57.630161+0000","last_active":"2024-09-17T05:25:57.635426+0000","last_peered":"2024-09-17T05:25:57.635426+0000","last_clean":"2024-09-17T05:25:57.635426+0000","last_became_active":"2024-09-17T05:25:57.629719+0000","last_became_peered":"2024-09-17T05:25:57.629719+0000","last_unstale":"2024-09-17T05:25:57.635426+0000","last_undegraded":"2024-09-17T05:25:57.635426+0000","last_fullsized":"2024-09-17T05:25:57.635426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:30.192084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609243+0000","last_change":"2024-09-17T05:25:56.605033+0000","last_active":"2024-09-17T05:25:58.609243+0000","last_peered":"2024-09-17T05:25:58.609243+0000","last_clean":"2024-09-17T05:25:58.609243+0000","last_became_active":"2024-09-17T05:25:56.604728+0000","last_became_peered":"2024-09-17T05:25:56.604728+0000","last_unstale":"2024-09-17T05:25:58.609243+0000","last_undegraded":"2024-09-17T05:25:58.609243+0000","last_fullsized":"2024-09-17T05:25:58.609243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:48.703259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610520+0000","last_change":"2024-09-17T05:25:57.618887+0000","last_active":"2024-09-17T05:25:58.610520+0000","last_peered":"2024-09-17T05:25:58.610520+0000","last_clean":"2024-09-17T05:25:58.610520+0000","last_became_active":"2024-09-17T05:25:57.618549+0000","last_became_peered":"2024-09-17T05:25:57.618549+0000","last_unstale":"2024-09-17T05:25:58.610520+0000","last_undegraded":"2024-09-17T05:25:58.610520+0000","last_fullsized":"2024-09-17T05:25:58.610520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:11.450539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601284+0000","last_change":"2024-09-17T05:25:56.612951+0000","last_active":"2024-09-17T05:25:57.601284+0000","last_peered":"2024-09-17T05:25:57.601284+0000","last_clean":"2024-09-17T05:25:57.601284+0000","last_became_active":"2024-09-17T05:25:56.612540+0000","last_became_peered":"2024-09-17T05:25:56.612540+0000","last_unstale":"2024-09-17T05:25:57.601284+0000","last_undegraded":"2024-09-17T05:25:57.601284+0000","last_fullsized":"2024-09-17T05:25:57.601284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:37.040947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611361+0000","last_change":"2024-09-17T05:25:57.628860+0000","last_active":"2024-09-17T05:25:58.611361+0000","last_peered":"2024-09-17T05:25:58.611361+0000","last_clean":"2024-09-17T05:25:58.611361+0000","last_became_active":"2024-09-17T05:25:57.628517+0000","last_became_peered":"2024-09-17T05:25:57.628517+0000","last_unstale":"2024-09-17T05:25:58.611361+0000","last_undegraded":"2024-09-17T05:25:58.611361+0000","last_fullsized":"2024-09-17T05:25:58.611361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:31.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.605250+0000","last_change":"2024-09-17T05:25:56.620974+0000","last_active":"2024-09-17T05:25:57.605250+0000","last_peered":"2024-09-17T05:25:57.605250+0000","last_clean":"2024-09-17T05:25:57.605250+0000","last_became_active":"2024-09-17T05:25:56.618268+0000","last_became_peered":"2024-09-17T05:25:56.618268+0000","last_unstale":"2024-09-17T05:25:57.605250+0000","last_undegraded":"2024-09-17T05:25:57.605250+0000","last_fullsized":"2024-09-17T05:25:57.605250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.860676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625281+0000","last_change":"2024-09-17T05:25:57.618334+0000","last_active":"2024-09-17T05:25:57.625281+0000","last_peered":"2024-09-17T05:25:57.625281+0000","last_clean":"2024-09-17T05:25:57.625281+0000","last_became_active":"2024-09-17T05:25:57.617488+0000","last_became_peered":"2024-09-17T05:25:57.617488+0000","last_unstale":"2024-09-17T05:25:57.625281+0000","last_undegraded":"2024-09-17T05:25:57.625281+0000","last_fullsized":"2024-09-17T05:25:57.625281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:02.356252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642900+0000","last_change":"2024-09-17T05:25:56.623672+0000","last_active":"2024-09-17T05:25:57.642900+0000","last_peered":"2024-09-17T05:25:57.642900+0000","last_clean":"2024-09-17T05:25:57.642900+0000","last_became_active":"2024-09-17T05:25:56.623101+0000","last_became_peered":"2024-09-17T05:25:56.623101+0000","last_unstale":"2024-09-17T05:25:57.642900+0000","last_undegraded":"2024-09-17T05:25:57.642900+0000","last_fullsized":"2024-09-17T05:25:57.642900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:19.717809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639462+0000","last_change":"2024-09-17T05:25:57.621965+0000","last_active":"2024-09-17T05:25:57.639462+0000","last_peered":"2024-09-17T05:25:57.639462+0000","last_clean":"2024-09-17T05:25:57.639462+0000","last_became_active":"2024-09-17T05:25:57.621619+0000","last_became_peered":"2024-09-17T05:25:57.621619+0000","last_unstale":"2024-09-17T05:25:57.639462+0000","last_undegraded":"2024-09-17T05:25:57.639462+0000","last_fullsized":"2024-09-17T05:25:57.639462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:46.532448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610189+0000","last_change":"2024-09-17T05:25:56.609748+0000","last_active":"2024-09-17T05:25:58.610189+0000","last_peered":"2024-09-17T05:25:58.610189+0000","last_clean":"2024-09-17T05:25:58.610189+0000","last_became_active":"2024-09-17T05:25:56.609389+0000","last_became_peered":"2024-09-17T05:25:56.609389+0000","last_unstale":"2024-09-17T05:25:58.610189+0000","last_undegraded":"2024-09-17T05:25:58.610189+0000","last_fullsized":"2024-09-17T05:25:58.610189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:01.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.624428+0000","last_change":"2024-09-17T05:25:57.614074+0000","last_active":"2024-09-17T05:25:57.624428+0000","last_peered":"2024-09-17T05:25:57.624428+0000","last_clean":"2024-09-17T05:25:57.624428+0000","last_became_active":"2024-09-17T05:25:57.613697+0000","last_became_peered":"2024-09-17T05:25:57.613697+0000","last_unstale":"2024-09-17T05:25:57.624428+0000","last_undegraded":"2024-09-17T05:25:57.624428+0000","last_fullsized":"2024-09-17T05:25:57.624428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:28.519789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639596+0000","last_change":"2024-09-17T05:25:56.615127+0000","last_active":"2024-09-17T05:25:57.639596+0000","last_peered":"2024-09-17T05:25:57.639596+0000","last_clean":"2024-09-17T05:25:57.639596+0000","last_became_active":"2024-09-17T05:25:56.614790+0000","last_became_peered":"2024-09-17T05:25:56.614790+0000","last_unstale":"2024-09-17T05:25:57.639596+0000","last_undegraded":"2024-09-17T05:25:57.639596+0000","last_fullsized":"2024-09-17T05:25:57.639596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:17.126953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610817+0000","last_change":"2024-09-17T05:25:57.618971+0000","last_active":"2024-09-17T05:25:58.610817+0000","last_peered":"2024-09-17T05:25:58.610817+0000","last_clean":"2024-09-17T05:25:58.610817+0000","last_became_active":"2024-09-17T05:25:57.618673+0000","last_became_peered":"2024-09-17T05:25:57.618673+0000","last_unstale":"2024-09-17T05:25:58.610817+0000","last_undegraded":"2024-09-17T05:25:58.610817+0000","last_fullsized":"2024-09-17T05:25:58.610817+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:03.730384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.614568+0000","last_change":"2024-09-17T05:25:56.613747+0000","last_active":"2024-09-17T05:25:57.614568+0000","last_peered":"2024-09-17T05:25:57.614568+0000","last_clean":"2024-09-17T05:25:57.614568+0000","last_became_active":"2024-09-17T05:25:56.613407+0000","last_became_peered":"2024-09-17T05:25:56.613407+0000","last_unstale":"2024-09-17T05:25:57.614568+0000","last_undegraded":"2024-09-17T05:25:57.614568+0000","last_fullsized":"2024-09-17T05:25:57.614568+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:09.169185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.618240+0000","last_change":"2024-09-17T05:25:57.610448+0000","last_active":"2024-09-17T05:25:57.618240+0000","last_peered":"2024-09-17T05:25:57.618240+0000","last_clean":"2024-09-17T05:25:57.618240+0000","last_became_active":"2024-09-17T05:25:57.610076+0000","last_became_peered":"2024-09-17T05:25:57.610076+0000","last_unstale":"2024-09-17T05:25:57.618240+0000","last_undegraded":"2024-09-17T05:25:57.618240+0000","last_fullsized":"2024-09-17T05:25:57.618240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:47.049648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.612535+0000","last_change":"2024-09-17T05:25:56.622751+0000","last_active":"2024-09-17T05:25:57.612535+0000","last_peered":"2024-09-17T05:25:57.612535+0000","last_clean":"2024-09-17T05:25:57.612535+0000","last_became_active":"2024-09-17T05:25:56.620388+0000","last_became_peered":"2024-09-17T05:25:56.620388+0000","last_unstale":"2024-09-17T05:25:57.612535+0000","last_undegraded":"2024-09-17T05:25:57.612535+0000","last_fullsized":"2024-09-17T05:25:57.612535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:12.620914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.641870+0000","last_change":"2024-09-17T05:25:57.635094+0000","last_active":"2024-09-17T05:25:57.641870+0000","last_peered":"2024-09-17T05:25:57.641870+0000","last_clean":"2024-09-17T05:25:57.641870+0000","last_became_active":"2024-09-17T05:25:57.634771+0000","last_became_peered":"2024-09-17T05:25:57.634771+0000","last_unstale":"2024-09-17T05:25:57.641870+0000","last_undegraded":"2024-09-17T05:25:57.641870+0000","last_fullsized":"2024-09-17T05:25:57.641870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:32.191787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642416+0000","last_change":"2024-09-17T05:25:56.619402+0000","last_active":"2024-09-17T05:25:57.642416+0000","last_peered":"2024-09-17T05:25:57.642416+0000","last_clean":"2024-09-17T05:25:57.642416+0000","last_became_active":"2024-09-17T05:25:56.619122+0000","last_became_peered":"2024-09-17T05:25:56.619122+0000","last_unstale":"2024-09-17T05:25:57.642416+0000","last_undegraded":"2024-09-17T05:25:57.642416+0000","last_fullsized":"2024-09-17T05:25:57.642416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:00.554948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611113+0000","last_change":"2024-09-17T05:25:57.630886+0000","last_active":"2024-09-17T05:25:58.611113+0000","last_peered":"2024-09-17T05:25:58.611113+0000","last_clean":"2024-09-17T05:25:58.611113+0000","last_became_active":"2024-09-17T05:25:57.630442+0000","last_became_peered":"2024-09-17T05:25:57.630442+0000","last_unstale":"2024-09-17T05:25:58.611113+0000","last_undegraded":"2024-09-17T05:25:58.611113+0000","last_fullsized":"2024-09-17T05:25:58.611113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:39.972695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636267+0000","last_change":"2024-09-17T05:25:56.624991+0000","last_active":"2024-09-17T05:25:57.636267+0000","last_peered":"2024-09-17T05:25:57.636267+0000","last_clean":"2024-09-17T05:25:57.636267+0000","last_became_active":"2024-09-17T05:25:56.624587+0000","last_became_peered":"2024-09-17T05:25:56.624587+0000","last_unstale":"2024-09-17T05:25:57.636267+0000","last_undegraded":"2024-09-17T05:25:57.636267+0000","last_fullsized":"2024-09-17T05:25:57.636267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:13.760914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611390+0000","last_change":"2024-09-17T05:25:57.627117+0000","last_active":"2024-09-17T05:25:58.611390+0000","last_peered":"2024-09-17T05:25:58.611390+0000","last_clean":"2024-09-17T05:25:58.611390+0000","last_became_active":"2024-09-17T05:25:57.624515+0000","last_became_peered":"2024-09-17T05:25:57.624515+0000","last_unstale":"2024-09-17T05:25:58.611390+0000","last_undegraded":"2024-09-17T05:25:58.611390+0000","last_fullsized":"2024-09-17T05:25:58.611390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:25.415314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.637321+0000","last_change":"2024-09-17T05:25:56.616994+0000","last_active":"2024-09-17T05:25:57.637321+0000","last_peered":"2024-09-17T05:25:57.637321+0000","last_clean":"2024-09-17T05:25:57.637321+0000","last_became_active":"2024-09-17T05:25:56.616603+0000","last_became_peered":"2024-09-17T05:25:56.616603+0000","last_unstale":"2024-09-17T05:25:57.637321+0000","last_undegraded":"2024-09-17T05:25:57.637321+0000","last_fullsized":"2024-09-17T05:25:57.637321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:36.954863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634325+0000","last_change":"2024-09-17T05:25:57.625878+0000","last_active":"2024-09-17T05:25:57.634325+0000","last_peered":"2024-09-17T05:25:57.634325+0000","last_clean":"2024-09-17T05:25:57.634325+0000","last_became_active":"2024-09-17T05:25:57.625479+0000","last_became_peered":"2024-09-17T05:25:57.625479+0000","last_unstale":"2024-09-17T05:25:57.634325+0000","last_undegraded":"2024-09-17T05:25:57.634325+0000","last_fullsized":"2024-09-17T05:25:57.634325+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:20.333175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611106+0000","last_change":"2024-09-17T05:25:56.618618+0000","last_active":"2024-09-17T05:25:58.611106+0000","last_peered":"2024-09-17T05:25:58.611106+0000","last_clean":"2024-09-17T05:25:58.611106+0000","last_became_active":"2024-09-17T05:25:56.617666+0000","last_became_peered":"2024-09-17T05:25:56.617666+0000","last_unstale":"2024-09-17T05:25:58.611106+0000","last_undegraded":"2024-09-17T05:25:58.611106+0000","last_fullsized":"2024-09-17T05:25:58.611106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.058611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639493+0000","last_change":"2024-09-17T05:25:57.626670+0000","last_active":"2024-09-17T05:25:57.639493+0000","last_peered":"2024-09-17T05:25:57.639493+0000","last_clean":"2024-09-17T05:25:57.639493+0000","last_became_active":"2024-09-17T05:25:57.626304+0000","last_became_peered":"2024-09-17T05:25:57.626304+0000","last_unstale":"2024-09-17T05:25:57.639493+0000","last_undegraded":"2024-09-17T05:25:57.639493+0000","last_fullsized":"2024-09-17T05:25:57.639493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:56.242997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604972+0000","last_change":"2024-09-17T05:25:56.612627+0000","last_active":"2024-09-17T05:25:57.604972+0000","last_peered":"2024-09-17T05:25:57.604972+0000","last_clean":"2024-09-17T05:25:57.604972+0000","last_became_active":"2024-09-17T05:25:56.612310+0000","last_became_peered":"2024-09-17T05:25:56.612310+0000","last_unstale":"2024-09-17T05:25:57.604972+0000","last_undegraded":"2024-09-17T05:25:57.604972+0000","last_fullsized":"2024-09-17T05:25:57.604972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:52.779414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630851+0000","last_change":"2024-09-17T05:25:57.624159+0000","last_active":"2024-09-17T05:25:57.630851+0000","last_peered":"2024-09-17T05:25:57.630851+0000","last_clean":"2024-09-17T05:25:57.630851+0000","last_became_active":"2024-09-17T05:25:57.621139+0000","last_became_peered":"2024-09-17T05:25:57.621139+0000","last_unstale":"2024-09-17T05:25:57.630851+0000","last_undegraded":"2024-09-17T05:25:57.630851+0000","last_fullsized":"2024-09-17T05:25:57.630851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:13.198875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636965+0000","last_change":"2024-09-17T05:25:56.612418+0000","last_active":"2024-09-17T05:25:57.636965+0000","last_peered":"2024-09-17T05:25:57.636965+0000","last_clean":"2024-09-17T05:25:57.636965+0000","last_became_active":"2024-09-17T05:25:56.612087+0000","last_became_peered":"2024-09-17T05:25:56.612087+0000","last_unstale":"2024-09-17T05:25:57.636965+0000","last_undegraded":"2024-09-17T05:25:57.636965+0000","last_fullsized":"2024-09-17T05:25:57.636965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:05.209454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.627680+0000","last_change":"2024-09-17T05:25:57.623746+0000","last_active":"2024-09-17T05:25:57.627680+0000","last_peered":"2024-09-17T05:25:57.627680+0000","last_clean":"2024-09-17T05:25:57.627680+0000","last_became_active":"2024-09-17T05:25:57.623459+0000","last_became_peered":"2024-09-17T05:25:57.623459+0000","last_unstale":"2024-09-17T05:25:57.627680+0000","last_undegraded":"2024-09-17T05:25:57.627680+0000","last_fullsized":"2024-09-17T05:25:57.627680+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:06.593568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604691+0000","last_change":"2024-09-17T05:25:56.618346+0000","last_active":"2024-09-17T05:25:57.604691+0000","last_peered":"2024-09-17T05:25:57.604691+0000","last_clean":"2024-09-17T05:25:57.604691+0000","last_became_active":"2024-09-17T05:25:56.617486+0000","last_became_peered":"2024-09-17T05:25:56.617486+0000","last_unstale":"2024-09-17T05:25:57.604691+0000","last_undegraded":"2024-09-17T05:25:57.604691+0000","last_fullsized":"2024-09-17T05:25:57.604691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:05.849904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.622194+0000","last_change":"2024-09-17T05:25:57.616146+0000","last_active":"2024-09-17T05:25:57.622194+0000","last_peered":"2024-09-17T05:25:57.622194+0000","last_clean":"2024-09-17T05:25:57.622194+0000","last_became_active":"2024-09-17T05:25:57.615768+0000","last_became_peered":"2024-09-17T05:25:57.615768+0000","last_unstale":"2024-09-17T05:25:57.622194+0000","last_undegraded":"2024-09-17T05:25:57.622194+0000","last_fullsized":"2024-09-17T05:25:57.622194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:04.473483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601698+0000","last_change":"2024-09-17T05:25:56.619091+0000","last_active":"2024-09-17T05:25:57.601698+0000","last_peered":"2024-09-17T05:25:57.601698+0000","last_clean":"2024-09-17T05:25:57.601698+0000","last_became_active":"2024-09-17T05:25:56.618457+0000","last_became_peered":"2024-09-17T05:25:56.618457+0000","last_unstale":"2024-09-17T05:25:57.601698+0000","last_undegraded":"2024-09-17T05:25:57.601698+0000","last_fullsized":"2024-09-17T05:25:57.601698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:55.581973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611244+0000","last_change":"2024-09-17T05:25:57.628995+0000","last_active":"2024-09-17T05:25:58.611244+0000","last_peered":"2024-09-17T05:25:58.611244+0000","last_clean":"2024-09-17T05:25:58.611244+0000","last_became_active":"2024-09-17T05:25:57.625672+0000","last_became_peered":"2024-09-17T05:25:57.625672+0000","last_unstale":"2024-09-17T05:25:58.611244+0000","last_undegraded":"2024-09-17T05:25:58.611244+0000","last_fullsized":"2024-09-17T05:25:58.611244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:53.781888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635059+0000","last_change":"2024-09-17T05:25:56.622397+0000","last_active":"2024-09-17T05:25:57.635059+0000","last_peered":"2024-09-17T05:25:57.635059+0000","last_clean":"2024-09-17T05:25:57.635059+0000","last_became_active":"2024-09-17T05:25:56.621549+0000","last_became_peered":"2024-09-17T05:25:56.621549+0000","last_unstale":"2024-09-17T05:25:57.635059+0000","last_undegraded":"2024-09-17T05:25:57.635059+0000","last_fullsized":"2024-09-17T05:25:57.635059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:57.433606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611580+0000","last_change":"2024-09-17T05:25:57.641404+0000","last_active":"2024-09-17T05:25:58.611580+0000","last_peered":"2024-09-17T05:25:58.611580+0000","last_clean":"2024-09-17T05:25:58.611580+0000","last_became_active":"2024-09-17T05:25:57.641106+0000","last_became_peered":"2024-09-17T05:25:57.641106+0000","last_unstale":"2024-09-17T05:25:58.611580+0000","last_undegraded":"2024-09-17T05:25:58.611580+0000","last_fullsized":"2024-09-17T05:25:58.611580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:24.314248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.614558+0000","last_change":"2024-09-17T05:25:56.622460+0000","last_active":"2024-09-17T05:25:57.614558+0000","last_peered":"2024-09-17T05:25:57.614558+0000","last_clean":"2024-09-17T05:25:57.614558+0000","last_became_active":"2024-09-17T05:25:56.620828+0000","last_became_peered":"2024-09-17T05:25:56.620828+0000","last_unstale":"2024-09-17T05:25:57.614558+0000","last_undegraded":"2024-09-17T05:25:57.614558+0000","last_fullsized":"2024-09-17T05:25:57.614558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:43.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633080+0000","last_change":"2024-09-17T05:25:57.622620+0000","last_active":"2024-09-17T05:25:57.633080+0000","last_peered":"2024-09-17T05:25:57.633080+0000","last_clean":"2024-09-17T05:25:57.633080+0000","last_became_active":"2024-09-17T05:25:57.622264+0000","last_became_peered":"2024-09-17T05:25:57.622264+0000","last_unstale":"2024-09-17T05:25:57.633080+0000","last_undegraded":"2024-09-17T05:25:57.633080+0000","last_fullsized":"2024-09-17T05:25:57.633080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:15.569272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.600120+0000","last_change":"2024-09-17T05:25:56.612032+0000","last_active":"2024-09-17T05:25:57.600120+0000","last_peered":"2024-09-17T05:25:57.600120+0000","last_clean":"2024-09-17T05:25:57.600120+0000","last_became_active":"2024-09-17T05:25:56.611692+0000","last_became_peered":"2024-09-17T05:25:56.611692+0000","last_unstale":"2024-09-17T05:25:57.600120+0000","last_undegraded":"2024-09-17T05:25:57.600120+0000","last_fullsized":"2024-09-17T05:25:57.600120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:09.864874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626184+0000","last_change":"2024-09-17T05:25:57.618798+0000","last_active":"2024-09-17T05:25:57.626184+0000","last_peered":"2024-09-17T05:25:57.626184+0000","last_clean":"2024-09-17T05:25:57.626184+0000","last_became_active":"2024-09-17T05:25:57.615624+0000","last_became_peered":"2024-09-17T05:25:57.615624+0000","last_unstale":"2024-09-17T05:25:57.626184+0000","last_undegraded":"2024-09-17T05:25:57.626184+0000","last_fullsized":"2024-09-17T05:25:57.626184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:44.367481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610827+0000","last_change":"2024-09-17T05:25:56.622123+0000","last_active":"2024-09-17T05:25:58.610827+0000","last_peered":"2024-09-17T05:25:58.610827+0000","last_clean":"2024-09-17T05:25:58.610827+0000","last_became_active":"2024-09-17T05:25:56.621736+0000","last_became_peered":"2024-09-17T05:25:56.621736+0000","last_unstale":"2024-09-17T05:25:58.610827+0000","last_undegraded":"2024-09-17T05:25:58.610827+0000","last_fullsized":"2024-09-17T05:25:58.610827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:00.720790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632333+0000","last_change":"2024-09-17T05:25:57.626610+0000","last_active":"2024-09-17T05:25:57.632333+0000","last_peered":"2024-09-17T05:25:57.632333+0000","last_clean":"2024-09-17T05:25:57.632333+0000","last_became_active":"2024-09-17T05:25:57.625895+0000","last_became_peered":"2024-09-17T05:25:57.625895+0000","last_unstale":"2024-09-17T05:25:57.632333+0000","last_undegraded":"2024-09-17T05:25:57.632333+0000","last_fullsized":"2024-09-17T05:25:57.632333+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:25.972778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.605529+0000","last_change":"2024-09-17T05:25:56.620365+0000","last_active":"2024-09-17T05:25:57.605529+0000","last_peered":"2024-09-17T05:25:57.605529+0000","last_clean":"2024-09-17T05:25:57.605529+0000","last_became_active":"2024-09-17T05:25:56.619802+0000","last_became_peered":"2024-09-17T05:25:56.619802+0000","last_unstale":"2024-09-17T05:25:57.605529+0000","last_undegraded":"2024-09-17T05:25:57.605529+0000","last_fullsized":"2024-09-17T05:25:57.605529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:48.498369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610141+0000","last_change":"2024-09-17T05:25:57.627811+0000","last_active":"2024-09-17T05:25:58.610141+0000","last_peered":"2024-09-17T05:25:58.610141+0000","last_clean":"2024-09-17T05:25:58.610141+0000","last_became_active":"2024-09-17T05:25:57.627460+0000","last_became_peered":"2024-09-17T05:25:57.627460+0000","last_unstale":"2024-09-17T05:25:58.610141+0000","last_undegraded":"2024-09-17T05:25:58.610141+0000","last_fullsized":"2024-09-17T05:25:58.610141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:31.804246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610311+0000","last_change":"2024-09-17T05:25:56.620199+0000","last_active":"2024-09-17T05:25:58.610311+0000","last_peered":"2024-09-17T05:25:58.610311+0000","last_clean":"2024-09-17T05:25:58.610311+0000","last_became_active":"2024-09-17T05:25:56.618305+0000","last_became_peered":"2024-09-17T05:25:56.618305+0000","last_unstale":"2024-09-17T05:25:58.610311+0000","last_undegraded":"2024-09-17T05:25:58.610311+0000","last_fullsized":"2024-09-17T05:25:58.610311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.702741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626709+0000","last_change":"2024-09-17T05:25:57.622968+0000","last_active":"2024-09-17T05:25:57.626709+0000","last_peered":"2024-09-17T05:25:57.626709+0000","last_clean":"2024-09-17T05:25:57.626709+0000","last_became_active":"2024-09-17T05:25:57.620376+0000","last_became_peered":"2024-09-17T05:25:57.620376+0000","last_unstale":"2024-09-17T05:25:57.626709+0000","last_undegraded":"2024-09-17T05:25:57.626709+0000","last_fullsized":"2024-09-17T05:25:57.626709+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:58.375489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.609469+0000","last_change":"2024-09-17T05:25:56.617986+0000","last_active":"2024-09-17T05:25:57.609469+0000","last_peered":"2024-09-17T05:25:57.609469+0000","last_clean":"2024-09-17T05:25:57.609469+0000","last_became_active":"2024-09-17T05:25:56.617362+0000","last_became_peered":"2024-09-17T05:25:56.617362+0000","last_unstale":"2024-09-17T05:25:57.609469+0000","last_undegraded":"2024-09-17T05:25:57.609469+0000","last_fullsized":"2024-09-17T05:25:57.609469+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.593668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634536+0000","last_change":"2024-09-17T05:25:57.631959+0000","last_active":"2024-09-17T05:25:57.634536+0000","last_peered":"2024-09-17T05:25:57.634536+0000","last_clean":"2024-09-17T05:25:57.634536+0000","last_became_active":"2024-09-17T05:25:57.625636+0000","last_became_peered":"2024-09-17T05:25:57.625636+0000","last_unstale":"2024-09-17T05:25:57.634536+0000","last_undegraded":"2024-09-17T05:25:57.634536+0000","last_fullsized":"2024-09-17T05:25:57.634536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:03.789409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604973+0000","last_change":"2024-09-17T05:25:56.623509+0000","last_active":"2024-09-17T05:25:57.604973+0000","last_peered":"2024-09-17T05:25:57.604973+0000","last_clean":"2024-09-17T05:25:57.604973+0000","last_became_active":"2024-09-17T05:25:56.623181+0000","last_became_peered":"2024-09-17T05:25:56.623181+0000","last_unstale":"2024-09-17T05:25:57.604973+0000","last_undegraded":"2024-09-17T05:25:57.604973+0000","last_fullsized":"2024-09-17T05:25:57.604973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:10.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633783+0000","last_change":"2024-09-17T05:25:57.628588+0000","last_active":"2024-09-17T05:25:57.633783+0000","last_peered":"2024-09-17T05:25:57.633783+0000","last_clean":"2024-09-17T05:25:57.633783+0000","last_became_active":"2024-09-17T05:25:57.628276+0000","last_became_peered":"2024-09-17T05:25:57.628276+0000","last_unstale":"2024-09-17T05:25:57.633783+0000","last_undegraded":"2024-09-17T05:25:57.633783+0000","last_fullsized":"2024-09-17T05:25:57.633783+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:13.958788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609538+0000","last_change":"2024-09-17T05:25:56.614603+0000","last_active":"2024-09-17T05:25:58.609538+0000","last_peered":"2024-09-17T05:25:58.609538+0000","last_clean":"2024-09-17T05:25:58.609538+0000","last_became_active":"2024-09-17T05:25:56.614263+0000","last_became_peered":"2024-09-17T05:25:56.614263+0000","last_unstale":"2024-09-17T05:25:58.609538+0000","last_undegraded":"2024-09-17T05:25:58.609538+0000","last_fullsized":"2024-09-17T05:25:58.609538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:36.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601494+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:57.601494+0000","last_peered":"2024-09-17T05:25:57.601494+0000","last_clean":"2024-09-17T05:25:57.601494+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:57.601494+0000","last_undegraded":"2024-09-17T05:25:57.601494+0000","last_fullsized":"2024-09-17T05:25:57.601494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":284923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":875233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":285447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":284921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":708608,"data_stored":286603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":875781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":286443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":278056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:26:01.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 -- 172.21.15.43:0/642073896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 msgr2=0x7f06a815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 --2- 172.21.15.43:0/642073896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815c7e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f06a80af0a0 tx=0x7f069801c620 comp rx=0 tx=0).stop 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 -- 172.21.15.43:0/642073896 shutdown_connections 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 --2- 172.21.15.43:0/642073896 >> v2:172.21.15.62:3300/0 conn(0x7f06a806f110 0x7f06a80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 --2- 172.21.15.43:0/642073896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815c7e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 -- 172.21.15.43:0/642073896 >> 172.21.15.43:0/642073896 conn(0x7f06a80b23d0 msgr2=0x7f06a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 -- 172.21.15.43:0/642073896 shutdown_connections 2024-09-17T05:26:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.000+0000 7f06af200640 1 -- 172.21.15.43:0/642073896 wait complete. 2024-09-17T05:26:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 Processor -- start 2024-09-17T05:26:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- start start 2024-09-17T05:26:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06a806f110 0x7f06a8164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b3920 0x7f06a808e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06a81689d0 con 0x7f06a8155d70 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8168850 con 0x7f06a80b3920 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f06a8168b50 con 0x7f06a806f110 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39948/0 (socket says 172.21.15.43:39948) 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 -- 172.21.15.43:0/1034283418 learned_addr learned my addr 172.21.15.43:0/1034283418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f069fe00640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b3920 0x7f06a808e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06a7e00640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06a806f110 0x7f06a8164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 -- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06a806f110 msgr2=0x7f06a8164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06a806f110 0x7f06a8164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 -- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b3920 msgr2=0x7f06a808e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b3920 0x7f06a808e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 -- 172.21.15.43:0/1034283418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a808ed30 con 0x7f06a8155d70 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06ad200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b9e0 tx=0x7f06a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0003b80 con 0x7f06a8155d70 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06a0003e10 con 0x7f06a8155d70 2024-09-17T05:26:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a808c110 con 0x7f06a8155d70 2024-09-17T05:26:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a000cb80 con 0x7f06a8155d70 2024-09-17T05:26:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.004+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a808c6d0 con 0x7f06a8155d70 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a808eaa0 con 0x7f06a8155d70 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f06a0006d10 con 0x7f06a8155d70 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06a4c00640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 0x7f0684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f06a0093dc0 con 0x7f06a8155d70 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06a7e00640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 0x7f0684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.008+0000 7f06a7e00640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 0x7f0684075710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06980ae000 tx=0x7f069801ae60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.012+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f06a005edd0 con 0x7f06a8155d70 2024-09-17T05:26:02.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.196+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06a808f010 con 0x7f0684073200 2024-09-17T05:26:02.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06a4c00640 1 -- 172.21.15.43:0/1034283418 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f06a808f010 con 0x7f0684073200 2024-09-17T05:26:02.210 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 msgr2=0x7f0684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 0x7f0684075710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06980ae000 tx=0x7f069801ae60 comp rx=0 tx=0).stop 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 msgr2=0x7f06a815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b9e0 tx=0x7f06a000bea0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 shutdown_connections 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0684073200 0x7f0684075710 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b3920 0x7f06a808e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a8155d70 0x7f06a815d2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.204+0000 7f06af200640 1 --2- 172.21.15.43:0/1034283418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f06a806f110 0x7f06a8164b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.208+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 >> 172.21.15.43:0/1034283418 conn(0x7f06a80b23d0 msgr2=0x7f06a815df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.208+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 shutdown_connections 2024-09-17T05:26:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.208+0000 7f06af200640 1 -- 172.21.15.43:0/1034283418 wait complete. 2024-09-17T05:26:02.231 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-17T05:26:01.879528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247776,"kb_used_data":6584,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726944,"statfs":{"total":773094113280,"available":772840390656,"internally_reserved":0,"allocated":6742016,"data_stored":3457407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001515"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635604+0000","last_change":"2024-09-17T05:25:57.631016+0000","last_active":"2024-09-17T05:25:57.635604+0000","last_peered":"2024-09-17T05:25:57.635604+0000","last_clean":"2024-09-17T05:25:57.635604+0000","last_became_active":"2024-09-17T05:25:57.627295+0000","last_became_peered":"2024-09-17T05:25:57.627295+0000","last_unstale":"2024-09-17T05:25:57.635604+0000","last_undegraded":"2024-09-17T05:25:57.635604+0000","last_fullsized":"2024-09-17T05:25:57.635604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:22.011741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633627+0000","last_change":"2024-09-17T05:25:57.623604+0000","last_active":"2024-09-17T05:25:57.633627+0000","last_peered":"2024-09-17T05:25:57.633627+0000","last_clean":"2024-09-17T05:25:57.633627+0000","last_became_active":"2024-09-17T05:25:57.623264+0000","last_became_peered":"2024-09-17T05:25:57.623264+0000","last_unstale":"2024-09-17T05:25:57.633627+0000","last_undegraded":"2024-09-17T05:25:57.633627+0000","last_fullsized":"2024-09-17T05:25:57.633627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:00.291409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611147+0000","last_change":"2024-09-17T05:25:57.631707+0000","last_active":"2024-09-17T05:25:58.611147+0000","last_peered":"2024-09-17T05:25:58.611147+0000","last_clean":"2024-09-17T05:25:58.611147+0000","last_became_active":"2024-09-17T05:25:57.631066+0000","last_became_peered":"2024-09-17T05:25:57.631066+0000","last_unstale":"2024-09-17T05:25:58.611147+0000","last_undegraded":"2024-09-17T05:25:58.611147+0000","last_fullsized":"2024-09-17T05:25:58.611147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:11.450539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638976+0000","last_change":"2024-09-17T05:25:57.625950+0000","last_active":"2024-09-17T05:25:57.638976+0000","last_peered":"2024-09-17T05:25:57.638976+0000","last_clean":"2024-09-17T05:25:57.638976+0000","last_became_active":"2024-09-17T05:25:57.625620+0000","last_became_peered":"2024-09-17T05:25:57.625620+0000","last_unstale":"2024-09-17T05:25:57.638976+0000","last_undegraded":"2024-09-17T05:25:57.638976+0000","last_fullsized":"2024-09-17T05:25:57.638976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:44.634325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639525+0000","last_change":"2024-09-17T05:25:57.636677+0000","last_active":"2024-09-17T05:25:57.639525+0000","last_peered":"2024-09-17T05:25:57.639525+0000","last_clean":"2024-09-17T05:25:57.639525+0000","last_became_active":"2024-09-17T05:25:57.636018+0000","last_became_peered":"2024-09-17T05:25:57.636018+0000","last_unstale":"2024-09-17T05:25:57.639525+0000","last_undegraded":"2024-09-17T05:25:57.639525+0000","last_fullsized":"2024-09-17T05:25:57.639525+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:55.616900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632365+0000","last_change":"2024-09-17T05:25:57.626965+0000","last_active":"2024-09-17T05:25:57.632365+0000","last_peered":"2024-09-17T05:25:57.632365+0000","last_clean":"2024-09-17T05:25:57.632365+0000","last_became_active":"2024-09-17T05:25:57.626268+0000","last_became_peered":"2024-09-17T05:25:57.626268+0000","last_unstale":"2024-09-17T05:25:57.632365+0000","last_undegraded":"2024-09-17T05:25:57.632365+0000","last_fullsized":"2024-09-17T05:25:57.632365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:31.263493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638603+0000","last_change":"2024-09-17T05:25:57.633222+0000","last_active":"2024-09-17T05:25:57.638603+0000","last_peered":"2024-09-17T05:25:57.638603+0000","last_clean":"2024-09-17T05:25:57.638603+0000","last_became_active":"2024-09-17T05:25:57.632854+0000","last_became_peered":"2024-09-17T05:25:57.632854+0000","last_unstale":"2024-09-17T05:25:57.638603+0000","last_undegraded":"2024-09-17T05:25:57.638603+0000","last_fullsized":"2024-09-17T05:25:57.638603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:57.721157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625974+0000","last_change":"2024-09-17T05:25:57.620215+0000","last_active":"2024-09-17T05:25:57.625974+0000","last_peered":"2024-09-17T05:25:57.625974+0000","last_clean":"2024-09-17T05:25:57.625974+0000","last_became_active":"2024-09-17T05:25:57.616145+0000","last_became_peered":"2024-09-17T05:25:57.616145+0000","last_unstale":"2024-09-17T05:25:57.625974+0000","last_undegraded":"2024-09-17T05:25:57.625974+0000","last_fullsized":"2024-09-17T05:25:57.625974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:25.171418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630959+0000","last_change":"2024-09-17T05:25:57.622118+0000","last_active":"2024-09-17T05:25:57.630959+0000","last_peered":"2024-09-17T05:25:57.630959+0000","last_clean":"2024-09-17T05:25:57.630959+0000","last_became_active":"2024-09-17T05:25:57.621310+0000","last_became_peered":"2024-09-17T05:25:57.621310+0000","last_unstale":"2024-09-17T05:25:57.630959+0000","last_undegraded":"2024-09-17T05:25:57.630959+0000","last_fullsized":"2024-09-17T05:25:57.630959+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:14.953015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620581+0000","last_change":"2024-09-17T05:25:57.615977+0000","last_active":"2024-09-17T05:25:57.620581+0000","last_peered":"2024-09-17T05:25:57.620581+0000","last_clean":"2024-09-17T05:25:57.620581+0000","last_became_active":"2024-09-17T05:25:57.615669+0000","last_became_peered":"2024-09-17T05:25:57.615669+0000","last_unstale":"2024-09-17T05:25:57.620581+0000","last_undegraded":"2024-09-17T05:25:57.620581+0000","last_fullsized":"2024-09-17T05:25:57.620581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:59.852702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626925+0000","last_change":"2024-09-17T05:25:57.621394+0000","last_active":"2024-09-17T05:25:57.626925+0000","last_peered":"2024-09-17T05:25:57.626925+0000","last_clean":"2024-09-17T05:25:57.626925+0000","last_became_active":"2024-09-17T05:25:57.620144+0000","last_became_peered":"2024-09-17T05:25:57.620144+0000","last_unstale":"2024-09-17T05:25:57.626925+0000","last_undegraded":"2024-09-17T05:25:57.626925+0000","last_fullsized":"2024-09-17T05:25:57.626925+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:25.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620544+0000","last_change":"2024-09-17T05:25:57.611565+0000","last_active":"2024-09-17T05:25:57.620544+0000","last_peered":"2024-09-17T05:25:57.620544+0000","last_clean":"2024-09-17T05:25:57.620544+0000","last_became_active":"2024-09-17T05:25:57.611202+0000","last_became_peered":"2024-09-17T05:25:57.611202+0000","last_unstale":"2024-09-17T05:25:57.620544+0000","last_undegraded":"2024-09-17T05:25:57.620544+0000","last_fullsized":"2024-09-17T05:25:57.620544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:07.436706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609911+0000","last_change":"2024-09-17T05:25:57.614032+0000","last_active":"2024-09-17T05:25:58.609911+0000","last_peered":"2024-09-17T05:25:58.609911+0000","last_clean":"2024-09-17T05:25:58.609911+0000","last_became_active":"2024-09-17T05:25:57.612857+0000","last_became_peered":"2024-09-17T05:25:57.612857+0000","last_unstale":"2024-09-17T05:25:58.609911+0000","last_undegraded":"2024-09-17T05:25:58.609911+0000","last_fullsized":"2024-09-17T05:25:58.609911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:30.858228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609971+0000","last_change":"2024-09-17T05:25:57.619743+0000","last_active":"2024-09-17T05:25:58.609971+0000","last_peered":"2024-09-17T05:25:58.609971+0000","last_clean":"2024-09-17T05:25:58.609971+0000","last_became_active":"2024-09-17T05:25:57.619444+0000","last_became_peered":"2024-09-17T05:25:57.619444+0000","last_unstale":"2024-09-17T05:25:58.609971+0000","last_undegraded":"2024-09-17T05:25:58.609971+0000","last_fullsized":"2024-09-17T05:25:58.609971+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.664202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609353+0000","last_change":"2024-09-17T05:25:57.618322+0000","last_active":"2024-09-17T05:25:58.609353+0000","last_peered":"2024-09-17T05:25:58.609353+0000","last_clean":"2024-09-17T05:25:58.609353+0000","last_became_active":"2024-09-17T05:25:57.618005+0000","last_became_peered":"2024-09-17T05:25:57.618005+0000","last_unstale":"2024-09-17T05:25:58.609353+0000","last_undegraded":"2024-09-17T05:25:58.609353+0000","last_fullsized":"2024-09-17T05:25:58.609353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:40.510229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.629375+0000","last_change":"2024-09-17T05:25:57.621062+0000","last_active":"2024-09-17T05:25:57.629375+0000","last_peered":"2024-09-17T05:25:57.629375+0000","last_clean":"2024-09-17T05:25:57.629375+0000","last_became_active":"2024-09-17T05:25:57.620738+0000","last_became_peered":"2024-09-17T05:25:57.620738+0000","last_unstale":"2024-09-17T05:25:57.629375+0000","last_undegraded":"2024-09-17T05:25:57.629375+0000","last_fullsized":"2024-09-17T05:25:57.629375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:21.005563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636295+0000","last_change":"2024-09-17T05:25:57.630743+0000","last_active":"2024-09-17T05:25:57.636295+0000","last_peered":"2024-09-17T05:25:57.636295+0000","last_clean":"2024-09-17T05:25:57.636295+0000","last_became_active":"2024-09-17T05:25:57.630402+0000","last_became_peered":"2024-09-17T05:25:57.630402+0000","last_unstale":"2024-09-17T05:25:57.636295+0000","last_undegraded":"2024-09-17T05:25:57.636295+0000","last_fullsized":"2024-09-17T05:25:57.636295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.191803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611142+0000","last_change":"2024-09-17T05:25:57.628918+0000","last_active":"2024-09-17T05:25:58.611142+0000","last_peered":"2024-09-17T05:25:58.611142+0000","last_clean":"2024-09-17T05:25:58.611142+0000","last_became_active":"2024-09-17T05:25:57.627711+0000","last_became_peered":"2024-09-17T05:25:57.627711+0000","last_unstale":"2024-09-17T05:25:58.611142+0000","last_undegraded":"2024-09-17T05:25:58.611142+0000","last_fullsized":"2024-09-17T05:25:58.611142+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:36.945516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633911+0000","last_change":"2024-09-17T05:25:57.625475+0000","last_active":"2024-09-17T05:25:57.633911+0000","last_peered":"2024-09-17T05:25:57.633911+0000","last_clean":"2024-09-17T05:25:57.633911+0000","last_became_active":"2024-09-17T05:25:57.625068+0000","last_became_peered":"2024-09-17T05:25:57.625068+0000","last_unstale":"2024-09-17T05:25:57.633911+0000","last_undegraded":"2024-09-17T05:25:57.633911+0000","last_fullsized":"2024-09-17T05:25:57.633911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:56.117332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626881+0000","last_change":"2024-09-17T05:25:57.619473+0000","last_active":"2024-09-17T05:25:57.626881+0000","last_peered":"2024-09-17T05:25:57.626881+0000","last_clean":"2024-09-17T05:25:57.626881+0000","last_became_active":"2024-09-17T05:25:57.619214+0000","last_became_peered":"2024-09-17T05:25:57.619214+0000","last_unstale":"2024-09-17T05:25:57.626881+0000","last_undegraded":"2024-09-17T05:25:57.626881+0000","last_fullsized":"2024-09-17T05:25:57.626881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:54.995659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610026+0000","last_change":"2024-09-17T05:25:57.609437+0000","last_active":"2024-09-17T05:25:58.610026+0000","last_peered":"2024-09-17T05:25:58.610026+0000","last_clean":"2024-09-17T05:25:58.610026+0000","last_became_active":"2024-09-17T05:25:57.609065+0000","last_became_peered":"2024-09-17T05:25:57.609065+0000","last_unstale":"2024-09-17T05:25:58.610026+0000","last_undegraded":"2024-09-17T05:25:58.610026+0000","last_fullsized":"2024-09-17T05:25:58.610026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:12.203019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626675+0000","last_change":"2024-09-17T05:25:57.614217+0000","last_active":"2024-09-17T05:25:57.626675+0000","last_peered":"2024-09-17T05:25:57.626675+0000","last_clean":"2024-09-17T05:25:57.626675+0000","last_became_active":"2024-09-17T05:25:57.613860+0000","last_became_peered":"2024-09-17T05:25:57.613860+0000","last_unstale":"2024-09-17T05:25:57.626675+0000","last_undegraded":"2024-09-17T05:25:57.626675+0000","last_fullsized":"2024-09-17T05:25:57.626675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:57.659052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.641900+0000","last_change":"2024-09-17T05:25:57.638376+0000","last_active":"2024-09-17T05:25:57.641900+0000","last_peered":"2024-09-17T05:25:57.641900+0000","last_clean":"2024-09-17T05:25:57.641900+0000","last_became_active":"2024-09-17T05:25:57.638064+0000","last_became_peered":"2024-09-17T05:25:57.638064+0000","last_unstale":"2024-09-17T05:25:57.641900+0000","last_undegraded":"2024-09-17T05:25:57.641900+0000","last_fullsized":"2024-09-17T05:25:57.641900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:16.470698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630529+0000","last_change":"2024-09-17T05:25:57.618378+0000","last_active":"2024-09-17T05:25:57.630529+0000","last_peered":"2024-09-17T05:25:57.630529+0000","last_clean":"2024-09-17T05:25:57.630529+0000","last_became_active":"2024-09-17T05:25:57.617935+0000","last_became_peered":"2024-09-17T05:25:57.617935+0000","last_unstale":"2024-09-17T05:25:57.630529+0000","last_undegraded":"2024-09-17T05:25:57.630529+0000","last_fullsized":"2024-09-17T05:25:57.630529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.312690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633216+0000","last_change":"2024-09-17T05:25:57.622847+0000","last_active":"2024-09-17T05:25:57.633216+0000","last_peered":"2024-09-17T05:25:57.633216+0000","last_clean":"2024-09-17T05:25:57.633216+0000","last_became_active":"2024-09-17T05:25:57.622508+0000","last_became_peered":"2024-09-17T05:25:57.622508+0000","last_unstale":"2024-09-17T05:25:57.633216+0000","last_undegraded":"2024-09-17T05:25:57.633216+0000","last_fullsized":"2024-09-17T05:25:57.633216+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:26.833060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610442+0000","last_change":"2024-09-17T05:25:57.622610+0000","last_active":"2024-09-17T05:25:58.610442+0000","last_peered":"2024-09-17T05:25:58.610442+0000","last_clean":"2024-09-17T05:25:58.610442+0000","last_became_active":"2024-09-17T05:25:57.622274+0000","last_became_peered":"2024-09-17T05:25:57.622274+0000","last_unstale":"2024-09-17T05:25:58.610442+0000","last_undegraded":"2024-09-17T05:25:58.610442+0000","last_fullsized":"2024-09-17T05:25:58.610442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:06.852378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633822+0000","last_change":"2024-09-17T05:25:57.624501+0000","last_active":"2024-09-17T05:25:57.633822+0000","last_peered":"2024-09-17T05:25:57.633822+0000","last_clean":"2024-09-17T05:25:57.633822+0000","last_became_active":"2024-09-17T05:25:57.624158+0000","last_became_peered":"2024-09-17T05:25:57.624158+0000","last_unstale":"2024-09-17T05:25:57.633822+0000","last_undegraded":"2024-09-17T05:25:57.633822+0000","last_fullsized":"2024-09-17T05:25:57.633822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:32.625863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625249+0000","last_change":"2024-09-17T05:25:57.616712+0000","last_active":"2024-09-17T05:25:57.625249+0000","last_peered":"2024-09-17T05:25:57.625249+0000","last_clean":"2024-09-17T05:25:57.625249+0000","last_became_active":"2024-09-17T05:25:57.616385+0000","last_became_peered":"2024-09-17T05:25:57.616385+0000","last_unstale":"2024-09-17T05:25:57.625249+0000","last_undegraded":"2024-09-17T05:25:57.625249+0000","last_fullsized":"2024-09-17T05:25:57.625249+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:33.942936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.627648+0000","last_change":"2024-09-17T05:25:57.622667+0000","last_active":"2024-09-17T05:25:57.627648+0000","last_peered":"2024-09-17T05:25:57.627648+0000","last_clean":"2024-09-17T05:25:57.627648+0000","last_became_active":"2024-09-17T05:25:57.622137+0000","last_became_peered":"2024-09-17T05:25:57.622137+0000","last_unstale":"2024-09-17T05:25:57.627648+0000","last_undegraded":"2024-09-17T05:25:57.627648+0000","last_fullsized":"2024-09-17T05:25:57.627648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:02.548171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.640261+0000","last_change":"2024-09-17T05:25:57.632398+0000","last_active":"2024-09-17T05:25:57.640261+0000","last_peered":"2024-09-17T05:25:57.640261+0000","last_clean":"2024-09-17T05:25:57.640261+0000","last_became_active":"2024-09-17T05:25:57.631992+0000","last_became_peered":"2024-09-17T05:25:57.631992+0000","last_unstale":"2024-09-17T05:25:57.640261+0000","last_undegraded":"2024-09-17T05:25:57.640261+0000","last_fullsized":"2024-09-17T05:25:57.640261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:41.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634331+0000","last_change":"2024-09-17T05:25:57.629493+0000","last_active":"2024-09-17T05:25:57.634331+0000","last_peered":"2024-09-17T05:25:57.634331+0000","last_clean":"2024-09-17T05:25:57.634331+0000","last_became_active":"2024-09-17T05:25:57.624157+0000","last_became_peered":"2024-09-17T05:25:57.624157+0000","last_unstale":"2024-09-17T05:25:57.634331+0000","last_undegraded":"2024-09-17T05:25:57.634331+0000","last_fullsized":"2024-09-17T05:25:57.634331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:50.170213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626934+0000","last_change":"2024-09-17T05:25:57.618377+0000","last_active":"2024-09-17T05:25:57.626934+0000","last_peered":"2024-09-17T05:25:57.626934+0000","last_clean":"2024-09-17T05:25:57.626934+0000","last_became_active":"2024-09-17T05:25:57.615243+0000","last_became_peered":"2024-09-17T05:25:57.615243+0000","last_unstale":"2024-09-17T05:25:57.626934+0000","last_undegraded":"2024-09-17T05:25:57.626934+0000","last_fullsized":"2024-09-17T05:25:57.626934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:08.640363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.620614+0000","last_change":"2024-09-17T05:25:57.617672+0000","last_active":"2024-09-17T05:25:57.620614+0000","last_peered":"2024-09-17T05:25:57.620614+0000","last_clean":"2024-09-17T05:25:57.620614+0000","last_became_active":"2024-09-17T05:25:57.617221+0000","last_became_peered":"2024-09-17T05:25:57.617221+0000","last_unstale":"2024-09-17T05:25:57.620614+0000","last_undegraded":"2024-09-17T05:25:57.620614+0000","last_fullsized":"2024-09-17T05:25:57.620614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:13.140113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.637380+0000","last_change":"2024-09-17T05:25:56.602138+0000","last_active":"2024-09-17T05:25:57.637380+0000","last_peered":"2024-09-17T05:25:57.637380+0000","last_clean":"2024-09-17T05:25:57.637380+0000","last_became_active":"2024-09-17T05:25:56.601770+0000","last_became_peered":"2024-09-17T05:25:56.601770+0000","last_unstale":"2024-09-17T05:25:57.637380+0000","last_undegraded":"2024-09-17T05:25:57.637380+0000","last_fullsized":"2024-09-17T05:25:57.637380+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:34.638935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.638893+0000","last_change":"2024-09-17T05:25:57.636046+0000","last_active":"2024-09-17T05:25:57.638893+0000","last_peered":"2024-09-17T05:25:57.638893+0000","last_clean":"2024-09-17T05:25:57.638893+0000","last_became_active":"2024-09-17T05:25:57.635614+0000","last_became_peered":"2024-09-17T05:25:57.635614+0000","last_unstale":"2024-09-17T05:25:57.638893+0000","last_undegraded":"2024-09-17T05:25:57.638893+0000","last_fullsized":"2024-09-17T05:25:57.638893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:42.351638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632973+0000","last_change":"2024-09-17T05:25:56.620724+0000","last_active":"2024-09-17T05:25:57.632973+0000","last_peered":"2024-09-17T05:25:57.632973+0000","last_clean":"2024-09-17T05:25:57.632973+0000","last_became_active":"2024-09-17T05:25:56.619901+0000","last_became_peered":"2024-09-17T05:25:56.619901+0000","last_unstale":"2024-09-17T05:25:57.632973+0000","last_undegraded":"2024-09-17T05:25:57.632973+0000","last_fullsized":"2024-09-17T05:25:57.632973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:50.563424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625308+0000","last_change":"2024-09-17T05:25:57.618668+0000","last_active":"2024-09-17T05:25:57.625308+0000","last_peered":"2024-09-17T05:25:57.625308+0000","last_clean":"2024-09-17T05:25:57.625308+0000","last_became_active":"2024-09-17T05:25:57.617954+0000","last_became_peered":"2024-09-17T05:25:57.617954+0000","last_unstale":"2024-09-17T05:25:57.625308+0000","last_undegraded":"2024-09-17T05:25:57.625308+0000","last_fullsized":"2024-09-17T05:25:57.625308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:55.254342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636015+0000","last_change":"2024-09-17T05:25:56.620360+0000","last_active":"2024-09-17T05:25:57.636015+0000","last_peered":"2024-09-17T05:25:57.636015+0000","last_clean":"2024-09-17T05:25:57.636015+0000","last_became_active":"2024-09-17T05:25:56.619964+0000","last_became_peered":"2024-09-17T05:25:56.619964+0000","last_unstale":"2024-09-17T05:25:57.636015+0000","last_undegraded":"2024-09-17T05:25:57.636015+0000","last_fullsized":"2024-09-17T05:25:57.636015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:20.504533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.629902+0000","last_change":"2024-09-17T05:25:57.624345+0000","last_active":"2024-09-17T05:25:57.629902+0000","last_peered":"2024-09-17T05:25:57.629902+0000","last_clean":"2024-09-17T05:25:57.629902+0000","last_became_active":"2024-09-17T05:25:57.623583+0000","last_became_peered":"2024-09-17T05:25:57.623583+0000","last_unstale":"2024-09-17T05:25:57.629902+0000","last_undegraded":"2024-09-17T05:25:57.629902+0000","last_fullsized":"2024-09-17T05:25:57.629902+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:31.160498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642788+0000","last_change":"2024-09-17T05:25:56.625871+0000","last_active":"2024-09-17T05:25:57.642788+0000","last_peered":"2024-09-17T05:25:57.642788+0000","last_clean":"2024-09-17T05:25:57.642788+0000","last_became_active":"2024-09-17T05:25:56.625569+0000","last_became_peered":"2024-09-17T05:25:56.625569+0000","last_unstale":"2024-09-17T05:25:57.642788+0000","last_undegraded":"2024-09-17T05:25:57.642788+0000","last_fullsized":"2024-09-17T05:25:57.642788+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:12.956313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633671+0000","last_change":"2024-09-17T05:25:57.627051+0000","last_active":"2024-09-17T05:25:57.633671+0000","last_peered":"2024-09-17T05:25:57.633671+0000","last_clean":"2024-09-17T05:25:57.633671+0000","last_became_active":"2024-09-17T05:25:57.626651+0000","last_became_peered":"2024-09-17T05:25:57.626651+0000","last_unstale":"2024-09-17T05:25:57.633671+0000","last_undegraded":"2024-09-17T05:25:57.633671+0000","last_fullsized":"2024-09-17T05:25:57.633671+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:01.557423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610766+0000","last_change":"2024-09-17T05:25:56.615735+0000","last_active":"2024-09-17T05:25:58.610766+0000","last_peered":"2024-09-17T05:25:58.610766+0000","last_clean":"2024-09-17T05:25:58.610766+0000","last_became_active":"2024-09-17T05:25:56.615440+0000","last_became_peered":"2024-09-17T05:25:56.615440+0000","last_unstale":"2024-09-17T05:25:58.610766+0000","last_undegraded":"2024-09-17T05:25:58.610766+0000","last_fullsized":"2024-09-17T05:25:58.610766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.890221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626217+0000","last_change":"2024-09-17T05:25:57.614450+0000","last_active":"2024-09-17T05:25:57.626217+0000","last_peered":"2024-09-17T05:25:57.626217+0000","last_clean":"2024-09-17T05:25:57.626217+0000","last_became_active":"2024-09-17T05:25:57.614111+0000","last_became_peered":"2024-09-17T05:25:57.614111+0000","last_unstale":"2024-09-17T05:25:57.626217+0000","last_undegraded":"2024-09-17T05:25:57.626217+0000","last_fullsized":"2024-09-17T05:25:57.626217+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:08.532142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642238+0000","last_change":"2024-09-17T05:25:56.620062+0000","last_active":"2024-09-17T05:25:57.642238+0000","last_peered":"2024-09-17T05:25:57.642238+0000","last_clean":"2024-09-17T05:25:57.642238+0000","last_became_active":"2024-09-17T05:25:56.619711+0000","last_became_peered":"2024-09-17T05:25:56.619711+0000","last_unstale":"2024-09-17T05:25:57.642238+0000","last_undegraded":"2024-09-17T05:25:57.642238+0000","last_fullsized":"2024-09-17T05:25:57.642238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:40.597798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609807+0000","last_change":"2024-09-17T05:25:57.636411+0000","last_active":"2024-09-17T05:25:58.609807+0000","last_peered":"2024-09-17T05:25:58.609807+0000","last_clean":"2024-09-17T05:25:58.609807+0000","last_became_active":"2024-09-17T05:25:57.636080+0000","last_became_peered":"2024-09-17T05:25:57.636080+0000","last_unstale":"2024-09-17T05:25:58.609807+0000","last_undegraded":"2024-09-17T05:25:58.609807+0000","last_fullsized":"2024-09-17T05:25:58.609807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:25.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.600496+0000","last_change":"2024-09-17T05:25:56.600998+0000","last_active":"2024-09-17T05:25:57.600496+0000","last_peered":"2024-09-17T05:25:57.600496+0000","last_clean":"2024-09-17T05:25:57.600496+0000","last_became_active":"2024-09-17T05:25:56.600504+0000","last_became_peered":"2024-09-17T05:25:56.600504+0000","last_unstale":"2024-09-17T05:25:57.600496+0000","last_undegraded":"2024-09-17T05:25:57.600496+0000","last_fullsized":"2024-09-17T05:25:57.600496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:52.590333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635426+0000","last_change":"2024-09-17T05:25:57.630161+0000","last_active":"2024-09-17T05:25:57.635426+0000","last_peered":"2024-09-17T05:25:57.635426+0000","last_clean":"2024-09-17T05:25:57.635426+0000","last_became_active":"2024-09-17T05:25:57.629719+0000","last_became_peered":"2024-09-17T05:25:57.629719+0000","last_unstale":"2024-09-17T05:25:57.635426+0000","last_undegraded":"2024-09-17T05:25:57.635426+0000","last_fullsized":"2024-09-17T05:25:57.635426+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:30.192084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609243+0000","last_change":"2024-09-17T05:25:56.605033+0000","last_active":"2024-09-17T05:25:58.609243+0000","last_peered":"2024-09-17T05:25:58.609243+0000","last_clean":"2024-09-17T05:25:58.609243+0000","last_became_active":"2024-09-17T05:25:56.604728+0000","last_became_peered":"2024-09-17T05:25:56.604728+0000","last_unstale":"2024-09-17T05:25:58.609243+0000","last_undegraded":"2024-09-17T05:25:58.609243+0000","last_fullsized":"2024-09-17T05:25:58.609243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:48.703259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610520+0000","last_change":"2024-09-17T05:25:57.618887+0000","last_active":"2024-09-17T05:25:58.610520+0000","last_peered":"2024-09-17T05:25:58.610520+0000","last_clean":"2024-09-17T05:25:58.610520+0000","last_became_active":"2024-09-17T05:25:57.618549+0000","last_became_peered":"2024-09-17T05:25:57.618549+0000","last_unstale":"2024-09-17T05:25:58.610520+0000","last_undegraded":"2024-09-17T05:25:58.610520+0000","last_fullsized":"2024-09-17T05:25:58.610520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:11.450539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601284+0000","last_change":"2024-09-17T05:25:56.612951+0000","last_active":"2024-09-17T05:25:57.601284+0000","last_peered":"2024-09-17T05:25:57.601284+0000","last_clean":"2024-09-17T05:25:57.601284+0000","last_became_active":"2024-09-17T05:25:56.612540+0000","last_became_peered":"2024-09-17T05:25:56.612540+0000","last_unstale":"2024-09-17T05:25:57.601284+0000","last_undegraded":"2024-09-17T05:25:57.601284+0000","last_fullsized":"2024-09-17T05:25:57.601284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:37.040947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611361+0000","last_change":"2024-09-17T05:25:57.628860+0000","last_active":"2024-09-17T05:25:58.611361+0000","last_peered":"2024-09-17T05:25:58.611361+0000","last_clean":"2024-09-17T05:25:58.611361+0000","last_became_active":"2024-09-17T05:25:57.628517+0000","last_became_peered":"2024-09-17T05:25:57.628517+0000","last_unstale":"2024-09-17T05:25:58.611361+0000","last_undegraded":"2024-09-17T05:25:58.611361+0000","last_fullsized":"2024-09-17T05:25:58.611361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:31.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.605250+0000","last_change":"2024-09-17T05:25:56.620974+0000","last_active":"2024-09-17T05:25:57.605250+0000","last_peered":"2024-09-17T05:25:57.605250+0000","last_clean":"2024-09-17T05:25:57.605250+0000","last_became_active":"2024-09-17T05:25:56.618268+0000","last_became_peered":"2024-09-17T05:25:56.618268+0000","last_unstale":"2024-09-17T05:25:57.605250+0000","last_undegraded":"2024-09-17T05:25:57.605250+0000","last_fullsized":"2024-09-17T05:25:57.605250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.860676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.625281+0000","last_change":"2024-09-17T05:25:57.618334+0000","last_active":"2024-09-17T05:25:57.625281+0000","last_peered":"2024-09-17T05:25:57.625281+0000","last_clean":"2024-09-17T05:25:57.625281+0000","last_became_active":"2024-09-17T05:25:57.617488+0000","last_became_peered":"2024-09-17T05:25:57.617488+0000","last_unstale":"2024-09-17T05:25:57.625281+0000","last_undegraded":"2024-09-17T05:25:57.625281+0000","last_fullsized":"2024-09-17T05:25:57.625281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:02.356252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642900+0000","last_change":"2024-09-17T05:25:56.623672+0000","last_active":"2024-09-17T05:25:57.642900+0000","last_peered":"2024-09-17T05:25:57.642900+0000","last_clean":"2024-09-17T05:25:57.642900+0000","last_became_active":"2024-09-17T05:25:56.623101+0000","last_became_peered":"2024-09-17T05:25:56.623101+0000","last_unstale":"2024-09-17T05:25:57.642900+0000","last_undegraded":"2024-09-17T05:25:57.642900+0000","last_fullsized":"2024-09-17T05:25:57.642900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:19.717809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639462+0000","last_change":"2024-09-17T05:25:57.621965+0000","last_active":"2024-09-17T05:25:57.639462+0000","last_peered":"2024-09-17T05:25:57.639462+0000","last_clean":"2024-09-17T05:25:57.639462+0000","last_became_active":"2024-09-17T05:25:57.621619+0000","last_became_peered":"2024-09-17T05:25:57.621619+0000","last_unstale":"2024-09-17T05:25:57.639462+0000","last_undegraded":"2024-09-17T05:25:57.639462+0000","last_fullsized":"2024-09-17T05:25:57.639462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:46.532448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610189+0000","last_change":"2024-09-17T05:25:56.609748+0000","last_active":"2024-09-17T05:25:58.610189+0000","last_peered":"2024-09-17T05:25:58.610189+0000","last_clean":"2024-09-17T05:25:58.610189+0000","last_became_active":"2024-09-17T05:25:56.609389+0000","last_became_peered":"2024-09-17T05:25:56.609389+0000","last_unstale":"2024-09-17T05:25:58.610189+0000","last_undegraded":"2024-09-17T05:25:58.610189+0000","last_fullsized":"2024-09-17T05:25:58.610189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:01.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.624428+0000","last_change":"2024-09-17T05:25:57.614074+0000","last_active":"2024-09-17T05:25:57.624428+0000","last_peered":"2024-09-17T05:25:57.624428+0000","last_clean":"2024-09-17T05:25:57.624428+0000","last_became_active":"2024-09-17T05:25:57.613697+0000","last_became_peered":"2024-09-17T05:25:57.613697+0000","last_unstale":"2024-09-17T05:25:57.624428+0000","last_undegraded":"2024-09-17T05:25:57.624428+0000","last_fullsized":"2024-09-17T05:25:57.624428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:28.519789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639596+0000","last_change":"2024-09-17T05:25:56.615127+0000","last_active":"2024-09-17T05:25:57.639596+0000","last_peered":"2024-09-17T05:25:57.639596+0000","last_clean":"2024-09-17T05:25:57.639596+0000","last_became_active":"2024-09-17T05:25:56.614790+0000","last_became_peered":"2024-09-17T05:25:56.614790+0000","last_unstale":"2024-09-17T05:25:57.639596+0000","last_undegraded":"2024-09-17T05:25:57.639596+0000","last_fullsized":"2024-09-17T05:25:57.639596+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:17.126953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610817+0000","last_change":"2024-09-17T05:25:57.618971+0000","last_active":"2024-09-17T05:25:58.610817+0000","last_peered":"2024-09-17T05:25:58.610817+0000","last_clean":"2024-09-17T05:25:58.610817+0000","last_became_active":"2024-09-17T05:25:57.618673+0000","last_became_peered":"2024-09-17T05:25:57.618673+0000","last_unstale":"2024-09-17T05:25:58.610817+0000","last_undegraded":"2024-09-17T05:25:58.610817+0000","last_fullsized":"2024-09-17T05:25:58.610817+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:03.730384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.614568+0000","last_change":"2024-09-17T05:25:56.613747+0000","last_active":"2024-09-17T05:25:57.614568+0000","last_peered":"2024-09-17T05:25:57.614568+0000","last_clean":"2024-09-17T05:25:57.614568+0000","last_became_active":"2024-09-17T05:25:56.613407+0000","last_became_peered":"2024-09-17T05:25:56.613407+0000","last_unstale":"2024-09-17T05:25:57.614568+0000","last_undegraded":"2024-09-17T05:25:57.614568+0000","last_fullsized":"2024-09-17T05:25:57.614568+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:09.169185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.618240+0000","last_change":"2024-09-17T05:25:57.610448+0000","last_active":"2024-09-17T05:25:57.618240+0000","last_peered":"2024-09-17T05:25:57.618240+0000","last_clean":"2024-09-17T05:25:57.618240+0000","last_became_active":"2024-09-17T05:25:57.610076+0000","last_became_peered":"2024-09-17T05:25:57.610076+0000","last_unstale":"2024-09-17T05:25:57.618240+0000","last_undegraded":"2024-09-17T05:25:57.618240+0000","last_fullsized":"2024-09-17T05:25:57.618240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:47.049648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.612535+0000","last_change":"2024-09-17T05:25:56.622751+0000","last_active":"2024-09-17T05:25:57.612535+0000","last_peered":"2024-09-17T05:25:57.612535+0000","last_clean":"2024-09-17T05:25:57.612535+0000","last_became_active":"2024-09-17T05:25:56.620388+0000","last_became_peered":"2024-09-17T05:25:56.620388+0000","last_unstale":"2024-09-17T05:25:57.612535+0000","last_undegraded":"2024-09-17T05:25:57.612535+0000","last_fullsized":"2024-09-17T05:25:57.612535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:12.620914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.641870+0000","last_change":"2024-09-17T05:25:57.635094+0000","last_active":"2024-09-17T05:25:57.641870+0000","last_peered":"2024-09-17T05:25:57.641870+0000","last_clean":"2024-09-17T05:25:57.641870+0000","last_became_active":"2024-09-17T05:25:57.634771+0000","last_became_peered":"2024-09-17T05:25:57.634771+0000","last_unstale":"2024-09-17T05:25:57.641870+0000","last_undegraded":"2024-09-17T05:25:57.641870+0000","last_fullsized":"2024-09-17T05:25:57.641870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:32.191787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.642416+0000","last_change":"2024-09-17T05:25:56.619402+0000","last_active":"2024-09-17T05:25:57.642416+0000","last_peered":"2024-09-17T05:25:57.642416+0000","last_clean":"2024-09-17T05:25:57.642416+0000","last_became_active":"2024-09-17T05:25:56.619122+0000","last_became_peered":"2024-09-17T05:25:56.619122+0000","last_unstale":"2024-09-17T05:25:57.642416+0000","last_undegraded":"2024-09-17T05:25:57.642416+0000","last_fullsized":"2024-09-17T05:25:57.642416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:00.554948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611113+0000","last_change":"2024-09-17T05:25:57.630886+0000","last_active":"2024-09-17T05:25:58.611113+0000","last_peered":"2024-09-17T05:25:58.611113+0000","last_clean":"2024-09-17T05:25:58.611113+0000","last_became_active":"2024-09-17T05:25:57.630442+0000","last_became_peered":"2024-09-17T05:25:57.630442+0000","last_unstale":"2024-09-17T05:25:58.611113+0000","last_undegraded":"2024-09-17T05:25:58.611113+0000","last_fullsized":"2024-09-17T05:25:58.611113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:39.972695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636267+0000","last_change":"2024-09-17T05:25:56.624991+0000","last_active":"2024-09-17T05:25:57.636267+0000","last_peered":"2024-09-17T05:25:57.636267+0000","last_clean":"2024-09-17T05:25:57.636267+0000","last_became_active":"2024-09-17T05:25:56.624587+0000","last_became_peered":"2024-09-17T05:25:56.624587+0000","last_unstale":"2024-09-17T05:25:57.636267+0000","last_undegraded":"2024-09-17T05:25:57.636267+0000","last_fullsized":"2024-09-17T05:25:57.636267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:13.760914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611390+0000","last_change":"2024-09-17T05:25:57.627117+0000","last_active":"2024-09-17T05:25:58.611390+0000","last_peered":"2024-09-17T05:25:58.611390+0000","last_clean":"2024-09-17T05:25:58.611390+0000","last_became_active":"2024-09-17T05:25:57.624515+0000","last_became_peered":"2024-09-17T05:25:57.624515+0000","last_unstale":"2024-09-17T05:25:58.611390+0000","last_undegraded":"2024-09-17T05:25:58.611390+0000","last_fullsized":"2024-09-17T05:25:58.611390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:25.415314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.637321+0000","last_change":"2024-09-17T05:25:56.616994+0000","last_active":"2024-09-17T05:25:57.637321+0000","last_peered":"2024-09-17T05:25:57.637321+0000","last_clean":"2024-09-17T05:25:57.637321+0000","last_became_active":"2024-09-17T05:25:56.616603+0000","last_became_peered":"2024-09-17T05:25:56.616603+0000","last_unstale":"2024-09-17T05:25:57.637321+0000","last_undegraded":"2024-09-17T05:25:57.637321+0000","last_fullsized":"2024-09-17T05:25:57.637321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:36.954863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634325+0000","last_change":"2024-09-17T05:25:57.625878+0000","last_active":"2024-09-17T05:25:57.634325+0000","last_peered":"2024-09-17T05:25:57.634325+0000","last_clean":"2024-09-17T05:25:57.634325+0000","last_became_active":"2024-09-17T05:25:57.625479+0000","last_became_peered":"2024-09-17T05:25:57.625479+0000","last_unstale":"2024-09-17T05:25:57.634325+0000","last_undegraded":"2024-09-17T05:25:57.634325+0000","last_fullsized":"2024-09-17T05:25:57.634325+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:20.333175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611106+0000","last_change":"2024-09-17T05:25:56.618618+0000","last_active":"2024-09-17T05:25:58.611106+0000","last_peered":"2024-09-17T05:25:58.611106+0000","last_clean":"2024-09-17T05:25:58.611106+0000","last_became_active":"2024-09-17T05:25:56.617666+0000","last_became_peered":"2024-09-17T05:25:56.617666+0000","last_unstale":"2024-09-17T05:25:58.611106+0000","last_undegraded":"2024-09-17T05:25:58.611106+0000","last_fullsized":"2024-09-17T05:25:58.611106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.058611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.639493+0000","last_change":"2024-09-17T05:25:57.626670+0000","last_active":"2024-09-17T05:25:57.639493+0000","last_peered":"2024-09-17T05:25:57.639493+0000","last_clean":"2024-09-17T05:25:57.639493+0000","last_became_active":"2024-09-17T05:25:57.626304+0000","last_became_peered":"2024-09-17T05:25:57.626304+0000","last_unstale":"2024-09-17T05:25:57.639493+0000","last_undegraded":"2024-09-17T05:25:57.639493+0000","last_fullsized":"2024-09-17T05:25:57.639493+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:56.242997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604972+0000","last_change":"2024-09-17T05:25:56.612627+0000","last_active":"2024-09-17T05:25:57.604972+0000","last_peered":"2024-09-17T05:25:57.604972+0000","last_clean":"2024-09-17T05:25:57.604972+0000","last_became_active":"2024-09-17T05:25:56.612310+0000","last_became_peered":"2024-09-17T05:25:56.612310+0000","last_unstale":"2024-09-17T05:25:57.604972+0000","last_undegraded":"2024-09-17T05:25:57.604972+0000","last_fullsized":"2024-09-17T05:25:57.604972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:52.779414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.630851+0000","last_change":"2024-09-17T05:25:57.624159+0000","last_active":"2024-09-17T05:25:57.630851+0000","last_peered":"2024-09-17T05:25:57.630851+0000","last_clean":"2024-09-17T05:25:57.630851+0000","last_became_active":"2024-09-17T05:25:57.621139+0000","last_became_peered":"2024-09-17T05:25:57.621139+0000","last_unstale":"2024-09-17T05:25:57.630851+0000","last_undegraded":"2024-09-17T05:25:57.630851+0000","last_fullsized":"2024-09-17T05:25:57.630851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:13.198875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.636965+0000","last_change":"2024-09-17T05:25:56.612418+0000","last_active":"2024-09-17T05:25:57.636965+0000","last_peered":"2024-09-17T05:25:57.636965+0000","last_clean":"2024-09-17T05:25:57.636965+0000","last_became_active":"2024-09-17T05:25:56.612087+0000","last_became_peered":"2024-09-17T05:25:56.612087+0000","last_unstale":"2024-09-17T05:25:57.636965+0000","last_undegraded":"2024-09-17T05:25:57.636965+0000","last_fullsized":"2024-09-17T05:25:57.636965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:05.209454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.627680+0000","last_change":"2024-09-17T05:25:57.623746+0000","last_active":"2024-09-17T05:25:57.627680+0000","last_peered":"2024-09-17T05:25:57.627680+0000","last_clean":"2024-09-17T05:25:57.627680+0000","last_became_active":"2024-09-17T05:25:57.623459+0000","last_became_peered":"2024-09-17T05:25:57.623459+0000","last_unstale":"2024-09-17T05:25:57.627680+0000","last_undegraded":"2024-09-17T05:25:57.627680+0000","last_fullsized":"2024-09-17T05:25:57.627680+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:06.593568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604691+0000","last_change":"2024-09-17T05:25:56.618346+0000","last_active":"2024-09-17T05:25:57.604691+0000","last_peered":"2024-09-17T05:25:57.604691+0000","last_clean":"2024-09-17T05:25:57.604691+0000","last_became_active":"2024-09-17T05:25:56.617486+0000","last_became_peered":"2024-09-17T05:25:56.617486+0000","last_unstale":"2024-09-17T05:25:57.604691+0000","last_undegraded":"2024-09-17T05:25:57.604691+0000","last_fullsized":"2024-09-17T05:25:57.604691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:05.849904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.622194+0000","last_change":"2024-09-17T05:25:57.616146+0000","last_active":"2024-09-17T05:25:57.622194+0000","last_peered":"2024-09-17T05:25:57.622194+0000","last_clean":"2024-09-17T05:25:57.622194+0000","last_became_active":"2024-09-17T05:25:57.615768+0000","last_became_peered":"2024-09-17T05:25:57.615768+0000","last_unstale":"2024-09-17T05:25:57.622194+0000","last_undegraded":"2024-09-17T05:25:57.622194+0000","last_fullsized":"2024-09-17T05:25:57.622194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:04.473483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601698+0000","last_change":"2024-09-17T05:25:56.619091+0000","last_active":"2024-09-17T05:25:57.601698+0000","last_peered":"2024-09-17T05:25:57.601698+0000","last_clean":"2024-09-17T05:25:57.601698+0000","last_became_active":"2024-09-17T05:25:56.618457+0000","last_became_peered":"2024-09-17T05:25:56.618457+0000","last_unstale":"2024-09-17T05:25:57.601698+0000","last_undegraded":"2024-09-17T05:25:57.601698+0000","last_fullsized":"2024-09-17T05:25:57.601698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:55.581973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611244+0000","last_change":"2024-09-17T05:25:57.628995+0000","last_active":"2024-09-17T05:25:58.611244+0000","last_peered":"2024-09-17T05:25:58.611244+0000","last_clean":"2024-09-17T05:25:58.611244+0000","last_became_active":"2024-09-17T05:25:57.625672+0000","last_became_peered":"2024-09-17T05:25:57.625672+0000","last_unstale":"2024-09-17T05:25:58.611244+0000","last_undegraded":"2024-09-17T05:25:58.611244+0000","last_fullsized":"2024-09-17T05:25:58.611244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:53.781888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.635059+0000","last_change":"2024-09-17T05:25:56.622397+0000","last_active":"2024-09-17T05:25:57.635059+0000","last_peered":"2024-09-17T05:25:57.635059+0000","last_clean":"2024-09-17T05:25:57.635059+0000","last_became_active":"2024-09-17T05:25:56.621549+0000","last_became_peered":"2024-09-17T05:25:56.621549+0000","last_unstale":"2024-09-17T05:25:57.635059+0000","last_undegraded":"2024-09-17T05:25:57.635059+0000","last_fullsized":"2024-09-17T05:25:57.635059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:57.433606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.611580+0000","last_change":"2024-09-17T05:25:57.641404+0000","last_active":"2024-09-17T05:25:58.611580+0000","last_peered":"2024-09-17T05:25:58.611580+0000","last_clean":"2024-09-17T05:25:58.611580+0000","last_became_active":"2024-09-17T05:25:57.641106+0000","last_became_peered":"2024-09-17T05:25:57.641106+0000","last_unstale":"2024-09-17T05:25:58.611580+0000","last_undegraded":"2024-09-17T05:25:58.611580+0000","last_fullsized":"2024-09-17T05:25:58.611580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:24.314248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.614558+0000","last_change":"2024-09-17T05:25:56.622460+0000","last_active":"2024-09-17T05:25:57.614558+0000","last_peered":"2024-09-17T05:25:57.614558+0000","last_clean":"2024-09-17T05:25:57.614558+0000","last_became_active":"2024-09-17T05:25:56.620828+0000","last_became_peered":"2024-09-17T05:25:56.620828+0000","last_unstale":"2024-09-17T05:25:57.614558+0000","last_undegraded":"2024-09-17T05:25:57.614558+0000","last_fullsized":"2024-09-17T05:25:57.614558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:43.793494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633080+0000","last_change":"2024-09-17T05:25:57.622620+0000","last_active":"2024-09-17T05:25:57.633080+0000","last_peered":"2024-09-17T05:25:57.633080+0000","last_clean":"2024-09-17T05:25:57.633080+0000","last_became_active":"2024-09-17T05:25:57.622264+0000","last_became_peered":"2024-09-17T05:25:57.622264+0000","last_unstale":"2024-09-17T05:25:57.633080+0000","last_undegraded":"2024-09-17T05:25:57.633080+0000","last_fullsized":"2024-09-17T05:25:57.633080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:15.569272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.600120+0000","last_change":"2024-09-17T05:25:56.612032+0000","last_active":"2024-09-17T05:25:57.600120+0000","last_peered":"2024-09-17T05:25:57.600120+0000","last_clean":"2024-09-17T05:25:57.600120+0000","last_became_active":"2024-09-17T05:25:56.611692+0000","last_became_peered":"2024-09-17T05:25:56.611692+0000","last_unstale":"2024-09-17T05:25:57.600120+0000","last_undegraded":"2024-09-17T05:25:57.600120+0000","last_fullsized":"2024-09-17T05:25:57.600120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:09.864874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626184+0000","last_change":"2024-09-17T05:25:57.618798+0000","last_active":"2024-09-17T05:25:57.626184+0000","last_peered":"2024-09-17T05:25:57.626184+0000","last_clean":"2024-09-17T05:25:57.626184+0000","last_became_active":"2024-09-17T05:25:57.615624+0000","last_became_peered":"2024-09-17T05:25:57.615624+0000","last_unstale":"2024-09-17T05:25:57.626184+0000","last_undegraded":"2024-09-17T05:25:57.626184+0000","last_fullsized":"2024-09-17T05:25:57.626184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:44.367481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610827+0000","last_change":"2024-09-17T05:25:56.622123+0000","last_active":"2024-09-17T05:25:58.610827+0000","last_peered":"2024-09-17T05:25:58.610827+0000","last_clean":"2024-09-17T05:25:58.610827+0000","last_became_active":"2024-09-17T05:25:56.621736+0000","last_became_peered":"2024-09-17T05:25:56.621736+0000","last_unstale":"2024-09-17T05:25:58.610827+0000","last_undegraded":"2024-09-17T05:25:58.610827+0000","last_fullsized":"2024-09-17T05:25:58.610827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:00.720790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.632333+0000","last_change":"2024-09-17T05:25:57.626610+0000","last_active":"2024-09-17T05:25:57.632333+0000","last_peered":"2024-09-17T05:25:57.632333+0000","last_clean":"2024-09-17T05:25:57.632333+0000","last_became_active":"2024-09-17T05:25:57.625895+0000","last_became_peered":"2024-09-17T05:25:57.625895+0000","last_unstale":"2024-09-17T05:25:57.632333+0000","last_undegraded":"2024-09-17T05:25:57.632333+0000","last_fullsized":"2024-09-17T05:25:57.632333+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:25.972778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.605529+0000","last_change":"2024-09-17T05:25:56.620365+0000","last_active":"2024-09-17T05:25:57.605529+0000","last_peered":"2024-09-17T05:25:57.605529+0000","last_clean":"2024-09-17T05:25:57.605529+0000","last_became_active":"2024-09-17T05:25:56.619802+0000","last_became_peered":"2024-09-17T05:25:56.619802+0000","last_unstale":"2024-09-17T05:25:57.605529+0000","last_undegraded":"2024-09-17T05:25:57.605529+0000","last_fullsized":"2024-09-17T05:25:57.605529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:48.498369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610141+0000","last_change":"2024-09-17T05:25:57.627811+0000","last_active":"2024-09-17T05:25:58.610141+0000","last_peered":"2024-09-17T05:25:58.610141+0000","last_clean":"2024-09-17T05:25:58.610141+0000","last_became_active":"2024-09-17T05:25:57.627460+0000","last_became_peered":"2024-09-17T05:25:57.627460+0000","last_unstale":"2024-09-17T05:25:58.610141+0000","last_undegraded":"2024-09-17T05:25:58.610141+0000","last_fullsized":"2024-09-17T05:25:58.610141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:31.804246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.610311+0000","last_change":"2024-09-17T05:25:56.620199+0000","last_active":"2024-09-17T05:25:58.610311+0000","last_peered":"2024-09-17T05:25:58.610311+0000","last_clean":"2024-09-17T05:25:58.610311+0000","last_became_active":"2024-09-17T05:25:56.618305+0000","last_became_peered":"2024-09-17T05:25:56.618305+0000","last_unstale":"2024-09-17T05:25:58.610311+0000","last_undegraded":"2024-09-17T05:25:58.610311+0000","last_fullsized":"2024-09-17T05:25:58.610311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:58.702741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.626709+0000","last_change":"2024-09-17T05:25:57.622968+0000","last_active":"2024-09-17T05:25:57.626709+0000","last_peered":"2024-09-17T05:25:57.626709+0000","last_clean":"2024-09-17T05:25:57.626709+0000","last_became_active":"2024-09-17T05:25:57.620376+0000","last_became_peered":"2024-09-17T05:25:57.620376+0000","last_unstale":"2024-09-17T05:25:57.626709+0000","last_undegraded":"2024-09-17T05:25:57.626709+0000","last_fullsized":"2024-09-17T05:25:57.626709+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:58.375489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.609469+0000","last_change":"2024-09-17T05:25:56.617986+0000","last_active":"2024-09-17T05:25:57.609469+0000","last_peered":"2024-09-17T05:25:57.609469+0000","last_clean":"2024-09-17T05:25:57.609469+0000","last_became_active":"2024-09-17T05:25:56.617362+0000","last_became_peered":"2024-09-17T05:25:56.617362+0000","last_unstale":"2024-09-17T05:25:57.609469+0000","last_undegraded":"2024-09-17T05:25:57.609469+0000","last_fullsized":"2024-09-17T05:25:57.609469+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.593668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.634536+0000","last_change":"2024-09-17T05:25:57.631959+0000","last_active":"2024-09-17T05:25:57.634536+0000","last_peered":"2024-09-17T05:25:57.634536+0000","last_clean":"2024-09-17T05:25:57.634536+0000","last_became_active":"2024-09-17T05:25:57.625636+0000","last_became_peered":"2024-09-17T05:25:57.625636+0000","last_unstale":"2024-09-17T05:25:57.634536+0000","last_undegraded":"2024-09-17T05:25:57.634536+0000","last_fullsized":"2024-09-17T05:25:57.634536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:03.789409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.604973+0000","last_change":"2024-09-17T05:25:56.623509+0000","last_active":"2024-09-17T05:25:57.604973+0000","last_peered":"2024-09-17T05:25:57.604973+0000","last_clean":"2024-09-17T05:25:57.604973+0000","last_became_active":"2024-09-17T05:25:56.623181+0000","last_became_peered":"2024-09-17T05:25:56.623181+0000","last_unstale":"2024-09-17T05:25:57.604973+0000","last_undegraded":"2024-09-17T05:25:57.604973+0000","last_fullsized":"2024-09-17T05:25:57.604973+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:10.448064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.633783+0000","last_change":"2024-09-17T05:25:57.628588+0000","last_active":"2024-09-17T05:25:57.633783+0000","last_peered":"2024-09-17T05:25:57.633783+0000","last_clean":"2024-09-17T05:25:57.633783+0000","last_became_active":"2024-09-17T05:25:57.628276+0000","last_became_peered":"2024-09-17T05:25:57.628276+0000","last_unstale":"2024-09-17T05:25:57.633783+0000","last_undegraded":"2024-09-17T05:25:57.633783+0000","last_fullsized":"2024-09-17T05:25:57.633783+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:56.583221+0000","last_clean_scrub_stamp":"2024-09-17T05:25:56.583221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:13.958788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T05:25:58.609538+0000","last_change":"2024-09-17T05:25:56.614603+0000","last_active":"2024-09-17T05:25:58.609538+0000","last_peered":"2024-09-17T05:25:58.609538+0000","last_clean":"2024-09-17T05:25:58.609538+0000","last_became_active":"2024-09-17T05:25:56.614263+0000","last_became_peered":"2024-09-17T05:25:56.614263+0000","last_unstale":"2024-09-17T05:25:58.609538+0000","last_undegraded":"2024-09-17T05:25:58.609538+0000","last_fullsized":"2024-09-17T05:25:58.609538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:55.580746+0000","last_clean_scrub_stamp":"2024-09-17T05:25:55.580746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:36.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:25:57.601494+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:25:57.601494+0000","last_peered":"2024-09-17T05:25:57.601494+0000","last_clean":"2024-09-17T05:25:57.601494+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:25:57.601494+0000","last_undegraded":"2024-09-17T05:25:57.601494+0000","last_fullsized":"2024-09-17T05:25:57.601494+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":284923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":875233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":285447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":284921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":708608,"data_stored":286603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":875781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":286443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":278056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:26:02.235 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:26:02.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 -- 172.21.15.43:0/3244314679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480b7ce0 msgr2=0x7fe6480d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 --2- 172.21.15.43:0/3244314679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480b7ce0 0x7fe6480d0840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe638005fd0 tx=0x7fe63801c740 comp rx=0 tx=0).stop 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 -- 172.21.15.43:0/3244314679 shutdown_connections 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 --2- 172.21.15.43:0/3244314679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480b7ce0 0x7fe6480d0840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 -- 172.21.15.43:0/3244314679 >> 172.21.15.43:0/3244314679 conn(0x7fe6480cee80 msgr2=0x7fe6480cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 -- 172.21.15.43:0/3244314679 shutdown_connections 2024-09-17T05:26:02.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.436+0000 7fe64de00640 1 -- 172.21.15.43:0/3244314679 wait complete. 2024-09-17T05:26:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 Processor -- start 2024-09-17T05:26:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- start start 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6480b7ce0 0x7fe6481566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 0x7fe6481f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe64816b030 con 0x7fe6480cd580 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe64816aeb0 con 0x7fe6480d0d80 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe64816b1b0 con 0x7fe6480b7ce0 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39952/0 (socket says 172.21.15.43:39952) 2024-09-17T05:26:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 -- 172.21.15.43:0/2658983430 learned_addr learned my addr 172.21.15.43:0/2658983430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 -- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6480b7ce0 msgr2=0x7fe6481566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6480b7ce0 0x7fe6481566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 -- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 msgr2=0x7fe6481f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe647400640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 0x7fe6481f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 0x7fe6481f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 -- 172.21.15.43:0/2658983430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6480b53d0 con 0x7fe6480cd580 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe647400640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 0x7fe6481f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63fe00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c3f0 tx=0x7fe63000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630017070 con 0x7fe6480cd580 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6300053c0 con 0x7fe6480cd580 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6480b56b0 con 0x7fe6480cd580 2024-09-17T05:26:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630003860 con 0x7fe6480cd580 2024-09-17T05:26:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64815c350 con 0x7fe6480cd580 2024-09-17T05:26:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.440+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6480d0840 con 0x7fe6480cd580 2024-09-17T05:26:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.444+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe63001a070 con 0x7fe6480cd580 2024-09-17T05:26:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.444+0000 7fe63f400640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 0x7fe624075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.444+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fe630093530 con 0x7fe6480cd580 2024-09-17T05:26:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.444+0000 7fe646a00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 0x7fe624075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.444+0000 7fe646a00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 0x7fe624075740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6380ae000 tx=0x7fe6380194e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.448+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe63005e550 con 0x7fe6480cd580 2024-09-17T05:26:02.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.684+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe648075f80 con 0x7fe6480cd580 2024-09-17T05:26:02.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe63f400640 1 -- 172.21.15.43:0/2658983430 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7fe63005de50 con 0x7fe6480cd580 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:26:02.693 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:26:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 msgr2=0x7fe624075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 0x7fe624075740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6380ae000 tx=0x7fe6380194e0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 msgr2=0x7fe648153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c3f0 tx=0x7fe63000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 shutdown_connections 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe624073230 0x7fe624075740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6480d0d80 0x7fe6481f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6480cd580 0x7fe648153aa0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 --2- 172.21.15.43:0/2658983430 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6480b7ce0 0x7fe6481566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.688+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 >> 172.21.15.43:0/2658983430 conn(0x7fe6480cee80 msgr2=0x7fe6480cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.692+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 shutdown_connections 2024-09-17T05:26:02.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.692+0000 7fe64de00640 1 -- 172.21.15.43:0/2658983430 wait complete. 2024-09-17T05:26:02.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 -- 172.21.15.43:0/1195587506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48e0003b40 msgr2=0x7f48e0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 --2- 172.21.15.43:0/1195587506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48e0003b40 0x7f48e0003fb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009ec0 tx=0x7f48ec01d5d0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 -- 172.21.15.43:0/1195587506 shutdown_connections 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 --2- 172.21.15.43:0/1195587506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48e0003b40 0x7f48e0003fb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 -- 172.21.15.43:0/1195587506 >> 172.21.15.43:0/1195587506 conn(0x7f48fc0b23d0 msgr2=0x7f48fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 -- 172.21.15.43:0/1195587506 shutdown_connections 2024-09-17T05:26:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.868+0000 7f4901400640 1 -- 172.21.15.43:0/1195587506 wait complete. 2024-09-17T05:26:02.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 Processor -- start 2024-09-17T05:26:02.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 -- start start 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48e0003b40 0x7f48fc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48fc1fce50 0x7f48fc1fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc06d490 con 0x7f48fc1492a0 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc06d310 con 0x7f48fc1fce50 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f4901400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48fc06d610 con 0x7f48e0003b40 2024-09-17T05:26:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f48f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f48f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39960/0 (socket says 172.21.15.43:39960) 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f48f1e00640 1 -- 172.21.15.43:0/2852830929 learned_addr learned my addr 172.21.15.43:0/2852830929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f48fa800640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48e0003b40 0x7f48fc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.872+0000 7f48fb200640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48fc1fce50 0x7f48fc1fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 -- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48e0003b40 msgr2=0x7f48fc099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48e0003b40 0x7f48fc099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 -- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48fc1fce50 msgr2=0x7f48fc1fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48fc1fce50 0x7f48fc1fd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 -- 172.21.15.43:0/2852830929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc15f550 con 0x7f48fc1492a0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f1e00640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f48e4003e50 tx=0x7f48e4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4015070 con 0x7f48fc1492a0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e4011840 con 0x7f48fc1492a0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48fc1e82c0 con 0x7f48fc1492a0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011c90 con 0x7f48fc1492a0 2024-09-17T05:26:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48fc1477f0 con 0x7f48fc1492a0 2024-09-17T05:26:02.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48fc1e87e0 con 0x7f48fc1492a0 2024-09-17T05:26:02.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48e4018070 con 0x7f48fc1492a0 2024-09-17T05:26:02.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.876+0000 7f48f3400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 0x7f48d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.880+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f48e4094a90 con 0x7f48fc1492a0 2024-09-17T05:26:02.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.880+0000 7f48fa800640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 0x7f48d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.880+0000 7f48fa800640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 0x7f48d8075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ae000 tx=0x7f48ec01a370 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:02.884+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48fc1e87e0 con 0x7f48fc1492a0 2024-09-17T05:26:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48fc1fd7a0 con 0x7f48fc1492a0 2024-09-17T05:26:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f48f3400640 1 -- 172.21.15.43:0/2852830929 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5214 (secure 0 0 0) 0x7f48e405f440 con 0x7f48fc1492a0 2024-09-17T05:26:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:03.100 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"btime":"2024-09-17T05:26:00:616105+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:00.035083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5838},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5838":{"gid":5838,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2376863639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2376863639},{"type":"v1","addr":"172.21.15.43:6834","nonce":2376863639}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5838,"qdb_cluster":[5838]},"id":4}]} 2024-09-17T05:26:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 46 2024-09-17T05:26:03.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 msgr2=0x7f48d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 0x7f48d8075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ae000 tx=0x7f48ec01a370 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 msgr2=0x7f48fc1fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f48e4003e50 tx=0x7f48e4001640 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 shutdown_connections 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48d8073200 0x7f48d8075710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48fc1fce50 0x7f48fc1fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48fc1492a0 0x7f48fc1fc910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 --2- 172.21.15.43:0/2852830929 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48e0003b40 0x7f48fc099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 >> 172.21.15.43:0/2852830929 conn(0x7f48fc0b23d0 msgr2=0x7f48fc158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 shutdown_connections 2024-09-17T05:26:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.096+0000 7f4901400640 1 -- 172.21.15.43:0/2852830929 wait complete. 2024-09-17T05:26:03.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:00.035083+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5838}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5838': {'gid': 5838, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/2376863639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 2376863639}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 2376863639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5838, 'qdb_cluster': [5838]} 2024-09-17T05:26:03.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:26:03.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:26:03.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 -- 172.21.15.43:0/1234000997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4415fe10 msgr2=0x7f5d440b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 --2- 172.21.15.43:0/1234000997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4415fe10 0x7f5d440b4b80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009870 tx=0x7f5d3401c700 comp rx=0 tx=0).stop 2024-09-17T05:26:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 -- 172.21.15.43:0/1234000997 shutdown_connections 2024-09-17T05:26:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 --2- 172.21.15.43:0/1234000997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4415fe10 0x7f5d440b4b80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 -- 172.21.15.43:0/1234000997 >> 172.21.15.43:0/1234000997 conn(0x7f5d440b23d0 msgr2=0x7f5d440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 -- 172.21.15.43:0/1234000997 shutdown_connections 2024-09-17T05:26:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.684+0000 7f5d49a00640 1 -- 172.21.15.43:0/1234000997 wait complete. 2024-09-17T05:26:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 Processor -- start 2024-09-17T05:26:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 -- start start 2024-09-17T05:26:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d4415fe10 0x7f5d4415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d4415ba00 0x7f5d440b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4416aff0 con 0x7f5d4406fc90 2024-09-17T05:26:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4416ae70 con 0x7f5d4415ba00 2024-09-17T05:26:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d49a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4416b170 con 0x7f5d4415fe10 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39982/0 (socket says 172.21.15.43:39982) 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 -- 172.21.15.43:0/845132104 learned_addr learned my addr 172.21.15.43:0/845132104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d43400640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d4415ba00 0x7f5d440b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42000640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d4415fe10 0x7f5d4415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 -- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d4415fe10 msgr2=0x7f5d4415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d4415fe10 0x7f5d4415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 -- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d4415ba00 msgr2=0x7f5d440b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d4415ba00 0x7f5d440b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.688+0000 7f5d42a00640 1 -- 172.21.15.43:0/845132104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440981b0 con 0x7f5d4406fc90 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d42a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009790 tx=0x7f5d340a67a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 -- 172.21.15.43:0/845132104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340bb020 con 0x7f5d4406fc90 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 -- 172.21.15.43:0/845132104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d34004370 con 0x7f5d4406fc90 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 -- 172.21.15.43:0/845132104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34005290 con 0x7f5d4406fc90 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d44098490 con 0x7f5d4406fc90 2024-09-17T05:26:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d44098a50 con 0x7f5d4406fc90 2024-09-17T05:26:03.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 -- 172.21.15.43:0/845132104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d340be020 con 0x7f5d4406fc90 2024-09-17T05:26:03.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 0x7f5d24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d3aa00640 1 -- 172.21.15.43:0/845132104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3412e660 con 0x7f5d4406fc90 2024-09-17T05:26:03.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.692+0000 7f5d42000640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 0x7f5d24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 Processor -- start 2024-09-17T05:26:03.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 -- start start 2024-09-17T05:26:03.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d42000640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 0x7f5d24075710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d300046c0 tx=0x7f5d30007ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b7185a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718450950 0x55b7185a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b718476410 con 0x55b718439ca0 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b7183b3b40 con 0x55b718450950 2024-09-17T05:26:03.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b7183a2f70 con 0x55b718060460 2024-09-17T05:26:03.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38992/0 (socket says 172.21.15.43:38992) 2024-09-17T05:26:03.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 -- 172.21.15.43:0/1784858537 learned_addr learned my addr 172.21.15.43:0/1784858537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d42a00640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b7185a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d42000640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718450950 0x55b7185a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 -- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 msgr2=0x55b7185a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b7185a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 -- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718450950 msgr2=0x55b7185a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718450950 0x55b7185a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 -- 172.21.15.43:0/1784858537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71836a5b0 con 0x55b718439ca0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d42a00640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b7185a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d43400640 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c001ee0 tx=0x7f5d3c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7eb3481c61c29e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d2f400640 1 -- 172.21.15.43:0/1784858537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00fe50 con 0x55b718439ca0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d2f400640 1 -- 172.21.15.43:0/1784858537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d3c018070 con 0x55b718439ca0 2024-09-17T05:26:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.696+0000 7f5d2f400640 1 -- 172.21.15.43:0/1784858537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003be0 con 0x55b718439ca0 2024-09-17T05:26:03.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- 172.21.15.43:0/1784858537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 msgr2=0x55b7185a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c001ee0 tx=0x7f5d3c001640 comp rx=0 tx=0).stop 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- 172.21.15.43:0/1784858537 shutdown_connections 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718450950 0x55b7185a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185a6580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1784858537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b7185a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- 172.21.15.43:0/1784858537 >> 172.21.15.43:0/1784858537 conn(0x55b718526d90 msgr2=0x55b718556690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- 172.21.15.43:0/1784858537 shutdown_connections 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- 172.21.15.43:0/1784858537 wait complete. 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- start start 2024-09-17T05:26:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 Processor -- start 2024-09-17T05:26:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b718594120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718439ca0 0x55b718594660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b71833d5f0 con 0x55b718450950 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b7183a6d80 con 0x55b718439ca0 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b718349520 con 0x55b718060460 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d511f7000 10 client.0 _ll_register_callbacks cb 0x55b71834d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38994/0 (socket says 172.21.15.43:38994) 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 -- 172.21.15.43:0/4160765593 learned_addr learned my addr 172.21.15.43:0/4160765593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d42a00640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b718594120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d42000640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718439ca0 0x55b718594660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 -- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 msgr2=0x55b718594120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b718594120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 -- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718439ca0 msgr2=0x55b718594660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718439ca0 0x55b718594660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d43400640 1 -- 172.21.15.43:0/4160765593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718341190 con 0x55b718450950 2024-09-17T05:26:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.700+0000 7f5d42a00640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b718594120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:03.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d43400640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c014000 tx=0x7f5d3c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce05e52e9207ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:03.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c006d60 con 0x55b718450950 2024-09-17T05:26:03.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b71833b110 con 0x55b718450950 2024-09-17T05:26:03.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c0052e0 con 0x55b718450950 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5d3c02c050 con 0x55b718450950 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 10 client.5973 fetch_fsmap learned FSMap version 46 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55b718427f10 con 0x55b718450950 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b71833b110 con 0x55b718450950 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f5d3c02f020 con 0x55b718450950 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 10 client.5973 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 10 client.5973 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 20 client.5973 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b7180457d0 0x55b7185955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 4 client.5973 mds_command: new command op to 5838 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 --> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7183b5a90 con 0x55b7180457d0 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d42a00640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b7180457d0 0x55b7185955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d42a00640 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b7180457d0 0x55b7185955b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6db72c88768b60 server_cookie=65521126c88158cd in_seq=0 out_seq=0 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 10 client.5973 ms_handle_connect on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 1 -- 172.21.15.43:0/4160765593 <== mds.0 v2:172.21.15.43:6832/2376863639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7183b5a90 con 0x55b7180457d0 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 10 client.5973 handle_command_reply: tid=0 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 client.5973 shutdown 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 msgr2=0x55b718594ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c014000 tx=0x7f5d3c003f70 comp rx=0 tx=0).stop 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 shutdown_connections 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b7180457d0 0x55b7185955b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718439ca0 0x55b718594660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718450950 0x55b718594ba0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 --2- 172.21.15.43:0/4160765593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718060460 0x55b718594120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d23e00640 0 client.5973 ms_handle_reset on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 >> 172.21.15.43:0/4160765593 conn(0x55b718526d90 msgr2=0x55b71855e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 shutdown_connections 2024-09-17T05:26:03.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 1 -- 172.21.15.43:0/4160765593 wait complete. 2024-09-17T05:26:03.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.704+0000 7f5d511f7000 20 client.5973 trim_cache size 0 max 16384 2024-09-17T05:26:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 Processor -- start 2024-09-17T05:26:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 -- start start 2024-09-17T05:26:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b718563190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b7185bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44201060 con 0x55b718439ca0 2024-09-17T05:26:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440cc510 con 0x55b718060460 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4415aee0 con 0x55b718450950 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.724+0000 7f5d42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39004/0 (socket says 172.21.15.43:39004) 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 -- 172.21.15.43:0/402731000 learned_addr learned my addr 172.21.15.43:0/402731000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42a00640 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b718563190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d43400640 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b7185bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 -- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 msgr2=0x55b7185bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b7185bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 -- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 msgr2=0x55b718563190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b718563190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 -- 172.21.15.43:0/402731000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7183fedb0 con 0x55b718439ca0 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d42000640 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000fd00 tx=0x7f5d30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b68fc4205a30d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d2ea00640 1 -- 172.21.15.43:0/402731000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001b040 con 0x55b718439ca0 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d2ea00640 1 -- 172.21.15.43:0/402731000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d3001c040 con 0x55b718439ca0 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d2ea00640 1 -- 172.21.15.43:0/402731000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30023680 con 0x55b718439ca0 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- 172.21.15.43:0/402731000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 msgr2=0x55b7185636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000fd00 tx=0x7f5d30012000 comp rx=0 tx=0).stop 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- 172.21.15.43:0/402731000 shutdown_connections 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b7185bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b7185636d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 --2- 172.21.15.43:0/402731000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b718563190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- 172.21.15.43:0/402731000 >> 172.21.15.43:0/402731000 conn(0x55b718526d90 msgr2=0x55b718562a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- 172.21.15.43:0/402731000 shutdown_connections 2024-09-17T05:26:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- 172.21.15.43:0/402731000 wait complete. 2024-09-17T05:26:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 -- start start 2024-09-17T05:26:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.728+0000 7f5d511f7000 1 Processor -- start 2024-09-17T05:26:03.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b71861b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b71861c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b71836eee0 con 0x55b718439ca0 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440cbec0 con 0x55b718060460 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b71833d5f0 con 0x55b718450950 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 10 client.0 _ll_register_callbacks cb 0x7f5d30001230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39016/0 (socket says 172.21.15.43:39016) 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 -- 172.21.15.43:0/1063724220 learned_addr learned my addr 172.21.15.43:0/1063724220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42a00640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b71861b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d43400640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b71861c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 -- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 msgr2=0x55b71861c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b71861c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 -- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 msgr2=0x55b71861b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b71861b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 -- 172.21.15.43:0/1063724220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718369ab0 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42000640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5d30001ec0 tx=0x7f5d300234c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f9c068afaa775f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001b050 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5d30023ac0 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3001c040 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5d30038050 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 10 client.5979 fetch_fsmap learned FSMap version 46 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55b718348070 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d3001c040 con 0x55b718439ca0 2024-09-17T05:26:03.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f5d3003b020 con 0x55b718439ca0 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 10 client.5979 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 10 client.5979 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 20 client.5979 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b718033830 0x55b7185b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 4 client.5979 mds_command: new command op to 5838 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 --> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] -- command(tid 0: {"prefix": "status"}) -- 0x55b7183b1540 con 0x55b718033830 2024-09-17T05:26:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42a00640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b718033830 0x55b7185b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d42a00640 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b718033830 0x55b7185b8be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acde8f97dbd515b server_cookie=f6f608bf499c5550 in_seq=0 out_seq=0 2024-09-17T05:26:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 10 client.5979 ms_handle_connect on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 1 -- 172.21.15.43:0/1063724220 <== mds.0 v2:172.21.15.43:6832/2376863639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b7183b1540 con 0x55b718033830 2024-09-17T05:26:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d23e00640 10 client.5979 handle_command_reply: tid=0 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 client.5979 shutdown 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 msgr2=0x55b71861bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.732+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5d30001ec0 tx=0x7f5d300234c0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 shutdown_connections 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x55b718033830 0x55b7185b8be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b718450950 0x55b71861c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718439ca0 0x55b71861bc10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 --2- 172.21.15.43:0/1063724220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b718060460 0x55b71861b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d23e00640 0 client.5979 ms_handle_reset on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 >> 172.21.15.43:0/1063724220 conn(0x55b718526d90 msgr2=0x55b718563440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 shutdown_connections 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 1 -- 172.21.15.43:0/1063724220 wait complete. 2024-09-17T05:26:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d511f7000 20 client.5979 trim_cache size 0 max 16384 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5838, 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.1460141009999996, 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 46, 2024-09-17T05:26:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 59, 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 59, 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.510814625 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 msgr2=0x7f5d24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 0x7f5d24075710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d300046c0 tx=0x7f5d30007ea0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 msgr2=0x7f5d44099250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009790 tx=0x7f5d340a67a0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 shutdown_connections 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d24073200 0x7f5d24075710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d4415ba00 0x7f5d440b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d4415fe10 0x7f5d4415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 --2- 172.21.15.43:0/845132104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d4406fc90 0x7f5d44099250 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 >> 172.21.15.43:0/845132104 conn(0x7f5d440b23d0 msgr2=0x7f5d440cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 shutdown_connections 2024-09-17T05:26:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.736+0000 7f5d49a00640 1 -- 172.21.15.43:0/845132104 wait complete. 2024-09-17T05:26:03.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:26:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- 172.21.15.43:0/2578213897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84280b3af0 msgr2=0x7f842815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 --2- 172.21.15.43:0/2578213897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84280b3af0 0x7f842815c7e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8410009840 tx=0x7f841001c790 comp rx=0 tx=0).stop 2024-09-17T05:26:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- 172.21.15.43:0/2578213897 shutdown_connections 2024-09-17T05:26:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 --2- 172.21.15.43:0/2578213897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84280b3af0 0x7f842815c7e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- 172.21.15.43:0/2578213897 >> 172.21.15.43:0/2578213897 conn(0x7f84280b23d0 msgr2=0x7f84280d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- 172.21.15.43:0/2578213897 shutdown_connections 2024-09-17T05:26:03.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- 172.21.15.43:0/2578213897 wait complete. 2024-09-17T05:26:03.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 Processor -- start 2024-09-17T05:26:03.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.916+0000 7f842d000640 1 -- start start 2024-09-17T05:26:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8428155d70 0x7f8428159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 0x7f842815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8428169da0 con 0x7f842806f110 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8428169c20 con 0x7f842815a230 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8428169f20 con 0x7f8428155d70 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39024/0 (socket says 172.21.15.43:39024) 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 -- 172.21.15.43:0/3858524066 learned_addr learned my addr 172.21.15.43:0/3858524066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f841d400640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8428155d70 0x7f8428159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8426800640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 0x7f842815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 -- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8428155d70 msgr2=0x7f8428159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8428155d70 0x7f8428159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 -- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 msgr2=0x7f842815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 0x7f842815a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 -- 172.21.15.43:0/3858524066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842815ab80 con 0x7f842806f110 2024-09-17T05:26:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8426800640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 0x7f842815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f8425e00640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f841800b3e0 tx=0x7f841800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418015020 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8418011920 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842808e0c0 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418011d70 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842808e680 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.920+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842815c7e0 con 0x7f842806f110 2024-09-17T05:26:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.924+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8418018020 con 0x7f842806f110 2024-09-17T05:26:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.924+0000 7f841ea00640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 0x7f8404075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.924+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f8418093df0 con 0x7f842806f110 2024-09-17T05:26:03.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.924+0000 7f841d400640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 0x7f8404075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.924+0000 7f841d400640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 0x7f8404075740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f84100af000 tx=0x7f841001afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:03.928+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f841801f030 con 0x7f842806f110 2024-09-17T05:26:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84280b00c0 con 0x7f842806f110 2024-09-17T05:26:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f841ea00640 1 -- 172.21.15.43:0/3858524066 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f842815ab80 con 0x7f842806f110 2024-09-17T05:26:04.148 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 msgr2=0x7f8404075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 0x7f8404075740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f84100af000 tx=0x7f841001afd0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 msgr2=0x7f84280c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f841800b3e0 tx=0x7f841800b8a0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 shutdown_connections 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8404073230 0x7f8404075740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f842815a230 0x7f842815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8428155d70 0x7f8428159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 --2- 172.21.15.43:0/3858524066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842806f110 0x7f84280c9ce0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 >> 172.21.15.43:0/3858524066 conn(0x7f84280b23d0 msgr2=0x7f842815d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 shutdown_connections 2024-09-17T05:26:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04.144+0000 7f842d000640 1 -- 172.21.15.43:0/3858524066 wait complete. 2024-09-17T05:26:04.170 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:26:04.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:26:04.170 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:26:04.170 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:26:04.171 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:04.171 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:26:04.171 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:26:04.171 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:26:04.171 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:04.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.171 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:26:04.182 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42280sec preferred_lft 42280sec 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:26:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:26:04.184 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.185 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:26:04.185 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:26:04.185 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:26:04.185 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:04.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.185 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:26:04.251 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:26:04.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.254 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:26:04.313 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:26:04.313 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:26:04.314 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:26:04.314 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:26:04.314 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:26:04.314 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:26:04.314 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:26:04.315 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:26:04.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:26:04.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:26:04.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:26:04.315 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:26:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:26:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:04.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.455 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:26:04.465 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:26:04.465 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:26:04.466 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:04.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:26:04.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:26:04.527 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:04.582 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:04.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:26:04.583 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:04.640 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:26:04.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:04.749 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:04.749 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:04.749 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:04.768 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:04.768 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:04.768 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:04.768 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:26:04.768 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:26:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:26:04.780 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:26:04.783 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:26:04.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:04.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:04.803 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:26:04.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:26:04.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:04.910 INFO:teuthology.orchestra.run.smithi043.stdout:client5498 2024-09-17T05:26:04.911 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:26:04.911 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:04.920 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-09-17T05:26:05.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:26:05.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:05.034 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:26:05.034 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:05.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:26:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 -- 172.21.15.43:0/1978946725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 msgr2=0x7f34440734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 --2- 172.21.15.43:0/1978946725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440734c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f343c009c80 tx=0x7f343c01cc80 comp rx=0 tx=0).stop 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 -- 172.21.15.43:0/1978946725 shutdown_connections 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 --2- 172.21.15.43:0/1978946725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440734c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 --2- 172.21.15.43:0/1978946725 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 0x7f34440700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 -- 172.21.15.43:0/1978946725 >> 172.21.15.43:0/1978946725 conn(0x7f34440b23d0 msgr2=0x7f34440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 -- 172.21.15.43:0/1978946725 shutdown_connections 2024-09-17T05:26:05.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.244+0000 7f344c600640 1 -- 172.21.15.43:0/1978946725 wait complete. 2024-09-17T05:26:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 Processor -- start 2024-09-17T05:26:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- start start 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 0x7f3444074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 0x7f344415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3444169fc0 con 0x7f34440730b0 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3444169e40 con 0x7f344415fe10 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f344416a140 con 0x7f344406fc90 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39052/0 (socket says 172.21.15.43:39052) 2024-09-17T05:26:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 -- 172.21.15.43:0/3041590582 learned_addr learned my addr 172.21.15.43:0/3041590582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449c00640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 0x7f3444074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344a600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 0x7f344415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 -- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 msgr2=0x7f3444074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 0x7f3444074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 -- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 msgr2=0x7f344415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 0x7f344415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 -- 172.21.15.43:0/3041590582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3444099430 con 0x7f34440730b0 2024-09-17T05:26:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344a600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 0x7f344415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3449200640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f343000d720 tx=0x7f343000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430017070 con 0x7f34440730b0 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3430009d20 con 0x7f34440730b0 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3444074840 con 0x7f34440730b0 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430005280 con 0x7f34440730b0 2024-09-17T05:26:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.248+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3444099d30 con 0x7f34440730b0 2024-09-17T05:26:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.252+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34440700a0 con 0x7f34440730b0 2024-09-17T05:26:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.252+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3430005420 con 0x7f34440730b0 2024-09-17T05:26:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.252+0000 7f3441600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.252+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5767+0+0 (secure 0 0 0) 0x7f3430094250 con 0x7f34440730b0 2024-09-17T05:26:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.252+0000 7f3449c00640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.256+0000 7f3449c00640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 0x7f3424075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3438007fd0 tx=0x7f3438009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.256+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f343005ec70 con 0x7f34440730b0 2024-09-17T05:26:05.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34440b1510 con 0x7f34440730b0 2024-09-17T05:26:05.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f3441600640 1 -- 172.21.15.43:0/3041590582 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) ==== 80+0+0 (secure 0 0 0) 0x7f34300636a0 con 0x7f34440730b0 2024-09-17T05:26:05.448 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 msgr2=0x7f3424075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 0x7f3424075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3438007fd0 tx=0x7f3438009040 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 msgr2=0x7f34440989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f343000d720 tx=0x7f343000dbe0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 shutdown_connections 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f344415fe10 0x7f344415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34440730b0 0x7f34440989b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 --2- 172.21.15.43:0/3041590582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f344406fc90 0x7f3444074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 >> 172.21.15.43:0/3041590582 conn(0x7f34440b23d0 msgr2=0x7f34440aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 shutdown_connections 2024-09-17T05:26:05.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.444+0000 7f344c600640 1 -- 172.21.15.43:0/3041590582 wait complete. 2024-09-17T05:26:05.471 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:26:05.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:05.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:05.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:26:05.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:05.501 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:26:05.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-17T05:26:05.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- 172.21.15.43:0/181474269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff41c003b40 msgr2=0x7ff41c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 --2- 172.21.15.43:0/181474269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff41c003b40 0x7ff41c003fb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff428009b40 tx=0x7ff42801d220 comp rx=0 tx=0).stop 2024-09-17T05:26:05.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- 172.21.15.43:0/181474269 shutdown_connections 2024-09-17T05:26:05.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 --2- 172.21.15.43:0/181474269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff41c003b40 0x7ff41c003fb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- 172.21.15.43:0/181474269 >> 172.21.15.43:0/181474269 conn(0x7ff4380b2de0 msgr2=0x7ff4380b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- 172.21.15.43:0/181474269 shutdown_connections 2024-09-17T05:26:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- 172.21.15.43:0/181474269 wait complete. 2024-09-17T05:26:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 Processor -- start 2024-09-17T05:26:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.760+0000 7ff43f200640 1 -- start start 2024-09-17T05:26:05.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff41c003b40 0x7ff438155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4380c9740 0x7ff4381590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380ce680 con 0x7ff438159620 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380ce500 con 0x7ff4380c9740 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380ce800 con 0x7ff41c003b40 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39070/0 (socket says 172.21.15.43:39070) 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 -- 172.21.15.43:0/2118798615 learned_addr learned my addr 172.21.15.43:0/2118798615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff42f400640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4380c9740 0x7ff4381590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff437e00640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff41c003b40 0x7ff438155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 -- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff41c003b40 msgr2=0x7ff438155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff41c003b40 0x7ff438155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 -- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4380c9740 msgr2=0x7ff4381590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4380c9740 0x7ff4381590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 -- 172.21.15.43:0/2118798615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438156010 con 0x7ff438159620 2024-09-17T05:26:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43d200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cc920 tx=0x7ff4300019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff434c00640 1 -- 172.21.15.43:0/2118798615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430011870 con 0x7ff438159620 2024-09-17T05:26:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff434c00640 1 -- 172.21.15.43:0/2118798615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff430017070 con 0x7ff438159620 2024-09-17T05:26:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381f70d0 con 0x7ff438159620 2024-09-17T05:26:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff434c00640 1 -- 172.21.15.43:0/2118798615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430011da0 con 0x7ff438159620 2024-09-17T05:26:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.764+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381f7690 con 0x7ff438159620 2024-09-17T05:26:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff434c00640 1 -- 172.21.15.43:0/2118798615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff43001c460 con 0x7ff438159620 2024-09-17T05:26:05.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff434c00640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 0x7ff414075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff434c00640 1 -- 172.21.15.43:0/2118798615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7ff430094900 con 0x7ff438159620 2024-09-17T05:26:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff437e00640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 0x7ff414075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 Processor -- start 2024-09-17T05:26:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 -- start start 2024-09-17T05:26:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff437e00640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 0x7ff414075740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ad000 tx=0x7ff428019fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f5586d0 0x563f8f6c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f273aa0 0x563f8f6c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563f8f563cd0 con 0x563f8f1af890 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563f8f479330 con 0x563f8f5586d0 2024-09-17T05:26:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563f8f4d6bb0 con 0x563f8f273aa0 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.768+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39082/0 (socket says 172.21.15.43:39082) 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 -- 172.21.15.43:0/3797076832 learned_addr learned my addr 172.21.15.43:0/3797076832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff43d200640 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f273aa0 0x563f8f6c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff437e00640 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f5586d0 0x563f8f6c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 -- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f273aa0 msgr2=0x563f8f6c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f273aa0 0x563f8f6c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 -- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f5586d0 msgr2=0x563f8f6c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f5586d0 0x563f8f6c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 -- 172.21.15.43:0/3797076832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8f47ac80 con 0x563f8f1af890 2024-09-17T05:26:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42f400640 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff4200063d0 tx=0x7ff420006c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5bbc1fcdcc2463 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42cc00640 1 -- 172.21.15.43:0/3797076832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004b90 con 0x563f8f1af890 2024-09-17T05:26:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42cc00640 1 -- 172.21.15.43:0/3797076832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420016070 con 0x563f8f1af890 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff42cc00640 1 -- 172.21.15.43:0/3797076832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200126a0 con 0x563f8f1af890 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- 172.21.15.43:0/3797076832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 msgr2=0x563f8f6c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff4200063d0 tx=0x7ff420006c20 comp rx=0 tx=0).stop 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- 172.21.15.43:0/3797076832 shutdown_connections 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f273aa0 0x563f8f6c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f1af890 0x563f8f6c9530 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 --2- 172.21.15.43:0/3797076832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f5586d0 0x563f8f6c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- 172.21.15.43:0/3797076832 >> 172.21.15.43:0/3797076832 conn(0x563f8f1835c0 msgr2=0x563f8f681ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- 172.21.15.43:0/3797076832 shutdown_connections 2024-09-17T05:26:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- 172.21.15.43:0/3797076832 wait complete. 2024-09-17T05:26:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 -- start start 2024-09-17T05:26:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.772+0000 7ff446a85000 1 Processor -- start 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f68b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563f8f4c9df0 con 0x563f8f273aa0 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563f8f4c5fe0 con 0x563f8f1af890 2024-09-17T05:26:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563f8f46b4d0 con 0x563f8f5586d0 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 10 client.0 _ll_register_callbacks cb 0x563f8f4497f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:57620/0 (socket says 172.21.15.43:57620) 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 -- 172.21.15.43:0/2142788134 learned_addr learned my addr 172.21.15.43:0/2142788134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39084/0 (socket says 172.21.15.43:39084) 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff43d200640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f68b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 -- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 msgr2=0x563f8f68b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f68b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 -- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 msgr2=0x563f8f68b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 -- 172.21.15.43:0/2142788134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8f464210 con 0x563f8f1af890 2024-09-17T05:26:05.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff42f400640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff437e00640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff428098c10 tx=0x7ff4280091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3123306435efde9b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-09-17T05:26:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428004a50 con 0x563f8f1af890 2024-09-17T05:26:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7ff428004bf0 con 0x563f8f1af890 2024-09-17T05:26:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f8f45e350 con 0x563f8f1af890 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff428004df0 con 0x563f8f1af890 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 10 client.5827 fetch_fsmap learned FSMap version 46 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x563f8f44afc0 con 0x563f8f1af890 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563f8f45e350 con 0x563f8f1af890 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7ff4280c5520 con 0x563f8f1af890 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 10 client.5827 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:26:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 10 client.5827 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 20 client.5827 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f168930 0x563f8f6da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 4 client.5827 mds_command: new command op to 5838 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 --> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8f4d8b00 con 0x563f8f168930 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.776+0000 7ff42f400640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f168930 0x563f8f6da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff42f400640 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f168930 0x563f8f6da5b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb78f59396cbc28 server_cookie=3dfb2cf972a241b in_seq=0 out_seq=0 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff425600640 10 client.5827 ms_handle_connect on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff425600640 1 -- 172.21.15.43:0/2142788134 <== mds.0 v2:172.21.15.43:6832/2376863639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f8f4d8b00 con 0x563f8f168930 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff425600640 10 client.5827 handle_command_reply: tid=0 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 client.5827 shutdown 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 msgr2=0x563f8f68ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff428098c10 tx=0x7ff4280091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 shutdown_connections 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f168930 0x563f8f6da5b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f68b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f68b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 --2- 172.21.15.43:0/2142788134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f68ac70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff425600640 0 client.5827 ms_handle_reset on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 >> 172.21.15.43:0/2142788134 conn(0x563f8f1835c0 msgr2=0x563f8f679360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 shutdown_connections 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 1 -- 172.21.15.43:0/2142788134 wait complete. 2024-09-17T05:26:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.780+0000 7ff446a85000 20 client.5827 trim_cache size 0 max 16384 2024-09-17T05:26:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 Processor -- start 2024-09-17T05:26:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 -- start start 2024-09-17T05:26:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f67a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f6798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563f8f688d40 con 0x563f8f273aa0 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff438201b60 con 0x563f8f1af890 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380d0760 con 0x563f8f5586d0 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39086/0 (socket says 172.21.15.43:39086) 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 -- 172.21.15.43:0/111702984 learned_addr learned my addr 172.21.15.43:0/111702984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff43d200640 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f6798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff437e00640 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f67a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 -- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 msgr2=0x563f8f6798a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f6798a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 -- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 msgr2=0x563f8f67a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f67a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.796+0000 7ff42f400640 1 -- 172.21.15.43:0/111702984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8f523680 con 0x563f8f273aa0 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff42f400640 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c10 tx=0x7ff420014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa543b3f76c7935 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff427e00640 1 -- 172.21.15.43:0/111702984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420004d10 con 0x563f8f273aa0 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff427e00640 1 -- 172.21.15.43:0/111702984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420016040 con 0x563f8f273aa0 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff427e00640 1 -- 172.21.15.43:0/111702984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42001b6c0 con 0x563f8f273aa0 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- 172.21.15.43:0/111702984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 msgr2=0x563f8f679360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c10 tx=0x7ff420014040 comp rx=0 tx=0).stop 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- 172.21.15.43:0/111702984 shutdown_connections 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f6798a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f679360 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- 172.21.15.43:0/111702984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f67a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- 172.21.15.43:0/111702984 >> 172.21.15.43:0/111702984 conn(0x563f8f1835c0 msgr2=0x563f8f68a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- 172.21.15.43:0/111702984 shutdown_connections 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- 172.21.15.43:0/111702984 wait complete. 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- start start 2024-09-17T05:26:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 Processor -- start 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f73e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f73f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380d0110 con 0x563f8f273aa0 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381fad80 con 0x563f8f1af890 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.800+0000 7ff446a85000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563f8f18b680 con 0x563f8f5586d0 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 10 client.0 _ll_register_callbacks cb 0x7ff41c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff437e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f73e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39090/0 (socket says 172.21.15.43:39090) 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 -- 172.21.15.43:0/2990580268 learned_addr learned my addr 172.21.15.43:0/2990580268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff43d200640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f73f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 -- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 msgr2=0x563f8f73f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f73f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 -- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 msgr2=0x563f8f73e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f73e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 -- 172.21.15.43:0/2990580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8f4900b0 con 0x563f8f273aa0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff42f400640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c10 tx=0x7ff420001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd004c4989231ced server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420037070 con 0x563f8f273aa0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7ff420030480 con 0x563f8f273aa0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff420016040 con 0x563f8f273aa0 2024-09-17T05:26:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff42003f050 con 0x563f8f273aa0 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 10 client.6012 fetch_fsmap learned FSMap version 46 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x563f8f45c610 con 0x563f8f273aa0 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420016040 con 0x563f8f273aa0 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7ff420042020 con 0x563f8f273aa0 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 10 client.6012 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 10 client.6012 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 20 client.6012 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f156990 0x563f8f6dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 4 client.6012 mds_command: new command op to 5838 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 --> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f8f4d45b0 con 0x563f8f156990 2024-09-17T05:26:05.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff437e00640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f156990 0x563f8f6dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:05.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff437e00640 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f156990 0x563f8f6dbab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515834e32690cca7 server_cookie=6a5fa775b2593dd2 in_seq=0 out_seq=0 2024-09-17T05:26:05.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.804+0000 7ff425600640 10 client.6012 ms_handle_connect on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:05.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff425600640 1 -- 172.21.15.43:0/2990580268 <== mds.0 v2:172.21.15.43:6832/2376863639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f8f4d45b0 con 0x563f8f156990 2024-09-17T05:26:05.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff425600640 10 client.6012 handle_command_reply: tid=0 2024-09-17T05:26:05.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 client.6012 shutdown 2024-09-17T05:26:05.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 msgr2=0x563f8f73ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c10 tx=0x7ff420001e50 comp rx=0 tx=0).stop 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 shutdown_connections 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:6832/2376863639,v1:172.21.15.43:6834/2376863639] conn(0x563f8f156990 0x563f8f6dbab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563f8f5586d0 0x563f8f73f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563f8f273aa0 0x563f8f73ec70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff425600640 0 client.6012 ms_handle_reset on v2:172.21.15.43:6832/2376863639 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 --2- 172.21.15.43:0/2990580268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563f8f1af890 0x563f8f73e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 >> 172.21.15.43:0/2990580268 conn(0x563f8f1835c0 msgr2=0x563f8f680e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 shutdown_connections 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 1 -- 172.21.15.43:0/2990580268 wait complete. 2024-09-17T05:26:05.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff446a85000 20 client.6012 trim_cache size 0 max 16384 2024-09-17T05:26:05.826 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:26:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:26:05.826 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:26:05.826 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 msgr2=0x7ff414075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 0x7ff414075740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ad000 tx=0x7ff428019fc0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 msgr2=0x7ff4380c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff4380cc920 tx=0x7ff4300019c0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 shutdown_connections 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff414073230 0x7ff414075740 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4380c9740 0x7ff4381590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff438159620 0x7ff4380c64b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 --2- 172.21.15.43:0/2118798615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff41c003b40 0x7ff438155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 >> 172.21.15.43:0/2118798615 conn(0x7ff4380b2de0 msgr2=0x7ff4380b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:05.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 shutdown_connections 2024-09-17T05:26:05.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:05.820+0000 7ff43f200640 1 -- 172.21.15.43:0/2118798615 wait complete. 2024-09-17T05:26:05.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:26:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- 172.21.15.43:0/829535918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 msgr2=0x7fc07c0b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 --2- 172.21.15.43:0/829535918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c0b4930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc074009bc0 tx=0x7fc07401cb10 comp rx=0 tx=0).stop 2024-09-17T05:26:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- 172.21.15.43:0/829535918 shutdown_connections 2024-09-17T05:26:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 --2- 172.21.15.43:0/829535918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c0b4930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 --2- 172.21.15.43:0/829535918 >> v2:172.21.15.62:3300/0 conn(0x7fc07c0cccb0 0x7fc07c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- 172.21.15.43:0/829535918 >> 172.21.15.43:0/829535918 conn(0x7fc07c0691b0 msgr2=0x7fc07c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- 172.21.15.43:0/829535918 shutdown_connections 2024-09-17T05:26:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- 172.21.15.43:0/829535918 wait complete. 2024-09-17T05:26:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 Processor -- start 2024-09-17T05:26:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.000+0000 7fc082000640 1 -- start start 2024-09-17T05:26:06.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07c0cccb0 0x7fc07c14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:06.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:06.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:06.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c153370 con 0x7fc07c15cea0 2024-09-17T05:26:06.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c1531f0 con 0x7fc07c0cccb0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc07c1534f0 con 0x7fc07c0cd0c0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39100/0 (socket says 172.21.15.43:39100) 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 -- 172.21.15.43:0/2322572334 learned_addr learned my addr 172.21.15.43:0/2322572334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc07be00640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc080c00640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07c0cccb0 0x7fc07c14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 -- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 msgr2=0x7fc07c15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c15b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 -- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07c0cccb0 msgr2=0x7fc07c14f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07c0cccb0 0x7fc07c14f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 -- 172.21.15.43:0/2322572334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c155a50 con 0x7fc07c15cea0 2024-09-17T05:26:06.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc081600640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc07401c580 tx=0x7fc074003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0740b2040 con 0x7fc07c15cea0 2024-09-17T05:26:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0740bb070 con 0x7fc07c15cea0 2024-09-17T05:26:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074004370 con 0x7fc07c15cea0 2024-09-17T05:26:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c155d30 con 0x7fc07c15cea0 2024-09-17T05:26:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c1562f0 con 0x7fc07c15cea0 2024-09-17T05:26:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.004+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c14ffa0 con 0x7fc07c15cea0 2024-09-17T05:26:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.008+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc074004510 con 0x7fc07c15cea0 2024-09-17T05:26:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.008+0000 7fc078c00640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 0x7fc05c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.008+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fc07412eb60 con 0x7fc07c15cea0 2024-09-17T05:26:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.008+0000 7fc080c00640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 0x7fc05c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.008+0000 7fc080c00640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 0x7fc05c075740 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0d2110 tx=0x7fc070009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:06.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.012+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc074016790 con 0x7fc07c15cea0 2024-09-17T05:26:06.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.220+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc07c06a1e0 con 0x7fc07c15cea0 2024-09-17T05:26:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.220+0000 7fc078c00640 1 -- 172.21.15.43:0/2322572334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7fc0740f9ab0 con 0x7fc07c15cea0 2024-09-17T05:26:06.226 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 msgr2=0x7fc05c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 0x7fc05c075740 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0d2110 tx=0x7fc070009040 comp rx=0 tx=0).stop 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 msgr2=0x7fc07c15bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc07401c580 tx=0x7fc074003f40 comp rx=0 tx=0).stop 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 shutdown_connections 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc05c073230 0x7fc05c075740 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07c15cea0 0x7fc07c15bca0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc07c0cd0c0 0x7fc07c15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 --2- 172.21.15.43:0/2322572334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc07c0cccb0 0x7fc07c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 >> 172.21.15.43:0/2322572334 conn(0x7fc07c0691b0 msgr2=0x7fc07c0b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 shutdown_connections 2024-09-17T05:26:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06.224+0000 7fc082000640 1 -- 172.21.15.43:0/2322572334 wait complete. 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:26:06.249 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:06.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.250 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:26:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42278sec preferred_lft 42278sec 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:26:06.260 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:26:06.261 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.262 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:26:06.262 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:26:06.262 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:26:06.262 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:26:06.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.263 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:26:06.325 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:26:06.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.327 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:26:06.384 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:26:06.384 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:26:06.384 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:26:06.384 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:26:06.385 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:26:06.385 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:26:06.385 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:26:06.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:26:06.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:06.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.522 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:26:06.571 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:26:06.571 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:26:06.572 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:06.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:26:06.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:26:06.635 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:06.691 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:26:06.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:26:06.692 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:06.742 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:26:06.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:06.848 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:06.848 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:06.848 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:26:06.867 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:26:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:26:06.878 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:26:06.878 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:26:06.881 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:26:06.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:06.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:06.901 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:26:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:26:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:07.000 INFO:teuthology.orchestra.run.smithi043.stdout:client5528 2024-09-17T05:26:07.001 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:26:07.001 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:07.010 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-09-17T05:26:07.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:26:07.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:07.115 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:26:07.115 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:07.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 -- 172.21.15.43:0/4020954682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c003ad0 msgr2=0x7f207c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 --2- 172.21.15.43:0/4020954682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c003ad0 0x7f207c003f00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f208800c260 tx=0x7f208801cf60 comp rx=0 tx=0).stop 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f2099800640 1 -- 172.21.15.43:0/4020954682 reap_dead start 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 -- 172.21.15.43:0/4020954682 shutdown_connections 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 -- 172.21.15.43:0/4020954682 >> 172.21.15.43:0/4020954682 conn(0x7f20940b23d0 msgr2=0x7f20940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 -- 172.21.15.43:0/4020954682 shutdown_connections 2024-09-17T05:26:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.324+0000 7f209b800640 1 -- 172.21.15.43:0/4020954682 wait complete. 2024-09-17T05:26:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 Processor -- start 2024-09-17T05:26:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 -- start start 2024-09-17T05:26:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f207c003ad0 0x7f20940c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 0x7f20940c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f209406d680 con 0x7f20941f7d00 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f209406d500 con 0x7f20941f89c0 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f209b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f209406d800 con 0x7f207c003ad0 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f2099800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f2099800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39122/0 (socket says 172.21.15.43:39122) 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f2099800640 1 -- 172.21.15.43:0/3677735034 learned_addr learned my addr 172.21.15.43:0/3677735034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f2093e00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 0x7f20940c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.328+0000 7f2098e00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f207c003ad0 0x7f20940c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 -- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f207c003ad0 msgr2=0x7f20940c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f207c003ad0 0x7f20940c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 -- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 msgr2=0x7f20940c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 0x7f20940c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 -- 172.21.15.43:0/3677735034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20941f8650 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2093e00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 0x7f20940c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2099800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f207c004790 tx=0x7f208c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c003b90 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f208c003e20 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c004d90 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20940c9c20 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20940c3a40 con 0x7f20941f7d00 2024-09-17T05:26:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20940ad0b0 con 0x7f20941f7d00 2024-09-17T05:26:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.332+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f208c01a020 con 0x7f20941f7d00 2024-09-17T05:26:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.336+0000 7f2090c00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 0x7f2074075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.336+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f208c094460 con 0x7f20941f7d00 2024-09-17T05:26:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.336+0000 7f2098e00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 0x7f2074075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.336+0000 7f2098e00640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 0x7f2074075740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f20880ac000 tx=0x7f2088019da0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:07.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.340+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20941f8650 con 0x7f20941f7d00 2024-09-17T05:26:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20940c51d0 con 0x7f20941f7d00 2024-09-17T05:26:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f2090c00640 1 -- 172.21.15.43:0/3677735034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) ==== 80+0+0 (secure 0 0 0) 0x7f208c05f430 con 0x7f20941f7d00 2024-09-17T05:26:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 msgr2=0x7f2074075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 0x7f2074075740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f20880ac000 tx=0x7f2088019da0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 msgr2=0x7f20941f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f207c004790 tx=0x7f208c00fed0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 shutdown_connections 2024-09-17T05:26:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2074073230 0x7f2074075740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20941f89c0 0x7f20940c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20941f7d00 0x7f20941f8110 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 --2- 172.21.15.43:0/3677735034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f207c003ad0 0x7f20940c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 >> 172.21.15.43:0/3677735034 conn(0x7f20940b23d0 msgr2=0x7f20940ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 shutdown_connections 2024-09-17T05:26:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.528+0000 7f209b800640 1 -- 172.21.15.43:0/3677735034 wait complete. 2024-09-17T05:26:07.554 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:26:07.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:07.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:07.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:26:07.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:07.584 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:26:07.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:26:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 -- 172.21.15.43:0/1196738232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f415fa70 msgr2=0x7f49f415fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 --2- 172.21.15.43:0/1196738232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f415fa70 0x7f49f415fe80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f49f40738f0 tx=0x7f49e001c890 comp rx=0 tx=0).stop 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 -- 172.21.15.43:0/1196738232 shutdown_connections 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 --2- 172.21.15.43:0/1196738232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f415fa70 0x7f49f415fe80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 --2- 172.21.15.43:0/1196738232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f40b0c20 0x7f49f40b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 --2- 172.21.15.43:0/1196738232 >> v2:172.21.15.62:3300/0 conn(0x7f49f40b5fe0 0x7f49f415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 -- 172.21.15.43:0/1196738232 >> 172.21.15.43:0/1196738232 conn(0x7f49f40b2e80 msgr2=0x7f49f40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 -- 172.21.15.43:0/1196738232 shutdown_connections 2024-09-17T05:26:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.836+0000 7f49fc600640 1 -- 172.21.15.43:0/1196738232 wait complete. 2024-09-17T05:26:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 Processor -- start 2024-09-17T05:26:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 -- start start 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b5fe0 0x7f49f41fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 0x7f49f415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49f406d780 con 0x7f49f40b0c20 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49f406d600 con 0x7f49f40b5fe0 2024-09-17T05:26:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fc600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f49f406d900 con 0x7f49f415fa70 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39140/0 (socket says 172.21.15.43:39140) 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 -- 172.21.15.43:0/1707249321 learned_addr learned my addr 172.21.15.43:0/1707249321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9200640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b5fe0 0x7f49f41fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fa600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 0x7f49f415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 -- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 msgr2=0x7f49f415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 0x7f49f415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 -- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b5fe0 msgr2=0x7f49f41fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b5fe0 0x7f49f41fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 -- 172.21.15.43:0/1707249321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f41fd7c0 con 0x7f49f40b0c20 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49fa600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 0x7f49f415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.840+0000 7f49f9c00640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49e800d780 tx=0x7f49e800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8017070 con 0x7f49f40b0c20 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49e800de90 con 0x7f49f40b0c20 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80099d0 con 0x7f49f40b0c20 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f4166860 con 0x7f49f40b0c20 2024-09-17T05:26:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f40c7c60 con 0x7f49f40b0c20 2024-09-17T05:26:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f415f530 con 0x7f49f40b0c20 2024-09-17T05:26:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f49e8009b70 con 0x7f49f40b0c20 2024-09-17T05:26:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 0x7f49d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f49e8094250 con 0x7f49f40b0c20 2024-09-17T05:26:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.844+0000 7f49f9200640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 0x7f49d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.848+0000 7f49f9200640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 0x7f49d4075710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49e00af000 tx=0x7f49e00a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:07.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:07.848+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f49f41fd7c0 con 0x7f49f40b0c20 2024-09-17T05:26:08.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.060+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49f40cf380 con 0x7f49f40b0c20 2024-09-17T05:26:08.066 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:26:08.060+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:26:08.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.064+0000 7f49f1600640 1 -- 172.21.15.43:0/1707249321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f49e805ec60 con 0x7f49f40b0c20 2024-09-17T05:26:08.069 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:26:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 msgr2=0x7f49d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 0x7f49d4075710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49e00af000 tx=0x7f49e00a7040 comp rx=0 tx=0).stop 2024-09-17T05:26:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 msgr2=0x7f49f4150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f49e800d780 tx=0x7f49e800dc40 comp rx=0 tx=0).stop 2024-09-17T05:26:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 shutdown_connections 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49d4073200 0x7f49d4075710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b5fe0 0x7f49f41fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49f415fa70 0x7f49f415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 --2- 172.21.15.43:0/1707249321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40b0c20 0x7f49f4150770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 >> 172.21.15.43:0/1707249321 conn(0x7f49f40b2e80 msgr2=0x7f49f40b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 shutdown_connections 2024-09-17T05:26:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.068+0000 7f49fc600640 1 -- 172.21.15.43:0/1707249321 wait complete. 2024-09-17T05:26:08.082 INFO:tasks.ceph.mds.c.smithi043.stderr: -5541> 2024-09-17T05:25:58.564+0000 7fa5ec600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:26:08.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-09-17T05:26:08.204 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 -- 172.21.15.43:0/3687449591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000b3af0 msgr2=0x7f68000ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 --2- 172.21.15.43:0/3687449591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000b3af0 0x7f68000ad0b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6800072b60 tx=0x7f67f801c800 comp rx=0 tx=0).stop 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 -- 172.21.15.43:0/3687449591 shutdown_connections 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 --2- 172.21.15.43:0/3687449591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000b3af0 0x7f68000ad0b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 --2- 172.21.15.43:0/3687449591 >> v2:172.21.15.62:3300/0 conn(0x7f6800155d70 0x7f680015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 -- 172.21.15.43:0/3687449591 >> 172.21.15.43:0/3687449591 conn(0x7f68000b23d0 msgr2=0x7f68000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 -- 172.21.15.43:0/3687449591 shutdown_connections 2024-09-17T05:26:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.252+0000 7f6805a00640 1 -- 172.21.15.43:0/3687449591 wait complete. 2024-09-17T05:26:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 Processor -- start 2024-09-17T05:26:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 -- start start 2024-09-17T05:26:08.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6800155d70 0x7f680009d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68000cd850 0x7f680009d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6800168c30 con 0x7f680006f110 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6800168ab0 con 0x7f68000cd850 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f6805a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6800168db0 con 0x7f6800155d70 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.256+0000 7f67fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39154/0 (socket says 172.21.15.43:39154) 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 -- 172.21.15.43:0/4190639670 learned_addr learned my addr 172.21.15.43:0/4190639670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67f6000640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6800155d70 0x7f680009d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67ff400640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68000cd850 0x7f680009d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 -- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6800155d70 msgr2=0x7f680009d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6800155d70 0x7f680009d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 -- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68000cd850 msgr2=0x7f680009d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68000cd850 0x7f680009d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 -- 172.21.15.43:0/4190639670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68000c3cf0 con 0x7f680006f110 2024-09-17T05:26:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67fea00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e9e0 tx=0x7f67f000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f000cd50 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67f0011470 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6800074280 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00118c0 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6800074840 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68000ad0b0 con 0x7f680006f110 2024-09-17T05:26:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.260+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67f0011a60 con 0x7f680006f110 2024-09-17T05:26:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.264+0000 7f67f7400640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 0x7f67dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.264+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f67f0095110 con 0x7f680006f110 2024-09-17T05:26:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.264+0000 7f67f6000640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 0x7f67dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.264+0000 7f67f6000640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 0x7f67dc075710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007fd0 tx=0x7f67e8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:08.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.264+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f67f0016030 con 0x7f680006f110 2024-09-17T05:26:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.444+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7f68000b0020 con 0x7f680006f110 2024-09-17T05:26:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.444+0000 7f67f7400640 1 -- 172.21.15.43:0/4190639670 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v40)=0 v40) ==== 139+0+0 (secure 0 0 0) 0x7f67f005f9b0 con 0x7f680006f110 2024-09-17T05:26:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 msgr2=0x7f67dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 0x7f67dc075710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007fd0 tx=0x7f67e8009040 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 msgr2=0x7f68000cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e9e0 tx=0x7f67f000eea0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 shutdown_connections 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67dc073200 0x7f67dc075710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68000cd850 0x7f680009d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6800155d70 0x7f680009d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 --2- 172.21.15.43:0/4190639670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680006f110 0x7f68000cd310 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 >> 172.21.15.43:0/4190639670 conn(0x7f68000b23d0 msgr2=0x7f68000b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 shutdown_connections 2024-09-17T05:26:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:08.448+0000 7f6805a00640 1 -- 172.21.15.43:0/4190639670 wait complete. 2024-09-17T05:26:13.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- 172.21.15.43:0/1893831661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 msgr2=0x7fbd400ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 --2- 172.21.15.43:0/1893831661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd400ae0b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbd40071120 tx=0x7fbd2c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- 172.21.15.43:0/1893831661 shutdown_connections 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 --2- 172.21.15.43:0/1893831661 >> v2:172.21.15.62:3300/0 conn(0x7fbd400ae5f0 0x7fbd400aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 --2- 172.21.15.43:0/1893831661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd400ae0b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- 172.21.15.43:0/1893831661 >> 172.21.15.43:0/1893831661 conn(0x7fbd4006d9f0 msgr2=0x7fbd400b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- 172.21.15.43:0/1893831661 shutdown_connections 2024-09-17T05:26:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- 172.21.15.43:0/1893831661 wait complete. 2024-09-17T05:26:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 Processor -- start 2024-09-17T05:26:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.648+0000 7fbd45800640 1 -- start start 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd400ae5f0 0x7fbd4014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd4015b9f0 0x7fbd4015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40168e20 con 0x7fbd4015c510 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40168ca0 con 0x7fbd4015b9f0 2024-09-17T05:26:13.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbd40168fa0 con 0x7fbd400ae5f0 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39174/0 (socket says 172.21.15.43:39174) 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 -- 172.21.15.43:0/2418590527 learned_addr learned my addr 172.21.15.43:0/2418590527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd3f400640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd4015b9f0 0x7fbd4015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd3fe00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd400ae5f0 0x7fbd4014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 -- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd400ae5f0 msgr2=0x7fbd4014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd400ae5f0 0x7fbd4014fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 -- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd4015b9f0 msgr2=0x7fbd4015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd4015b9f0 0x7fbd4015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 -- 172.21.15.43:0/2418590527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4009d880 con 0x7fbd4015c510 2024-09-17T05:26:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd44e00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c970 tx=0x7fbd3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30007bb0 con 0x7fbd4015c510 2024-09-17T05:26:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd3000f450 con 0x7fbd4015c510 2024-09-17T05:26:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30007e60 con 0x7fbd4015c510 2024-09-17T05:26:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd401f7820 con 0x7fbd4015c510 2024-09-17T05:26:13.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4009cc30 con 0x7fbd4015c510 2024-09-17T05:26:13.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.652+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd401f7650 con 0x7fbd4015c510 2024-09-17T05:26:13.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.656+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbd3000f8a0 con 0x7fbd4015c510 2024-09-17T05:26:13.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.656+0000 7fbd37e00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:13.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.656+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fbd30094940 con 0x7fbd4015c510 2024-09-17T05:26:13.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.656+0000 7fbd3fe00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:13.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.656+0000 7fbd3fe00640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 0x7fbd20075710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b450 tx=0x7fbd3800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:13.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.660+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd30014030 con 0x7fbd4015c510 2024-09-17T05:26:13.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:13.872+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbd400aea00 con 0x7fbd4015c510 2024-09-17T05:26:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.088+0000 7fbd37e00640 1 -- 172.21.15.43:0/2418590527 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v49) ==== 162+0+0 (secure 0 0 0) 0x7fbd3005f270 con 0x7fbd4015c510 2024-09-17T05:26:14.095 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:26:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 msgr2=0x7fbd20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 0x7fbd20075710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b450 tx=0x7fbd3800d040 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 msgr2=0x7fbd4015e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c970 tx=0x7fbd3000ce30 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 shutdown_connections 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd4015b9f0 0x7fbd4015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd4015c510 0x7fbd4015e140 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 --2- 172.21.15.43:0/2418590527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbd400ae5f0 0x7fbd4014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 >> 172.21.15.43:0/2418590527 conn(0x7fbd4006d9f0 msgr2=0x7fbd400cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 shutdown_connections 2024-09-17T05:26:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.092+0000 7fbd45800640 1 -- 172.21.15.43:0/2418590527 wait complete. 2024-09-17T05:26:14.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:14.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 -- 172.21.15.43:0/2806837100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d0910 msgr2=0x7f5f9006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 --2- 172.21.15.43:0/2806837100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d0910 0x7f5f9006ce60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009900 tx=0x7f5f7801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 -- 172.21.15.43:0/2806837100 shutdown_connections 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 --2- 172.21.15.43:0/2806837100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d0910 0x7f5f9006ce60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 --2- 172.21.15.43:0/2806837100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f90070620 0x7f5f900b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 -- 172.21.15.43:0/2806837100 >> 172.21.15.43:0/2806837100 conn(0x7f5f900b23d0 msgr2=0x7f5f900d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 -- 172.21.15.43:0/2806837100 shutdown_connections 2024-09-17T05:26:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.276+0000 7f5f96800640 1 -- 172.21.15.43:0/2806837100 wait complete. 2024-09-17T05:26:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 Processor -- start 2024-09-17T05:26:14.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- start start 2024-09-17T05:26:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f900d0910 0x7f5f900c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f900d0d20 0x7f5f900c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:14.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900ce6b0 con 0x7f5f90070620 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900ce530 con 0x7f5f900d0910 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900ce830 con 0x7f5f900d0d20 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59364/0 (socket says 172.21.15.43:59364) 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 -- 172.21.15.43:0/1585743751 learned_addr learned my addr 172.21.15.43:0/1585743751 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8fe00640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f900d0d20 0x7f5f900c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f86a00640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f900d0910 0x7f5f900c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 -- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f900d0d20 msgr2=0x7f5f900c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f900d0d20 0x7f5f900c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 -- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f900d0910 msgr2=0x7f5f900c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f900d0910 0x7f5f900c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 -- 172.21.15.43:0/1585743751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f90075000 con 0x7f5f90070620 2024-09-17T05:26:14.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f8f400640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000e7c0 tx=0x7f5f8000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000cc60 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f8000ce20 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005280 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f900c3e60 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f900c4100 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f900cede0 con 0x7f5f90070620 2024-09-17T05:26:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f80005420 con 0x7f5f90070620 2024-09-17T05:26:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 0x7f5f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f5f80094b10 con 0x7f5f90070620 2024-09-17T05:26:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.284+0000 7f5f86a00640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 0x7f5f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.288+0000 7f5f86a00640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 0x7f5f6c075710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f780af000 tx=0x7f5f7801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.288+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5f8005f4e0 con 0x7f5f90070620 2024-09-17T05:26:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f9006cf00 con 0x7f5f90070620 2024-09-17T05:26:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f87e00640 1 -- 172.21.15.43:0/1585743751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5202 (secure 0 0 0) 0x7f5f80063f10 con 0x7f5f90070620 2024-09-17T05:26:14.504 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:14.504 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":50,"btime":"2024-09-17T05:26:14:098365+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:14.098351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-17T05:26:14.504 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 50 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 msgr2=0x7f5f6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 0x7f5f6c075710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f780af000 tx=0x7f5f7801b090 comp rx=0 tx=0).stop 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 msgr2=0x7f5f9009dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000e7c0 tx=0x7f5f8000ec80 comp rx=0 tx=0).stop 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 shutdown_connections 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f6c073200 0x7f5f6c075710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f900d0910 0x7f5f900c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f900d0d20 0x7f5f900c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 --2- 172.21.15.43:0/1585743751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90070620 0x7f5f9009dcd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 >> 172.21.15.43:0/1585743751 conn(0x7f5f900b23d0 msgr2=0x7f5f900cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 shutdown_connections 2024-09-17T05:26:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:14.500+0000 7f5f96800640 1 -- 172.21.15.43:0/1585743751 wait complete. 2024-09-17T05:26:14.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:14.098351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'c', 'rank': 0, 'incarnation': 50, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6834/61022679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 61022679}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 61022679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:26:14.928 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6036:up:replay 2024-09-17T05:26:15.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:16.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 -- 172.21.15.43:0/2771016210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff8070620 msgr2=0x7f9ff80b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 --2- 172.21.15.43:0/2771016210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff8070620 0x7f9ff80b30d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8073040 tx=0x7f9fe801c770 comp rx=0 tx=0).stop 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 -- 172.21.15.43:0/2771016210 shutdown_connections 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 --2- 172.21.15.43:0/2771016210 >> v2:172.21.15.62:3300/0 conn(0x7f9ff80cb860 0x7f9ff80cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 --2- 172.21.15.43:0/2771016210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff8070620 0x7f9ff80b30d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 -- 172.21.15.43:0/2771016210 >> 172.21.15.43:0/2771016210 conn(0x7f9ff80b23d0 msgr2=0x7f9ff80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 -- 172.21.15.43:0/2771016210 shutdown_connections 2024-09-17T05:26:16.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.100+0000 7f9ffe800640 1 -- 172.21.15.43:0/2771016210 wait complete. 2024-09-17T05:26:16.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 Processor -- start 2024-09-17T05:26:16.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 -- start start 2024-09-17T05:26:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 0x7f9ff809d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ff80cb860 0x7f9ff809d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff80ce5a0 con 0x7f9ff806ce60 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff80ce420 con 0x7f9ff8070620 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ffe800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff80ce720 con 0x7f9ff80cb860 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59390/0 (socket says 172.21.15.43:59390) 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 -- 172.21.15.43:0/3365746921 learned_addr learned my addr 172.21.15.43:0/3365746921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7e00640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ff80cb860 0x7f9ff809d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9fefe00640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 0x7f9ff809d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 -- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ff80cb860 msgr2=0x7f9ff809d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ff80cb860 0x7f9ff809d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 -- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 msgr2=0x7f9ff809d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 0x7f9ff809d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 -- 172.21.15.43:0/3365746921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff8150440 con 0x7f9ff806ce60 2024-09-17T05:26:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9fefe00640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 0x7f9ff809d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.104+0000 7f9ff7400640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c10 tx=0x7f9fe80a8580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80a8c70 con 0x7f9ff806ce60 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fe80a8e10 con 0x7f9ff806ce60 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8004290 con 0x7f9ff806ce60 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8150610 con 0x7f9ff806ce60 2024-09-17T05:26:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81529b0 con 0x7f9ff806ce60 2024-09-17T05:26:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff80ced10 con 0x7f9ff806ce60 2024-09-17T05:26:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.108+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9fe8005070 con 0x7f9ff806ce60 2024-09-17T05:26:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.112+0000 7f9fef400640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 0x7f9fd4075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:16.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.112+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f9fe812e610 con 0x7f9ff806ce60 2024-09-17T05:26:16.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.112+0000 7f9fefe00640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 0x7f9fd4075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.112+0000 7f9fefe00640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 0x7f9fd4075770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80c6aa0 tx=0x7f9fe0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:16.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.116+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9fe80ac030 con 0x7f9ff806ce60 2024-09-17T05:26:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ff815ba60 con 0x7f9ff806ce60 2024-09-17T05:26:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9fef400640 1 -- 172.21.15.43:0/3365746921 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5202 (secure 0 0 0) 0x7f9fe80163f0 con 0x7f9ff806ce60 2024-09-17T05:26:16.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:16.329 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"btime":"2024-09-17T05:26:16:105673+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:15.107420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-17T05:26:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2024-09-17T05:26:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 msgr2=0x7f9fd4075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 0x7f9fd4075770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80c6aa0 tx=0x7f9fe0009040 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 msgr2=0x7f9ff808c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c10 tx=0x7f9fe80a8580 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 shutdown_connections 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9fd4073260 0x7f9fd4075770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff8070620 0x7f9ff809d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ff80cb860 0x7f9ff809d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 --2- 172.21.15.43:0/3365746921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ff806ce60 0x7f9ff808c990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 >> 172.21.15.43:0/3365746921 conn(0x7f9ff80b23d0 msgr2=0x7f9ff80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 shutdown_connections 2024-09-17T05:26:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:16.328+0000 7f9ffe800640 1 -- 172.21.15.43:0/3365746921 wait complete. 2024-09-17T05:26:16.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:15.107420+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'c', 'rank': 0, 'incarnation': 50, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.43:6834/61022679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 61022679}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 61022679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:26:16.754 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6036:up:rejoin 2024-09-17T05:26:17.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 -- 172.21.15.43:0/306889413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240d0f50 msgr2=0x7f6b240cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 --2- 172.21.15.43:0/306889413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240d0f50 0x7f6b240cce70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6b2409bbe0 tx=0x7f6b1001c820 comp rx=0 tx=0).stop 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 -- 172.21.15.43:0/306889413 shutdown_connections 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 --2- 172.21.15.43:0/306889413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240d0f50 0x7f6b240cce70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 -- 172.21.15.43:0/306889413 >> 172.21.15.43:0/306889413 conn(0x7f6b240cee80 msgr2=0x7f6b240cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 -- 172.21.15.43:0/306889413 shutdown_connections 2024-09-17T05:26:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.924+0000 7f6b29800640 1 -- 172.21.15.43:0/306889413 wait complete. 2024-09-17T05:26:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 Processor -- start 2024-09-17T05:26:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 -- start start 2024-09-17T05:26:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b240b7ce0 0x7f6b2414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 0x7f6b24151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2416b030 con 0x7f6b240cd3b0 2024-09-17T05:26:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2416aeb0 con 0x7f6b240d0f50 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b29800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2416b1b0 con 0x7f6b240b7ce0 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b22000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b22000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59396/0 (socket says 172.21.15.43:59396) 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.928+0000 7f6b22000640 1 -- 172.21.15.43:0/3065274954 learned_addr learned my addr 172.21.15.43:0/3065274954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22a00640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b240b7ce0 0x7f6b2414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b23400640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 0x7f6b24151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 -- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b240b7ce0 msgr2=0x7f6b2414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b240b7ce0 0x7f6b2414cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 -- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 msgr2=0x7f6b24151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 0x7f6b24151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 -- 172.21.15.43:0/3065274954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2415df00 con 0x7f6b240cd3b0 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b23400640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 0x7f6b24151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b22000640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a450 tx=0x7f6b100a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b100be070 con 0x7f6b240cd3b0 2024-09-17T05:26:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b10005040 con 0x7f6b240cd3b0 2024-09-17T05:26:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10005470 con 0x7f6b240cd3b0 2024-09-17T05:26:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2415fe70 con 0x7f6b240cd3b0 2024-09-17T05:26:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2415e1a0 con 0x7f6b240cd3b0 2024-09-17T05:26:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b24159560 con 0x7f6b240cd3b0 2024-09-17T05:26:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.932+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b100046d0 con 0x7f6b240cd3b0 2024-09-17T05:26:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.936+0000 7f6b1aa00640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 0x7f6b04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.936+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f6b1012dfe0 con 0x7f6b240cd3b0 2024-09-17T05:26:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.936+0000 7f6b22a00640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 0x7f6b04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.936+0000 7f6b22a00640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 0x7f6b04075710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b24071220 tx=0x7f6b14009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:17.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:17.940+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b100ac030 con 0x7f6b240cd3b0 2024-09-17T05:26:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.152+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b240d0c50 con 0x7f6b240cd3b0 2024-09-17T05:26:18.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.152+0000 7f6b1aa00640 1 -- 172.21.15.43:0/3065274954 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5209 (secure 0 0 0) 0x7f6b10016410 con 0x7f6b240cd3b0 2024-09-17T05:26:18.153 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":53,"btime":"2024-09-17T05:26:17:112057+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:17.112047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:18.154 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 53 2024-09-17T05:26:18.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 msgr2=0x7f6b04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 0x7f6b04075710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b24071220 tx=0x7f6b14009040 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 msgr2=0x7f6b241511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a450 tx=0x7f6b100a6570 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 shutdown_connections 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6b04073200 0x7f6b04075710 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b240d0f50 0x7f6b24151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b240cd3b0 0x7f6b241511c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 --2- 172.21.15.43:0/3065274954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6b240b7ce0 0x7f6b2414cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 >> 172.21.15.43:0/3065274954 conn(0x7f6b240cee80 msgr2=0x7f6b240cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 shutdown_connections 2024-09-17T05:26:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.156+0000 7f6b29800640 1 -- 172.21.15.43:0/3065274954 wait complete. 2024-09-17T05:26:18.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:17.112047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'c', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.43:6834/61022679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 61022679}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 61022679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6036, 'qdb_cluster': [6036]} 2024-09-17T05:26:18.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:26:18.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- 172.21.15.43:0/2162354655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd4406f110 msgr2=0x7fcd440b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 --2- 172.21.15.43:0/2162354655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd4406f110 0x7fcd440b33e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009900 tx=0x7fcd3001c640 comp rx=0 tx=0).stop 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- 172.21.15.43:0/2162354655 shutdown_connections 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 --2- 172.21.15.43:0/2162354655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd4406f110 0x7fcd440b33e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 --2- 172.21.15.43:0/2162354655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd44155d70 0x7fcd4415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- 172.21.15.43:0/2162354655 >> 172.21.15.43:0/2162354655 conn(0x7fcd440b23d0 msgr2=0x7fcd440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- 172.21.15.43:0/2162354655 shutdown_connections 2024-09-17T05:26:18.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- 172.21.15.43:0/2162354655 wait complete. 2024-09-17T05:26:18.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 Processor -- start 2024-09-17T05:26:18.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.740+0000 7fcd4c200640 1 -- start start 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4406f110 0x7fcd440cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd440b3920 0x7fcd441f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44168a60 con 0x7fcd44155d70 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd441688e0 con 0x7fcd4406f110 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcd44168be0 con 0x7fcd440b3920 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59412/0 (socket says 172.21.15.43:59412) 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 -- 172.21.15.43:0/456835078 learned_addr learned my addr 172.21.15.43:0/456835078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd48e00640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd440b3920 0x7fcd441f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd49800640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4406f110 0x7fcd440cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 -- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd440b3920 msgr2=0x7fcd441f97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd440b3920 0x7fcd441f97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 -- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4406f110 msgr2=0x7fcd440cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4406f110 0x7fcd440cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 -- 172.21.15.43:0/456835078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd441f9cf0 con 0x7fcd44155d70 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4a200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b9e0 tx=0x7fcd3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd41600640 1 -- 172.21.15.43:0/456835078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c019070 con 0x7fcd44155d70 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd41600640 1 -- 172.21.15.43:0/456835078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd3c00cbb0 con 0x7fcd44155d70 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd41600640 1 -- 172.21.15.43:0/456835078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c014440 con 0x7fcd44155d70 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd440c7650 con 0x7fcd44155d70 2024-09-17T05:26:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.744+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd440c7c10 con 0x7fcd44155d70 2024-09-17T05:26:18.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd41600640 1 -- 172.21.15.43:0/456835078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd3c01d3f0 con 0x7fcd44155d70 2024-09-17T05:26:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd53952000 1 Processor -- start 2024-09-17T05:26:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd53952000 1 -- start start 2024-09-17T05:26:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd41600640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 0x7fcd24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd41600640 1 -- 172.21.15.43:0/456835078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fcd3c093e30 con 0x7fcd44155d70 2024-09-17T05:26:18.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd48e00640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 0x7fcd24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.748+0000 7fcd48e00640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 0x7fcd24075740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd300ae000 tx=0x7fcd3001ae80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603996fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603996fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603995d0410 con 0x5603991ba460 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56039950db40 con 0x560399593ca0 2024-09-17T05:26:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5603994fcf70 con 0x5603995aa950 2024-09-17T05:26:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59426/0 (socket says 172.21.15.43:59426) 2024-09-17T05:26:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 -- 172.21.15.43:0/1267364444 learned_addr learned my addr 172.21.15.43:0/1267364444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd48e00640 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603996fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd4a200640 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603996fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 -- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 msgr2=0x5603996fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603996fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 -- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 msgr2=0x5603996fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603996fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 -- 172.21.15.43:0/1267364444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603994c45b0 con 0x5603991ba460 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd49800640 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800b770 tx=0x7fcd3800bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca9980591d47692 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd36000640 1 -- 172.21.15.43:0/1267364444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800d950 con 0x5603991ba460 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd36000640 1 -- 172.21.15.43:0/1267364444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd3800dbe0 con 0x5603991ba460 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd36000640 1 -- 172.21.15.43:0/1267364444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800ddd0 con 0x5603991ba460 2024-09-17T05:26:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- 172.21.15.43:0/1267364444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 msgr2=0x5603996fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800b770 tx=0x7fcd3800bc30 comp rx=0 tx=0).stop 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- 172.21.15.43:0/1267364444 shutdown_connections 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603996fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603996fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 --2- 172.21.15.43:0/1267364444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x5603996fb6a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- 172.21.15.43:0/1267364444 >> 172.21.15.43:0/1267364444 conn(0x560399680d90 msgr2=0x5603996b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- 172.21.15.43:0/1267364444 shutdown_connections 2024-09-17T05:26:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- 172.21.15.43:0/1267364444 wait complete. 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 -- start start 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.752+0000 7fcd53952000 1 Processor -- start 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x560399702e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603997033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603994975f0 con 0x5603991ba460 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560399500d80 con 0x560399593ca0 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5603994a3520 con 0x5603995aa950 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 10 client.0 _ll_register_callbacks cb 0x5603994a7830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59442/0 (socket says 172.21.15.43:59442) 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 -- 172.21.15.43:0/481637554 learned_addr learned my addr 172.21.15.43:0/481637554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd48e00640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x560399702e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd4a200640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603997033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 -- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 msgr2=0x5603997033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603997033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 -- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 msgr2=0x560399702e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x560399702e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 -- 172.21.15.43:0/481637554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039949b190 con 0x5603991ba460 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd49800640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcd38016040 tx=0x7fcd3800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b10f6f9e192b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38027440 con 0x5603991ba460 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560399495110 con 0x5603991ba460 2024-09-17T05:26:18.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fcd380275e0 con 0x5603991ba460 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fcd3802f050 con 0x5603991ba460 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 10 client.6084 fetch_fsmap learned FSMap version 53 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x560399581f10 con 0x5603991ba460 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560399495110 con 0x5603991ba460 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fcd38032020 con 0x5603991ba460 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 10 client.6084 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 10 client.6084 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 20 client.6084 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039919f7d0 0x56039970cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 4 client.6084 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 --> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039950fa90 con 0x56039919f7d0 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd48e00640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039919f7d0 0x56039970cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd2aa00640 10 client.6084 ms_handle_connect on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:18.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.756+0000 7fcd48e00640 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039919f7d0 0x56039970cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5664dbbd959adc7e server_cookie=35d9d52309093fa7 in_seq=0 out_seq=0 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd2aa00640 1 -- 172.21.15.43:0/481637554 <== mds.0 v2:172.21.15.43:6832/61022679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56039950fa90 con 0x56039919f7d0 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd2aa00640 10 client.6084 handle_command_reply: tid=0 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 client.6084 shutdown 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 msgr2=0x560399702920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcd38016040 tx=0x7fcd3800dc00 comp rx=0 tx=0).stop 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 shutdown_connections 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039919f7d0 0x56039970cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x560399702e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x5603997033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd2aa00640 0 client.6084 ms_handle_reset on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:18.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 --2- 172.21.15.43:0/481637554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399702920 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 >> 172.21.15.43:0/481637554 conn(0x560399680d90 msgr2=0x5603996b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 shutdown_connections 2024-09-17T05:26:18.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 1 -- 172.21.15.43:0/481637554 wait complete. 2024-09-17T05:26:18.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.760+0000 7fcd53952000 20 client.6084 trim_cache size 0 max 16384 2024-09-17T05:26:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 Processor -- start 2024-09-17T05:26:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 -- start start 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 0x56039976d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x56039976e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd442054d0 con 0x560399593ca0 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd441fd200 con 0x5603991ba460 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcd441fc560 con 0x5603995aa950 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59444/0 (socket says 172.21.15.43:59444) 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 -- 172.21.15.43:0/1273292895 learned_addr learned my addr 172.21.15.43:0/1273292895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd49800640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 0x56039976d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd4a200640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x56039976e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 -- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 msgr2=0x56039976e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x56039976e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 -- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 msgr2=0x56039976d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 0x56039976d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 -- 172.21.15.43:0/1273292895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560399558db0 con 0x560399593ca0 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd49800640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 0x56039976d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd4a200640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x56039976e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:18.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd48e00640 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009850 tx=0x7fcd300aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d54711090457630 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd35600640 1 -- 172.21.15.43:0/1273292895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300a6d70 con 0x560399593ca0 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd35600640 1 -- 172.21.15.43:0/1273292895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd300046d0 con 0x560399593ca0 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.780+0000 7fcd35600640 1 -- 172.21.15.43:0/1273292895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004a80 con 0x560399593ca0 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- 172.21.15.43:0/1273292895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 msgr2=0x56039976db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009850 tx=0x7fcd300aa050 comp rx=0 tx=0).stop 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- 172.21.15.43:0/1273292895 shutdown_connections 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x56039976e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560399593ca0 0x56039976db60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- 172.21.15.43:0/1273292895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5603991ba460 0x56039976d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- 172.21.15.43:0/1273292895 >> 172.21.15.43:0/1273292895 conn(0x560399680d90 msgr2=0x5603996f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- 172.21.15.43:0/1273292895 shutdown_connections 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- 172.21.15.43:0/1273292895 wait complete. 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- start start 2024-09-17T05:26:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 Processor -- start 2024-09-17T05:26:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603997718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x560399771e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5603995053b0 con 0x5603991ba460 2024-09-17T05:26:18.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560399500090 con 0x560399593ca0 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560399581a80 con 0x5603995aa950 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd53952000 10 client.0 _ll_register_callbacks cb 0x7fcd30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59450/0 (socket says 172.21.15.43:59450) 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 -- 172.21.15.43:0/483728530 learned_addr learned my addr 172.21.15.43:0/483728530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd48e00640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603997718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd4a200640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x560399771e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 -- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 msgr2=0x560399771e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x560399771e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 -- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 msgr2=0x5603997718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603997718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd49800640 1 -- 172.21.15.43:0/483728530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603994c3ab0 con 0x5603991ba460 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.784+0000 7fcd4a200640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x560399771e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:18.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd49800640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcd38004780 tx=0x7fcd38015770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0e99990ad47e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38037020 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd300046d0 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fcd3800f040 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fcd38041050 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 10 client.6090 fetch_fsmap learned FSMap version 53 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5603994a2070 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd300046d0 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fcd38044020 con 0x5603991ba460 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 10 client.6090 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 10 client.6090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 20 client.6090 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039918d830 0x56039972d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 4 client.6090 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:26:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 --> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] -- command(tid 0: {"prefix": "status"}) -- 0x56039950b540 con 0x56039918d830 2024-09-17T05:26:18.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd48e00640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039918d830 0x56039972d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 10 client.6090 ms_handle_connect on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd48e00640 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039918d830 0x56039972d610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9fb44fbd7025c0 server_cookie=8425af857bb7dcf in_seq=0 out_seq=0 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 1 -- 172.21.15.43:0/483728530 <== mds.0 v2:172.21.15.43:6832/61022679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56039950b540 con 0x56039918d830 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 10 client.6090 handle_command_reply: tid=0 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 client.6090 shutdown 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 msgr2=0x560399771390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcd38004780 tx=0x7fcd38015770 comp rx=0 tx=0).stop 2024-09-17T05:26:18.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 shutdown_connections 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x56039918d830 0x56039972d610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560399593ca0 0x5603997718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5603995aa950 0x560399771e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 --2- 172.21.15.43:0/483728530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5603991ba460 0x560399771390 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 >> 172.21.15.43:0/483728530 conn(0x560399680d90 msgr2=0x5603996f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd2aa00640 0 client.6090 ms_handle_reset on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 shutdown_connections 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 1 -- 172.21.15.43:0/483728530 wait complete. 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd53952000 20 client.6090 trim_cache size 0 max 16384 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:26:18.793 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6036, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.6899540560000004, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 53, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 62, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 62, 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.581976998 2024-09-17T05:26:18.794 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 msgr2=0x7fcd24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 0x7fcd24075740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd300ae000 tx=0x7fcd3001ae80 comp rx=0 tx=0).stop 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 msgr2=0x7fcd44164770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b9e0 tx=0x7fcd3c00bea0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 shutdown_connections 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd24073230 0x7fcd24075740 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd44155d70 0x7fcd44164770 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd440b3920 0x7fcd441f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 --2- 172.21.15.43:0/456835078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4406f110 0x7fcd440cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 >> 172.21.15.43:0/456835078 conn(0x7fcd440b23d0 msgr2=0x7fcd440d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 shutdown_connections 2024-09-17T05:26:18.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.788+0000 7fcd4c200640 1 -- 172.21.15.43:0/456835078 wait complete. 2024-09-17T05:26:18.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 -- 172.21.15.43:0/3180553567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806f110 msgr2=0x7f58780ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 --2- 172.21.15.43:0/3180553567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806f110 0x7f58780ad0b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f58780af0a0 tx=0x7f587001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 -- 172.21.15.43:0/3180553567 shutdown_connections 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 --2- 172.21.15.43:0/3180553567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587806f110 0x7f58780ad0b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 --2- 172.21.15.43:0/3180553567 >> v2:172.21.15.62:3300/0 conn(0x7f58780b3af0 0x7f587815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 -- 172.21.15.43:0/3180553567 >> 172.21.15.43:0/3180553567 conn(0x7f58780b23d0 msgr2=0x7f58780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 -- 172.21.15.43:0/3180553567 shutdown_connections 2024-09-17T05:26:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.968+0000 7f587da00640 1 -- 172.21.15.43:0/3180553567 wait complete. 2024-09-17T05:26:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 Processor -- start 2024-09-17T05:26:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- start start 2024-09-17T05:26:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 0x7f5878155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 0x7f58780ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f587806d640 con 0x7f58781555c0 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f587806d4c0 con 0x7f5878155d70 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f587806d7c0 con 0x7f587806f110 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59468/0 (socket says 172.21.15.43:59468) 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 -- 172.21.15.43:0/3392632403 learned_addr learned my addr 172.21.15.43:0/3392632403 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f586e000640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 0x7f58780ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5876a00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 0x7f5878155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 -- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 msgr2=0x7f5878155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 0x7f5878155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 -- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 msgr2=0x7f58780ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 0x7f58780ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 -- 172.21.15.43:0/3392632403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780cc960 con 0x7f58781555c0 2024-09-17T05:26:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5876a00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 0x7f5878155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f586e000640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 0x7f58780ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f5877400640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5870009bd0 tx=0x7f58700a5b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700bb070 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5870004e00 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58780ccc20 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700b2040 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.972+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58780cd1e0 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58780afa00 con 0x7f58781555c0 2024-09-17T05:26:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58700be070 con 0x7f58781555c0 2024-09-17T05:26:18.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f586f400640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 0x7f5854075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:18.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f587012e200 con 0x7f58781555c0 2024-09-17T05:26:18.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f5876a00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 0x7f5854075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.976+0000 7f5876a00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 0x7f5854075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58780715c0 tx=0x7f5868008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:18.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:18.980+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5870016970 con 0x7f58781555c0 2024-09-17T05:26:19.078 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:26:19.076+0000 7f683a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:26:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.188+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58780d1150 con 0x7f58781555c0 2024-09-17T05:26:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f586f400640 1 -- 172.21.15.43:0/3392632403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5209 (secure 0 0 0) 0x7f58780d1150 con 0x7f58781555c0 2024-09-17T05:26:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:19.191 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":53,"btime":"2024-09-17T05:26:17:112057+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:17.112047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 53 2024-09-17T05:26:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 msgr2=0x7f5854075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 0x7f5854075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58780715c0 tx=0x7f5868008040 comp rx=0 tx=0).stop 2024-09-17T05:26:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 msgr2=0x7f58780cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5870009bd0 tx=0x7f58700a5b50 comp rx=0 tx=0).stop 2024-09-17T05:26:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 shutdown_connections 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5854073200 0x7f5854075710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5878155d70 0x7f58780ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58781555c0 0x7f58780cae20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 --2- 172.21.15.43:0/3392632403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f587806f110 0x7f5878155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 >> 172.21.15.43:0/3392632403 conn(0x7f58780b23d0 msgr2=0x7f58780b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 shutdown_connections 2024-09-17T05:26:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.192+0000 7f587da00640 1 -- 172.21.15.43:0/3392632403 wait complete. 2024-09-17T05:26:19.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-09-17T05:26:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2943483184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce006f110 msgr2=0x7f8ce00b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2943483184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce006f110 0x7f8ce00b33e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0071050 tx=0x7f8ccc01c800 comp rx=0 tx=0).stop 2024-09-17T05:26:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2943483184 shutdown_connections 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2943483184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce006f110 0x7f8ce00b33e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2943483184 >> v2:172.21.15.62:3300/0 conn(0x7f8ce00b3920 0x7f8ce00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2943483184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce0155d70 0x7f8ce015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2943483184 >> 172.21.15.43:0/2943483184 conn(0x7f8ce00b23d0 msgr2=0x7f8ce00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2943483184 shutdown_connections 2024-09-17T05:26:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.372+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2943483184 wait complete. 2024-09-17T05:26:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 Processor -- start 2024-09-17T05:26:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 -- start start 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce006f110 0x7f8ce009b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ce0155d70 0x7f8ce00c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce015f8d0 con 0x7f8ce00b3920 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce015f750 con 0x7f8ce0155d70 2024-09-17T05:26:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8ce4e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce015fa50 con 0x7f8ce006f110 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59494/0 (socket says 172.21.15.43:59494) 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 -- 172.21.15.43:0/2385363914 learned_addr learned my addr 172.21.15.43:0/2385363914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cde800640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ce0155d70 0x7f8ce00c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdde00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce006f110 0x7f8ce009b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 -- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce006f110 msgr2=0x7f8ce009b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce006f110 0x7f8ce009b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 -- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ce0155d70 msgr2=0x7f8ce00c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ce0155d70 0x7f8ce00c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 -- 172.21.15.43:0/2385363914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce00c4850 con 0x7f8ce00b3920 2024-09-17T05:26:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cdd400640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0b0040 tx=0x7f8ccc005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0b2040 con 0x7f8ce00b3920 2024-09-17T05:26:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.376+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ccc004330 con 0x7f8ce00b3920 2024-09-17T05:26:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce00c4b30 con 0x7f8ce00b3920 2024-09-17T05:26:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc005290 con 0x7f8ce00b3920 2024-09-17T05:26:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce00ca120 con 0x7f8ce00b3920 2024-09-17T05:26:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce00ca3e0 con 0x7f8ce00b3920 2024-09-17T05:26:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ccc0bd060 con 0x7f8ce00b3920 2024-09-17T05:26:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8cd6000640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 0x7f8cc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f8ccc12d890 con 0x7f8ce00b3920 2024-09-17T05:26:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.380+0000 7f8cdde00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 0x7f8cc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.384+0000 7f8cdde00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 0x7f8cc0075710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00715c0 tx=0x7f8cd00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:19.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.384+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ccc0164b0 con 0x7f8ce00b3920 2024-09-17T05:26:19.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:19.624+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7f8ce015ff00 con 0x7f8ce00b3920 2024-09-17T05:26:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8cd6000640 1 -- 172.21.15.43:0/2385363914 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6036 v54) ==== 112+0+0 (secure 0 0 0) 0x7f8ccc0f8780 con 0x7f8ce00b3920 2024-09-17T05:26:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:freezing mds.6036 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 msgr2=0x7f8cc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 0x7f8cc0075710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00715c0 tx=0x7f8cd00091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 msgr2=0x7f8ce00c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0b0040 tx=0x7f8ccc005e70 comp rx=0 tx=0).stop 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 shutdown_connections 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cc0073200 0x7f8cc0075710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ce0155d70 0x7f8ce00c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ce00b3920 0x7f8ce00c6340 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 --2- 172.21.15.43:0/2385363914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ce006f110 0x7f8ce009b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.128+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 >> 172.21.15.43:0/2385363914 conn(0x7f8ce00b23d0 msgr2=0x7f8ce00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.132+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 shutdown_connections 2024-09-17T05:26:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.132+0000 7f8ce4e00640 1 -- 172.21.15.43:0/2385363914 wait complete. 2024-09-17T05:26:20.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-17T05:26:20.156 DEBUG:teuthology.orchestra.run.smithi043:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-09-17T05:26:20.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-17T05:26:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 -- 172.21.15.43:0/1446503582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 msgr2=0x7f771c0ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 --2- 172.21.15.43:0/1446503582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0ae160 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f770401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 -- 172.21.15.43:0/1446503582 shutdown_connections 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 --2- 172.21.15.43:0/1446503582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c0ae6a0 0x7f771c0d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 --2- 172.21.15.43:0/1446503582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0ae160 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 -- 172.21.15.43:0/1446503582 >> 172.21.15.43:0/1446503582 conn(0x7f771c0b2e80 msgr2=0x7f771c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 -- 172.21.15.43:0/1446503582 shutdown_connections 2024-09-17T05:26:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.324+0000 7f7720e00640 1 -- 172.21.15.43:0/1446503582 wait complete. 2024-09-17T05:26:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.328+0000 7f7720e00640 1 Processor -- start 2024-09-17T05:26:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.328+0000 7f7720e00640 1 -- start start 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c0ae6a0 0x7f771c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c15ee30 0x7f771c0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f771c16c510 con 0x7f771c0b69f0 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f771c16c390 con 0x7f771c15ee30 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f771c16c690 con 0x7f771c0ae6a0 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59518/0 (socket says 172.21.15.43:59518) 2024-09-17T05:26:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 -- 172.21.15.43:0/3483581452 learned_addr learned my addr 172.21.15.43:0/3483581452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7719e00640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c0ae6a0 0x7f771c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f771a800640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c15ee30 0x7f771c0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 -- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c0ae6a0 msgr2=0x7f771c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f771c0ae6a0 0x7f771c1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 -- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c15ee30 msgr2=0x7f771c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f771c15ee30 0x7f771c0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 -- 172.21.15.43:0/3483581452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c154b10 con 0x7f771c0b69f0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7711400640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771c0b69f0 0x7f771c0d1260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f77040b0040 tx=0x7f7704005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7712a00640 1 -- 172.21.15.43:0/3483581452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040b2040 con 0x7f771c0b69f0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7712a00640 1 -- 172.21.15.43:0/3483581452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7704005320 con 0x7f771c0b69f0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 -- 172.21.15.43:0/3483581452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1598b0 con 0x7f771c0b69f0 2024-09-17T05:26:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7712a00640 1 -- 172.21.15.43:0/3483581452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040046d0 con 0x7f771c0b69f0 2024-09-17T05:26:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.332+0000 7f7720e00640 1 -- 172.21.15.43:0/3483581452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c1fcec0 con 0x7f771c0b69f0 2024-09-17T05:26:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f7712a00640 1 -- 172.21.15.43:0/3483581452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77040be060 con 0x7f771c0b69f0 2024-09-17T05:26:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f7712a00640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76f8073200 0x7f76f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f7712a00640 1 -- 172.21.15.43:0/3483581452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f770412ddb0 con 0x7f771c0b69f0 2024-09-17T05:26:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f7719e00640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76f8073200 0x7f76f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f7719e00640 1 --2- 172.21.15.43:0/3483581452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76f8073200 0x7f76f8075710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f770c001e40 tx=0x7f770c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.336+0000 7f77284a9000 1 Processor -- start 2024-09-17T05:26:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 -- start start 2024-09-17T05:26:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f44f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f44f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651f4302ce0 con 0x5651f4392440 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5651f42f2110 con 0x5651f437e7e0 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5651f42f5f20 con 0x5651f4376cc0 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59526/0 (socket says 172.21.15.43:59526) 2024-09-17T05:26:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 -- 172.21.15.43:0/3603229946 learned_addr learned my addr 172.21.15.43:0/3603229946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7711400640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f44f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f771a800640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f44f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 -- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 msgr2=0x5651f44f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f44f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 -- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 msgr2=0x5651f44f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f44f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 -- 172.21.15.43:0/3603229946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f42904d0 con 0x5651f4392440 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7711400640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f44f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.340+0000 7f7719e00640 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f770c00fd00 tx=0x7f770c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a98e2404c23c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f770aa00640 1 -- 172.21.15.43:0/3603229946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c025070 con 0x5651f4392440 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f770aa00640 1 -- 172.21.15.43:0/3603229946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f770c01b050 con 0x5651f4392440 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f770aa00640 1 -- 172.21.15.43:0/3603229946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c020610 con 0x5651f4392440 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- 172.21.15.43:0/3603229946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 msgr2=0x5651f44f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f770c00fd00 tx=0x7f770c012000 comp rx=0 tx=0).stop 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- 172.21.15.43:0/3603229946 shutdown_connections 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f44f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f44f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 --2- 172.21.15.43:0/3603229946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44f1610 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- 172.21.15.43:0/3603229946 >> 172.21.15.43:0/3603229946 conn(0x5651f3fb05d0 msgr2=0x5651f44a61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- 172.21.15.43:0/3603229946 shutdown_connections 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- 172.21.15.43:0/3603229946 wait complete. 2024-09-17T05:26:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 -- start start 2024-09-17T05:26:20.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.344+0000 7f77284a9000 1 Processor -- start 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 0x5651f44b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 0x5651f44b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651f3fb7880 con 0x5651f4392440 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5651f428c920 con 0x5651f4376cc0 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5651f42a7c80 con 0x5651f437e7e0 2024-09-17T05:26:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 10 client.0 _ll_register_callbacks cb 0x5651f4493360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f771a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7719e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 0x5651f44b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 0x5651f44b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7719e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 0x5651f44b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:53302/0 (socket says 172.21.15.43:53302) 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7719e00640 1 -- 172.21.15.43:0/1672699792 learned_addr learned my addr 172.21.15.43:0/1672699792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 -- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 msgr2=0x5651f44b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 0x5651f44b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 -- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 msgr2=0x5651f44b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 -- 172.21.15.43:0/1672699792 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5651f42af4d0 con 0x5651f437e7e0 2024-09-17T05:26:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f771a800640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7711400640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 0x5651f44b3d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f770412c210 tx=0x7f77040f8230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ccf334edb26a10a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3301/0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040f8860 con 0x5651f437e7e0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mon.2 v2:172.21.15.62:3301/0 2 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f7704101cb0 con 0x5651f437e7e0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651f428a450 con 0x5651f437e7e0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f77040fd290 con 0x5651f437e7e0 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 10 client.5597 fetch_fsmap learned FSMap version 54 2024-09-17T05:26:20.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5651f42b5df0 con 0x5651f437e7e0 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mon.2 v2:172.21.15.62:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5651f428a450 con 0x5651f437e7e0 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mon.2 v2:172.21.15.62:3301/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f77040f7ac0 con 0x5651f437e7e0 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 10 client.5597 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 10 client.5597 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 20 client.5597 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f94b30 0x5651f4502bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 4 client.5597 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 --> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651f4304c30 con 0x5651f3f94b30 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.348+0000 7f7719e00640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f94b30 0x5651f4502bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f7719e00640 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f94b30 0x5651f4502bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1633e93afb5d436 server_cookie=933f30df6648e1b0 in_seq=0 out_seq=0 2024-09-17T05:26:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f76ff400640 10 client.5597 ms_handle_connect on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f76ff400640 1 -- 172.21.15.43:0/1672699792 <== mds.0 v2:172.21.15.43:6832/61022679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651f4304c30 con 0x5651f3f94b30 2024-09-17T05:26:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f76ff400640 10 client.5597 handle_command_reply: tid=0 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 client.5597 shutdown 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 msgr2=0x5651f44b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 0x5651f44b3d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f770412c210 tx=0x7f77040f8230 comp rx=0 tx=0).stop 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 shutdown_connections 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f44b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f437e7e0 0x5651f44b3d70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f94b30 0x5651f4502bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 --2- 172.21.15.43:0/1672699792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4376cc0 0x5651f44b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f76ff400640 0 client.5597 ms_handle_reset on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 >> 172.21.15.43:0/1672699792 conn(0x5651f3fb05d0 msgr2=0x5651f44ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 shutdown_connections 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 1 -- 172.21.15.43:0/1672699792 wait complete. 2024-09-17T05:26:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f77284a9000 20 client.5597 trim_cache size 0 max 16384 2024-09-17T05:26:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 -- 172.21.15.43:0/4224823543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 msgr2=0x7f2b680ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 --2- 172.21.15.43:0/4224823543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 0x7f2b680ccea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005fd0 tx=0x7f2b5c01c700 comp rx=0 tx=0).stop 2024-09-17T05:26:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 -- 172.21.15.43:0/4224823543 shutdown_connections 2024-09-17T05:26:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 --2- 172.21.15.43:0/4224823543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 0x7f2b680ccea0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 -- 172.21.15.43:0/4224823543 >> 172.21.15.43:0/4224823543 conn(0x7f2b680cee80 msgr2=0x7f2b680cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 -- 172.21.15.43:0/4224823543 shutdown_connections 2024-09-17T05:26:20.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.352+0000 7f2b70200640 1 -- 172.21.15.43:0/4224823543 wait complete. 2024-09-17T05:26:20.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 Processor -- start 2024-09-17T05:26:20.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 -- start start 2024-09-17T05:26:20.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680b2d80 0x7f2b680b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b680cd3e0 0x7f2b6815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 0x7f2b6815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6816b630 con 0x7f2b680cd3e0 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6816b4b0 con 0x7f2b680cffd0 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.356+0000 7f2b70200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6816b7b0 con 0x7f2b680b2d80 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b680cd3e0 0x7f2b6815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b680cd3e0 0x7f2b6815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59540/0 (socket says 172.21.15.43:59540) 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3013468311 learned_addr learned my addr 172.21.15.43:0/3013468311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6d800640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680b2d80 0x7f2b680b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6e200640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 0x7f2b6815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680b2d80 msgr2=0x7f2b680b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b680b2d80 0x7f2b680b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 msgr2=0x7f2b6815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b680cffd0 0x7f2b6815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 -- 172.21.15.43:0/3013468311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680c8400 con 0x7f2b680cd3e0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b6ce00640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b680cd3e0 0x7f2b6815d900 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c3f0 tx=0x7f2b5400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54017070 con 0x7f2b680cd3e0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b54003600 con 0x7f2b680cd3e0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003a50 con 0x7f2b680cd3e0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b70200640 1 -- 172.21.15.43:0/3013468311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b680c86e0 con 0x7f2b680cd3e0 2024-09-17T05:26:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b70200640 1 -- 172.21.15.43:0/3013468311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b680c8ca0 con 0x7f2b680cd3e0 2024-09-17T05:26:20.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.360+0000 7f2b70200640 1 -- 172.21.15.43:0/3013468311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2b680ccea0 con 0x7f2b680cd3e0 2024-09-17T05:26:20.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b5401a070 con 0x7f2b680cd3e0 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b65600640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073230 0x7f2b48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f2b54093530 con 0x7f2b680cd3e0 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+515 (secure 0 0 0) 0x7f2b54095ad0 con 0x7f2b680cd3e0 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b6d800640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073230 0x7f2b48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b70200640 1 -- 172.21.15.43:0/3013468311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f2b680b82c0 con 0x7f2b680cd3e0 2024-09-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 7 ==== log(1 entries from seq 580 at 2024-09-17T05:26:19.627651+0000) ==== 308+0+0 (secure 0 0 0) 0x7f2b54001070 con 0x7f2b680cd3e0 2024-09-17T05:26:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.364+0000 7f2b6d800640 1 --2- 172.21.15.43:0/3013468311 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2b48073230 0x7f2b48075740 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0ae000 tx=0x7f2b5c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.368+0000 7f77284a9000 1 Processor -- start 2024-09-17T05:26:20.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.368+0000 7f77284a9000 1 -- start start 2024-09-17T05:26:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4562b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f4563060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f771c0cc5e0 con 0x5651f4392440 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f771c0cb970 con 0x5651f437e7e0 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f771c0cbf90 con 0x5651f4376cc0 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f7711400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f4563060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f7719e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4562b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59544/0 (socket says 172.21.15.43:59544) 2024-09-17T05:26:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 -- 172.21.15.43:0/1599894110 learned_addr learned my addr 172.21.15.43:0/1599894110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 -- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 msgr2=0x5651f4562b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4562b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 -- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 msgr2=0x5651f4563060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f4563060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 -- 172.21.15.43:0/1599894110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f434eb90 con 0x5651f4392440 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f771a800640 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7714003790 tx=0x7f7714003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804cb99d592b4d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f770a000640 1 -- 172.21.15.43:0/1599894110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714003ea0 con 0x5651f4392440 2024-09-17T05:26:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f770a000640 1 -- 172.21.15.43:0/1599894110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f771400ba40 con 0x5651f4392440 2024-09-17T05:26:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f770a000640 1 -- 172.21.15.43:0/1599894110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400be10 con 0x5651f4392440 2024-09-17T05:26:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- 172.21.15.43:0/1599894110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 msgr2=0x5651f45635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7714003790 tx=0x7f7714003c50 comp rx=0 tx=0).stop 2024-09-17T05:26:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- 172.21.15.43:0/1599894110 shutdown_connections 2024-09-17T05:26:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f437e7e0 0x5651f4563060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f4392440 0x5651f45635a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 --2- 172.21.15.43:0/1599894110 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4562b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- 172.21.15.43:0/1599894110 >> 172.21.15.43:0/1599894110 conn(0x5651f3fb05d0 msgr2=0x5651f44a57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- 172.21.15.43:0/1599894110 shutdown_connections 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.372+0000 7f77284a9000 1 -- 172.21.15.43:0/1599894110 wait complete. 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 -- start start 2024-09-17T05:26:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 Processor -- start 2024-09-17T05:26:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4566f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f437e7e0 0x5651f4567440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4392440 0x5651f4567980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651f42ba8d0 con 0x5651f437e7e0 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5651f42a9de0 con 0x5651f4392440 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5651f42f5230 con 0x5651f4376cc0 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 10 client.0 _ll_register_callbacks cb 0x7f77040f8630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:20.378 INFO:teuthology.orchestra.run.smithi043.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-17T05:26:20.379 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-17T05:26:20.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f437e7e0 0x5651f4567440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7719e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4566f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f437e7e0 0x5651f4567440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59550/0 (socket says 172.21.15.43:59550) 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 -- 172.21.15.43:0/2571976674 learned_addr learned my addr 172.21.15.43:0/2571976674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f771a800640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4392440 0x5651f4567980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 msgr2=0x5651f4566f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4566f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4392440 msgr2=0x5651f4567980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651f4392440 0x5651f4567980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 -- 172.21.15.43:0/2571976674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f427baf0 con 0x5651f437e7e0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5651f4376cc0 0x5651f4566f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f7711400640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651f437e7e0 0x5651f4567440 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f77040f5ad0 tx=0x7f7704005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdf12cadf1aa521 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f76ff400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f76ff400640 1 -- 172.21.15.43:0/2571976674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704101eb0 con 0x5651f437e7e0 2024-09-17T05:26:20.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f76ff400640 1 -- 172.21.15.43:0/2571976674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f770411dbc0 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f77284a9000 1 -- 172.21.15.43:0/2571976674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771400ba40 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.376+0000 7f76ff400640 1 -- 172.21.15.43:0/2571976674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7704102140 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 10 client.6126 fetch_fsmap learned FSMap version 54 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 1 -- 172.21.15.43:0/2571976674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5651f4297350 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f76ff400640 1 -- 172.21.15.43:0/2571976674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f771400ba40 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f76ff400640 1 -- 172.21.15.43:0/2571976674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f77040f9ae0 con 0x5651f437e7e0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 10 client.6126 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 10 client.6126 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 20 client.6126 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 4 client.6126 mds_command: new command op to 6036 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f77284a9000 1 -- 172.21.15.43:0/2571976674 --> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651f43006e0 con 0x5651f3f82b90 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e3ddafe450e505 server_cookie=46f0a442259e9071 in_seq=0 out_seq=0 2024-09-17T05:26:20.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.380+0000 7f76ff400640 10 client.6126 ms_handle_connect on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:20.384 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:26:20.384+0000 7f6835200640 -1 mds.0.cache.den(0x10000000001 d) newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] with next_snap:2 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:26:20.384+0000 7f6835200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr:./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6835200640 time 2024-09-17T05:26:20.384379+0000 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f684464330e] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.385 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.386 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:26:20.384+0000 7f6835200640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6835200640 time 2024-09-17T05:26:20.384379+0000 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:26:20.388 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f684464330e] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.389 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.390 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.391 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.391 INFO:tasks.ceph.mds.c.smithi043.stderr:*** Caught signal (Aborted) ** 2024-09-17T05:26:20.391 INFO:tasks.ceph.mds.c.smithi043.stderr: in thread 7f6835200640 thread_name:MR_Finisher 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6843c42520] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: raise() 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: abort() 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6844643480] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.392 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.393 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 29: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 30: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:26:20.384+0000 7f6835200640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: in thread 7f6835200640 thread_name:MR_Finisher 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6843c42520] 2024-09-17T05:26:20.394 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: raise() 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: abort() 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6844643480] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.395 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.396 INFO:tasks.ceph.mds.c.smithi043.stderr: 29: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: 30: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: -211> 2024-09-17T05:26:19.076+0000 7f683a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: -12> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 mds.0.cache.den(0x10000000001 d) newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] with next_snap:2 2024-09-17T05:26:20.397 INFO:tasks.ceph.mds.c.smithi043.stderr: -10> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: -2> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6835200640 time 2024-09-17T05:26:20.384379+0000 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f684464330e] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.398 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.399 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 0> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: in thread 7f6835200640 thread_name:MR_Finisher 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6843c42520] 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: raise() 2024-09-17T05:26:20.400 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: abort() 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6844643480] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.401 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 29: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 30: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:26:20.402 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: -211> 2024-09-17T05:26:19.076+0000 7f683a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: -12> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 mds.0.cache.den(0x10000000001 d) newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] with next_snap:2 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: -10> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: -2> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f6835200640 time 2024-09-17T05:26:20.384379+0000 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.403 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f684464330e] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.404 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.405 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 0> 2024-09-17T05:26:20.384+0000 7f6835200640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: in thread 7f6835200640 thread_name:MR_Finisher 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6843c42520] 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 3: raise() 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 4: abort() 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f6844643480] 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x560eb6b8309b] 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x560eb6e597d0] 2024-09-17T05:26:20.406 INFO:tasks.ceph.mds.c.smithi043.stderr: 8: ceph-mds(+0x404f9b) [0x560eb6e77f9b] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 9: (CDir::_omap_commit(int)+0x7ff) [0x560eb6e78d53] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x560eb6e79af6] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11a) [0x560eb6e69834] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x560eb6bf0f1f] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x560eb6f2cbd7] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 14: (MDLog::trim_to(unsigned long)+0x922) [0x560eb6f32be2] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 15: ceph-mds(+0x111547) [0x560eb6b84547] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 16: ceph-mds(+0x1116d0) [0x560eb6b846d0] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 17: ceph-mds(+0x15a41b) [0x560eb6bcd41b] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 18: ceph-mds(+0xf027f) [0x560eb6b6327f] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x560eb6f20c2e] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 20: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 21: ceph-mds(+0x4adc87) [0x560eb6f20c87] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 22: (MDSContext::complete(int)+0x7c) [0x560eb6f27196] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x560eb6f27815] 2024-09-17T05:26:20.407 INFO:tasks.ceph.mds.c.smithi043.stderr: 24: (MDSLogContextBase::complete(int)+0x46) [0x560eb6f27892] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f68445d3d95] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 26: ceph-mds(+0x108223) [0x560eb6b7b223] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f684460ce65] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f684460ce81] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 29: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f6843c94b43] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 30: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f6843d26a00] 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:26:20.408 INFO:tasks.ceph.mds.c.smithi043.stderr: 2024-09-17T05:26:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 -- 172.21.15.43:0/3996441662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140d0f50 msgr2=0x7f1e140cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 --2- 172.21.15.43:0/3996441662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140d0f50 0x7f1e140cce70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1e1409bbe0 tx=0x7f1e0001c820 comp rx=0 tx=0).stop 2024-09-17T05:26:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 -- 172.21.15.43:0/3996441662 shutdown_connections 2024-09-17T05:26:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 --2- 172.21.15.43:0/3996441662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140d0f50 0x7f1e140cce70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 -- 172.21.15.43:0/3996441662 >> 172.21.15.43:0/3996441662 conn(0x7f1e140cee80 msgr2=0x7f1e140cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 -- 172.21.15.43:0/3996441662 shutdown_connections 2024-09-17T05:26:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.544+0000 7f1e1c000640 1 -- 172.21.15.43:0/3996441662 wait complete. 2024-09-17T05:26:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 Processor -- start 2024-09-17T05:26:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 -- start start 2024-09-17T05:26:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140b7ce0 0x7f1e1414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e140d0f50 0x7f1e1415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1416b030 con 0x7f1e140cd3b0 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1416aeb0 con 0x7f1e140b7ce0 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1c000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1416b1b0 con 0x7f1e140d0f50 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59566/0 (socket says 172.21.15.43:59566) 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 -- 172.21.15.43:0/2872271274 learned_addr learned my addr 172.21.15.43:0/2872271274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e19600640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140b7ce0 0x7f1e1414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e1a000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e140d0f50 0x7f1e1415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 -- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e140d0f50 msgr2=0x7f1e1415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e140d0f50 0x7f1e1415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 -- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140b7ce0 msgr2=0x7f1e1414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140b7ce0 0x7f1e1414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.548+0000 7f1e18c00640 1 -- 172.21.15.43:0/2872271274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14160490 con 0x7f1e140cd3b0 2024-09-17T05:26:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e18c00640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e0001cd00 tx=0x7f1e000a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000be070 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e00005040 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e14150bf0 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00005470 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e14150e90 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e140cce70 con 0x7f1e140cd3b0 2024-09-17T05:26:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.552+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e000046d0 con 0x7f1e140cd3b0 2024-09-17T05:26:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.556+0000 7f1e11600640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 0x7f1df4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.556+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f1e0012df30 con 0x7f1e140cd3b0 2024-09-17T05:26:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.556+0000 7f1e19600640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 0x7f1df4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.556+0000 7f1e19600640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 0x7f1df4075740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000fc0 tx=0x7f1e08009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.560+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e000ac030 con 0x7f1e140cd3b0 2024-09-17T05:26:20.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk reading from fd=18 : -104 Unknown error -104 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-09-17T05:26:20.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.608+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T05:26:20.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.780+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e1415ff40 con 0x7f1e140cd3b0 2024-09-17T05:26:20.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.780+0000 7f1e11600640 1 -- 172.21.15.43:0/2872271274 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5209 (secure 0 0 0) 0x7f1e00016410 con 0x7f1e140cd3b0 2024-09-17T05:26:20.782 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:20.782 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"btime":"2024-09-17T05:26:20:122817+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:19.627756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:20.782 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-09-17T05:26:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 msgr2=0x7f1df4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 0x7f1df4075740 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000fc0 tx=0x7f1e08009040 comp rx=0 tx=0).stop 2024-09-17T05:26:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 msgr2=0x7f1e1415f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e0001cd00 tx=0x7f1e000a6570 comp rx=0 tx=0).stop 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 shutdown_connections 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1df4073230 0x7f1df4075740 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1e140d0f50 0x7f1e1415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cd3b0 0x7f1e1415f680 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 --2- 172.21.15.43:0/2872271274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e140b7ce0 0x7f1e1414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 >> 172.21.15.43:0/2872271274 conn(0x7f1e140cee80 msgr2=0x7f1e140712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 shutdown_connections 2024-09-17T05:26:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.784+0000 7f1e1c000640 1 -- 172.21.15.43:0/2872271274 wait complete. 2024-09-17T05:26:20.804 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:26:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.812+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.812+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:20.812+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T05:26:20.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:21.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:21.128+0000 7f2b65600640 1 -- 172.21.15.43:0/3013468311 <== mon.0 v2:172.21.15.43:3300/0 8 ==== log(2 entries from seq 581 at 2024-09-17T05:26:20.128582+0000) ==== 638+0+0 (secure 0 0 0) 0x7f2b54091a40 con 0x7f2b680cd3e0 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: cluster: 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: id: 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: health: HEALTH_OK 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: services: 2024-09-17T05:26:21.131 INFO:teuthology.orchestra.run.smithi043.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: mgr: x(active, since 3m), standbys: y 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: mds: 1/1 daemons up, 3 standby 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: data: 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: volumes: 1/1 healthy 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: pools: 3 pools, 97 pgs 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: objects: 29 objects, 601 KiB 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: usage: 242 MiB used, 720 GiB / 720 GiB avail 2024-09-17T05:26:21.132 INFO:teuthology.orchestra.run.smithi043.stdout: pgs: 97 active+clean 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout: io: 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout: client: 7.8 KiB/s rd, 3.2 KiB/s wr, 6 op/s rd, 3 op/s wr 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:21.133 INFO:teuthology.orchestra.run.smithi043.stdout:2024-09-17T05:26:20.384347+0000 mds.c [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/b/d [fffffffffffffff6,head] auth (dversion lock) v=4 ino=0x10000000002 state=1610612736 | inodepin=1 dirty=1 0x560ed2567400] 2024-09-17T05:26:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:21.216+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:21.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:21.216+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:21.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:21.216+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T05:26:21.219 INFO:tasks.ceph.mds.c.smithi043.stderr:daemon-helper: command crashed with signal 6 2024-09-17T05:26:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:22.020+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:22.020+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:22.020+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T05:26:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:23.624+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:23.624+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:23.624+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-17T05:26:24.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T05:26:25.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 -- 172.21.15.43:0/2431938130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa280040c0 msgr2=0x7ffa28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 --2- 172.21.15.43:0/2431938130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa280040c0 0x7ffa28004810 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009b60 tx=0x7ffa2c01d280 comp rx=0 tx=0).stop 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa46400640 1 -- 172.21.15.43:0/2431938130 reap_dead start 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 -- 172.21.15.43:0/2431938130 shutdown_connections 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 -- 172.21.15.43:0/2431938130 >> 172.21.15.43:0/2431938130 conn(0x7ffa400b23d0 msgr2=0x7ffa400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:25.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 -- 172.21.15.43:0/2431938130 shutdown_connections 2024-09-17T05:26:25.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.984+0000 7ffa48400640 1 -- 172.21.15.43:0/2431938130 wait complete. 2024-09-17T05:26:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 Processor -- start 2024-09-17T05:26:25.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 -- start start 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa280040c0 0x7ffa400cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400c50c0 0x7ffa400ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401667f0 con 0x7ffa400cae20 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40166670 con 0x7ffa280040c0 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa48400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffa40166970 con 0x7ffa400c50c0 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58760/0 (socket says 172.21.15.43:58760) 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 -- 172.21.15.43:0/184486667 learned_addr learned my addr 172.21.15.43:0/184486667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa45000640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400c50c0 0x7ffa400ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa45a00640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa280040c0 0x7ffa400cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 -- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400c50c0 msgr2=0x7ffa400ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400c50c0 0x7ffa400ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 -- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa280040c0 msgr2=0x7ffa400cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa280040c0 0x7ffa400cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 -- 172.21.15.43:0/184486667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400c5740 con 0x7ffa400cae20 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.988+0000 7ffa46400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800fa00 con 0x7ffa400cae20 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa38019070 con 0x7ffa400cae20 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa400c4340 con 0x7ffa400cae20 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800fe60 con 0x7ffa400cae20 2024-09-17T05:26:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa400c45e0 con 0x7ffa400cae20 2024-09-17T05:26:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa400c5910 con 0x7ffa400cae20 2024-09-17T05:26:25.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa3801e460 con 0x7ffa400cae20 2024-09-17T05:26:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 0x7ffa20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:25.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7ffa38094470 con 0x7ffa400cae20 2024-09-17T05:26:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.992+0000 7ffa45a00640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 0x7ffa20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:25.996+0000 7ffa45a00640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 0x7ffa20075740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa400715c0 tx=0x7ffa340091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:25.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.000+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa400c5910 con 0x7ffa400cae20 2024-09-17T05:26:26.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.204+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa400b33e0 con 0x7ffa400cae20 2024-09-17T05:26:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa3d600640 1 -- 172.21.15.43:0/184486667 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5209 (secure 0 0 0) 0x7ffa400b3d30 con 0x7ffa400cae20 2024-09-17T05:26:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:26.207 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"btime":"2024-09-17T05:26:20:122817+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:19.627756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 msgr2=0x7ffa20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 0x7ffa20075740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa400715c0 tx=0x7ffa340091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 msgr2=0x7ffa4009d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38006660 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 shutdown_connections 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa20073230 0x7ffa20075740 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa400cae20 0x7ffa4009d0f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa400c50c0 0x7ffa400ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 --2- 172.21.15.43:0/184486667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa280040c0 0x7ffa400cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 >> 172.21.15.43:0/184486667 conn(0x7ffa400b23d0 msgr2=0x7ffa400b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:26.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 shutdown_connections 2024-09-17T05:26:26.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.208+0000 7ffa48400640 1 -- 172.21.15.43:0/184486667 wait complete. 2024-09-17T05:26:26.230 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:26:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.828+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.828+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:26.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:26.828+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-17T05:26:30.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-17T05:26:31.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 -- 172.21.15.43:0/2902344088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c815e880 msgr2=0x7fb5c80713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 --2- 172.21.15.43:0/2902344088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c815e880 0x7fb5c80713a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009900 tx=0x7fb5bc01c6d0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 -- 172.21.15.43:0/2902344088 shutdown_connections 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 --2- 172.21.15.43:0/2902344088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c8073ec0 0x7fb5c80742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 --2- 172.21.15.43:0/2902344088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c815e880 0x7fb5c80713a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 -- 172.21.15.43:0/2902344088 >> 172.21.15.43:0/2902344088 conn(0x7fb5c80d1100 msgr2=0x7fb5c80d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 -- 172.21.15.43:0/2902344088 shutdown_connections 2024-09-17T05:26:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.400+0000 7fb5ce800640 1 -- 172.21.15.43:0/2902344088 wait complete. 2024-09-17T05:26:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 Processor -- start 2024-09-17T05:26:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 -- start start 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 0x7fb5c813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 0x7fb5c8157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8168b40 con 0x7fb5c8073ec0 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c81689c0 con 0x7fb5c815e880 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5ce800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c8168cc0 con 0x7fb5c815ec90 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58776/0 (socket says 172.21.15.43:58776) 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.404+0000 7fb5cd400640 1 -- 172.21.15.43:0/430480142 learned_addr learned my addr 172.21.15.43:0/430480142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cde00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 0x7fb5c8157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 -- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 msgr2=0x7fb5c8157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cca00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 0x7fb5c813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 0x7fb5c8157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 -- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 msgr2=0x7fb5c813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 0x7fb5c813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 -- 172.21.15.43:0/430480142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c806da30 con 0x7fb5c8073ec0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cde00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 0x7fb5c8157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cca00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 0x7fb5c813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5cd400640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b0040 tx=0x7fb5bc0a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0bb070 con 0x7fb5c8073ec0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5bc0043b0 con 0x7fb5c8073ec0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0b2040 con 0x7fb5c8073ec0 2024-09-17T05:26:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c815a180 con 0x7fb5c8073ec0 2024-09-17T05:26:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c815a740 con 0x7fb5c8073ec0 2024-09-17T05:26:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c80742d0 con 0x7fb5c8073ec0 2024-09-17T05:26:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.408+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5bc0be070 con 0x7fb5c8073ec0 2024-09-17T05:26:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.412+0000 7fb5c5600640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 0x7fb5a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.412+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fb5bc12dbd0 con 0x7fb5c8073ec0 2024-09-17T05:26:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.412+0000 7fb5cca00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 0x7fb5a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:31.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.412+0000 7fb5cca00640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 0x7fb5a8075710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8145e30 tx=0x7fb5b4008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.416+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb5c8074810 con 0x7fb5c8073ec0 2024-09-17T05:26:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5c8070ac0 con 0x7fb5c8073ec0 2024-09-17T05:26:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5c5600640 1 -- 172.21.15.43:0/430480142 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5209 (secure 0 0 0) 0x7fb5bc0f8b50 con 0x7fb5c8073ec0 2024-09-17T05:26:31.624 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:31.624 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"btime":"2024-09-17T05:26:20:122817+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:19.627756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 msgr2=0x7fb5a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 0x7fb5a8075710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8145e30 tx=0x7fb5b4008040 comp rx=0 tx=0).stop 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 msgr2=0x7fb5c81653d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b0040 tx=0x7fb5bc0a5510 comp rx=0 tx=0).stop 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 shutdown_connections 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb5a8073200 0x7fb5a8075710 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5c815e880 0x7fb5c813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb5c815ec90 0x7fb5c8157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 --2- 172.21.15.43:0/430480142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c8073ec0 0x7fb5c81653d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:31.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 >> 172.21.15.43:0/430480142 conn(0x7fb5c80d1100 msgr2=0x7fb5c80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:31.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 shutdown_connections 2024-09-17T05:26:31.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:31.624+0000 7fb5ce800640 1 -- 172.21.15.43:0/430480142 wait complete. 2024-09-17T05:26:31.647 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:26:33.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:33.232+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:6832/61022679 2024-09-17T05:26:33.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:33.232+0000 7f76ff400640 1 client.6126 ms_handle_refused on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:33.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:33.232+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-17T05:26:36.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 -- 172.21.15.43:0/3733132365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5406fc90 msgr2=0x7f5b540700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 --2- 172.21.15.43:0/3733132365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5406fc90 0x7f5b540700a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5b540d0e90 tx=0x7f5b3c01d200 comp rx=0 tx=0).stop 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 -- 172.21.15.43:0/3733132365 shutdown_connections 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 --2- 172.21.15.43:0/3733132365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5406fc90 0x7f5b540700a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 --2- 172.21.15.43:0/3733132365 >> v2:172.21.15.62:3300/0 conn(0x7f5b540730b0 0x7f5b540734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 --2- 172.21.15.43:0/3733132365 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5415fe10 0x7f5b540b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 -- 172.21.15.43:0/3733132365 >> 172.21.15.43:0/3733132365 conn(0x7f5b540b23d0 msgr2=0x7f5b540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 -- 172.21.15.43:0/3733132365 shutdown_connections 2024-09-17T05:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.820+0000 7f5b58c00640 1 -- 172.21.15.43:0/3733132365 wait complete. 2024-09-17T05:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 Processor -- start 2024-09-17T05:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- start start 2024-09-17T05:26:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5406fc90 0x7f5b54162b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 0x7f5b5415a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540cea10 con 0x7f5b540730b0 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540ce890 con 0x7f5b5415fe10 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540ceb90 con 0x7f5b5406fc90 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58488/0 (socket says 172.21.15.43:58488) 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 -- 172.21.15.43:0/288982905 learned_addr learned my addr 172.21.15.43:0/288982905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b51e00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5406fc90 0x7f5b54162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b52800640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 0x7f5b5415a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 -- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5406fc90 msgr2=0x7f5b54162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5406fc90 0x7f5b54162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 -- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 msgr2=0x7f5b5415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 0x7f5b5415a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 -- 172.21.15.43:0/288982905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541fcd00 con 0x7f5b540730b0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b52800640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 0x7f5b5415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4be00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c01ae50 tx=0x7f5b3c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0ba020 con 0x7f5b540730b0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b3c004130 con 0x7f5b540730b0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0b2040 con 0x7f5b540730b0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541fcfe0 con 0x7f5b540730b0 2024-09-17T05:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.824+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541fc3b0 con 0x7f5b540730b0 2024-09-17T05:26:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.828+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b541fd500 con 0x7f5b540730b0 2024-09-17T05:26:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.828+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5b3c0bd070 con 0x7f5b540730b0 2024-09-17T05:26:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.828+0000 7f5b4a000640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 0x7f5b30075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.828+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f5b3c12d9b0 con 0x7f5b540730b0 2024-09-17T05:26:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.828+0000 7f5b51e00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 0x7f5b30075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:36.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.832+0000 7f5b51e00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 0x7f5b30075740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009b40 tx=0x7f5b440091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:36.832+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5b541fd500 con 0x7f5b540730b0 2024-09-17T05:26:37.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b540734c0 con 0x7f5b540730b0 2024-09-17T05:26:37.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b4a000640 1 -- 172.21.15.43:0/288982905 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5209 (secure 0 0 0) 0x7f5b3c0164b0 con 0x7f5b540730b0 2024-09-17T05:26:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:37.044 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"btime":"2024-09-17T05:26:20:122817+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:19.627756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:37.044 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 msgr2=0x7f5b30075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 0x7f5b30075740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009b40 tx=0x7f5b440091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 msgr2=0x7f5b541fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c01ae50 tx=0x7f5b3c005760 comp rx=0 tx=0).stop 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 shutdown_connections 2024-09-17T05:26:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b30073230 0x7f5b30075740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5415fe10 0x7f5b5415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b540730b0 0x7f5b541fbe70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 --2- 172.21.15.43:0/288982905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5b5406fc90 0x7f5b54162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 >> 172.21.15.43:0/288982905 conn(0x7f5b540b23d0 msgr2=0x7f5b5415e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 shutdown_connections 2024-09-17T05:26:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:37.043+0000 7f5b58c00640 1 -- 172.21.15.43:0/288982905 wait complete. 2024-09-17T05:26:37.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:26:37.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-09-17T05:26:42.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- 172.21.15.43:0/2697078332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c0737c0 msgr2=0x7fa76c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 --2- 172.21.15.43:0/2697078332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c0737c0 0x7fa76c0700a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa764009c80 tx=0x7fa76401cc40 comp rx=0 tx=0).stop 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- 172.21.15.43:0/2697078332 shutdown_connections 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 --2- 172.21.15.43:0/2697078332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c0737c0 0x7fa76c0700a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 --2- 172.21.15.43:0/2697078332 >> v2:172.21.15.62:3300/0 conn(0x7fa76c15fe10 0x7fa76c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- 172.21.15.43:0/2697078332 >> 172.21.15.43:0/2697078332 conn(0x7fa76c0b23d0 msgr2=0x7fa76c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- 172.21.15.43:0/2697078332 shutdown_connections 2024-09-17T05:26:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- 172.21.15.43:0/2697078332 wait complete. 2024-09-17T05:26:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 Processor -- start 2024-09-17T05:26:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.239+0000 7fa773c00640 1 -- start start 2024-09-17T05:26:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 0x7fa76c097790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa76c0b4bc0 0x7fa76c1fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c169fc0 con 0x7fa76c15fe10 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c169e40 con 0x7fa76c06fc90 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c16a140 con 0x7fa76c0b4bc0 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa771200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 0x7fa76c097790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58510/0 (socket says 172.21.15.43:58510) 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 -- 172.21.15.43:0/3655066481 learned_addr learned my addr 172.21.15.43:0/3655066481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa771c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa76c0b4bc0 0x7fa76c1fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 -- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa76c0b4bc0 msgr2=0x7fa76c1fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa76c0b4bc0 0x7fa76c1fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 -- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 msgr2=0x7fa76c097790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 0x7fa76c097790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 -- 172.21.15.43:0/3655066481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c099ec0 con 0x7fa76c15fe10 2024-09-17T05:26:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa771200640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 0x7fa76c097790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa76be00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c3c0 tx=0x7fa75800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758017070 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7580053c0 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c0cc210 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758003860 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c0cc7d0 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.243+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c0700a0 con 0x7fa76c15fe10 2024-09-17T05:26:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.247+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa75801a070 con 0x7fa76c15fe10 2024-09-17T05:26:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.247+0000 7fa768c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 0x7fa74c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.247+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fa7580935c0 con 0x7fa76c15fe10 2024-09-17T05:26:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.247+0000 7fa771200640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 0x7fa74c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.247+0000 7fa771200640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 0x7fa74c075740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa760009f30 tx=0x7fa7600091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.251+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa758001090 con 0x7fa76c15fe10 2024-09-17T05:26:42.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.459+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa76c0aee40 con 0x7fa76c15fe10 2024-09-17T05:26:42.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.459+0000 7fa768c00640 1 -- 172.21.15.43:0/3655066481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5257 (secure 0 0 0) 0x7fa75805de60 con 0x7fa76c15fe10 2024-09-17T05:26:42.464 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":55,"btime":"2024-09-17T05:26:38:174680+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:38.174651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6834/61022679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":61022679},{"type":"v1","addr":"172.21.15.43:6834","nonce":61022679}]},"join_fscid":-1,"laggy_since":"2024-09-17T05:26:38.174651+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6036,"qdb_cluster":[6036]},"id":4}]} 2024-09-17T05:26:42.465 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 55 2024-09-17T05:26:42.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 msgr2=0x7fa74c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 0x7fa74c075740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa760009f30 tx=0x7fa7600091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 msgr2=0x7fa76c1537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c3c0 tx=0x7fa75800c880 comp rx=0 tx=0).stop 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 shutdown_connections 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa74c073230 0x7fa74c075740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa76c0b4bc0 0x7fa76c1fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c15fe10 0x7fa76c1537c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 --2- 172.21.15.43:0/3655066481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa76c06fc90 0x7fa76c097790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 >> 172.21.15.43:0/3655066481 conn(0x7fa76c0b23d0 msgr2=0x7fa76c073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 shutdown_connections 2024-09-17T05:26:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.463+0000 7fa773c00640 1 -- 172.21.15.43:0/3655066481 wait complete. 2024-09-17T05:26:42.485 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-17T05:26:42.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- 172.21.15.43:0/73549223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc06fc90 msgr2=0x7f53dc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 --2- 172.21.15.43:0/73549223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc06fc90 0x7f53dc0700a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009900 tx=0x7f53c401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- 172.21.15.43:0/73549223 shutdown_connections 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 --2- 172.21.15.43:0/73549223 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 0x7f53dc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 --2- 172.21.15.43:0/73549223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc06fc90 0x7f53dc0700a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- 172.21.15.43:0/73549223 >> 172.21.15.43:0/73549223 conn(0x7f53dc0b23d0 msgr2=0x7f53dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- 172.21.15.43:0/73549223 shutdown_connections 2024-09-17T05:26:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- 172.21.15.43:0/73549223 wait complete. 2024-09-17T05:26:42.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 Processor -- start 2024-09-17T05:26:42.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.747+0000 7f53e0c00640 1 -- start start 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 0x7f53dc15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 0x7f53dc0992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc16a060 con 0x7f53dc15fe10 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc169ee0 con 0x7f53dc06fc90 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc16a1e0 con 0x7f53dc0730b0 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58528/0 (socket says 172.21.15.43:58528) 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 -- 172.21.15.43:0/1688042683 learned_addr learned my addr 172.21.15.43:0/1688042683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d3e00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 0x7f53dc0992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d9e00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 0x7f53dc15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 -- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 msgr2=0x7f53dc0992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 0x7f53dc0992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 -- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 msgr2=0x7f53dc15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 0x7f53dc15f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 -- 172.21.15.43:0/1688042683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc158220 con 0x7f53dc15fe10 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d9e00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 0x7f53dc15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53da800640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f53d400eab0 tx=0x7f53d400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:42.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400cde0 con 0x7f53dc15fe10 2024-09-17T05:26:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d4004b30 con 0x7f53dc15fe10 2024-09-17T05:26:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4005020 con 0x7f53dc15fe10 2024-09-17T05:26:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc0b1850 con 0x7f53dc15fe10 2024-09-17T05:26:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc098ae0 con 0x7f53dc15fe10 2024-09-17T05:26:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.751+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc0700a0 con 0x7f53dc15fe10 2024-09-17T05:26:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.755+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f53d40051c0 con 0x7f53dc15fe10 2024-09-17T05:26:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.755+0000 7f53d2000640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 0x7f53b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.755+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f53d4094d90 con 0x7f53dc15fe10 2024-09-17T05:26:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.755+0000 7f53d9e00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 0x7f53b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:42.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.755+0000 7f53d9e00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 0x7f53b8075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0016b0 tx=0x7f53cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.759+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53d405fc80 con 0x7f53dc15fe10 2024-09-17T05:26:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.931+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7f53dc158e60 con 0x7f53dc15fe10 2024-09-17T05:26:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.935+0000 7f53d2000640 1 -- 172.21.15.43:0/1688042683 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f53d405f580 con 0x7f53dc15fe10 2024-09-17T05:26:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 msgr2=0x7f53b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 0x7f53b8075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0016b0 tx=0x7f53cc009040 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 msgr2=0x7f53dc099820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f53d400eab0 tx=0x7f53d400ef70 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 shutdown_connections 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f53b8073230 0x7f53b8075740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53dc15fe10 0x7f53dc099820 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f53dc0730b0 0x7f53dc0992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 --2- 172.21.15.43:0/1688042683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53dc06fc90 0x7f53dc15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 >> 172.21.15.43:0/1688042683 conn(0x7f53dc0b23d0 msgr2=0x7f53dc0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 shutdown_connections 2024-09-17T05:26:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:42.939+0000 7f53e0c00640 1 -- 172.21.15.43:0/1688042683 wait complete. 2024-09-17T05:26:42.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 -- 172.21.15.43:0/1066990890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d0003ad0 msgr2=0x7fe9d0003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/1066990890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9d0003ad0 0x7fe9d0003f00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0096d0 tx=0x7fe9dc01f460 comp rx=0 tx=0).stop 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f0a00640 1 -- 172.21.15.43:0/1066990890 reap_dead start 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 -- 172.21.15.43:0/1066990890 shutdown_connections 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 -- 172.21.15.43:0/1066990890 >> 172.21.15.43:0/1066990890 conn(0x7fe9ec0b23d0 msgr2=0x7fe9ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:43.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 -- 172.21.15.43:0/1066990890 shutdown_connections 2024-09-17T05:26:43.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.119+0000 7fe9f2a00640 1 -- 172.21.15.43:0/1066990890 wait complete. 2024-09-17T05:26:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 Processor -- start 2024-09-17T05:26:43.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 -- start start 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0003ad0 0x7fe9ec0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ec1f8040 0x7fe9ec0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec06d580 con 0x7fe9ec1f8450 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec06d400 con 0x7fe9ec1f8040 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f2a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec06d700 con 0x7fe9d0003ad0 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58538/0 (socket says 172.21.15.43:58538) 2024-09-17T05:26:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 -- 172.21.15.43:0/3568177313 learned_addr learned my addr 172.21.15.43:0/3568177313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9ebe00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0003ad0 0x7fe9ec0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9eb400640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ec1f8040 0x7fe9ec0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 -- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0003ad0 msgr2=0x7fe9ec0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0003ad0 0x7fe9ec0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 -- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ec1f8040 msgr2=0x7fe9ec0c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ec1f8040 0x7fe9ec0c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 -- 172.21.15.43:0/3568177313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec09dd40 con 0x7fe9ec1f8450 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.123+0000 7fe9f0a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0b5290 tx=0x7fe9e400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4010890 con 0x7fe9ec1f8450 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e4017070 con 0x7fe9ec1f8450 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0c3b80 con 0x7fe9ec1f8450 2024-09-17T05:26:43.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4010de0 con 0x7fe9ec1f8450 2024-09-17T05:26:43.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0c4140 con 0x7fe9ec1f8450 2024-09-17T05:26:43.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec06dc90 con 0x7fe9ec1f8450 2024-09-17T05:26:43.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.127+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9e401c460 con 0x7fe9ec1f8450 2024-09-17T05:26:43.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.131+0000 7fe9e3e00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 0x7fe9cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:43.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.131+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7fe9e4094490 con 0x7fe9ec1f8450 2024-09-17T05:26:43.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.131+0000 7fe9ebe00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 0x7fe9cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:43.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.131+0000 7fe9ebe00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 0x7fe9cc075740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0b1000 tx=0x7fe9dc01c200 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:43.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.135+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe9ec09dd40 con 0x7fe9ec1f8450 2024-09-17T05:26:43.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:43.375+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7fe9ec0738b0 con 0x7fe9ec1f8450 2024-09-17T05:26:44.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2024-09-17T05:26:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.191+0000 7fe9e3e00640 1 -- 172.21.15.43:0/3568177313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6036 v56) ==== 115+0+0 (secure 0 0 0) 0x7fe9e405ecb0 con 0x7fe9ec1f8450 2024-09-17T05:26:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:unfreezing mds.6036 2024-09-17T05:26:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 msgr2=0x7fe9cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 0x7fe9cc075740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0b1000 tx=0x7fe9dc01c200 comp rx=0 tx=0).stop 2024-09-17T05:26:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 msgr2=0x7fe9ec0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0b5290 tx=0x7fe9e400ea10 comp rx=0 tx=0).stop 2024-09-17T05:26:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 shutdown_connections 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe9cc073230 0x7fe9cc075740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9ec1f8040 0x7fe9ec0c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ec1f8450 0x7fe9ec0c9c90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 --2- 172.21.15.43:0/3568177313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9d0003ad0 0x7fe9ec0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 >> 172.21.15.43:0/3568177313 conn(0x7fe9ec0b23d0 msgr2=0x7fe9ec0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 shutdown_connections 2024-09-17T05:26:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.195+0000 7fe9f2a00640 1 -- 172.21.15.43:0/3568177313 wait complete. 2024-09-17T05:26:44.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -n kernel.core_pattern 2024-09-17T05:26:44.245 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:26:44.247 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:26:44.247 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-17T05:26:44.338 INFO:teuthology.orchestra.run.smithi043.stdout:1726550780.39430.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-17T05:26:44.341 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726550780.39430.core'] 2024-09-17T05:26:44.341 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726550780.39430.core, deleting it 2024-09-17T05:26:44.342 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726550780.39430.core 2024-09-17T05:26:44.439 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:26:44.439 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:26:44.439 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:26:44.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:26:44.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:26:44.440 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi043 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-17T05:26:44.441 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:26:44.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:26:44.444 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:26:44.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:44.547 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:26:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 -- 172.21.15.43:0/1286705870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3af0 msgr2=0x7f54f815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 --2- 172.21.15.43:0/1286705870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3af0 0x7f54f815c7e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009840 tx=0x7f54e001c760 comp rx=0 tx=0).stop 2024-09-17T05:26:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 -- 172.21.15.43:0/1286705870 shutdown_connections 2024-09-17T05:26:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 --2- 172.21.15.43:0/1286705870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f806f110 0x7f54f80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 --2- 172.21.15.43:0/1286705870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f80b3af0 0x7f54f815c7e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 -- 172.21.15.43:0/1286705870 >> 172.21.15.43:0/1286705870 conn(0x7f54f80b23d0 msgr2=0x7f54f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 -- 172.21.15.43:0/1286705870 shutdown_connections 2024-09-17T05:26:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.607+0000 7f54fd800640 1 -- 172.21.15.43:0/1286705870 wait complete. 2024-09-17T05:26:44.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 Processor -- start 2024-09-17T05:26:44.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- start start 2024-09-17T05:26:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f8155d70 0x7f54f80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f809c790 0x7f54f80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54f8168c60 con 0x7f54f806f110 2024-09-17T05:26:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f54f8168ae0 con 0x7f54f8155d70 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f54f8168de0 con 0x7f54f809c790 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41716/0 (socket says 172.21.15.43:41716) 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 -- 172.21.15.43:0/2490934995 learned_addr learned my addr 172.21.15.43:0/2490934995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54ee000640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f8155d70 0x7f54f80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f7400640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f809c790 0x7f54f80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 -- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f809c790 msgr2=0x7f54f80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f809c790 0x7f54f80ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 -- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f8155d70 msgr2=0x7f54f80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f8155d70 0x7f54f80cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 -- 172.21.15.43:0/2490934995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f80c9840 con 0x7f54f806f110 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54f6a00640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c400 tx=0x7f54e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8011790 con 0x7f54f806f110 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e8017070 con 0x7f54f806f110 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8011cc0 con 0x7f54f806f110 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f80c7450 con 0x7f54f806f110 2024-09-17T05:26:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.611+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f80c9b10 con 0x7f54f806f110 2024-09-17T05:26:44.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.615+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f80ad0b0 con 0x7f54f806f110 2024-09-17T05:26:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.615+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54e801c460 con 0x7f54f806f110 2024-09-17T05:26:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.615+0000 7f54ef400640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 0x7f54d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.615+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f54e8093fb0 con 0x7f54f806f110 2024-09-17T05:26:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.615+0000 7f54ee000640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 0x7f54d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.619+0000 7f54ee000640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 0x7f54d4075710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54e00af000 tx=0x7f54e001afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.623+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f54e805e880 con 0x7f54f806f110 2024-09-17T05:26:44.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.831+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f806d9c0 con 0x7f54f806f110 2024-09-17T05:26:44.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.831+0000 7f54ef400640 1 -- 172.21.15.43:0/2490934995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+4383 (secure 0 0 0) 0x7f54e80632b0 con 0x7f54f806f110 2024-09-17T05:26:44.834 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:44.834 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":57,"btime":"2024-09-17T05:26:44:197540+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:44.197531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-17T05:26:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 57 2024-09-17T05:26:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 msgr2=0x7f54d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 0x7f54d4075710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54e00af000 tx=0x7f54e001afa0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 msgr2=0x7f54f80ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c400 tx=0x7f54e800c8c0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 shutdown_connections 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f54d4073200 0x7f54d4075710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54f8155d70 0x7f54f80cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f54f809c790 0x7f54f80ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 --2- 172.21.15.43:0/2490934995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54f806f110 0x7f54f80ca2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 >> 172.21.15.43:0/2490934995 conn(0x7f54f80b23d0 msgr2=0x7f54f80b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 shutdown_connections 2024-09-17T05:26:44.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:44.835+0000 7f54fd800640 1 -- 172.21.15.43:0/2490934995 wait complete. 2024-09-17T05:26:45.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:44.197531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5695}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5695': {'gid': 5695, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/48203361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 48203361}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 48203361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:26:45.259 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5695:up:replay 2024-09-17T05:26:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-09-17T05:26:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-09-17T05:26:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f76ff400640 0 client.6126 ms_handle_remote_reset on v2:172.21.15.43:6832/61022679 2024-09-17T05:26:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:26:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:26:46.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:26:46.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.047+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:26:46.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- 172.21.15.43:0/17556315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa48004d50 msgr2=0x7faa48005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 --2- 172.21.15.43:0/17556315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa48004d50 0x7faa48005160 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c610 tx=0x7faa5c01faa0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa66800640 1 -- 172.21.15.43:0/17556315 reap_dead start 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- 172.21.15.43:0/17556315 shutdown_connections 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- 172.21.15.43:0/17556315 >> 172.21.15.43:0/17556315 conn(0x7faa600cee80 msgr2=0x7faa600cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- 172.21.15.43:0/17556315 shutdown_connections 2024-09-17T05:26:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- 172.21.15.43:0/17556315 wait complete. 2024-09-17T05:26:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 Processor -- start 2024-09-17T05:26:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.431+0000 7faa68800640 1 -- start start 2024-09-17T05:26:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 0x7faa6015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa601625c0 0x7faa600b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa6006f130 con 0x7faa601e82a0 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa6006efb0 con 0x7faa601625c0 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faa600b7dd0 con 0x7faa48004d50 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41726/0 (socket says 172.21.15.43:41726) 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 -- 172.21.15.43:0/3164656437 learned_addr learned my addr 172.21.15.43:0/3164656437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa66800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa601625c0 0x7faa600b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65e00640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 0x7faa6015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 -- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 msgr2=0x7faa6015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 0x7faa6015c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 -- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa601625c0 msgr2=0x7faa600b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:46.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa601625c0 0x7faa600b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 -- 172.21.15.43:0/3164656437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601e86f0 con 0x7faa601e82a0 2024-09-17T05:26:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65e00640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 0x7faa6015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa65400640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa50011ae0 tx=0x7faa50009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:46.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5001e070 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa50004460 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5000fbe0 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa601ea4f0 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa600cbc10 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.435+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa600b75a0 con 0x7faa601e82a0 2024-09-17T05:26:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.439+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa50021070 con 0x7faa601e82a0 2024-09-17T05:26:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.439+0000 7faa5a000640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 0x7faa44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.439+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7faa50094d20 con 0x7faa601e82a0 2024-09-17T05:26:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.439+0000 7faa65e00640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 0x7faa44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:46.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.439+0000 7faa65e00640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 0x7faa44075710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa54005fd0 tx=0x7faa54005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:46.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.443+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faa5005f5f0 con 0x7faa601e82a0 2024-09-17T05:26:46.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.651+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa601ea260 con 0x7faa601e82a0 2024-09-17T05:26:46.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.651+0000 7faa5a000640 1 -- 172.21.15.43:0/3164656437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5209 (secure 0 0 0) 0x7faa50066020 con 0x7faa601e82a0 2024-09-17T05:26:46.653 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:46.654 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":59,"btime":"2024-09-17T05:26:46:202644+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:45.205119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-17T05:26:46.654 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 59 2024-09-17T05:26:46.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 msgr2=0x7faa44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:46.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 0x7faa44075710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faa54005fd0 tx=0x7faa54005f40 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 msgr2=0x7faa600b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa50011ae0 tx=0x7faa50009e10 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 shutdown_connections 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa44073200 0x7faa44075710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa601625c0 0x7faa600b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa601e82a0 0x7faa600b3230 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 --2- 172.21.15.43:0/3164656437 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa48004d50 0x7faa6015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 >> 172.21.15.43:0/3164656437 conn(0x7faa600cee80 msgr2=0x7faa6015eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 shutdown_connections 2024-09-17T05:26:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:46.655+0000 7faa68800640 1 -- 172.21.15.43:0/3164656437 wait complete. 2024-09-17T05:26:47.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:45.205119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5695}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5695': {'gid': 5695, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:rejoin', 'state_seq': 15, 'addr': '172.21.15.62:6837/48203361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 48203361}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 48203361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:26:47.079 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5695:up:rejoin 2024-09-17T05:26:48.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 -- 172.21.15.43:0/984280797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 msgr2=0x7f4784004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 --2- 172.21.15.43:0/984280797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f4784004c20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4788009b40 tx=0x7f478801d270 comp rx=0 tx=0).stop 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a5000640 1 -- 172.21.15.43:0/984280797 reap_dead start 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 -- 172.21.15.43:0/984280797 shutdown_connections 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 -- 172.21.15.43:0/984280797 >> 172.21.15.43:0/984280797 conn(0x7f47a00b23d0 msgr2=0x7f47a00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 -- 172.21.15.43:0/984280797 shutdown_connections 2024-09-17T05:26:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.251+0000 7f47a7000640 1 -- 172.21.15.43:0/984280797 wait complete. 2024-09-17T05:26:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 Processor -- start 2024-09-17T05:26:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 -- start start 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 0x7f47a015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a015ac50 0x7f47a0160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a01673c0 con 0x7f4784004810 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47a015cc10 con 0x7f47a015eb00 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f47a7000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f47a0167540 con 0x7f47a015ac50 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f479fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f479fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41754/0 (socket says 172.21.15.43:41754) 2024-09-17T05:26:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.255+0000 7f479fe00640 1 -- 172.21.15.43:0/2606910205 learned_addr learned my addr 172.21.15.43:0/2606910205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 -- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a015ac50 msgr2=0x7f47a0160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a015ac50 0x7f47a0160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 -- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 msgr2=0x7f47a015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f4797e00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 0x7f47a015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 0x7f47a015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 -- 172.21.15.43:0/2606910205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a015b060 con 0x7f4784004810 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f4797e00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 0x7f47a015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479fe00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4790006630 tx=0x7f47900039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790015070 con 0x7f4784004810 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4790011900 con 0x7f4784004810 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a0155880 con 0x7f4784004810 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790011d50 con 0x7f4784004810 2024-09-17T05:26:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a008fae0 con 0x7f4784004810 2024-09-17T05:26:48.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00ad0b0 con 0x7f4784004810 2024-09-17T05:26:48.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.259+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4790018070 con 0x7f4784004810 2024-09-17T05:26:48.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.263+0000 7f479cc00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 0x7f477c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:48.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.263+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f47900950d0 con 0x7f4784004810 2024-09-17T05:26:48.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.263+0000 7f4797e00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 0x7f477c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:48.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.263+0000 7f4797e00640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 0x7f477c075740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47880ad000 tx=0x7f478801a010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.267+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f47a015b060 con 0x7f4784004810 2024-09-17T05:26:48.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.475+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47a00aec60 con 0x7f4784004810 2024-09-17T05:26:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.475+0000 7f479cc00640 1 -- 172.21.15.43:0/2606910205 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5216 (secure 0 0 0) 0x7f479005ff60 con 0x7f4784004810 2024-09-17T05:26:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:26:48.480 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"btime":"2024-09-17T05:26:47:209080+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:47.209072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-17T05:26:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 60 2024-09-17T05:26:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 msgr2=0x7f477c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 0x7f477c075740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47880ad000 tx=0x7f478801a010 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 msgr2=0x7f47a0155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4790006630 tx=0x7f47900039d0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 shutdown_connections 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f477c073230 0x7f477c075740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47a015eb00 0x7f47a015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f47a015ac50 0x7f47a0160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 --2- 172.21.15.43:0/2606910205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4784004810 0x7f47a0155340 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 >> 172.21.15.43:0/2606910205 conn(0x7f47a00b23d0 msgr2=0x7f47a00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:48.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 shutdown_connections 2024-09-17T05:26:48.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:48.479+0000 7f47a7000640 1 -- 172.21.15.43:0/2606910205 wait complete. 2024-09-17T05:26:48.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:57.018678+0000', 'modified': '2024-09-17T05:26:47.209072+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 63, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5695}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5695': {'gid': 5695, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.62:6837/48203361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 48203361}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 48203361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5695, 'qdb_cluster': [5695]} 2024-09-17T05:26:48.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:26:48.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:26:49.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 -- 172.21.15.43:0/1574416782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80b3af0 msgr2=0x7f02c815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 --2- 172.21.15.43:0/1574416782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80b3af0 0x7f02c815c7e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009900 tx=0x7f02b401c820 comp rx=0 tx=0).stop 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 -- 172.21.15.43:0/1574416782 shutdown_connections 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 --2- 172.21.15.43:0/1574416782 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 0x7f02c80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 --2- 172.21.15.43:0/1574416782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80b3af0 0x7f02c815c7e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 -- 172.21.15.43:0/1574416782 >> 172.21.15.43:0/1574416782 conn(0x7f02c80b23d0 msgr2=0x7f02c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 -- 172.21.15.43:0/1574416782 shutdown_connections 2024-09-17T05:26:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.063+0000 7f02d0200640 1 -- 172.21.15.43:0/1574416782 wait complete. 2024-09-17T05:26:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 Processor -- start 2024-09-17T05:26:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 -- start start 2024-09-17T05:26:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 0x7f02c80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80c68f0 0x7f02c81fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8168bc0 con 0x7f02c8155d70 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8168a40 con 0x7f02c80c68f0 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02d0200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f02c8168d40 con 0x7f02c806f110 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cd800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 0x7f02c80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41766/0 (socket says 172.21.15.43:41766) 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 -- 172.21.15.43:0/3265718857 learned_addr learned my addr 172.21.15.43:0/3265718857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02ce200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80c68f0 0x7f02c81fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 -- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 msgr2=0x7f02c80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 0x7f02c80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 -- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80c68f0 msgr2=0x7f02c81fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80c68f0 0x7f02c81fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.067+0000 7f02cce00640 1 -- 172.21.15.43:0/3265718857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c815d080 con 0x7f02c8155d70 2024-09-17T05:26:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02cce00640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f02b40b0040 tx=0x7f02b40a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02c5600640 1 -- 172.21.15.43:0/3265718857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b40b2040 con 0x7f02c8155d70 2024-09-17T05:26:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02c5600640 1 -- 172.21.15.43:0/3265718857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b40041b0 con 0x7f02c8155d70 2024-09-17T05:26:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02c5600640 1 -- 172.21.15.43:0/3265718857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4003600 con 0x7f02c8155d70 2024-09-17T05:26:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c8073700 con 0x7f02c8155d70 2024-09-17T05:26:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c8155980 con 0x7f02c8155d70 2024-09-17T05:26:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.071+0000 7f02c5600640 1 -- 172.21.15.43:0/3265718857 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f02b40bf020 con 0x7f02c8155d70 2024-09-17T05:26:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02c5600640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 0x7f02a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02c5600640 1 -- 172.21.15.43:0/3265718857 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f02b412e2e0 con 0x7f02c8155d70 2024-09-17T05:26:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02cd800640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 0x7f02a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 Processor -- start 2024-09-17T05:26:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 -- start start 2024-09-17T05:26:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02cd800640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 0x7f02a8075740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02bc007fd0 tx=0x7f02bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 0x55adfb4266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb426be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adfb2fb410 con 0x55adfb2beca0 2024-09-17T05:26:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55adfb238b40 con 0x55adfb2d5950 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55adfb227f70 con 0x55adfaee5460 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41770/0 (socket says 172.21.15.43:41770) 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 -- 172.21.15.43:0/108798270 learned_addr learned my addr 172.21.15.43:0/108798270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02cce00640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb426be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02cd800640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 0x55adfb4266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 -- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 msgr2=0x55adfb4266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 0x55adfb4266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 -- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 msgr2=0x55adfb426be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb426be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 -- 172.21.15.43:0/108798270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adfb1ef5b0 con 0x55adfb2beca0 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02cd800640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 0x55adfb4266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ce200640 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009f60 tx=0x7f02c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e7eded11cb6080 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.075+0000 7f02ba000640 1 -- 172.21.15.43:0/108798270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0004d30 con 0x55adfb2beca0 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02ba000640 1 -- 172.21.15.43:0/108798270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02c0003980 con 0x55adfb2beca0 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02ba000640 1 -- 172.21.15.43:0/108798270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0003d30 con 0x55adfb2beca0 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- 172.21.15.43:0/108798270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 msgr2=0x55adfb427120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009f60 tx=0x7f02c0009f90 comp rx=0 tx=0).stop 2024-09-17T05:26:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- 172.21.15.43:0/108798270 shutdown_connections 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb426be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb427120 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- 172.21.15.43:0/108798270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfaee5460 0x55adfb4266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- 172.21.15.43:0/108798270 >> 172.21.15.43:0/108798270 conn(0x55adfb3abd90 msgr2=0x55adfb3db690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- 172.21.15.43:0/108798270 shutdown_connections 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- 172.21.15.43:0/108798270 wait complete. 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- start start 2024-09-17T05:26:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 Processor -- start 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb42d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2d5950 0x55adfb42e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adfb1c25f0 con 0x55adfb2beca0 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55adfb22bd80 con 0x55adfaee5460 2024-09-17T05:26:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.079+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55adfb1ce520 con 0x55adfb2d5950 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 10 client.0 _ll_register_callbacks cb 0x55adfb1d2830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41778/0 (socket says 172.21.15.43:41778) 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 -- 172.21.15.43:0/599443670 learned_addr learned my addr 172.21.15.43:0/599443670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cd800640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb42d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02ce200640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2d5950 0x55adfb42e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 -- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2d5950 msgr2=0x55adfb42e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2d5950 0x55adfb42e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 -- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 msgr2=0x55adfb42d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb42d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 -- 172.21.15.43:0/599443670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adfb1c6190 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cce00640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f02b40f5eb0 tx=0x7f02b40f8470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0436906d247614 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4102290 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55adfb1c0110 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f02b40f7ea0 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f02b412e0b0 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 10 client.6201 fetch_fsmap learned FSMap version 60 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55adfb2acf10 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55adfb1c0110 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f02b411e0b0 con 0x55adfb2beca0 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 10 client.6201 fetch_fsmap finished waiting for FSMap version 60 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 10 client.6201 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 20 client.6201 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeca7d0 0x55adfb437d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 4 client.6201 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:26:49.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 --> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adfb23aa90 con 0x55adfaeca7d0 2024-09-17T05:26:49.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cd800640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeca7d0 0x55adfb437d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02aea00640 10 client.6201 ms_handle_connect on v2:172.21.15.62:6835/48203361 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.083+0000 7f02cd800640 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeca7d0 0x55adfb437d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c843981bfa63aa01 server_cookie=8125622d760db805 in_seq=0 out_seq=0 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02aea00640 1 -- 172.21.15.43:0/599443670 <== mds.0 v2:172.21.15.62:6835/48203361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adfb23aa90 con 0x55adfaeca7d0 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02aea00640 10 client.6201 handle_command_reply: tid=0 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 client.6201 shutdown 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 msgr2=0x55adfb42de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f02b40f5eb0 tx=0x7f02b40f8470 comp rx=0 tx=0).stop 2024-09-17T05:26:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 shutdown_connections 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeca7d0 0x55adfb437d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2d5950 0x55adfb42e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2beca0 0x55adfb42de60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 --2- 172.21.15.43:0/599443670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb42d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02aea00640 0 client.6201 ms_handle_reset on v2:172.21.15.62:6835/48203361 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 >> 172.21.15.43:0/599443670 conn(0x55adfb3abd90 msgr2=0x55adfb3e2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 shutdown_connections 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 1 -- 172.21.15.43:0/599443670 wait complete. 2024-09-17T05:26:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.087+0000 7f02d78f3000 20 client.6201 trim_cache size 0 max 16384 2024-09-17T05:26:49.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.103+0000 7f02d78f3000 1 Processor -- start 2024-09-17T05:26:49.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.103+0000 7f02d78f3000 1 -- start start 2024-09-17T05:26:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb498b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb4990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8205380 con 0x55adfaee5460 2024-09-17T05:26:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8204430 con 0x55adfb2d5950 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80cd870 con 0x55adfb2beca0 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41792/0 (socket says 172.21.15.43:41792) 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 -- 172.21.15.43:0/3452456371 learned_addr learned my addr 172.21.15.43:0/3452456371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 -- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 msgr2=0x55adfb498b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cce00640 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb498b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02ce200640 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb4990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb498b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 -- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 msgr2=0x55adfb4990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb4990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 -- 172.21.15.43:0/3452456371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adfb283db0 con 0x55adfaee5460 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02cd800640 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f70 tx=0x7f02bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfa76840b0d3420 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02b9600640 1 -- 172.21.15.43:0/3452456371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00fdd0 con 0x55adfaee5460 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02b9600640 1 -- 172.21.15.43:0/3452456371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02bc02f070 con 0x55adfaee5460 2024-09-17T05:26:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02b9600640 1 -- 172.21.15.43:0/3452456371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc014450 con 0x55adfaee5460 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- 172.21.15.43:0/3452456371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 msgr2=0x55adfb498620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f70 tx=0x7f02bc012000 comp rx=0 tx=0).stop 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- 172.21.15.43:0/3452456371 shutdown_connections 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfb2d5950 0x55adfb4990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb498b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 --2- 172.21.15.43:0/3452456371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfaee5460 0x55adfb498620 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- 172.21.15.43:0/3452456371 >> 172.21.15.43:0/3452456371 conn(0x55adfb3abd90 msgr2=0x55adfb424620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- 172.21.15.43:0/3452456371 shutdown_connections 2024-09-17T05:26:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.107+0000 7f02d78f3000 1 -- 172.21.15.43:0/3452456371 wait complete. 2024-09-17T05:26:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- start start 2024-09-17T05:26:49.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 Processor -- start 2024-09-17T05:26:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb49c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb49c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55adfb22bd80 con 0x55adfb2d5950 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815ae10 con 0x55adfaee5460 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55adfb1c25f0 con 0x55adfb2beca0 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 10 client.0 _ll_register_callbacks cb 0x7f02b40f4b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41796/0 (socket says 172.21.15.43:41796) 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 -- 172.21.15.43:0/2710321130 learned_addr learned my addr 172.21.15.43:0/2710321130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02cd800640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb49c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02cce00640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb49c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 -- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 msgr2=0x55adfb49c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb49c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 -- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 msgr2=0x55adfb49c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:26:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb49c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 -- 172.21.15.43:0/2710321130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adfb1eeab0 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02ce200640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55adfb2327e0 tx=0x7f02c0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c791ebdadbe259db server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02aea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0028030 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02bc02f070 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f02c0004600 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f02c0034050 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 10 client.6207 fetch_fsmap learned FSMap version 60 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55adfb1cd070 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.111+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02bc02f070 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f02c0037020 con 0x55adfb2d5950 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 10 client.6207 fetch_fsmap finished waiting for FSMap version 60 2024-09-17T05:26:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 10 client.6207 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:26:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 20 client.6207 populate_metadata read hostname 'smithi043' 2024-09-17T05:26:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeb8830 0x55adfb458670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 4 client.6207 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:26:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 --> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] -- command(tid 0: {"prefix": "status"}) -- 0x55adfb236540 con 0x55adfaeb8830 2024-09-17T05:26:49.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02cd800640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeb8830 0x55adfb458670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02cd800640 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeb8830 0x55adfb458670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a8f5ebccb14154 server_cookie=40430dbf636bfeb9 in_seq=0 out_seq=0 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02aea00640 10 client.6207 ms_handle_connect on v2:172.21.15.62:6835/48203361 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02aea00640 1 -- 172.21.15.43:0/2710321130 <== mds.0 v2:172.21.15.62:6835/48203361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55adfb236540 con 0x55adfaeb8830 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02aea00640 10 client.6207 handle_command_reply: tid=0 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 client.6207 shutdown 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 msgr2=0x55adfb49ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55adfb2327e0 tx=0x7f02c0004280 comp rx=0 tx=0).stop 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 shutdown_connections 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55adfaeb8830 0x55adfb458670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55adfb2d5950 0x55adfb49ce10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55adfb2beca0 0x55adfb49c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 --2- 172.21.15.43:0/2710321130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55adfaee5460 0x55adfb49c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02aea00640 0 client.6207 ms_handle_reset on v2:172.21.15.62:6835/48203361 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 >> 172.21.15.43:0/2710321130 conn(0x55adfb3abd90 msgr2=0x55adfb424fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 shutdown_connections 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 1 -- 172.21.15.43:0/2710321130 wait complete. 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d78f3000 20 client.6207 trim_cache size 0 max 16384 2024-09-17T05:26:49.120 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5695, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.9150522409999997, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 60, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 63, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 63, 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 55.867237326999998 2024-09-17T05:26:49.121 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 msgr2=0x7f02a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 0x7f02a8075740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02bc007fd0 tx=0x7f02bc009040 comp rx=0 tx=0).stop 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 msgr2=0x7f02c8155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f02b40b0040 tx=0x7f02b40a56e0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 shutdown_connections 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f02a8073230 0x7f02a8075740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02c80c68f0 0x7f02c81fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c8155d70 0x7f02c8155440 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 --2- 172.21.15.43:0/3265718857 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f02c806f110 0x7f02c80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 >> 172.21.15.43:0/3265718857 conn(0x7f02c80b23d0 msgr2=0x7f02c815d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 shutdown_connections 2024-09-17T05:26:49.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:26:49.115+0000 7f02d0200640 1 -- 172.21.15.43:0/3265718857 wait complete. 2024-09-17T05:26:49.586 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-09-17T05:26:49.584+0000 7f73b1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:26:50.722 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T05:26:51.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:51.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:01.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:01.063+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:01.063+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:27:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:01.063+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:27:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:01.063+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:27:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:01.063+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:27:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:16.071+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:16.071+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:27:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:16.071+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:27:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:16.071+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:27:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:16.071+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:27:21.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:21.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:31.087+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:31.087+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:27:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:31.087+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:27:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:31.087+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:27:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:31.087+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:27:46.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:46.103+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:46.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:46.103+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:27:46.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:46.103+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:27:46.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:46.103+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:27:46.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:27:46.103+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:27:51.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:51.439 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:01.115+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:01.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:01.115+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:28:01.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:01.115+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:28:01.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:01.115+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:28:01.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:01.115+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:28:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:16.131+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:16.131+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-17T05:28:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:16.131+0000 7f7719e00640 1 -- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 msgr2=0x5651f4522640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T05:28:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:16.131+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T05:28:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:16.131+0000 7f7719e00640 1 --2- 172.21.15.43:0/2571976674 >> [v2:172.21.15.43:6832/61022679,v1:172.21.15.43:6834/61022679] conn(0x5651f3f82b90 0x5651f4522640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-17T05:28:20.186 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-17T05:28:20.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:28:20.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 -- 172.21.15.43:0/298565302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 msgr2=0x7f176000bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 --2- 172.21.15.43:0/298565302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f176000bde0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f177400c610 tx=0x7f177401d2a0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 -- 172.21.15.43:0/298565302 shutdown_connections 2024-09-17T05:28:20.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 --2- 172.21.15.43:0/298565302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f176000bde0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 -- 172.21.15.43:0/298565302 >> 172.21.15.43:0/298565302 conn(0x7f177c0b23d0 msgr2=0x7f177c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 -- 172.21.15.43:0/298565302 shutdown_connections 2024-09-17T05:28:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.358+0000 7f1780c00640 1 -- 172.21.15.43:0/298565302 wait complete. 2024-09-17T05:28:20.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 Processor -- start 2024-09-17T05:28:20.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- start start 2024-09-17T05:28:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f177c0cd4c0 0x7f177c099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 0x7f177c1fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0730b0 con 0x7f17600099c0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f177c070420 con 0x7f177c0cd4c0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f177c073230 con 0x7f177c1fbdd0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59362/0 (socket says 172.21.15.43:59362) 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 -- 172.21.15.43:0/1509158644 learned_addr learned my addr 172.21.15.43:0/1509158644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779400640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f177c0cd4c0 0x7f177c099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f177a800640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 0x7f177c1fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 -- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 msgr2=0x7f177c1fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 0x7f177c1fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 -- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f177c0cd4c0 msgr2=0x7f177c099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f177c0cd4c0 0x7f177c099890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 -- 172.21.15.43:0/1509158644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0cd8d0 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f177a800640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 0x7f177c1fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1779e00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f176c011ab0 tx=0x7f176c011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c01e070 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f176c004460 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c00fbe0 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1fc720 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.362+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1550b0 con 0x7f17600099c0 2024-09-17T05:28:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.366+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c0cf7e0 con 0x7f17600099c0 2024-09-17T05:28:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.366+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f176c021070 con 0x7f17600099c0 2024-09-17T05:28:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.366+0000 7f1772000640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 0x7f175c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.366+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f176c094d70 con 0x7f17600099c0 2024-09-17T05:28:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.366+0000 7f1779400640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 0x7f175c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.370+0000 7f1779400640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 0x7f175c075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f177c160bb0 tx=0x7f1768006b80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.370+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f177c1fc720 con 0x7f17600099c0 2024-09-17T05:28:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f177c06d1e0 con 0x7f17600099c0 2024-09-17T05:28:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1772000640 1 -- 172.21.15.43:0/1509158644 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f177c0cd8d0 con 0x7f17600099c0 2024-09-17T05:28:20.588 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 msgr2=0x7f175c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 0x7f175c075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f177c160bb0 tx=0x7f1768006b80 comp rx=0 tx=0).stop 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 msgr2=0x7f177c15b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f176c011ab0 tx=0x7f176c011eb0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 shutdown_connections 2024-09-17T05:28:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f175c073200 0x7f175c075710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f177c0cd4c0 0x7f177c099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f177c1fbdd0 0x7f177c1fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 --2- 172.21.15.43:0/1509158644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17600099c0 0x7f177c15b5c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 >> 172.21.15.43:0/1509158644 conn(0x7f177c0b23d0 msgr2=0x7f177c0b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 shutdown_connections 2024-09-17T05:28:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20.586+0000 7f1780c00640 1 -- 172.21.15.43:0/1509158644 wait complete. 2024-09-17T05:28:20.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:28:20.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:28:20.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:28:20.610 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:28:20.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:20.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:28:20.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:28:20.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:28:20.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:20.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.612 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:28:20.621 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:28:20.621 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:28:20.621 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42143sec preferred_lft 42143sec 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.622 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.623 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:28:20.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:20.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.625 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:28:20.685 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:28:20.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.687 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:28:20.743 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:28:20.744 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:28:20.744 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:28:20.744 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:28:20.744 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:28:20.744 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:28:20.744 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:28:20.744 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:28:20.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:28:20.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:28:20.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:28:20.745 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:28:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:28:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:20.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.886 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:28:20.895 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:28:20.895 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:28:20.895 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:20.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:28:20.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:20.958 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:21.014 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:21.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:21.014 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:21.066 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:28:21.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:21.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:21.168 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:21.169 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:21.169 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:28:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:21.199 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:28:21.200 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:28:21.200 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:28:21.203 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:28:21.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:21.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:21.223 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:28:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:28:21.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:21.324 INFO:teuthology.orchestra.run.smithi043.stdout:client5681 2024-09-17T05:28:21.325 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:21.325 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:21.334 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-17T05:28:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:28:21.435 INFO:teuthology.orchestra.run.smithi043.stdout: File: a/ 2024-09-17T05:28:21.435 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-17T05:28:21.435 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 2ch/44d Inode: 1099511627776 Links: 4 2024-09-17T05:28:21.435 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2024-09-17T05:28:21.436 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:26:05.016330269 +0000 2024-09-17T05:28:21.436 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:26:05.020330227 +0000 2024-09-17T05:28:21.436 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:26:05.020330227 +0000 2024-09-17T05:28:21.436 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:26:05.016330269 +0000 2024-09-17T05:28:21.443 INFO:teuthology.orchestra.run.smithi043.stdout:a/ 2024-09-17T05:28:21.443 INFO:teuthology.orchestra.run.smithi043.stdout:a/c 2024-09-17T05:28:21.444 INFO:teuthology.orchestra.run.smithi043.stdout:a/c/d 2024-09-17T05:28:21.444 INFO:teuthology.orchestra.run.smithi043.stdout:a/b 2024-09-17T05:28:21.444 INFO:teuthology.orchestra.run.smithi043.stdout:a/b/d 2024-09-17T05:28:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:21.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-17T05:28:21.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:21.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 -- 172.21.15.43:0/1755090495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40099c0 msgr2=0x7f01e400bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 --2- 172.21.15.43:0/1755090495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01e40099c0 0x7f01e400bde0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f02000b3bf0 tx=0x7f01f801d610 comp rx=0 tx=0).stop 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f01ffe00640 1 -- 172.21.15.43:0/1755090495 reap_dead start 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 -- 172.21.15.43:0/1755090495 shutdown_connections 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 -- 172.21.15.43:0/1755090495 >> 172.21.15.43:0/1755090495 conn(0x7f02000b23d0 msgr2=0x7f02000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 -- 172.21.15.43:0/1755090495 shutdown_connections 2024-09-17T05:28:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.610+0000 7f0206200640 1 -- 172.21.15.43:0/1755090495 wait complete. 2024-09-17T05:28:21.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 Processor -- start 2024-09-17T05:28:21.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 -- start start 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01e40099c0 0x7f020009dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 0x7f02001fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0200168aa0 con 0x7f02000997d0 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0200168920 con 0x7f020009cf00 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f0206200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0200168c20 con 0x7f01e40099c0 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f01f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f01f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59366/0 (socket says 172.21.15.43:59366) 2024-09-17T05:28:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f01f7e00640 1 -- 172.21.15.43:0/2958031698 learned_addr learned my addr 172.21.15.43:0/2958031698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f01ff400640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01e40099c0 0x7f020009dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.614+0000 7f01ffe00640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 0x7f02001fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 -- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01e40099c0 msgr2=0x7f020009dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01e40099c0 0x7f020009dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 -- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 msgr2=0x7f02001fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 0x7f02001fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 -- 172.21.15.43:0/2958031698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020009c1a0 con 0x7f02000997d0 2024-09-17T05:28:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01ffe00640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 0x7f02001fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7e00640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f01e8001490 tx=0x7f01e8003600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7400640 1 -- 172.21.15.43:0/2958031698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8015070 con 0x7f02000997d0 2024-09-17T05:28:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7400640 1 -- 172.21.15.43:0/2958031698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01e8011840 con 0x7f02000997d0 2024-09-17T05:28:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f01f7400640 1 -- 172.21.15.43:0/2958031698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8011c90 con 0x7f02000997d0 2024-09-17T05:28:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02000cd6f0 con 0x7f02000997d0 2024-09-17T05:28:21.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.618+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020009c700 con 0x7f02000997d0 2024-09-17T05:28:21.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01f7400640 1 -- 172.21.15.43:0/2958031698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f01e8018070 con 0x7f02000997d0 2024-09-17T05:28:21.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01f7400640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 0x7f01dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01f7400640 1 -- 172.21.15.43:0/2958031698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f01e8094d90 con 0x7f02000997d0 2024-09-17T05:28:21.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ff400640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 0x7f01dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 Processor -- start 2024-09-17T05:28:21.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 -- start start 2024-09-17T05:28:21.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ff400640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 0x7f01dc075710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f01f0006fd0 tx=0x7f01f0006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9be2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b912890 0x55df9be2c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df9bcc6cd0 con 0x55df9b9d6aa0 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df9bbdc330 con 0x55df9bcbb6d0 2024-09-17T05:28:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55df9bc39bb0 con 0x55df9b912890 2024-09-17T05:28:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59378/0 (socket says 172.21.15.43:59378) 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ffe00640 1 -- 172.21.15.43:0/3853388452 learned_addr learned my addr 172.21.15.43:0/3853388452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01ff400640 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9be2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.622+0000 7f01f7e00640 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b912890 0x55df9be2c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 -- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b912890 msgr2=0x55df9be2c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b912890 0x55df9be2c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 -- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 msgr2=0x55df9be2bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9be2bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 -- 172.21.15.43:0/3853388452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9bbddc80 con 0x55df9b9d6aa0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ffe00640 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f01f801d440 tx=0x7f01f801a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7017dfb29e8e61c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ef400640 1 -- 172.21.15.43:0/3853388452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8004a50 con 0x55df9b9d6aa0 2024-09-17T05:28:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ef400640 1 -- 172.21.15.43:0/3853388452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f8004bf0 con 0x55df9b9d6aa0 2024-09-17T05:28:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f01ef400640 1 -- 172.21.15.43:0/3853388452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8004e00 con 0x55df9b9d6aa0 2024-09-17T05:28:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- 172.21.15.43:0/3853388452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 msgr2=0x55df9be2c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f01f801d440 tx=0x7f01f801a3b0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- 172.21.15.43:0/3853388452 shutdown_connections 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b9d6aa0 0x55df9be2c860 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b912890 0x55df9be2c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 --2- 172.21.15.43:0/3853388452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9be2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- 172.21.15.43:0/3853388452 >> 172.21.15.43:0/3853388452 conn(0x55df9b8e65c0 msgr2=0x55df9bde1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- 172.21.15.43:0/3853388452 shutdown_connections 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- 172.21.15.43:0/3853388452 wait complete. 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 -- start start 2024-09-17T05:28:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.626+0000 7f020da18000 1 Processor -- start 2024-09-17T05:28:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be19610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be1a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df9bc2cdf0 con 0x55df9bcbb6d0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df9bc28fe0 con 0x55df9b912890 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55df9bbce4d0 con 0x55df9b9d6aa0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 10 client.0 _ll_register_callbacks cb 0x55df9bbac7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59384/0 (socket says 172.21.15.43:59384) 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be19610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 -- 172.21.15.43:0/1395206226 learned_addr learned my addr 172.21.15.43:0/1395206226 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01f7e00640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be1a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 -- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 msgr2=0x55df9be1a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be1a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 -- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 msgr2=0x55df9be19610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be19610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 -- 172.21.15.43:0/1395206226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9bbc7210 con 0x55df9bcbb6d0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ffe00640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01f801be90 tx=0x7f01f80aa550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8523063f6a8d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80aa7d0 con 0x55df9bcbb6d0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f01f8002040 con 0x55df9bcbb6d0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55df9bbc1350 con 0x55df9bcbb6d0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f01f80aa970 con 0x55df9bcbb6d0 2024-09-17T05:28:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 10 client.6225 fetch_fsmap learned FSMap version 60 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55df9bbadfc0 con 0x55df9bcbb6d0 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55df9bbc1350 con 0x55df9bcbb6d0 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f01f80aac00 con 0x55df9bcbb6d0 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 10 client.6225 fetch_fsmap finished waiting for FSMap version 60 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 10 client.6225 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 20 client.6225 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8cb930 0x55df9be1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 4 client.6225 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 --> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df9bc3bb00 con 0x55df9b8cb930 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.630+0000 7f01ff400640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8cb930 0x55df9be1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f01ff400640 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8cb930 0x55df9be1b020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeed13bce1a42c41 server_cookie=466a7f10d9cf7b18 in_seq=0 out_seq=0 2024-09-17T05:28:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f01e3e00640 10 client.6225 ms_handle_connect on v2:172.21.15.62:6835/48203361 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f01e3e00640 1 -- 172.21.15.43:0/1395206226 <== mds.0 v2:172.21.15.62:6835/48203361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df9bc3bb00 con 0x55df9b8cb930 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f01e3e00640 10 client.6225 handle_command_reply: tid=0 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 client.6225 shutdown 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 msgr2=0x55df9be1a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01f801be90 tx=0x7f01f80aa550 comp rx=0 tx=0).stop 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 shutdown_connections 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8cb930 0x55df9be1b020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9be1a610 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be1a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 --2- 172.21.15.43:0/1395206226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be19610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f01e3e00640 0 client.6225 ms_handle_reset on v2:172.21.15.62:6835/48203361 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 >> 172.21.15.43:0/1395206226 conn(0x55df9b8e65c0 msgr2=0x55df9bde1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 shutdown_connections 2024-09-17T05:28:21.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 1 -- 172.21.15.43:0/1395206226 wait complete. 2024-09-17T05:28:21.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.634+0000 7f020da18000 20 client.6225 trim_cache size 0 max 16384 2024-09-17T05:28:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 Processor -- start 2024-09-17T05:28:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 -- start start 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be42810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be42d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0200201740 con 0x55df9bcbb6d0 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02000cc0b0 con 0x55df9b912890 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f02001f7470 con 0x55df9b9d6aa0 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59386/0 (socket says 172.21.15.43:59386) 2024-09-17T05:28:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 -- 172.21.15.43:0/889134898 learned_addr learned my addr 172.21.15.43:0/889134898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:21.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ff400640 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be42810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01f7e00640 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be42d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 -- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 msgr2=0x55df9be42d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be42d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 -- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 msgr2=0x55df9be42810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be42810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.650+0000 7f01ffe00640 1 -- 172.21.15.43:0/889134898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9bc86680 con 0x55df9bcbb6d0 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f01ffe00640 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f90 tx=0x7f01f80c3720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d71c7f25c4e757 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f01eea00640 1 -- 172.21.15.43:0/889134898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80aab10 con 0x55df9bcbb6d0 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f01eea00640 1 -- 172.21.15.43:0/889134898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f80d2020 con 0x55df9bcbb6d0 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f01eea00640 1 -- 172.21.15.43:0/889134898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8004a50 con 0x55df9bcbb6d0 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- 172.21.15.43:0/889134898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 msgr2=0x55df9bddf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f90 tx=0x7f01f80c3720 comp rx=0 tx=0).stop 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- 172.21.15.43:0/889134898 shutdown_connections 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9bcbb6d0 0x55df9bddf120 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9be42d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- 172.21.15.43:0/889134898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9b912890 0x55df9be42810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- 172.21.15.43:0/889134898 >> 172.21.15.43:0/889134898 conn(0x55df9b8e65c0 msgr2=0x55df9be2b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- 172.21.15.43:0/889134898 shutdown_connections 2024-09-17T05:28:21.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- 172.21.15.43:0/889134898 wait complete. 2024-09-17T05:28:21.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- start start 2024-09-17T05:28:21.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 Processor -- start 2024-09-17T05:28:21.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9bea1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9bea1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.654+0000 7f020da18000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df9bc2cdf0 con 0x55df9b912890 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df9bc28fe0 con 0x55df9bcbb6d0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55df9bde7d60 con 0x55df9b9d6aa0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 10 client.0 _ll_register_callbacks cb 0x7f01e805bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59398/0 (socket says 172.21.15.43:59398) 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 -- 172.21.15.43:0/1217467245 learned_addr learned my addr 172.21.15.43:0/1217467245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ffe00640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9bea1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01f7e00640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9bea1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 -- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 msgr2=0x55df9bea1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9bea1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 -- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 msgr2=0x55df9bea1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:21.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9bea1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 -- 172.21.15.43:0/1217467245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9bbf30b0 con 0x55df9b912890 2024-09-17T05:28:21.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01ff400640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f01f00019d0 tx=0x7f01f0001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4729f56e8ab9cd09 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:21.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0032400 con 0x55df9b912890 2024-09-17T05:28:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f01f0033d40 con 0x55df9b912890 2024-09-17T05:28:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f80d2020 con 0x55df9b912890 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f01f00325a0 con 0x55df9b912890 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 10 client.6231 fetch_fsmap learned FSMap version 60 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55df9bbbf610 con 0x55df9b912890 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f80d2020 con 0x55df9b912890 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f01f001f400 con 0x55df9b912890 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 10 client.6231 fetch_fsmap finished waiting for FSMap version 60 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 10 client.6231 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 20 client.6231 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8b9990 0x55df9be40590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 4 client.6231 mds_command: new command op to 5695 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:28:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 --> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df9bc375b0 con 0x55df9b8b9990 2024-09-17T05:28:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01f7e00640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8b9990 0x55df9be40590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01e3e00640 10 client.6231 ms_handle_connect on v2:172.21.15.62:6835/48203361 2024-09-17T05:28:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.658+0000 7f01f7e00640 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8b9990 0x55df9be40590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c65f35b012aa88 server_cookie=79148f518549e680 in_seq=0 out_seq=0 2024-09-17T05:28:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f01e3e00640 1 -- 172.21.15.43:0/1217467245 <== mds.0 v2:172.21.15.62:6835/48203361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df9bc375b0 con 0x55df9b8b9990 2024-09-17T05:28:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f01e3e00640 10 client.6231 handle_command_reply: tid=0 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 client.6231 shutdown 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 msgr2=0x55df9bea0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f01f00019d0 tx=0x7f01f0001370 comp rx=0 tx=0).stop 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 shutdown_connections 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:6835/48203361,v1:172.21.15.62:6837/48203361] conn(0x55df9b8b9990 0x55df9be40590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df9bcbb6d0 0x55df9bea1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f01e3e00640 0 client.6231 ms_handle_reset on v2:172.21.15.62:6835/48203361 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df9b9d6aa0 0x55df9bea1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 --2- 172.21.15.43:0/1217467245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df9b912890 0x55df9bea0dd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 >> 172.21.15.43:0/1217467245 conn(0x55df9b8e65c0 msgr2=0x55df9be38050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 shutdown_connections 2024-09-17T05:28:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 1 -- 172.21.15.43:0/1217467245 wait complete. 2024-09-17T05:28:21.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f020da18000 20 client.6231 trim_cache size 0 max 16384 2024-09-17T05:28:21.677 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:28:21.677 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:28:21.677 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:28:21.677 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 msgr2=0x7f01dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 0x7f01dc075710 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f01f0006fd0 tx=0x7f01f0006b20 comp rx=0 tx=0).stop 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 msgr2=0x7f0200073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f01e8001490 tx=0x7f01e8003600 comp rx=0 tx=0).stop 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 shutdown_connections 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f01dc073200 0x7f01dc075710 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f020009cf00 0x7f02001fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02000997d0 0x7f0200073be0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 --2- 172.21.15.43:0/2958031698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f01e40099c0 0x7f020009dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 >> 172.21.15.43:0/2958031698 conn(0x7f02000b23d0 msgr2=0x7f02000b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 shutdown_connections 2024-09-17T05:28:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:21.674+0000 7f0206200640 1 -- 172.21.15.43:0/2958031698 wait complete. 2024-09-17T05:28:21.696 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:28:21.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:28:21.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:28:21.871 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:28:21.871 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:28:21.871 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:21.871 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:21.880 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:21.880 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:21.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 -- 172.21.15.43:0/457858783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 msgr2=0x7fb79c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/457858783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c0b33e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb79c072b60 tx=0x7fb78801c620 comp rx=0 tx=0).stop 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 -- 172.21.15.43:0/457858783 shutdown_connections 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/457858783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c0b33e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/457858783 >> v2:172.21.15.62:3300/0 conn(0x7fb79c155d70 0x7fb79c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 -- 172.21.15.43:0/457858783 >> 172.21.15.43:0/457858783 conn(0x7fb79c0b23d0 msgr2=0x7fb79c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 -- 172.21.15.43:0/457858783 shutdown_connections 2024-09-17T05:28:22.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.134+0000 7fb7a3a00640 1 -- 172.21.15.43:0/457858783 wait complete. 2024-09-17T05:28:22.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 Processor -- start 2024-09-17T05:28:22.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 -- start start 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb79c0b3920 0x7fb79c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79c155d70 0x7fb79c073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c168a50 con 0x7fb79c06f110 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1688d0 con 0x7fb79c0b3920 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a3a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c168bd0 con 0x7fb79c155d70 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59410/0 (socket says 172.21.15.43:59410) 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 -- 172.21.15.43:0/3206054545 learned_addr learned my addr 172.21.15.43:0/3206054545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb79be00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb79c0b3920 0x7fb79c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79c155d70 0x7fb79c073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 -- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79c155d70 msgr2=0x7fb79c073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79c155d70 0x7fb79c073410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 -- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb79c0b3920 msgr2=0x7fb79c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb79c0b3920 0x7fb79c153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 -- 172.21.15.43:0/3206054545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0cc8a0 con 0x7fb79c06f110 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.138+0000 7fb7a1000640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b420 tx=0x7fb79000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790011830 con 0x7fb79c06f110 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb790016070 con 0x7fb79c06f110 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c0ce4e0 con 0x7fb79c06f110 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790011d80 con 0x7fb79c06f110 2024-09-17T05:28:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c1fc460 con 0x7fb79c06f110 2024-09-17T05:28:22.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c0b33e0 con 0x7fb79c06f110 2024-09-17T05:28:22.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb79001b460 con 0x7fb79c06f110 2024-09-17T05:28:22.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 0x7fb77c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7fb790093e90 con 0x7fb79c06f110 2024-09-17T05:28:22.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.142+0000 7fb79be00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 0x7fb77c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.146+0000 7fb79be00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 0x7fb77c075740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae000 tx=0x7fb78801ae60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.146+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb79005e760 con 0x7fb79c06f110 2024-09-17T05:28:22.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb79c0af280 con 0x7fb79c06f110 2024-09-17T05:28:22.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb798c00640 1 -- 172.21.15.43:0/3206054545 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v63) ==== 80+0+240 (secure 0 0 0) 0x7fb790095080 con 0x7fb79c06f110 2024-09-17T05:28:22.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/61022679 2024-09-18T05:26:44.197145+0000 2024-09-17T05:28:22.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6832/61022679 2024-09-18T05:26:44.197145+0000 2024-09-17T05:28:22.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/2376863639 2024-09-18T05:26:08.067813+0000 2024-09-17T05:28:22.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6832/2376863639 2024-09-18T05:26:08.067813+0000 2024-09-17T05:28:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:listed 4 entries 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 msgr2=0x7fb77c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 0x7fb77c075740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae000 tx=0x7fb78801ae60 comp rx=0 tx=0).stop 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 msgr2=0x7fb79c1fcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b420 tx=0x7fb79000b8e0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 shutdown_connections 2024-09-17T05:28:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb77c073230 0x7fb77c075740 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb79c0b3920 0x7fb79c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb79c155d70 0x7fb79c073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 --2- 172.21.15.43:0/3206054545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79c06f110 0x7fb79c1fcf40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 >> 172.21.15.43:0/3206054545 conn(0x7fb79c0b23d0 msgr2=0x7fb79c09db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 shutdown_connections 2024-09-17T05:28:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.334+0000 7fb7a3a00640 1 -- 172.21.15.43:0/3206054545 wait complete. 2024-09-17T05:28:22.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:28:22.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:22.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:22.386 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:28:22.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:22.386 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:28:22.439 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:28:22.439 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:28:22.439 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:22.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:28:22.449 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:28:22.449 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:22.449 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:22.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:28:22.495 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:28:22.495 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:28:22.495 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:22.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:28:22.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 -- 172.21.15.43:0/869802885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc004810 msgr2=0x7f66dc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 --2- 172.21.15.43:0/869802885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc004810 0x7f66dc004c20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c2e0 tx=0x7f66e001d980 comp rx=0 tx=0).stop 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 -- 172.21.15.43:0/869802885 shutdown_connections 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 --2- 172.21.15.43:0/869802885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc004810 0x7f66dc004c20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 -- 172.21.15.43:0/869802885 >> 172.21.15.43:0/869802885 conn(0x7f66f80b23d0 msgr2=0x7f66f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 -- 172.21.15.43:0/869802885 shutdown_connections 2024-09-17T05:28:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.698+0000 7f66fe000640 1 -- 172.21.15.43:0/869802885 wait complete. 2024-09-17T05:28:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 Processor -- start 2024-09-17T05:28:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 -- start start 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 0x7f66f81f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f80c6190 0x7f66f81f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80ad6e0 con 0x7f66f80c4980 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80ad560 con 0x7f66f80c6190 2024-09-17T05:28:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66fe000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f66f80ad860 con 0x7f66dc004810 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59424/0 (socket says 172.21.15.43:59424) 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 -- 172.21.15.43:0/738144349 learned_addr learned my addr 172.21.15.43:0/738144349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66f7400640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 0x7f66f81f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66f7e00640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f80c6190 0x7f66f81f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 -- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 msgr2=0x7f66f81f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 0x7f66f81f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 -- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f80c6190 msgr2=0x7f66f81f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f80c6190 0x7f66f81f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 -- 172.21.15.43:0/738144349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f80c50f0 con 0x7f66f80c4980 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66f7400640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 0x7f66f81f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66efe00640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66e001b620 tx=0x7f66e0006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.702+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00014c0 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66e0001660 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f80c53d0 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0006330 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f80c5990 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f80addc0 con 0x7f66f80c4980 2024-09-17T05:28:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66e00a5d00 con 0x7f66f80c4980 2024-09-17T05:28:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66ef400640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 0x7f66d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f66e012d6d0 con 0x7f66f80c4980 2024-09-17T05:28:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.706+0000 7f66f7400640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 0x7f66d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:22.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.710+0000 7f66f7400640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 0x7f66d4075710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66f806a400 tx=0x7f66e80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.710+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f66e0016cd0 con 0x7f66f80c4980 2024-09-17T05:28:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f80c4e80 con 0x7f66f80c4980 2024-09-17T05:28:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66ef400640 1 -- 172.21.15.43:0/738144349 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5216 (secure 0 0 0) 0x7f66e00f85f0 con 0x7f66f80c4980 2024-09-17T05:28:22.928 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:22.928 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"btime":"2024-09-17T05:26:47:209080+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:47.209072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-17T05:28:22.929 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 60 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 msgr2=0x7f66d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 0x7f66d4075710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66f806a400 tx=0x7f66e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 msgr2=0x7f66f81f6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66e001b620 tx=0x7f66e0006d50 comp rx=0 tx=0).stop 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 shutdown_connections 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66d4073200 0x7f66d4075710 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f80c6190 0x7f66f81f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f80c4980 0x7f66f81f6690 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 --2- 172.21.15.43:0/738144349 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66dc004810 0x7f66f81f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.926+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 >> 172.21.15.43:0/738144349 conn(0x7f66f80b23d0 msgr2=0x7f66f80cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.930+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 shutdown_connections 2024-09-17T05:28:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:22.930+0000 7f66fe000640 1 -- 172.21.15.43:0/738144349 wait complete. 2024-09-17T05:28:22.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- 172.21.15.43:0/1382008406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3490004810 msgr2=0x7f3490004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 --2- 172.21.15.43:0/1382008406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3490004810 0x7f3490004c20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3494009b60 tx=0x7f349401d290 comp rx=0 tx=0).stop 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b0e00640 1 -- 172.21.15.43:0/1382008406 reap_dead start 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- 172.21.15.43:0/1382008406 shutdown_connections 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- 172.21.15.43:0/1382008406 >> 172.21.15.43:0/1382008406 conn(0x7f34ac0cee60 msgr2=0x7f34ac0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- 172.21.15.43:0/1382008406 shutdown_connections 2024-09-17T05:28:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- 172.21.15.43:0/1382008406 wait complete. 2024-09-17T05:28:23.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 Processor -- start 2024-09-17T05:28:23.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.110+0000 7f34b2e00640 1 -- start start 2024-09-17T05:28:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3490004810 0x7f34ac15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ac15c830 0x7f34ac0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac0cd1c0 con 0x7f34ac15cd80 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac0cd040 con 0x7f34ac15c830 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0cd340 con 0x7f3490004810 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59434/0 (socket says 172.21.15.43:59434) 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 -- 172.21.15.43:0/3899067716 learned_addr learned my addr 172.21.15.43:0/3899067716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34abe00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3490004810 0x7f34ac15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b0e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ac15c830 0x7f34ac0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 -- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3490004810 msgr2=0x7f34ac15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3490004810 0x7f34ac15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 -- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ac15c830 msgr2=0x7f34ac0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ac15c830 0x7f34ac0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 -- 172.21.15.43:0/3899067716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac15d9d0 con 0x7f34ac15cd80 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a3e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34940af040 tx=0x7f3494005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940b1040 con 0x7f34ac15cd80 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34940bb070 con 0x7f34ac15cd80 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac15dcb0 con 0x7f34ac15cd80 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494009730 con 0x7f34ac15cd80 2024-09-17T05:28:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac15e270 con 0x7f34ac15cd80 2024-09-17T05:28:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.114+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac15d740 con 0x7f34ac15cd80 2024-09-17T05:28:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.118+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3494004290 con 0x7f34ac15cd80 2024-09-17T05:28:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.118+0000 7f34a8c00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 0x7f3488075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.118+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f349412f380 con 0x7f34ac15cd80 2024-09-17T05:28:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.118+0000 7f34abe00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 0x7f3488075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.118+0000 7f34abe00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 0x7f3488075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f349c009be0 tx=0x7f349c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.122+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3494016fa0 con 0x7f34ac15cd80 2024-09-17T05:28:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34ac0b3130 con 0x7f34ac15cd80 2024-09-17T05:28:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34a8c00640 1 -- 172.21.15.43:0/3899067716 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5216 (secure 0 0 0) 0x7f34940fa2a0 con 0x7f34ac15cd80 2024-09-17T05:28:23.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:23.340 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"btime":"2024-09-17T05:26:47:209080+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:47.209072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-17T05:28:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 60 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 msgr2=0x7f3488075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 0x7f3488075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f349c009be0 tx=0x7f349c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 msgr2=0x7f34ac0cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34940af040 tx=0x7f3494005540 comp rx=0 tx=0).stop 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 shutdown_connections 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3488073200 0x7f3488075710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ac15c830 0x7f34ac0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac15cd80 0x7f34ac0cbc90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 --2- 172.21.15.43:0/3899067716 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3490004810 0x7f34ac15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 >> 172.21.15.43:0/3899067716 conn(0x7f34ac0cee60 msgr2=0x7f34ac0c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 shutdown_connections 2024-09-17T05:28:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.338+0000 7f34b2e00640 1 -- 172.21.15.43:0/3899067716 wait complete. 2024-09-17T05:28:23.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 -- 172.21.15.43:0/4027906842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8638003af0 msgr2=0x7f8638003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 --2- 172.21.15.43:0/4027906842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8638003af0 0x7f8638003f60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8644009b30 tx=0x7f864401d220 comp rx=0 tx=0).stop 2024-09-17T05:28:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f8653e00640 1 -- 172.21.15.43:0/4027906842 reap_dead start 2024-09-17T05:28:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 -- 172.21.15.43:0/4027906842 shutdown_connections 2024-09-17T05:28:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 -- 172.21.15.43:0/4027906842 >> 172.21.15.43:0/4027906842 conn(0x7f86540b2810 msgr2=0x7f86540b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 -- 172.21.15.43:0/4027906842 shutdown_connections 2024-09-17T05:28:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.518+0000 7f865a400640 1 -- 172.21.15.43:0/4027906842 wait complete. 2024-09-17T05:28:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 Processor -- start 2024-09-17T05:28:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 -- start start 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8638003af0 0x7f86541643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86541fa280 0x7f8654152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8654166f80 con 0x7f865415b0e0 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8654166e00 con 0x7f86541fa280 2024-09-17T05:28:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f865a400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8654167100 con 0x7f8638003af0 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59466/0 (socket says 172.21.15.43:59466) 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 -- 172.21.15.43:0/3981805838 learned_addr learned my addr 172.21.15.43:0/3981805838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f864aa00640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86541fa280 0x7f8654152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8638003af0 0x7f86541643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 -- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8638003af0 msgr2=0x7f86541643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8638003af0 0x7f86541643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 -- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86541fa280 msgr2=0x7f8654152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86541fa280 0x7f8654152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.522+0000 7f8653e00640 1 -- 172.21.15.43:0/3981805838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865415ba30 con 0x7f865415b0e0 2024-09-17T05:28:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f8653e00640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f864c001340 tx=0x7f864c006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c011870 con 0x7f865415b0e0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f864c017070 con 0x7f865415b0e0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c011da0 con 0x7f865415b0e0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8654164b70 con 0x7f865415b0e0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8654165130 con 0x7f865415b0e0 2024-09-17T05:28:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86540b5a20 con 0x7f865415b0e0 2024-09-17T05:28:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f864c01c460 con 0x7f865415b0e0 2024-09-17T05:28:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 0x7f8630075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.526+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f864c094a10 con 0x7f865415b0e0 2024-09-17T05:28:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.530+0000 7f8653400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 0x7f8630075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.530+0000 7f8653400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 0x7f8630075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae000 tx=0x7f8644019fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.534+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f865415ba30 con 0x7f865415b0e0 2024-09-17T05:28:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.706+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8654164900 con 0x7f865415b0e0 2024-09-17T05:28:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.706+0000 7f864be00640 1 -- 172.21.15.43:0/3981805838 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17016 (secure 0 0 0) 0x7f86540b5f60 con 0x7f865415b0e0 2024-09-17T05:28:23.710 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:23.710 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":63,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:26:44.197224+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:55.383804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:55.976123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.37","mappings":[{"from":6,"to":4}]},{"pgid":"9.3e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6832/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6834/2376863639":"2024-09-18T05:26:08.067813+0000","172.21.15.43:6832/2376863639":"2024-09-18T05:26:08.067813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 msgr2=0x7f8630075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 0x7f8630075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae000 tx=0x7f8644019fc0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 msgr2=0x7f865415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f864c001340 tx=0x7f864c006420 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 shutdown_connections 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8630073200 0x7f8630075710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86541fa280 0x7f8654152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865415b0e0 0x7f865415b4f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 --2- 172.21.15.43:0/3981805838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8638003af0 0x7f86541643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 >> 172.21.15.43:0/3981805838 conn(0x7f86540b2810 msgr2=0x7f8654165c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 shutdown_connections 2024-09-17T05:28:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.710+0000 7f865a400640 1 -- 172.21.15.43:0/3981805838 wait complete. 2024-09-17T05:28:23.733 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:28:23.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- 172.21.15.43:0/2941322250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 msgr2=0x7fbc280ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2941322250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc280ccc40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009900 tx=0x7fbc1c01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- 172.21.15.43:0/2941322250 shutdown_connections 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2941322250 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2941322250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc280ccc40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- 172.21.15.43:0/2941322250 >> 172.21.15.43:0/2941322250 conn(0x7fbc28068fe0 msgr2=0x7fbc28069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- 172.21.15.43:0/2941322250 shutdown_connections 2024-09-17T05:28:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- 172.21.15.43:0/2941322250 wait complete. 2024-09-17T05:28:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 Processor -- start 2024-09-17T05:28:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.890+0000 7fbc2e000640 1 -- start start 2024-09-17T05:28:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2814c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc281605e0 0x7fbc2814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc28153680 con 0x7fbc280d1cb0 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc28153500 con 0x7fbc281605e0 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbc28153800 con 0x7fbc2815ca80 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39848/0 (socket says 172.21.15.43:39848) 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 -- 172.21.15.43:0/2938291892 learned_addr learned my addr 172.21.15.43:0/2938291892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc27e00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2814c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2d600640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc281605e0 0x7fbc2814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 -- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 msgr2=0x7fbc2814c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2814c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 -- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc281605e0 msgr2=0x7fbc2814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc281605e0 0x7fbc2814c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 -- 172.21.15.43:0/2938291892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc280c9e50 con 0x7fbc280d1cb0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc27e00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2cc00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009850 tx=0x7fbc1c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0b2040 con 0x7fbc280d1cb0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc1c005320 con 0x7fbc280d1cb0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0046d0 con 0x7fbc280d1cb0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2806ff40 con 0x7fbc280d1cb0 2024-09-17T05:28:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.894+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2806ed90 con 0x7fbc280d1cb0 2024-09-17T05:28:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2806efd0 con 0x7fbc280d1cb0 2024-09-17T05:28:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc1c0bd060 con 0x7fbc280d1cb0 2024-09-17T05:28:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc24c00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 0x7fbc08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7fbc1c12e0f0 con 0x7fbc280d1cb0 2024-09-17T05:28:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc27e00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 0x7fbc08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.898+0000 7fbc27e00640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 0x7fbc08075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc2806fd30 tx=0x7fbc140091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:23.902+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc2806efd0 con 0x7fbc280d1cb0 2024-09-17T05:28:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.114+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc280ccc40 con 0x7fbc280d1cb0 2024-09-17T05:28:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.114+0000 7fbc24c00640 1 -- 172.21.15.43:0/2938291892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5216 (secure 0 0 0) 0x7fbc1c0164d0 con 0x7fbc280d1cb0 2024-09-17T05:28:24.118 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:24.119 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"btime":"2024-09-17T05:26:47:209080+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:47.209072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-17T05:28:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 60 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 msgr2=0x7fbc08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 0x7fbc08075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc2806fd30 tx=0x7fbc140091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 msgr2=0x7fbc28148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009850 tx=0x7fbc1c005de0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 shutdown_connections 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc08073200 0x7fbc08075710 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc281605e0 0x7fbc2814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc2815ca80 0x7fbc2814c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 --2- 172.21.15.43:0/2938291892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc280d1cb0 0x7fbc28148a10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 >> 172.21.15.43:0/2938291892 conn(0x7fbc28068fe0 msgr2=0x7fbc2814b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:24.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 shutdown_connections 2024-09-17T05:28:24.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.118+0000 7fbc2e000640 1 -- 172.21.15.43:0/2938291892 wait complete. 2024-09-17T05:28:24.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 -- 172.21.15.43:0/277956771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32f8003b40 msgr2=0x7f32f8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 --2- 172.21.15.43:0/277956771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32f8003b40 0x7f32f8003fb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3304009b40 tx=0x7f330401d270 comp rx=0 tx=0).stop 2024-09-17T05:28:24.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 -- 172.21.15.43:0/277956771 shutdown_connections 2024-09-17T05:28:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 --2- 172.21.15.43:0/277956771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32f8003b40 0x7f32f8003fb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 -- 172.21.15.43:0/277956771 >> 172.21.15.43:0/277956771 conn(0x7f33140b23d0 msgr2=0x7f33140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 -- 172.21.15.43:0/277956771 shutdown_connections 2024-09-17T05:28:24.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.298+0000 7f3318a00640 1 -- 172.21.15.43:0/277956771 wait complete. 2024-09-17T05:28:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 Processor -- start 2024-09-17T05:28:24.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- start start 2024-09-17T05:28:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32f8003b40 0x7f331415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33140c3da0 0x7f33141ec080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f331406d490 con 0x7f33140c41b0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f331406d310 con 0x7f32f8003b40 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f331406d610 con 0x7f33140c3da0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39862/0 (socket says 172.21.15.43:39862) 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 -- 172.21.15.43:0/3762693610 learned_addr learned my addr 172.21.15.43:0/3762693610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3311e00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32f8003b40 0x7f331415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f330be00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33140c3da0 0x7f33141ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 -- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33140c3da0 msgr2=0x7f33141ec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33140c3da0 0x7f33141ec080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 -- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32f8003b40 msgr2=0x7f331415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32f8003b40 0x7f331415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 -- 172.21.15.43:0/3762693610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33141f9890 con 0x7f33140c41b0 2024-09-17T05:28:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3312800640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f33140b3940 tx=0x7f330c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c011900 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f330c011aa0 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00f430 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33141f9b70 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.302+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33141fa130 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.306+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33140ce310 con 0x7f33140c41b0 2024-09-17T05:28:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.306+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f330c00f5d0 con 0x7f33140c41b0 2024-09-17T05:28:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.306+0000 7f330a000640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 0x7f32f0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.306+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f330c0948f0 con 0x7f33140c41b0 2024-09-17T05:28:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.306+0000 7f3311e00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 0x7f32f0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.310+0000 7f3311e00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 0x7f32f0075740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f33040ad000 tx=0x7f330401a010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.310+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f330c05f250 con 0x7f33140c41b0 2024-09-17T05:28:24.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33141f9620 con 0x7f33140c41b0 2024-09-17T05:28:24.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f330a000640 1 -- 172.21.15.43:0/3762693610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5216 (secure 0 0 0) 0x7f330c063c80 con 0x7f33140c41b0 2024-09-17T05:28:24.525 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:24.525 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"btime":"2024-09-17T05:26:47:209080+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:57.018678+0000","modified":"2024-09-17T05:26:47.209072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5695},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5695":{"gid":5695,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/48203361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":48203361},{"type":"v1","addr":"172.21.15.62:6837","nonce":48203361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5695,"qdb_cluster":[5695]},"id":4}]} 2024-09-17T05:28:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 60 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 msgr2=0x7f32f0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 0x7f32f0075740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f33040ad000 tx=0x7f330401a010 comp rx=0 tx=0).stop 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 msgr2=0x7f33141ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f33140b3940 tx=0x7f330c006430 comp rx=0 tx=0).stop 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 shutdown_connections 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f32f0073230 0x7f32f0075740 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33140c41b0 0x7f33141ec5c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33140c3da0 0x7f33141ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 --2- 172.21.15.43:0/3762693610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32f8003b40 0x7f331415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.522+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 >> 172.21.15.43:0/3762693610 conn(0x7f33140b23d0 msgr2=0x7f33140b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.526+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 shutdown_connections 2024-09-17T05:28:24.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.526+0000 7f3318a00640 1 -- 172.21.15.43:0/3762693610 wait complete. 2024-09-17T05:28:24.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:24.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 -- 172.21.15.43:0/3332543941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c15ec90 msgr2=0x7f651c0ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 --2- 172.21.15.43:0/3332543941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c15ec90 0x7f651c0ceea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f651c0b3ea0 tx=0x7f650401c890 comp rx=0 tx=0).stop 2024-09-17T05:28:24.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 -- 172.21.15.43:0/3332543941 shutdown_connections 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 --2- 172.21.15.43:0/3332543941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c15ec90 0x7f651c0ceea0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 --2- 172.21.15.43:0/3332543941 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c0cf3e0 0x7f651c073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 --2- 172.21.15.43:0/3332543941 >> v2:172.21.15.62:3300/0 conn(0x7f651c15e880 0x7f651c0713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 -- 172.21.15.43:0/3332543941 >> 172.21.15.43:0/3332543941 conn(0x7f651c0d1100 msgr2=0x7f651c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 -- 172.21.15.43:0/3332543941 shutdown_connections 2024-09-17T05:28:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.702+0000 7f6522400640 1 -- 172.21.15.43:0/3332543941 wait complete. 2024-09-17T05:28:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 Processor -- start 2024-09-17T05:28:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 -- start start 2024-09-17T05:28:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 0x7f651c15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 0x7f651c15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f651c15fc50 con 0x7f651c0cf3e0 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f651c15fad0 con 0x7f651c15ec90 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6522400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f651c15fdd0 con 0x7f651c15e880 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39892/0 (socket says 172.21.15.43:39892) 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 -- 172.21.15.43:0/4250242836 learned_addr learned my addr 172.21.15.43:0/4250242836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6513e00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 0x7f651c15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521a00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 0x7f651c15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 -- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 msgr2=0x7f651c15ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 0x7f651c15ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 -- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 msgr2=0x7f651c15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 0x7f651c15b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521000640 1 -- 172.21.15.43:0/4250242836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c15dde0 con 0x7f651c0cf3e0 2024-09-17T05:28:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6521a00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 0x7f651c15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.706+0000 7f6513e00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 0x7f651c15ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6521000640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f650c00ca20 tx=0x7f650c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c007c70 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f650c00f450 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00f8a0 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c1573f0 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c151600 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c150c60 con 0x7f651c0cf3e0 2024-09-17T05:28:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f650c00fa40 con 0x7f651c0cf3e0 2024-09-17T05:28:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 0x7f64f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.710+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f650c094b50 con 0x7f651c0cf3e0 2024-09-17T05:28:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.714+0000 7f6513e00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 0x7f64f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.714+0000 7f6513e00640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 0x7f64f8075740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65040af000 tx=0x7f650401b0d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.714+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f650c014030 con 0x7f651c0cf3e0 2024-09-17T05:28:24.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.890+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f651c1519e0 con 0x7f651c0cf3e0 2024-09-17T05:28:24.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.890+0000 7f6519600640 1 -- 172.21.15.43:0/4250242836 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17016 (secure 0 0 0) 0x7f650c05fa70 con 0x7f651c0cf3e0 2024-09-17T05:28:24.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:24.894 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":63,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:26:44.197224+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:55.383804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:55.976123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.37","mappings":[{"from":6,"to":4}]},{"pgid":"9.3e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6832/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6834/2376863639":"2024-09-18T05:26:08.067813+0000","172.21.15.43:6832/2376863639":"2024-09-18T05:26:08.067813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 msgr2=0x7f64f8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 0x7f64f8075740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65040af000 tx=0x7f650401b0d0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 msgr2=0x7f651c150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f650c00ca20 tx=0x7f650c00cee0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 shutdown_connections 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64f8073230 0x7f64f8075740 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f651c15ec90 0x7f651c15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f651c15e880 0x7f651c15ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 --2- 172.21.15.43:0/4250242836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f651c0cf3e0 0x7f651c150720 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 >> 172.21.15.43:0/4250242836 conn(0x7f651c0d1100 msgr2=0x7f651c1549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 shutdown_connections 2024-09-17T05:28:24.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:24.894+0000 7f6522400640 1 -- 172.21.15.43:0/4250242836 wait complete. 2024-09-17T05:28:24.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 -- 172.21.15.43:0/1635633163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 msgr2=0x7f439400bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1635633163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f439400bde0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009b10 tx=0x7f43a801d270 comp rx=0 tx=0).stop 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 -- 172.21.15.43:0/1635633163 shutdown_connections 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1635633163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f439400bde0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 -- 172.21.15.43:0/1635633163 >> 172.21.15.43:0/1635633163 conn(0x7f43b00b23d0 msgr2=0x7f43b00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 -- 172.21.15.43:0/1635633163 shutdown_connections 2024-09-17T05:28:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.074+0000 7f43b4a00640 1 -- 172.21.15.43:0/1635633163 wait complete. 2024-09-17T05:28:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 Processor -- start 2024-09-17T05:28:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- start start 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b0157820 0x7f43b015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 0x7f43b015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00b4f70 con 0x7f43940099c0 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00b4df0 con 0x7f43b0157c30 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43b00b50f0 con 0x7f43b0157820 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39912/0 (socket says 172.21.15.43:39912) 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 -- 172.21.15.43:0/1300739024 learned_addr learned my addr 172.21.15.43:0/1300739024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ad400640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b0157820 0x7f43b015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ae800640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 0x7f43b015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 -- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b0157820 msgr2=0x7f43b015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b0157820 0x7f43b015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 -- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 msgr2=0x7f43b015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 0x7f43b015e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 -- 172.21.15.43:0/1300739024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b0158450 con 0x7f43940099c0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ae800640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 0x7f43b015e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43ade00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e590 tx=0x7f43a000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0015020 con 0x7f43940099c0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a0010980 con 0x7f43940099c0 2024-09-17T05:28:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b0165900 con 0x7f43940099c0 2024-09-17T05:28:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0010dd0 con 0x7f43940099c0 2024-09-17T05:28:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.078+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b0165ec0 con 0x7f43940099c0 2024-09-17T05:28:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b00733c0 con 0x7f43940099c0 2024-09-17T05:28:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43a0018020 con 0x7f43940099c0 2024-09-17T05:28:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43a6000640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 0x7f438c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5939+0+0 (secure 0 0 0) 0x7f43a00948b0 con 0x7f43940099c0 2024-09-17T05:28:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43ad400640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 0x7f438c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.082+0000 7f43ad400640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 0x7f438c075710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f90 tx=0x7f4398006ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.086+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43a001f020 con 0x7f43940099c0 2024-09-17T05:28:25.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.294+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43b00742a0 con 0x7f43940099c0 2024-09-17T05:28:25.298 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:28:25.298+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:28:25.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.298+0000 7f43a6000640 1 -- 172.21.15.43:0/1300739024 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7f43a005f870 con 0x7f43940099c0 2024-09-17T05:28:25.301 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:28:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 msgr2=0x7f438c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 0x7f438c075710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f90 tx=0x7f4398006ea0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 msgr2=0x7f43b015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f43a000e590 tx=0x7f43a000e990 comp rx=0 tx=0).stop 2024-09-17T05:28:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 shutdown_connections 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f438c073200 0x7f438c075710 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f90 tx=0x7f4398006ea0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43b0157c30 0x7f43b015e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43b0157820 0x7f43b015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 --2- 172.21.15.43:0/1300739024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43940099c0 0x7f43b015f3e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 >> 172.21.15.43:0/1300739024 conn(0x7f43b00b23d0 msgr2=0x7f43b00b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 shutdown_connections 2024-09-17T05:28:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.302+0000 7f43b4a00640 1 -- 172.21.15.43:0/1300739024 wait complete. 2024-09-17T05:28:25.317 INFO:tasks.ceph.mds.b.smithi062.stderr: -4301> 2024-09-17T05:26:49.584+0000 7f73b1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:28:25.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:28:25.439 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 -- 172.21.15.43:0/1610157490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c155d70 msgr2=0x7fc45c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 --2- 172.21.15.43:0/1610157490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c155d70 0x7fc45c15c7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc450009870 tx=0x7fc45001c7d0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 -- 172.21.15.43:0/1610157490 shutdown_connections 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 --2- 172.21.15.43:0/1610157490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c155d70 0x7fc45c15c7e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 -- 172.21.15.43:0/1610157490 >> 172.21.15.43:0/1610157490 conn(0x7fc45c0b23d0 msgr2=0x7fc45c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 -- 172.21.15.43:0/1610157490 shutdown_connections 2024-09-17T05:28:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.482+0000 7fc463600640 1 -- 172.21.15.43:0/1610157490 wait complete. 2024-09-17T05:28:25.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 Processor -- start 2024-09-17T05:28:25.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 -- start start 2024-09-17T05:28:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c06f110 0x7fc45c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c155d70 0x7fc45c0c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c169e90 con 0x7fc45c0c5c60 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c169d10 con 0x7fc45c155d70 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc463600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc45c16a010 con 0x7fc45c06f110 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc461600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc461600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39914/0 (socket says 172.21.15.43:39914) 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc461600640 1 -- 172.21.15.43:0/3612316958 learned_addr learned my addr 172.21.15.43:0/3612316958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.486+0000 7fc460c00640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c06f110 0x7fc45c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc45be00640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c155d70 0x7fc45c0c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 -- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c06f110 msgr2=0x7fc45c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c06f110 0x7fc45c0c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 -- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c155d70 msgr2=0x7fc45c0c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c155d70 0x7fc45c0c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 -- 172.21.15.43:0/3612316958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c0c4b90 con 0x7fc45c0c5c60 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc461600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c760 tx=0x7fc45400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454003b80 con 0x7fc45c0c5c60 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc454003d20 con 0x7fc45c0c5c60 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c0c4e70 con 0x7fc45c0c5c60 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400aa50 con 0x7fc45c0c5c60 2024-09-17T05:28:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c0c9160 con 0x7fc45c0c5c60 2024-09-17T05:28:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c0c93c0 con 0x7fc45c0c5c60 2024-09-17T05:28:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.490+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc45400abf0 con 0x7fc45c0c5c60 2024-09-17T05:28:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.494+0000 7fc458c00640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 0x7fc43c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.494+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6025+0+0 (secure 0 0 0) 0x7fc4540940e0 con 0x7fc45c0c5c60 2024-09-17T05:28:25.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.494+0000 7fc460c00640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 0x7fc43c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:25.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.494+0000 7fc460c00640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 0x7fc43c075710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc4500af000 tx=0x7fc45001b010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:25.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.498+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc45405e8d0 con 0x7fc45c0c5c60 2024-09-17T05:28:25.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:25.706+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc45c15c7e0 con 0x7fc45c0c5c60 2024-09-17T05:28:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.426+0000 7fc458c00640 1 -- 172.21.15.43:0/3612316958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) ==== 104+0+0 (secure 0 0 0) 0x7fc454063300 con 0x7fc45c0c5c60 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 msgr2=0x7fc43c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 0x7fc43c075710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc4500af000 tx=0x7fc45001b010 comp rx=0 tx=0).stop 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 msgr2=0x7fc45c0c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc45400c760 tx=0x7fc45400cc20 comp rx=0 tx=0).stop 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 shutdown_connections 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc43c073200 0x7fc43c075710 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc45c155d70 0x7fc45c0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc45c0c5c60 0x7fc45c0c8b60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 --2- 172.21.15.43:0/3612316958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc45c06f110 0x7fc45c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 >> 172.21.15.43:0/3612316958 conn(0x7fc45c0b23d0 msgr2=0x7fc45c15b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 shutdown_connections 2024-09-17T05:28:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.430+0000 7fc463600640 1 -- 172.21.15.43:0/3612316958 wait complete. 2024-09-17T05:28:26.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 -- 172.21.15.43:0/3437096515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740015b7e0 msgr2=0x7f74000cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 --2- 172.21.15.43:0/3437096515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740015b7e0 0x7f74000cf270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7400072bd0 tx=0x7f73e801c600 comp rx=0 tx=0).stop 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 -- 172.21.15.43:0/3437096515 shutdown_connections 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 --2- 172.21.15.43:0/3437096515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740015b7e0 0x7f74000cf270 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 --2- 172.21.15.43:0/3437096515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 0x7f740015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 --2- 172.21.15.43:0/3437096515 >> v2:172.21.15.62:3300/0 conn(0x7f7400074c60 0x7f74000b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 -- 172.21.15.43:0/3437096515 >> 172.21.15.43:0/3437096515 conn(0x7f740006ce60 msgr2=0x7f740006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 -- 172.21.15.43:0/3437096515 shutdown_connections 2024-09-17T05:28:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 -- 172.21.15.43:0/3437096515 wait complete. 2024-09-17T05:28:26.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.618+0000 7f7406e00640 1 Processor -- start 2024-09-17T05:28:26.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- start start 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 0x7f74001fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740015b7e0 0x7f74001fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74000ce9f0 con 0x7f7400074c60 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74000ce870 con 0x7f740015b7e0 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f74000ceb70 con 0x7f74000cf7b0 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:26.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39942/0 (socket says 172.21.15.43:39942) 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 -- 172.21.15.43:0/3043505607 learned_addr learned my addr 172.21.15.43:0/3043505607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7404e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740015b7e0 0x7f74001fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73f7e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 0x7f74001fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 -- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 msgr2=0x7f74001fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 0x7f74001fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 -- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740015b7e0 msgr2=0x7f74001fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740015b7e0 0x7f74001fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 -- 172.21.15.43:0/3043505607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74001557c0 con 0x7f7400074c60 2024-09-17T05:28:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73ffe00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d6a0 tx=0x7f73f000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0017070 con 0x7f7400074c60 2024-09-17T05:28:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73f0004310 con 0x7f7400074c60 2024-09-17T05:28:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0005280 con 0x7f7400074c60 2024-09-17T05:28:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7400144b70 con 0x7f7400074c60 2024-09-17T05:28:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001fd1b0 con 0x7f7400074c60 2024-09-17T05:28:26.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.622+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74000b3b80 con 0x7f7400074c60 2024-09-17T05:28:26.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.626+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f73f0005420 con 0x7f7400074c60 2024-09-17T05:28:26.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.626+0000 7f73fcc00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 0x7f73dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:26.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.626+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6025+0+0 (secure 0 0 0) 0x7f73f0094840 con 0x7f7400074c60 2024-09-17T05:28:26.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.626+0000 7f73f7e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 0x7f73dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:26.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.626+0000 7f73f7e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 0x7f73dc075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e801ae40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:26.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.630+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f73f0020020 con 0x7f7400074c60 2024-09-17T05:28:26.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:26.806+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7400154830 con 0x7f7400074c60 2024-09-17T05:28:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.434+0000 7f73fcc00640 1 -- 172.21.15.43:0/3043505607 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7f73f005f680 con 0x7f7400074c60 2024-09-17T05:28:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:28:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 msgr2=0x7f73dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 0x7f73dc075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e801ae40 comp rx=0 tx=0).stop 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 msgr2=0x7f7400144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d6a0 tx=0x7f73f000db60 comp rx=0 tx=0).stop 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 shutdown_connections 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f73dc073200 0x7f73dc075710 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740015b7e0 0x7f74001fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74000cf7b0 0x7f74001fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 --2- 172.21.15.43:0/3043505607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7400074c60 0x7f7400144290 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 >> 172.21.15.43:0/3043505607 conn(0x7f740006ce60 msgr2=0x7f7400154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 shutdown_connections 2024-09-17T05:28:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.438+0000 7f7406e00640 1 -- 172.21.15.43:0/3043505607 wait complete. 2024-09-17T05:28:27.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.634+0000 7fd8b1800640 1 -- 172.21.15.43:0/4217274407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac0cf980 msgr2=0x7fd8ac0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.634+0000 7fd8b1800640 1 --2- 172.21.15.43:0/4217274407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac0cf980 0x7fd8ac0731d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009c80 tx=0x7fd8a401cda0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- 172.21.15.43:0/4217274407 shutdown_connections 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 --2- 172.21.15.43:0/4217274407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac0cf980 0x7fd8ac0731d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 --2- 172.21.15.43:0/4217274407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac074c60 0x7fd8ac0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- 172.21.15.43:0/4217274407 >> 172.21.15.43:0/4217274407 conn(0x7fd8ac06ce60 msgr2=0x7fd8ac06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- 172.21.15.43:0/4217274407 shutdown_connections 2024-09-17T05:28:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- 172.21.15.43:0/4217274407 wait complete. 2024-09-17T05:28:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 Processor -- start 2024-09-17T05:28:27.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- start start 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ac15b7e0 0x7fd8ac159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac1f7b80 0x7fd8ac15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac168fa0 con 0x7fd8ac074c60 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac168e20 con 0x7fd8ac15b7e0 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.638+0000 7fd8b1800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac169120 con 0x7fd8ac1f7b80 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39950/0 (socket says 172.21.15.43:39950) 2024-09-17T05:28:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 -- 172.21.15.43:0/785967326 learned_addr learned my addr 172.21.15.43:0/785967326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8ab400640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac1f7b80 0x7fd8ac15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8a3e00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ac15b7e0 0x7fd8ac159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 -- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac1f7b80 msgr2=0x7fd8ac15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac1f7b80 0x7fd8ac15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 -- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ac15b7e0 msgr2=0x7fd8ac159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ac15b7e0 0x7fd8ac159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 -- 172.21.15.43:0/785967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac154800 con 0x7fd8ac074c60 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8aaa00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e7a0 tx=0x7fd89c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cc60 con 0x7fd8ac074c60 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd89c00ee50 con 0x7fd8ac074c60 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c010670 con 0x7fd8ac074c60 2024-09-17T05:28:27.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac154aa0 con 0x7fd8ac074c60 2024-09-17T05:28:27.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac155060 con 0x7fd8ac074c60 2024-09-17T05:28:27.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac0b3b80 con 0x7fd8ac074c60 2024-09-17T05:28:27.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.642+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd89c010810 con 0x7fd8ac074c60 2024-09-17T05:28:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.646+0000 7fd8a2a00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 0x7fd888075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.646+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5587+0+0 (secure 0 0 0) 0x7fd89c094a60 con 0x7fd8ac074c60 2024-09-17T05:28:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.646+0000 7fd8a3e00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 0x7fd888075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.646+0000 7fd8a3e00640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 0x7fd888075710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0cc880 tx=0x7fd894009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:27.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.650+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd89c05f4a0 con 0x7fd8ac074c60 2024-09-17T05:28:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:27.822+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd8ac0701b0 con 0x7fd8ac074c60 2024-09-17T05:28:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.438+0000 7fd8a2a00640 1 -- 172.21.15.43:0/785967326 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7fd89c063ed0 con 0x7fd8ac074c60 2024-09-17T05:28:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:28:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.442+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 msgr2=0x7fd888075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.442+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 0x7fd888075710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0cc880 tx=0x7fd894009040 comp rx=0 tx=0).stop 2024-09-17T05:28:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.442+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 msgr2=0x7fd8ac14a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.442+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e7a0 tx=0x7fd89c00ec60 comp rx=0 tx=0).stop 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 shutdown_connections 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd888073200 0x7fd888075710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8ac15b7e0 0x7fd8ac159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8ac1f7b80 0x7fd8ac15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 --2- 172.21.15.43:0/785967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8ac074c60 0x7fd8ac14a4a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 >> 172.21.15.43:0/785967326 conn(0x7fd8ac06ce60 msgr2=0x7fd8ac0cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 shutdown_connections 2024-09-17T05:28:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.446+0000 7fd8b1800640 1 -- 172.21.15.43:0/785967326 wait complete. 2024-09-17T05:28:28.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:28:28.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- 172.21.15.43:0/1683573135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f94155d70 msgr2=0x7f8f9415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/1683573135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f94155d70 0x7f8f9415c7e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009870 tx=0x7f8f8401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- 172.21.15.43:0/1683573135 shutdown_connections 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/1683573135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f94155d70 0x7f8f9415c7e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- 172.21.15.43:0/1683573135 >> 172.21.15.43:0/1683573135 conn(0x7f8f940b23d0 msgr2=0x7f8f940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- 172.21.15.43:0/1683573135 shutdown_connections 2024-09-17T05:28:28.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- 172.21.15.43:0/1683573135 wait complete. 2024-09-17T05:28:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 Processor -- start 2024-09-17T05:28:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.638+0000 7f8f9ac00640 1 -- start start 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f94155d70 0x7f8f940cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f941fc470 0x7f8f940cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94169df0 con 0x7f8f9406f110 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94169c70 con 0x7f8f94155d70 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8f94169f70 con 0x7f8f941fc470 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39960/0 (socket says 172.21.15.43:39960) 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 -- 172.21.15.43:0/3814998995 learned_addr learned my addr 172.21.15.43:0/3814998995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f98c00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f941fc470 0x7f8f940cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:28.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f8be00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f94155d70 0x7f8f940cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 -- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f941fc470 msgr2=0x7f8f940cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f941fc470 0x7f8f940cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 -- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f94155d70 msgr2=0x7f8f940cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f94155d70 0x7f8f940cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 -- 172.21.15.43:0/3814998995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f94154360 con 0x7f8f9406f110 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f93e00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c10 tx=0x7f8f840a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840bb070 con 0x7f8f9406f110 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f84004330 con 0x7f8f9406f110 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840b2040 con 0x7f8f9406f110 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f94154640 con 0x7f8f9406f110 2024-09-17T05:28:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f94154c00 con 0x7f8f9406f110 2024-09-17T05:28:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.642+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9415c7e0 con 0x7f8f9406f110 2024-09-17T05:28:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.646+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8f84005290 con 0x7f8f9406f110 2024-09-17T05:28:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.646+0000 7f8f90c00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 0x7f8f70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.646+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f8f8412d800 con 0x7f8f9406f110 2024-09-17T05:28:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.646+0000 7f8f8be00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 0x7f8f70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.646+0000 7f8f8be00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 0x7f8f70075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c007f30 tx=0x7f8f7c007ec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.650+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8f84016450 con 0x7f8f9406f110 2024-09-17T05:28:28.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.826+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f8f941501b0 con 0x7f8f9406f110 2024-09-17T05:28:28.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f90c00640 1 -- 172.21.15.43:0/3814998995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v42) ==== 71+0+0 (secure 0 0 0) 0x7f8f840f8410 con 0x7f8f9406f110 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 msgr2=0x7f8f70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 0x7f8f70075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c007f30 tx=0x7f8f7c007ec0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 msgr2=0x7f8f941f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000c10 tx=0x7f8f840a5f90 comp rx=0 tx=0).stop 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 shutdown_connections 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8f70073200 0x7f8f70075710 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f94155d70 0x7f8f940cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8f941fc470 0x7f8f940cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 --2- 172.21.15.43:0/3814998995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f9406f110 0x7f8f941f85f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.830+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 >> 172.21.15.43:0/3814998995 conn(0x7f8f940b23d0 msgr2=0x7f8f940ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.834+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 shutdown_connections 2024-09-17T05:28:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:28.834+0000 7f8f9ac00640 1 -- 172.21.15.43:0/3814998995 wait complete. 2024-09-17T05:28:28.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 -- 172.21.15.43:0/2297561659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c15ca80 msgr2=0x7f687c15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 --2- 172.21.15.43:0/2297561659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c15ca80 0x7f687c15ce90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f687c074a50 tx=0x7f686401cd50 comp rx=0 tx=0).stop 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 -- 172.21.15.43:0/2297561659 shutdown_connections 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 --2- 172.21.15.43:0/2297561659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c15ca80 0x7f687c15ce90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 --2- 172.21.15.43:0/2297561659 >> v2:172.21.15.62:3300/0 conn(0x7f687c0d1cb0 0x7f687c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 -- 172.21.15.43:0/2297561659 >> 172.21.15.43:0/2297561659 conn(0x7f687c068fe0 msgr2=0x7f687c069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 -- 172.21.15.43:0/2297561659 shutdown_connections 2024-09-17T05:28:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.010+0000 7f687b200640 1 -- 172.21.15.43:0/2297561659 wait complete. 2024-09-17T05:28:29.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 Processor -- start 2024-09-17T05:28:29.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 -- start start 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f687c15ca80 0x7f687c1f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 0x7f687c0d0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1536a0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f687c153520 con 0x7f687c15ca80 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f687b200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f687c153820 con 0x7f687c1605e0 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f6879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f6879e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39982/0 (socket says 172.21.15.43:39982) 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.014+0000 7f6879e00640 1 -- 172.21.15.43:0/119393313 learned_addr learned my addr 172.21.15.43:0/119393313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6871400640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f687c15ca80 0x7f687c1f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f687a800640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 0x7f687c0d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 -- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 msgr2=0x7f687c0d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 0x7f687c0d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 -- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f687c15ca80 msgr2=0x7f687c1f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f687c15ca80 0x7f687c1f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 -- 172.21.15.43:0/119393313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0c9f00 con 0x7f687c0d1cb0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f687a800640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 0x7f687c0d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6879e00640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e7f0 tx=0x7f686c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00ccc0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f686c00ce80 con 0x7f687c0d1cb0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c005280 con 0x7f687c0d1cb0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c0ca1e0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c0ca7a0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c0ccc40 con 0x7f687c0d1cb0 2024-09-17T05:28:29.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.018+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f686c005420 con 0x7f687c0d1cb0 2024-09-17T05:28:29.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.022+0000 7f6872a00640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 0x7f6858075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.022+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f686c094970 con 0x7f687c0d1cb0 2024-09-17T05:28:29.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.022+0000 7f6871400640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 0x7f6858075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.022+0000 7f6871400640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 0x7f6858075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f68640af000 tx=0x7f686401b590 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:29.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.026+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f686c05fbd0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.262+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7f687c0cee30 con 0x7f687c0d1cb0 2024-09-17T05:28:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f6872a00640 1 -- 172.21.15.43:0/119393313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v250) ==== 142+0+0 (secure 0 0 0) 0x7f686c05f4d0 con 0x7f687c0d1cb0 2024-09-17T05:28:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 msgr2=0x7f6858075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 0x7f6858075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f68640af000 tx=0x7f686401b590 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 msgr2=0x7f687c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e7f0 tx=0x7f686c00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 shutdown_connections 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6858073200 0x7f6858075710 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f687c15ca80 0x7f687c1f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f687c1605e0 0x7f687c0d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 --2- 172.21.15.43:0/119393313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f687c0d1cb0 0x7f687c0cafa0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 >> 172.21.15.43:0/119393313 conn(0x7f687c068fe0 msgr2=0x7f687c15df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 shutdown_connections 2024-09-17T05:28:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.442+0000 7f687b200640 1 -- 172.21.15.43:0/119393313 wait complete. 2024-09-17T05:28:29.467 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-17T05:28:29.467 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-17T05:28:29.467 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:28:29.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:29.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 -- 172.21.15.43:0/2863056715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c440d0650 msgr2=0x7f7c440cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2863056715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c440d0650 0x7f7c440cd750 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009bc0 tx=0x7f7c3c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 -- 172.21.15.43:0/2863056715 shutdown_connections 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2863056715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c440d0650 0x7f7c440cd750 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 -- 172.21.15.43:0/2863056715 >> 172.21.15.43:0/2863056715 conn(0x7f7c440cee60 msgr2=0x7f7c440cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 -- 172.21.15.43:0/2863056715 shutdown_connections 2024-09-17T05:28:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.622+0000 7f7c4a800640 1 -- 172.21.15.43:0/2863056715 wait complete. 2024-09-17T05:28:29.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 Processor -- start 2024-09-17T05:28:29.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 -- start start 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 0x7f7c441f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c440d0650 0x7f7c441670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4416aa80 con 0x7f7c4409ba60 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4416a900 con 0x7f7c440d0240 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c4a800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4416ac00 con 0x7f7c440d0650 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c43400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39990/0 (socket says 172.21.15.43:39990) 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.626+0000 7f7c43400640 1 -- 172.21.15.43:0/2290498521 learned_addr learned my addr 172.21.15.43:0/2290498521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43e00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c440d0650 0x7f7c441670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 -- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c440d0650 msgr2=0x7f7c441670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3aa00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 0x7f7c441f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c440d0650 0x7f7c441670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 -- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 msgr2=0x7f7c441f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 0x7f7c441f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 -- 172.21.15.43:0/2290498521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c440c7380 con 0x7f7c4409ba60 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3aa00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 0x7f7c441f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c43400640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c420 tx=0x7f7c3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34017070 con 0x7f7c4409ba60 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c340053c0 con 0x7f7c4409ba60 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34003860 con 0x7f7c4409ba60 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441f94e0 con 0x7f7c4409ba60 2024-09-17T05:28:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441f8f60 con 0x7f7c4409ba60 2024-09-17T05:28:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c440b3130 con 0x7f7c4409ba60 2024-09-17T05:28:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.630+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c3401a070 con 0x7f7c4409ba60 2024-09-17T05:28:29.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.634+0000 7f7c3be00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 0x7f7c20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:29.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.634+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f7c34093230 con 0x7f7c4409ba60 2024-09-17T05:28:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.634+0000 7f7c3aa00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 0x7f7c20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.634+0000 7f7c3aa00640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 0x7f7c20075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c44160ac0 tx=0x7f7c2c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.638+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c3405e530 con 0x7f7c4409ba60 2024-09-17T05:28:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c440c7160 con 0x7f7c4409ba60 2024-09-17T05:28:29.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c3be00640 1 -- 172.21.15.43:0/2290498521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13224 (secure 0 0 0) 0x7f7c3405de30 con 0x7f7c4409ba60 2024-09-17T05:28:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:29.816 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":66,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:28:28.436528+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/48203361":"2024-09-18T05:28:25.299885+0000","172.21.15.62:6837/48203361":"2024-09-18T05:28:25.299885+0000","172.21.15.43:6834/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6832/61022679":"2024-09-18T05:26:44.197145+0000","172.21.15.43:6834/2376863639":"2024-09-18T05:26:08.067813+0000","172.21.15.43:6832/2376863639":"2024-09-18T05:26:08.067813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 msgr2=0x7f7c20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 0x7f7c20075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c44160ac0 tx=0x7f7c2c009040 comp rx=0 tx=0).stop 2024-09-17T05:28:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 msgr2=0x7f7c4414b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c420 tx=0x7f7c3400c8e0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 shutdown_connections 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c20073230 0x7f7c20075740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c440d0240 0x7f7c441f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c440d0650 0x7f7c441670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 --2- 172.21.15.43:0/2290498521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c4409ba60 0x7f7c4414b870 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 >> 172.21.15.43:0/2290498521 conn(0x7f7c440cee60 msgr2=0x7f7c44072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 shutdown_connections 2024-09-17T05:28:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.814+0000 7f7c4a800640 1 -- 172.21.15.43:0/2290498521 wait complete. 2024-09-17T05:28:29.839 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:28:29.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:28:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 -- 172.21.15.43:0/607133514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01040b3af0 msgr2=0x7f010415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 --2- 172.21.15.43:0/607133514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01040b3af0 0x7f010415c7e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f01040d0c40 tx=0x7f00ec01c740 comp rx=0 tx=0).stop 2024-09-17T05:28:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 -- 172.21.15.43:0/607133514 shutdown_connections 2024-09-17T05:28:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 --2- 172.21.15.43:0/607133514 >> v2:172.21.15.62:3300/0 conn(0x7f010406f110 0x7f01040ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 --2- 172.21.15.43:0/607133514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01040b3af0 0x7f010415c7e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 -- 172.21.15.43:0/607133514 >> 172.21.15.43:0/607133514 conn(0x7f01040b23d0 msgr2=0x7f01040d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 -- 172.21.15.43:0/607133514 shutdown_connections 2024-09-17T05:28:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:29.998+0000 7f0109200640 1 -- 172.21.15.43:0/607133514 wait complete. 2024-09-17T05:28:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.002+0000 7f0109200640 1 Processor -- start 2024-09-17T05:28:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.002+0000 7f0109200640 1 -- start start 2024-09-17T05:28:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.002+0000 7f0109200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0104155d70 0x7f01041fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010409ac70 0x7f01041f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0104168b90 con 0x7f010406f110 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0104168a10 con 0x7f010409ac70 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0104168d10 con 0x7f0104155d70 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39996/0 (socket says 172.21.15.43:39996) 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 -- 172.21.15.43:0/4187941162 learned_addr learned my addr 172.21.15.43:0/4187941162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0102800640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010409ac70 0x7f01041f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f00f9400640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0104155d70 0x7f01041fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 -- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0104155d70 msgr2=0x7f01041fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0104155d70 0x7f01041fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 -- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010409ac70 msgr2=0x7f01041f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010409ac70 0x7f01041f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 -- 172.21.15.43:0/4187941162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01040ae350 con 0x7f010406f110 2024-09-17T05:28:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0101e00640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f00f400c3c0 tx=0x7f00f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4011790 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00f4017070 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4011ce0 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041fd1a0 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01040ce3e0 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01040ad0b0 con 0x7f010406f110 2024-09-17T05:28:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.006+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00f401c460 con 0x7f010406f110 2024-09-17T05:28:30.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.010+0000 7f00faa00640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 0x7f00e0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.010+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f00f4093c30 con 0x7f010406f110 2024-09-17T05:28:30.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.010+0000 7f00f9400640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 0x7f00e0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.010+0000 7f00f9400640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 0x7f00e0075740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0af000 tx=0x7f00ec01af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:30.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.014+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00f409a080 con 0x7f010406f110 2024-09-17T05:28:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.190+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01040ca3e0 con 0x7f010406f110 2024-09-17T05:28:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.190+0000 7f00faa00640 1 -- 172.21.15.43:0/4187941162 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f00f405efa0 con 0x7f010406f110 2024-09-17T05:28:30.194 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 msgr2=0x7f00e0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 0x7f00e0075740 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0af000 tx=0x7f00ec01af80 comp rx=0 tx=0).stop 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 msgr2=0x7f0104155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f00f400c3c0 tx=0x7f00f400c880 comp rx=0 tx=0).stop 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 shutdown_connections 2024-09-17T05:28:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00e0073230 0x7f00e0075740 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010409ac70 0x7f01041f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0104155d70 0x7f01041fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 --2- 172.21.15.43:0/4187941162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f010406f110 0x7f0104155440 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 >> 172.21.15.43:0/4187941162 conn(0x7f01040b23d0 msgr2=0x7f01040ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 shutdown_connections 2024-09-17T05:28:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.194+0000 7f0109200640 1 -- 172.21.15.43:0/4187941162 wait complete. 2024-09-17T05:28:30.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-17T05:28:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- 172.21.15.43:0/1452319335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c0cb860 msgr2=0x7f4a7c0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- 172.21.15.43:0/1452319335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0cbc70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400aa10 tx=0x7f4a7401db40 comp rx=0 tx=0).stop 2024-09-17T05:28:30.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- 172.21.15.43:0/1452319335 shutdown_connections 2024-09-17T05:28:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- 172.21.15.43:0/1452319335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0cbc70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- 172.21.15.43:0/1452319335 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c06ce60 0x7f4a7c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- 172.21.15.43:0/1452319335 >> 172.21.15.43:0/1452319335 conn(0x7f4a7c0b23d0 msgr2=0x7f4a7c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- 172.21.15.43:0/1452319335 shutdown_connections 2024-09-17T05:28:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- 172.21.15.43:0/1452319335 wait complete. 2024-09-17T05:28:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 Processor -- start 2024-09-17T05:28:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 -- start start 2024-09-17T05:28:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c070620 0x7f4a7c09d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.374+0000 7f4a81000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0ce690 con 0x7f4a7c06ce60 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0ce510 con 0x7f4a7c0cb860 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c0ce810 con 0x7f4a7c070620 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40022/0 (socket says 172.21.15.43:40022) 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 -- 172.21.15.43:0/2617477765 learned_addr learned my addr 172.21.15.43:0/2617477765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a71400640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c070620 0x7f4a7c09d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a7a800640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 -- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c070620 msgr2=0x7f4a7c09d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c070620 0x7f4a7c09d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 -- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 msgr2=0x7f4a7c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 -- 172.21.15.43:0/2617477765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c0aeea0 con 0x7f4a7c06ce60 2024-09-17T05:28:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a7a800640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a79e00640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00c760 tx=0x7f4a6c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c003b80 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a6c003d40 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00aab0 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c1f9310 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c09def0 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c1f9100 con 0x7f4a7c06ce60 2024-09-17T05:28:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.378+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a6c00ac50 con 0x7f4a7c06ce60 2024-09-17T05:28:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.382+0000 7f4a72a00640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 0x7f4a58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.382+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f4a6c093d70 con 0x7f4a7c06ce60 2024-09-17T05:28:30.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.382+0000 7f4a71400640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 0x7f4a58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:30.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.382+0000 7f4a71400640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 0x7f4a58075740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c1fd7b0 tx=0x7f4a6400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:30.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.386+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4a7c06d7b0 con 0x7f4a7c06ce60 2024-09-17T05:28:30.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:30.622+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f4a7c0ceda0 con 0x7f4a7c06ce60 2024-09-17T05:28:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.450+0000 7f4a72a00640 1 -- 172.21.15.43:0/2617477765 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v252) ==== 144+0+0 (secure 0 0 0) 0x7f4a6c05efd0 con 0x7f4a7c06ce60 2024-09-17T05:28:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 msgr2=0x7f4a58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 0x7f4a58075740 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c1fd7b0 tx=0x7f4a6400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 msgr2=0x7f4a7c0ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00c760 tx=0x7f4a6c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 shutdown_connections 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a58073230 0x7f4a58075740 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a7c0cb860 0x7f4a7c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a7c070620 0x7f4a7c09d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 --2- 172.21.15.43:0/2617477765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a7c06ce60 0x7f4a7c0ae670 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 >> 172.21.15.43:0/2617477765 conn(0x7f4a7c0b23d0 msgr2=0x7f4a7c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 shutdown_connections 2024-09-17T05:28:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.454+0000 7f4a81000640 1 -- 172.21.15.43:0/2617477765 wait complete. 2024-09-17T05:28:31.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:28:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- 172.21.15.43:0/1838892777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 msgr2=0x7f85580ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 --2- 172.21.15.43:0/1838892777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f85580ad0b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f85580af0a0 tx=0x7f855001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- 172.21.15.43:0/1838892777 shutdown_connections 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 --2- 172.21.15.43:0/1838892777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f85580ad0b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 --2- 172.21.15.43:0/1838892777 >> v2:172.21.15.62:3300/0 conn(0x7f85580b3af0 0x7f855815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- 172.21.15.43:0/1838892777 >> 172.21.15.43:0/1838892777 conn(0x7f85580b23d0 msgr2=0x7f85580d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- 172.21.15.43:0/1838892777 shutdown_connections 2024-09-17T05:28:31.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- 172.21.15.43:0/1838892777 wait complete. 2024-09-17T05:28:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 Processor -- start 2024-09-17T05:28:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.642+0000 7f855e600640 1 -- start start 2024-09-17T05:28:31.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 0x7f85580c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f855815cde0 0x7f855815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8558168c30 con 0x7f855806f110 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8558168ab0 con 0x7f855815cde0 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8558168db0 con 0x7f8558155d70 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40044/0 (socket says 172.21.15.43:40044) 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 -- 172.21.15.43:0/2105368102 learned_addr learned my addr 172.21.15.43:0/2105368102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8556a00640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 0x7f85580c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557e00640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f855815cde0 0x7f855815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 -- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 msgr2=0x7f85580c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 0x7f85580c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 -- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f855815cde0 msgr2=0x7f855815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f855815cde0 0x7f855815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 -- 172.21.15.43:0/2105368102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85580c88f0 con 0x7f855806f110 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8556a00640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 0x7f85580c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f8557400640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f854800b3e0 tx=0x7f854800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548015020 con 0x7f855806f110 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8548011920 con 0x7f855806f110 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85580c8bd0 con 0x7f855806f110 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548011db0 con 0x7f855806f110 2024-09-17T05:28:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85580c9190 con 0x7f855806f110 2024-09-17T05:28:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85580ad0b0 con 0x7f855806f110 2024-09-17T05:28:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.646+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8548018020 con 0x7f855806f110 2024-09-17T05:28:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.650+0000 7f854f400640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 0x7f8538075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:31.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.650+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7f8548093990 con 0x7f855806f110 2024-09-17T05:28:31.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.650+0000 7f8556a00640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 0x7f8538075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:31.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.650+0000 7f8556a00640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 0x7f8538075710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8544005fd0 tx=0x7f8544005f20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:31.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.654+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f854801f070 con 0x7f855806f110 2024-09-17T05:28:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.830+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f855815bd90 con 0x7f855806f110 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.830+0000 7f854f400640 1 -- 172.21.15.43:0/2105368102 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) ==== 59+0+2294 (secure 0 0 0) 0x7f854805ec00 con 0x7f855806f110 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:28:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:28:31.834 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:28:31.835 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:28:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 msgr2=0x7f8538075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 0x7f8538075710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8544005fd0 tx=0x7f8544005f20 comp rx=0 tx=0).stop 2024-09-17T05:28:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 msgr2=0x7f855809c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f854800b3e0 tx=0x7f854800b8a0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 shutdown_connections 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8538073200 0x7f8538075710 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f855815cde0 0x7f855815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8558155d70 0x7f85580c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 --2- 172.21.15.43:0/2105368102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855806f110 0x7f855809c600 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 >> 172.21.15.43:0/2105368102 conn(0x7f85580b23d0 msgr2=0x7f85580ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 shutdown_connections 2024-09-17T05:28:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:31.834+0000 7f855e600640 1 -- 172.21.15.43:0/2105368102 wait complete. 2024-09-17T05:28:31.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:28:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 -- 172.21.15.43:0/447980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6806f2a0 msgr2=0x7fbc680ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 --2- 172.21.15.43:0/447980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6806f2a0 0x7fbc680ad150 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbc680b5300 tx=0x7fbc6001cd50 comp rx=0 tx=0).stop 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 -- 172.21.15.43:0/447980692 shutdown_connections 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 --2- 172.21.15.43:0/447980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6806f2a0 0x7fbc680ad150 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 --2- 172.21.15.43:0/447980692 >> v2:172.21.15.62:3300/0 conn(0x7fbc680733b0 0x7fbc680737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 -- 172.21.15.43:0/447980692 >> 172.21.15.43:0/447980692 conn(0x7fbc6806d9f0 msgr2=0x7fbc680b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 -- 172.21.15.43:0/447980692 shutdown_connections 2024-09-17T05:28:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.014+0000 7fbc6d600640 1 -- 172.21.15.43:0/447980692 wait complete. 2024-09-17T05:28:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 Processor -- start 2024-09-17T05:28:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 -- start start 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806ee90 0x7fbc68160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc6806f2a0 0x7fbc6814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68169dd0 con 0x7fbc680733b0 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68169c50 con 0x7fbc6806ee90 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6d600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbc68169f50 con 0x7fbc6806f2a0 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40050/0 (socket says 172.21.15.43:40050) 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 -- 172.21.15.43:0/1202077006 learned_addr learned my addr 172.21.15.43:0/1202077006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806ee90 0x7fbc68160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc67400640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc6806f2a0 0x7fbc6814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 -- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc6806f2a0 msgr2=0x7fbc6814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc6806f2a0 0x7fbc6814da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 -- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806ee90 msgr2=0x7fbc68160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806ee90 0x7fbc68160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.018+0000 7fbc6cc00640 1 -- 172.21.15.43:0/1202077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6814dfa0 con 0x7fbc680733b0 2024-09-17T05:28:32.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc6cc00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbc6001b400 tx=0x7fbc600092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600bc070 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc60004330 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6814e3e0 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600b2040 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc68153c20 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6815fc80 con 0x7fbc680733b0 2024-09-17T05:28:32.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc600bf070 con 0x7fbc680733b0 2024-09-17T05:28:32.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.022+0000 7fbc5fe00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 0x7fbc48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.026+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7fbc6012dc70 con 0x7fbc680733b0 2024-09-17T05:28:32.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.026+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 0x7fbc48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.026+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 0x7fbc48075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc58004660 tx=0x7fbc5800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:32.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.030+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc60016970 con 0x7fbc680733b0 2024-09-17T05:28:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.202+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbc680ad150 con 0x7fbc680733b0 2024-09-17T05:28:32.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.206+0000 7fbc5fe00640 1 -- 172.21.15.43:0/1202077006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fbc600f8880 con 0x7fbc680733b0 2024-09-17T05:28:32.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 msgr2=0x7fbc48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 0x7fbc48075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc58004660 tx=0x7fbc5800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 msgr2=0x7fbc68154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbc6001b400 tx=0x7fbc600092c0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 shutdown_connections 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbc48073200 0x7fbc48075710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680733b0 0x7fbc68154450 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc6806f2a0 0x7fbc6814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 --2- 172.21.15.43:0/1202077006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806ee90 0x7fbc68160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 >> 172.21.15.43:0/1202077006 conn(0x7fbc6806d9f0 msgr2=0x7fbc680cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 shutdown_connections 2024-09-17T05:28:32.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.210+0000 7fbc6d600640 1 -- 172.21.15.43:0/1202077006 wait complete. 2024-09-17T05:28:32.231 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:28:32.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:28:32.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:28:32.427 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:28:32.427 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:32.427 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:32.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:28:32.436 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:28:32.436 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:32.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:28:32.446 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:32.446 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:32.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:28:32.487 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:28:32.487 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:32.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:28:32.497 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:28:32.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 -- 172.21.15.43:0/2070455669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec8004d50 msgr2=0x7faec8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 --2- 172.21.15.43:0/2070455669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec8004d50 0x7faec8005160 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ee0 tx=0x7faedc01d630 comp rx=0 tx=0).stop 2024-09-17T05:28:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faee9200640 1 -- 172.21.15.43:0/2070455669 reap_dead start 2024-09-17T05:28:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 -- 172.21.15.43:0/2070455669 shutdown_connections 2024-09-17T05:28:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 -- 172.21.15.43:0/2070455669 >> 172.21.15.43:0/2070455669 conn(0x7faee40b2e80 msgr2=0x7faee40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 -- 172.21.15.43:0/2070455669 shutdown_connections 2024-09-17T05:28:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.690+0000 7faeeb200640 1 -- 172.21.15.43:0/2070455669 wait complete. 2024-09-17T05:28:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 Processor -- start 2024-09-17T05:28:32.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 -- start start 2024-09-17T05:28:32.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8004d50 0x7faee41516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 0x7faee40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faee40b2ad0 con 0x7faee41f8d00 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faee40b2950 con 0x7faee41f68d0 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faeeb200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faee40ce2f0 con 0x7faec8004d50 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40060/0 (socket says 172.21.15.43:40060) 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 -- 172.21.15.43:0/3432671229 learned_addr learned my addr 172.21.15.43:0/3432671229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee3e00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8004d50 0x7faee41516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faedbe00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 0x7faee40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 -- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8004d50 msgr2=0x7faee41516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8004d50 0x7faee41516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 -- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 msgr2=0x7faee40b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 0x7faee40b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faee9200640 1 -- 172.21.15.43:0/3432671229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee4151c30 con 0x7faee41f8d00 2024-09-17T05:28:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.694+0000 7faedbe00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 0x7faee40b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee9200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ee0 tx=0x7faedc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc009200 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faedc0aa500 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc0aacc0 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee40b0d10 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee40b12d0 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee40b5720 con 0x7faee41f8d00 2024-09-17T05:28:32.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faedc003980 con 0x7faee41f8d00 2024-09-17T05:28:32.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:32.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5115+0+0 (secure 0 0 0) 0x7faedc12ed40 con 0x7faee41f8d00 2024-09-17T05:28:32.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.698+0000 7faee3e00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:32.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.702+0000 7faee3e00640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 0x7faec0075710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faed4000fc0 tx=0x7faed4006f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.702+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faedc0c3070 con 0x7faee41f8d00 2024-09-17T05:28:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.918+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee40aea70 con 0x7faee41f8d00 2024-09-17T05:28:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.918+0000 7faee0c00640 1 -- 172.21.15.43:0/3432671229 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+3905 (secure 0 0 0) 0x7faedc0ae020 con 0x7faee41f8d00 2024-09-17T05:28:32.921 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:32.922 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"btime":"2024-09-17T05:28:26:423102+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/198979112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":198979112},{"type":"v1","addr":"172.21.15.43:6835","nonce":198979112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/4199022131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4199022131},{"type":"v1","addr":"172.21.15.62:6836","nonce":4199022131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5735,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/558708353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":558708353},{"type":"v1","addr":"172.21.15.62:6837","nonce":558708353}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1714304294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1714304294},{"type":"v1","addr":"172.21.15.43:6834","nonce":1714304294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[]} 2024-09-17T05:28:32.922 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 62 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 msgr2=0x7faec0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 0x7faec0075710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faed4000fc0 tx=0x7faed4006f60 comp rx=0 tx=0).stop 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 msgr2=0x7faee41f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ee0 tx=0x7faedc009780 comp rx=0 tx=0).stop 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 shutdown_connections 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faee41f68d0 0x7faee40b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faee41f8d00 0x7faee41f9110 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 --2- 172.21.15.43:0/3432671229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8004d50 0x7faee41516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 >> 172.21.15.43:0/3432671229 conn(0x7faee40b2e80 msgr2=0x7faee40b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 shutdown_connections 2024-09-17T05:28:32.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:32.922+0000 7faeeb200640 1 -- 172.21.15.43:0/3432671229 wait complete. 2024-09-17T05:28:32.945 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:28:32.945 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:28:32.945 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:28:32.945 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:28:32.945 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:28:32.945 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:28:32.946 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:28:32.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:28:32.946 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:28:32.946 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:28:32.946 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:28:32.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:28:32.947 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:28:32.947 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:28:32.947 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:28:32.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:28:32.959 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:28:32.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:28:32.961 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:28:32.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:28:32.968 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:28:32.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:28:33.005 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:28:33.005 DEBUG:teuthology.parallel:result is None 2024-09-17T05:28:33.014 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:28:33.015 DEBUG:teuthology.parallel:result is None 2024-09-17T05:28:33.017 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:28:33.017 DEBUG:teuthology.parallel:result is None 2024-09-17T05:28:33.103 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:28:33.126 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:28:33.126 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:28:38.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T05:28:38.960 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:28:38.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:28:38.964 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:28:38.964 DEBUG:teuthology.parallel:result is None 2024-09-17T05:28:38.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:28:39.073 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 -- 172.21.15.43:0/204026084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d40cc6d0 msgr2=0x7f74d40ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 --2- 172.21.15.43:0/204026084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d40cc6d0 0x7f74d40ce100 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f74d406da80 tx=0x7f74c001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 -- 172.21.15.43:0/204026084 shutdown_connections 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 --2- 172.21.15.43:0/204026084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d40cc6d0 0x7f74d40ce100 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 --2- 172.21.15.43:0/204026084 >> v2:172.21.15.62:3300/0 conn(0x7f74d415adc0 0x7f74d406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 --2- 172.21.15.43:0/204026084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 0x7f74d406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 -- 172.21.15.43:0/204026084 >> 172.21.15.43:0/204026084 conn(0x7f74d40b23d0 msgr2=0x7f74d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 -- 172.21.15.43:0/204026084 shutdown_connections 2024-09-17T05:28:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 -- 172.21.15.43:0/204026084 wait complete. 2024-09-17T05:28:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.138+0000 7f74da400640 1 Processor -- start 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- start start 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 0x7f74d40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74d40cc6d0 0x7f74d40c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74d406fc70 con 0x7f74d415adc0 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74d406faf0 con 0x7f74d40cc6d0 2024-09-17T05:28:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f74d406fdf0 con 0x7f74d40711a0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 0x7f74d40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47742/0 (socket says 172.21.15.43:47742) 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 -- 172.21.15.43:0/2537049199 learned_addr learned my addr 172.21.15.43:0/2537049199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74d40cc6d0 0x7f74d40c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 -- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 msgr2=0x7f74d40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 0x7f74d40b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 -- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74d40cc6d0 msgr2=0x7f74d40c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74d40cc6d0 0x7f74d40c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 -- 172.21.15.43:0/2537049199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d409d560 con 0x7f74d415adc0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74d3e00640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009850 tx=0x7f74c00a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00b2040 con 0x7f74d415adc0 2024-09-17T05:28:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c00041d0 con 0x7f74d415adc0 2024-09-17T05:28:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0003600 con 0x7f74d415adc0 2024-09-17T05:28:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d4156770 con 0x7f74d415adc0 2024-09-17T05:28:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d40b1260 con 0x7f74d415adc0 2024-09-17T05:28:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.142+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d40ce100 con 0x7f74d415adc0 2024-09-17T05:28:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.146+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f74c00bf020 con 0x7f74d415adc0 2024-09-17T05:28:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.146+0000 7f74cb400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 0x7f74b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.146+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5115+0+0 (secure 0 0 0) 0x7f74c012e1e0 con 0x7f74d415adc0 2024-09-17T05:28:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.146+0000 7f74d3400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 0x7f74b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.146+0000 7f74d3400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 0x7f74b4075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009dd0 tx=0x7f74cc009520 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.150+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74c00c6020 con 0x7f74d415adc0 2024-09-17T05:28:39.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:39.342+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f74d40cf450 con 0x7f74d415adc0 2024-09-17T05:28:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.526+0000 7f74cb400640 1 -- 172.21.15.43:0/2537049199 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) ==== 97+0+0 (secure 0 0 0) 0x7f74c00164b0 con 0x7f74d415adc0 2024-09-17T05:28:40.530 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:28:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 msgr2=0x7f74b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 0x7f74b4075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009dd0 tx=0x7f74cc009520 comp rx=0 tx=0).stop 2024-09-17T05:28:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 msgr2=0x7f74d40c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009850 tx=0x7f74c00a56d0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 shutdown_connections 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f74b4073200 0x7f74b4075710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74d40cc6d0 0x7f74d40c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74d415adc0 0x7f74d40c7e40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 --2- 172.21.15.43:0/2537049199 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f74d40711a0 0x7f74d40b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 >> 172.21.15.43:0/2537049199 conn(0x7f74d40b23d0 msgr2=0x7f74d40adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 shutdown_connections 2024-09-17T05:28:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.530+0000 7f74da400640 1 -- 172.21.15.43:0/2537049199 wait complete. 2024-09-17T05:28:40.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- 172.21.15.43:0/3396285736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe28006fc90 msgr2=0x7fe2800700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 --2- 172.21.15.43:0/3396285736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe28006fc90 0x7fe2800700a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe2800ccb60 tx=0x7fe26801c7a0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- 172.21.15.43:0/3396285736 shutdown_connections 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 --2- 172.21.15.43:0/3396285736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe28006fc90 0x7fe2800700a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 --2- 172.21.15.43:0/3396285736 >> v2:172.21.15.62:3300/0 conn(0x7fe28015fe10 0x7fe2800b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- 172.21.15.43:0/3396285736 >> 172.21.15.43:0/3396285736 conn(0x7fe2800b23d0 msgr2=0x7fe2800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- 172.21.15.43:0/3396285736 shutdown_connections 2024-09-17T05:28:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- 172.21.15.43:0/3396285736 wait complete. 2024-09-17T05:28:40.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 Processor -- start 2024-09-17T05:28:40.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.722+0000 7fe286200640 1 -- start start 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 0x7fe2801fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 0x7fe2800cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe28016a0a0 con 0x7fe2800730b0 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe280169f20 con 0x7fe28015fe10 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe28016a220 con 0x7fe28006fc90 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47754/0 (socket says 172.21.15.43:47754) 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 -- 172.21.15.43:0/1609349358 learned_addr learned my addr 172.21.15.43:0/1609349358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe27fe00640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 0x7fe2800cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe27f400640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 0x7fe2801fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:40.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 -- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 msgr2=0x7fe2801fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 0x7fe2801fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 -- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 msgr2=0x7fe2800cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 0x7fe2800cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 -- 172.21.15.43:0/1609349358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28015d9b0 con 0x7fe2800730b0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe27fe00640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 0x7fe2800cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe27f400640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 0x7fe2801fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277e00640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe2680b0040 tx=0x7fe2680a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ba020 con 0x7fe2800730b0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2680043b0 con 0x7fe2800730b0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680b2040 con 0x7fe2800730b0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2801f94d0 con 0x7fe2800730b0 2024-09-17T05:28:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2801f9a90 con 0x7fe2800730b0 2024-09-17T05:28:40.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.726+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800700a0 con 0x7fe2800730b0 2024-09-17T05:28:40.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.730+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2680bd070 con 0x7fe2800730b0 2024-09-17T05:28:40.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.730+0000 7fe277400640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 0x7fe25c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:40.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.730+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4857+0+0 (secure 0 0 0) 0x7fe26812d890 con 0x7fe2800730b0 2024-09-17T05:28:40.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.730+0000 7fe27f400640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 0x7fe25c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:40.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.730+0000 7fe27f400640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 0x7fe25c075710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe270005fd0 tx=0x7fe27000a540 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:40.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.734+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe2680164b0 con 0x7fe2800730b0 2024-09-17T05:28:40.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe280074de0 con 0x7fe2800730b0 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe277400640 1 -- 172.21.15.43:0/1609349358 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fe2680f8c80 con 0x7fe2800730b0 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.972 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:28:40.975 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:28:40.977 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:28:40.979 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.981 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:28:40.984 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:28:40.985 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:28:40.986 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:28:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:40.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:28:40.987 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 msgr2=0x7fe25c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 0x7fe25c075710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe270005fd0 tx=0x7fe27000a540 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 msgr2=0x7fe2800c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe2680b0040 tx=0x7fe2680a5530 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 shutdown_connections 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe25c073200 0x7fe25c075710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe28015fe10 0x7fe2800cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2800730b0 0x7fe2800c9770 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 --2- 172.21.15.43:0/1609349358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe28006fc90 0x7fe2801fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 >> 172.21.15.43:0/1609349358 conn(0x7fe2800b23d0 msgr2=0x7fe2800ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:40.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 shutdown_connections 2024-09-17T05:28:40.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:40.970+0000 7fe286200640 1 -- 172.21.15.43:0/1609349358 wait complete. 2024-09-17T05:28:40.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 -- 172.21.15.43:0/979715371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 msgr2=0x7f088c0b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 --2- 172.21.15.43:0/979715371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0b5cf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f088c073b10 tx=0x7f088401caa0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 -- 172.21.15.43:0/979715371 shutdown_connections 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 --2- 172.21.15.43:0/979715371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0b5cf0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 --2- 172.21.15.43:0/979715371 >> v2:172.21.15.62:3300/0 conn(0x7f088c06ce60 0x7f088c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.150+0000 7f0892c00640 1 -- 172.21.15.43:0/979715371 >> 172.21.15.43:0/979715371 conn(0x7f088c0b2e80 msgr2=0x7f088c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- 172.21.15.43:0/979715371 shutdown_connections 2024-09-17T05:28:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- 172.21.15.43:0/979715371 wait complete. 2024-09-17T05:28:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 Processor -- start 2024-09-17T05:28:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- start start 2024-09-17T05:28:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f088c06ce60 0x7f088c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f088c0b4180 0x7f088c071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f088c169dc0 con 0x7f088c0b58e0 2024-09-17T05:28:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f088c169c40 con 0x7f088c06ce60 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.154+0000 7f0892c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f088c169f40 con 0x7f088c0b4180 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f088be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f088c06ce60 0x7f088c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47784/0 (socket says 172.21.15.43:47784) 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 -- 172.21.15.43:0/3786944848 learned_addr learned my addr 172.21.15.43:0/3786944848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0883e00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f088c0b4180 0x7f088c071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 -- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f088c0b4180 msgr2=0x7f088c071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f088c0b4180 0x7f088c071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 -- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f088c06ce60 msgr2=0x7f088c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f088c06ce60 0x7f088c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 -- 172.21.15.43:0/3786944848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c15dcf0 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0890c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0884009bd0 tx=0x7f0884005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840bb070 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0884004d40 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840b2040 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c15dfd0 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c15e590 con 0x7f088c0b58e0 2024-09-17T05:28:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c06d270 con 0x7f088c0b58e0 2024-09-17T05:28:41.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.158+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08840be070 con 0x7f088c0b58e0 2024-09-17T05:28:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.162+0000 7f0888c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 0x7f0868075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.162+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4857+0+0 (secure 0 0 0) 0x7f088412dc70 con 0x7f088c0b58e0 2024-09-17T05:28:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.162+0000 7f088be00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 0x7f0868075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.162+0000 7f088be00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 0x7f0868075740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f087c0098e0 tx=0x7f087c009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.166+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f088c06d7b0 con 0x7f088c0b58e0 2024-09-17T05:28:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.338+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f088c15da80 con 0x7f088c0b58e0 2024-09-17T05:28:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0888c00640 1 -- 172.21.15.43:0/3786944848 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f08840f8980 con 0x7f088c0b58e0 2024-09-17T05:28:41.343 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:28:41.344 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:28:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 msgr2=0x7f0868075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 0x7f0868075740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f087c0098e0 tx=0x7f087c009060 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 msgr2=0x7f088c0c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0884009bd0 tx=0x7f0884005b00 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 shutdown_connections 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0868073230 0x7f0868075740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b58e0 0x7f088c0c4ba0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f088c0b4180 0x7f088c071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 --2- 172.21.15.43:0/3786944848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f088c06ce60 0x7f088c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 >> 172.21.15.43:0/3786944848 conn(0x7f088c0b2e80 msgr2=0x7f088c0c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 shutdown_connections 2024-09-17T05:28:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.342+0000 7f0892c00640 1 -- 172.21.15.43:0/3786944848 wait complete. 2024-09-17T05:28:41.366 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:28:41.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:28:41.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 -- 172.21.15.43:0/469537871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40d05b0 msgr2=0x7f2ee40d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 --2- 172.21.15.43:0/469537871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40d05b0 0x7f2ee40d09c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005fd0 tx=0x7f2ed401c820 comp rx=0 tx=0).stop 2024-09-17T05:28:41.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 -- 172.21.15.43:0/469537871 shutdown_connections 2024-09-17T05:28:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 --2- 172.21.15.43:0/469537871 >> v2:172.21.15.62:3300/0 conn(0x7f2ee40b2c30 0x7f2ee40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 --2- 172.21.15.43:0/469537871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40d05b0 0x7f2ee40d09c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 -- 172.21.15.43:0/469537871 >> 172.21.15.43:0/469537871 conn(0x7f2ee40cef80 msgr2=0x7f2ee40cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 -- 172.21.15.43:0/469537871 shutdown_connections 2024-09-17T05:28:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.526+0000 7f2eea400640 1 -- 172.21.15.43:0/469537871 wait complete. 2024-09-17T05:28:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 Processor -- start 2024-09-17T05:28:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 -- start start 2024-09-17T05:28:41.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40b2c30 0x7f2ee41fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40d05b0 0x7f2ee415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee416a080 con 0x7f2ee40d0f00 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4169f00 con 0x7f2ee40d05b0 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2eea400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee416a200 con 0x7f2ee40b2c30 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47788/0 (socket says 172.21.15.43:47788) 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 -- 172.21.15.43:0/1847467838 learned_addr learned my addr 172.21.15.43:0/1847467838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee2a00640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40d05b0 0x7f2ee415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40b2c30 0x7f2ee41fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 -- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40b2c30 msgr2=0x7f2ee41fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40b2c30 0x7f2ee41fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 -- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40d05b0 msgr2=0x7f2ee415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40d05b0 0x7f2ee415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.530+0000 7f2ee3e00640 1 -- 172.21.15.43:0/1847467838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee415ca70 con 0x7f2ee40d0f00 2024-09-17T05:28:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2ee3e00640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ec30 tx=0x7f2edc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc003b80 con 0x7f2ee40d0f00 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2edc003e10 con 0x7f2ee40d0f00 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004280 con 0x7f2ee40d0f00 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41faaf0 con 0x7f2ee40d0f00 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee40c8420 con 0x7f2ee40d0f00 2024-09-17T05:28:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee40d09c0 con 0x7f2ee40d0f00 2024-09-17T05:28:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.534+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2edc004420 con 0x7f2ee40d0f00 2024-09-17T05:28:41.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.538+0000 7f2edb400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 0x7f2ec4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:41.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.538+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4857+0+0 (secure 0 0 0) 0x7f2edc094b70 con 0x7f2ee40d0f00 2024-09-17T05:28:41.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.538+0000 7f2ee3400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 0x7f2ec4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.538+0000 7f2ee3400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 0x7f2ec4075740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ae000 tx=0x7f2ed401b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:41.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.542+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2edc05f880 con 0x7f2ee40d0f00 2024-09-17T05:28:41.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:41.718+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2ee41fa880 con 0x7f2ee40d0f00 2024-09-17T05:28:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.542+0000 7f2edb400640 1 -- 172.21.15.43:0/1847467838 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f2edc0642b0 con 0x7f2ee40d0f00 2024-09-17T05:28:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 msgr2=0x7f2ec4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 0x7f2ec4075740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ae000 tx=0x7f2ed401b060 comp rx=0 tx=0).stop 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 msgr2=0x7f2ee415cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ec30 tx=0x7f2edc00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 shutdown_connections 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ec4073230 0x7f2ec4075740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee40d05b0 0x7f2ee415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee40d0f00 0x7f2ee415cc80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 --2- 172.21.15.43:0/1847467838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2ee40b2c30 0x7f2ee41fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 >> 172.21.15.43:0/1847467838 conn(0x7f2ee40cef80 msgr2=0x7f2ee4072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 shutdown_connections 2024-09-17T05:28:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.546+0000 7f2eea400640 1 -- 172.21.15.43:0/1847467838 wait complete. 2024-09-17T05:28:42.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:28:42.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.746+0000 7fc5f3000640 1 -- 172.21.15.43:0/805329737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0004d50 msgr2=0x7fc5d0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:42.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.746+0000 7fc5f3000640 1 --2- 172.21.15.43:0/805329737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0004d50 0x7fc5d0005160 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009eb0 tx=0x7fc5e401d5f0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.746+0000 7fc5f1000640 1 -- 172.21.15.43:0/805329737 reap_dead start 2024-09-17T05:28:42.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- 172.21.15.43:0/805329737 shutdown_connections 2024-09-17T05:28:42.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- 172.21.15.43:0/805329737 >> 172.21.15.43:0/805329737 conn(0x7fc5ec0b24c0 msgr2=0x7fc5ec0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:42.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- 172.21.15.43:0/805329737 shutdown_connections 2024-09-17T05:28:42.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- 172.21.15.43:0/805329737 wait complete. 2024-09-17T05:28:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 Processor -- start 2024-09-17T05:28:42.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- start start 2024-09-17T05:28:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5ec152cc0 0x7fc5ec1e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5ec1460b0 0x7fc5ec1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:42.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec070360 con 0x7fc5ec145760 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec0704e0 con 0x7fc5ec1460b0 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.750+0000 7fc5f3000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ec15d760 con 0x7fc5ec152cc0 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47794/0 (socket says 172.21.15.43:47794) 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 -- 172.21.15.43:0/1906784514 learned_addr learned my addr 172.21.15.43:0/1906784514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5ebe00640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5ec152cc0 0x7fc5ec1e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5f1000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5ec1460b0 0x7fc5ec1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 -- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5ec152cc0 msgr2=0x7fc5ec1e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5ec152cc0 0x7fc5ec1e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 -- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5ec1460b0 msgr2=0x7fc5ec1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5ec1460b0 0x7fc5ec1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:42.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 -- 172.21.15.43:0/1906784514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ec1e6910 con 0x7fc5ec145760 2024-09-17T05:28:42.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5eb400640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001870 tx=0x7fc5d8003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:42.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8015070 con 0x7fc5ec145760 2024-09-17T05:28:42.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5d800f960 con 0x7fc5ec145760 2024-09-17T05:28:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800fd90 con 0x7fc5ec145760 2024-09-17T05:28:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ec1e6bf0 con 0x7fc5ec145760 2024-09-17T05:28:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ec1464c0 con 0x7fc5ec145760 2024-09-17T05:28:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ec06fe20 con 0x7fc5ec145760 2024-09-17T05:28:42.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.754+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5d8025440 con 0x7fc5ec145760 2024-09-17T05:28:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.758+0000 7fc5e3e00640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 0x7fc5cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:42.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.758+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5227+0+0 (secure 0 0 0) 0x7fc5d8095300 con 0x7fc5ec145760 2024-09-17T05:28:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.758+0000 7fc5ebe00640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 0x7fc5cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.758+0000 7fc5ebe00640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 0x7fc5cc075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec146740 tx=0x7fc5dc0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.762+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5d8060600 con 0x7fc5ec145760 2024-09-17T05:28:42.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:42.934+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc5ec073b60 con 0x7fc5ec145760 2024-09-17T05:28:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.546+0000 7fc5e3e00640 1 -- 172.21.15.43:0/1906784514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fc5d805ff00 con 0x7fc5ec145760 2024-09-17T05:28:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:28:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 msgr2=0x7fc5cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 0x7fc5cc075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec146740 tx=0x7fc5dc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 msgr2=0x7fc5ec145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001870 tx=0x7fc5d8003ab0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 shutdown_connections 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5cc073230 0x7fc5cc075740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5ec1460b0 0x7fc5ec1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ec145760 0x7fc5ec145b70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 --2- 172.21.15.43:0/1906784514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5ec152cc0 0x7fc5ec1e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 >> 172.21.15.43:0/1906784514 conn(0x7fc5ec0b24c0 msgr2=0x7fc5ec15bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 shutdown_connections 2024-09-17T05:28:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.550+0000 7fc5f3000640 1 -- 172.21.15.43:0/1906784514 wait complete. 2024-09-17T05:28:43.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 -- 172.21.15.43:0/4255755852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35940053e0 msgr2=0x7f359400ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 --2- 172.21.15.43:0/4255755852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35940053e0 0x7f359400ba90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c610 tx=0x7f35a801fa00 comp rx=0 tx=0).stop 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 -- 172.21.15.43:0/4255755852 shutdown_connections 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 --2- 172.21.15.43:0/4255755852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35940053e0 0x7f359400ba90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 -- 172.21.15.43:0/4255755852 >> 172.21.15.43:0/4255755852 conn(0x7f35b00b23d0 msgr2=0x7f35b00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 -- 172.21.15.43:0/4255755852 shutdown_connections 2024-09-17T05:28:43.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.762+0000 7f35b5400640 1 -- 172.21.15.43:0/4255755852 wait complete. 2024-09-17T05:28:43.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 Processor -- start 2024-09-17T05:28:43.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- start start 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35940053e0 0x7f35b006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35b009c180 0x7f35b01f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35b015cb70 con 0x7f35b01f88b0 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35b015c9f0 con 0x7f35940053e0 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f35b00b33e0 con 0x7f35b009c180 2024-09-17T05:28:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37114/0 (socket says 172.21.15.43:37114) 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 -- 172.21.15.43:0/1002965436 learned_addr learned my addr 172.21.15.43:0/1002965436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35ae800640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35940053e0 0x7f35b006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35af200640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35b009c180 0x7f35b01f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 -- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35b009c180 msgr2=0x7f35b01f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35b009c180 0x7f35b01f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 -- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35940053e0 msgr2=0x7f35b006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35940053e0 0x7f35b006d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 -- 172.21.15.43:0/1002965436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01f9090 con 0x7f35b01f88b0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a5e00640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3598003ea0 tx=0x7f3598001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598015070 con 0x7f35b01f88b0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3598011630 con 0x7f35b01f88b0 2024-09-17T05:28:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598011a80 con 0x7f35b01f88b0 2024-09-17T05:28:43.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01f9370 con 0x7f35b01f88b0 2024-09-17T05:28:43.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.766+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01f55f0 con 0x7f35b01f88b0 2024-09-17T05:28:43.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.770+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b006f800 con 0x7f35b01f88b0 2024-09-17T05:28:43.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.770+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3598018070 con 0x7f35b01f88b0 2024-09-17T05:28:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.770+0000 7f35a7400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 0x7f358c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.770+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5609+0+0 (secure 0 0 0) 0x7f3598094e20 con 0x7f35b01f88b0 2024-09-17T05:28:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.770+0000 7f35ae800640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 0x7f358c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.774+0000 7f35ae800640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 0x7f358c075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35b00715c0 tx=0x7f35a00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.774+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f359805f8c0 con 0x7f35b01f88b0 2024-09-17T05:28:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:43.986+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f35b015b700 con 0x7f35b01f88b0 2024-09-17T05:28:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.562+0000 7f35a7400640 1 -- 172.21.15.43:0/1002965436 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v66) ==== 174+0+0 (secure 0 0 0) 0x7f35980642f0 con 0x7f35b01f88b0 2024-09-17T05:28:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 10 and data pool 11 2024-09-17T05:28:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 msgr2=0x7f358c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 0x7f358c075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35b00715c0 tx=0x7f35a00091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 msgr2=0x7f35b009d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3598003ea0 tx=0x7f3598001640 comp rx=0 tx=0).stop 2024-09-17T05:28:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 shutdown_connections 2024-09-17T05:28:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f358c073200 0x7f358c075710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35b009c180 0x7f35b01f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35b01f88b0 0x7f35b009d220 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 --2- 172.21.15.43:0/1002965436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35940053e0 0x7f35b006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 >> 172.21.15.43:0/1002965436 conn(0x7f35b00b23d0 msgr2=0x7f35b00736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:44.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.566+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 shutdown_connections 2024-09-17T05:28:44.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.570+0000 7f35b5400640 1 -- 172.21.15.43:0/1002965436 wait complete. 2024-09-17T05:28:44.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:44.745 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T05:28:44.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 -- 172.21.15.43:0/634222636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa80737c0 msgr2=0x7effa80700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 --2- 172.21.15.43:0/634222636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa80737c0 0x7effa80700a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7effa0009bc0 tx=0x7effa001ca80 comp rx=0 tx=0).stop 2024-09-17T05:28:44.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 -- 172.21.15.43:0/634222636 shutdown_connections 2024-09-17T05:28:44.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 --2- 172.21.15.43:0/634222636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa80737c0 0x7effa80700a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 --2- 172.21.15.43:0/634222636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa815fe10 0x7effa80b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 -- 172.21.15.43:0/634222636 >> 172.21.15.43:0/634222636 conn(0x7effa80b23d0 msgr2=0x7effa80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 -- 172.21.15.43:0/634222636 shutdown_connections 2024-09-17T05:28:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.782+0000 7effaf800640 1 -- 172.21.15.43:0/634222636 wait complete. 2024-09-17T05:28:44.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 Processor -- start 2024-09-17T05:28:44.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- start start 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa806fc90 0x7effa8149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 0x7effa8146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effa8169f30 con 0x7effa814a4b0 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7effa8169db0 con 0x7effa815fe10 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7effa816a0b0 con 0x7effa806fc90 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37126/0 (socket says 172.21.15.43:37126) 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 -- 172.21.15.43:0/3092669948 learned_addr learned my addr 172.21.15.43:0/3092669948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7efface00640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa806fc90 0x7effa8149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7eff9fe00640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 0x7effa8146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 -- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa806fc90 msgr2=0x7effa8149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa806fc90 0x7effa8149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 -- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 msgr2=0x7effa8146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 0x7effa8146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 -- 172.21.15.43:0/3092669948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa815c2b0 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7eff9fe00640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 0x7effa8146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effad800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7effa00b0040 tx=0x7effa0003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00b2040 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effa0004040 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa815c590 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004430 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.786+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa815cb50 con 0x7effa814a4b0 2024-09-17T05:28:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa815c020 con 0x7effa814a4b0 2024-09-17T05:28:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7effa00bd030 con 0x7effa814a4b0 2024-09-17T05:28:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7effa5600640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 0x7eff84075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7effa012e0d0 con 0x7effa814a4b0 2024-09-17T05:28:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7efface00640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 0x7eff84075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.790+0000 7efface00640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 0x7eff84075740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7eff98009b40 tx=0x7eff980091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.794+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effa00c4060 con 0x7effa814a4b0 2024-09-17T05:28:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.974+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effa80700a0 con 0x7effa814a4b0 2024-09-17T05:28:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.974+0000 7effa5600640 1 -- 172.21.15.43:0/3092669948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16621 (secure 0 0 0) 0x7effa0016790 con 0x7effa814a4b0 2024-09-17T05:28:44.978 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:44.978 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":74,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:28:43.992558+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:41.731155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:42.948479+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 msgr2=0x7eff84075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 0x7eff84075740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7eff98009b40 tx=0x7eff980091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 msgr2=0x7effa8146dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7effa00b0040 tx=0x7effa0003f40 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 shutdown_connections 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff84073230 0x7eff84075740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effa815fe10 0x7effa8146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effa814a4b0 0x7effa8146dc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 --2- 172.21.15.43:0/3092669948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effa806fc90 0x7effa8149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 >> 172.21.15.43:0/3092669948 conn(0x7effa80b23d0 msgr2=0x7effa80ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 shutdown_connections 2024-09-17T05:28:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:44.978+0000 7effaf800640 1 -- 172.21.15.43:0/3092669948 wait complete. 2024-09-17T05:28:45.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 -- 172.21.15.43:0/3608211991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 msgr2=0x7fd1fc004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 --2- 172.21.15.43:0/3608211991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd1fc004ce0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd200001860 tx=0x7fd20001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd217e00640 1 -- 172.21.15.43:0/3608211991 reap_dead start 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 -- 172.21.15.43:0/3608211991 shutdown_connections 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 -- 172.21.15.43:0/3608211991 >> 172.21.15.43:0/3608211991 conn(0x7fd2180b23d0 msgr2=0x7fd2180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 -- 172.21.15.43:0/3608211991 shutdown_connections 2024-09-17T05:28:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.158+0000 7fd21e600640 1 -- 172.21.15.43:0/3608211991 wait complete. 2024-09-17T05:28:45.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 Processor -- start 2024-09-17T05:28:45.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- start start 2024-09-17T05:28:45.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 0x7fd2180ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 0x7fd2180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd218167ad0 con 0x7fd1fc004810 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd218167950 con 0x7fd2180c72b0 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd218167c50 con 0x7fd21815fb60 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37154/0 (socket says 172.21.15.43:37154) 2024-09-17T05:28:45.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 -- 172.21.15.43:0/3960456094 learned_addr learned my addr 172.21.15.43:0/3960456094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:45.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217e00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 0x7fd2180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd20fe00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 0x7fd2180ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 -- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 msgr2=0x7fd2180ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 0x7fd2180ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 -- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 msgr2=0x7fd2180ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 0x7fd2180ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 -- 172.21.15.43:0/3960456094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2181fc1b0 con 0x7fd1fc004810 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd20fe00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 0x7fd2180ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217e00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 0x7fd2180ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd217400640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd2080119b0 tx=0x7fd208011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20800fcf0 con 0x7fd1fc004810 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd208005020 con 0x7fd1fc004810 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208005470 con 0x7fd1fc004810 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2181fc490 con 0x7fd1fc004810 2024-09-17T05:28:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.162+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2180cda60 con 0x7fd1fc004810 2024-09-17T05:28:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2180ad0b0 con 0x7fd1fc004810 2024-09-17T05:28:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd2080047b0 con 0x7fd1fc004810 2024-09-17T05:28:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd20f400640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 0x7fd1f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fd208094dd0 con 0x7fd1fc004810 2024-09-17T05:28:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd20fe00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 0x7fd1f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.166+0000 7fd20fe00640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 0x7fd1f4075740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd200001830 tx=0x7fd2000a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:45.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.170+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd20805fe60 con 0x7fd1fc004810 2024-09-17T05:28:45.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.346+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2180af6a0 con 0x7fd1fc004810 2024-09-17T05:28:45.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.346+0000 7fd20f400640 1 -- 172.21.15.43:0/3960456094 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16621 (secure 0 0 0) 0x7fd208096020 con 0x7fd1fc004810 2024-09-17T05:28:45.349 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:45.350 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":74,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:28:43.992558+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:41.731155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:42.948479+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 msgr2=0x7fd1f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 0x7fd1f4075740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd200001830 tx=0x7fd2000a7040 comp rx=0 tx=0).stop 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 msgr2=0x7fd21809ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd2080119b0 tx=0x7fd208011e70 comp rx=0 tx=0).stop 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 shutdown_connections 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1f4073230 0x7fd1f4075740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2180c72b0 0x7fd2180ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd21815fb60 0x7fd2180ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 --2- 172.21.15.43:0/3960456094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1fc004810 0x7fd21809ca50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 >> 172.21.15.43:0/3960456094 conn(0x7fd2180b23d0 msgr2=0x7fd2180d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:45.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 shutdown_connections 2024-09-17T05:28:45.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.350+0000 7fd21e600640 1 -- 172.21.15.43:0/3960456094 wait complete. 2024-09-17T05:28:45.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:28:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.530+0000 7fbe2ca00640 1 -- 172.21.15.43:0/2838710190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe28155d70 msgr2=0x7fbe2815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.530+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/2838710190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe28155d70 0x7fbe2815c7e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009900 tx=0x7fbe1801c850 comp rx=0 tx=0).stop 2024-09-17T05:28:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.530+0000 7fbe2ca00640 1 -- 172.21.15.43:0/2838710190 shutdown_connections 2024-09-17T05:28:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.530+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/2838710190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe28155d70 0x7fbe2815c7e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.530+0000 7fbe2ca00640 1 -- 172.21.15.43:0/2838710190 >> 172.21.15.43:0/2838710190 conn(0x7fbe280b23d0 msgr2=0x7fbe280d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- 172.21.15.43:0/2838710190 shutdown_connections 2024-09-17T05:28:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- 172.21.15.43:0/2838710190 wait complete. 2024-09-17T05:28:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 Processor -- start 2024-09-17T05:28:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- start start 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2806f110 0x7fbe28152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe28155d70 0x7fbe28152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28169d70 con 0x7fbe2809c0e0 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28169bf0 con 0x7fbe28155d70 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe2ca00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbe28169ef0 con 0x7fbe2806f110 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.534+0000 7fbe26800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37180/0 (socket says 172.21.15.43:37180) 2024-09-17T05:28:45.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 -- 172.21.15.43:0/1430245850 learned_addr learned my addr 172.21.15.43:0/1430245850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe1d400640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe28155d70 0x7fbe28152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe25e00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2806f110 0x7fbe28152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 -- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2806f110 msgr2=0x7fbe28152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2806f110 0x7fbe28152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 -- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe28155d70 msgr2=0x7fbe28152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe28155d70 0x7fbe28152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 -- 172.21.15.43:0/1430245850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2809a8a0 con 0x7fbe2809c0e0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe26800640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c3c0 tx=0x7fbe2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20017070 con 0x7fbe2809c0e0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe200053c0 con 0x7fbe2809c0e0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2809ab80 con 0x7fbe2809c0e0 2024-09-17T05:28:45.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20003860 con 0x7fbe2809c0e0 2024-09-17T05:28:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2809b140 con 0x7fbe2809c0e0 2024-09-17T05:28:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2815c7e0 con 0x7fbe2809c0e0 2024-09-17T05:28:45.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.538+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe2001a070 con 0x7fbe2809c0e0 2024-09-17T05:28:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.542+0000 7fbe1ea00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 0x7fbe04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.542+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fbe200934c0 con 0x7fbe2809c0e0 2024-09-17T05:28:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.542+0000 7fbe25e00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 0x7fbe04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:45.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.542+0000 7fbe25e00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 0x7fbe04075710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe180af000 tx=0x7fbe1801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.546+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe2005df40 con 0x7fbe2809c0e0 2024-09-17T05:28:45.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:45.762+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe2809a5c0 con 0x7fbe2809c0e0 2024-09-17T05:28:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.590+0000 7fbe1ea00640 1 -- 172.21.15.43:0/1430245850 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) ==== 118+0+0 (secure 0 0 0) 0x7fbe20062970 con 0x7fbe2809c0e0 2024-09-17T05:28:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 msgr2=0x7fbe04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 0x7fbe04075710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe180af000 tx=0x7fbe1801b090 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 msgr2=0x7fbe2809c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000c3c0 tx=0x7fbe2000c880 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 shutdown_connections 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe04073200 0x7fbe04075710 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe28155d70 0x7fbe28152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2809c0e0 0x7fbe2809c4f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 --2- 172.21.15.43:0/1430245850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2806f110 0x7fbe28152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 >> 172.21.15.43:0/1430245850 conn(0x7fbe280b23d0 msgr2=0x7fbe280b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 shutdown_connections 2024-09-17T05:28:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.594+0000 7fbe2ca00640 1 -- 172.21.15.43:0/1430245850 wait complete. 2024-09-17T05:28:46.622 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:28:46.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:28:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- 172.21.15.43:0/2873659669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f260006ce60 msgr2=0x7f260006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 --2- 172.21.15.43:0/2873659669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f260006ce60 0x7f260006d270 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009900 tx=0x7f25e801c670 comp rx=0 tx=0).stop 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- 172.21.15.43:0/2873659669 shutdown_connections 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 --2- 172.21.15.43:0/2873659669 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26000cb860 0x7f26000cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 --2- 172.21.15.43:0/2873659669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f260006ce60 0x7f260006d270 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- 172.21.15.43:0/2873659669 >> 172.21.15.43:0/2873659669 conn(0x7f26000b23d0 msgr2=0x7f26000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- 172.21.15.43:0/2873659669 shutdown_connections 2024-09-17T05:28:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- 172.21.15.43:0/2873659669 wait complete. 2024-09-17T05:28:46.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 Processor -- start 2024-09-17T05:28:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.786+0000 7f2607000640 1 -- start start 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f260006ce60 0x7f26000c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26000cb860 0x7f260014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26000cee40 con 0x7f2600070620 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26000cecc0 con 0x7f26000cb860 2024-09-17T05:28:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2600168b40 con 0x7f260006ce60 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37204/0 (socket says 172.21.15.43:37204) 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 -- 172.21.15.43:0/3355243255 learned_addr learned my addr 172.21.15.43:0/3355243255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25ffe00640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f260006ce60 0x7f26000c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2605000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26000cb860 0x7f260014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 -- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f260006ce60 msgr2=0x7f26000c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f260006ce60 0x7f26000c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 -- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26000cb860 msgr2=0x7f260014b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26000cb860 0x7f260014b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 -- 172.21.15.43:0/3355243255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26000c4270 con 0x7f2600070620 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25f7400640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009850 tx=0x7f25e80aa570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e80aad00 con 0x7f2600070620 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e8003600 con 0x7f2600070620 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8003a10 con 0x7f2600070620 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26000ae410 con 0x7f2600070620 2024-09-17T05:28:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.790+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260014ae30 con 0x7f2600070620 2024-09-17T05:28:46.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260006d270 con 0x7f2600070620 2024-09-17T05:28:46.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25e8003bb0 con 0x7f2600070620 2024-09-17T05:28:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f25fcc00640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 0x7f25dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f25e812f3a0 con 0x7f2600070620 2024-09-17T05:28:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f25ffe00640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 0x7f25dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:46.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.794+0000 7f25ffe00640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 0x7f25dc075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098e0 tx=0x7f25f0009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:46.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:46.798+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f260006d7b0 con 0x7f2600070620 2024-09-17T05:28:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.006+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f26000c6f70 con 0x7f2600070620 2024-09-17T05:28:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.594+0000 7f25fcc00640 1 -- 172.21.15.43:0/3355243255 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) ==== 122+0+0 (secure 0 0 0) 0x7f25e80f9d60 con 0x7f2600070620 2024-09-17T05:28:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 msgr2=0x7f25dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 0x7f25dc075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f25f00098e0 tx=0x7f25f0009060 comp rx=0 tx=0).stop 2024-09-17T05:28:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 msgr2=0x7f2600154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009850 tx=0x7f25e80aa570 comp rx=0 tx=0).stop 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 shutdown_connections 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25dc073200 0x7f25dc075710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26000cb860 0x7f260014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2600070620 0x7f2600154d50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 --2- 172.21.15.43:0/3355243255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f260006ce60 0x7f26000c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 >> 172.21.15.43:0/3355243255 conn(0x7f26000b23d0 msgr2=0x7f26001591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 shutdown_connections 2024-09-17T05:28:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.598+0000 7f2607000640 1 -- 172.21.15.43:0/3355243255 wait complete. 2024-09-17T05:28:47.625 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:28:47.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 -- 172.21.15.43:0/4112217308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f161406f110 msgr2=0x7f16140b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 --2- 172.21.15.43:0/4112217308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f161406f110 0x7f16140b33e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1600009840 tx=0x7f160001c620 comp rx=0 tx=0).stop 2024-09-17T05:28:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 -- 172.21.15.43:0/4112217308 shutdown_connections 2024-09-17T05:28:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 --2- 172.21.15.43:0/4112217308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f161406f110 0x7f16140b33e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 --2- 172.21.15.43:0/4112217308 >> v2:172.21.15.62:3300/0 conn(0x7f1614155d70 0x7f161415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 -- 172.21.15.43:0/4112217308 >> 172.21.15.43:0/4112217308 conn(0x7f16140b23d0 msgr2=0x7f16140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:47.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 -- 172.21.15.43:0/4112217308 shutdown_connections 2024-09-17T05:28:47.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 -- 172.21.15.43:0/4112217308 wait complete. 2024-09-17T05:28:47.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.786+0000 7f161be00640 1 Processor -- start 2024-09-17T05:28:47.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- start start 2024-09-17T05:28:47.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16140b3920 0x7f161409d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 0x7f161415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16140adb50 con 0x7f161406f110 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16140ad9d0 con 0x7f1614155d70 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f16140adcd0 con 0x7f16140b3920 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37206/0 (socket says 172.21.15.43:37206) 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 -- 172.21.15.43:0/1483048887 learned_addr learned my addr 172.21.15.43:0/1483048887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1618a00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16140b3920 0x7f161409d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619e00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 0x7f161415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 -- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16140b3920 msgr2=0x7f161409d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16140b3920 0x7f161409d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 -- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 msgr2=0x7f161415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 0x7f161415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 -- 172.21.15.43:0/1483048887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16140c72b0 con 0x7f161406f110 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619e00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 0x7f161415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1619400640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f160800ea20 tx=0x7f160800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160800cd80 con 0x7f161406f110 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1608010430 con 0x7f161406f110 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608010860 con 0x7f161406f110 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16140caa10 con 0x7f161406f110 2024-09-17T05:28:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.790+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16140caf90 con 0x7f161406f110 2024-09-17T05:28:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16140b33e0 con 0x7f161406f110 2024-09-17T05:28:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1608010a00 con 0x7f161406f110 2024-09-17T05:28:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f1611600640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 0x7f15f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f1608094dc0 con 0x7f161406f110 2024-09-17T05:28:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f1618a00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 0x7f15f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.794+0000 7f1618a00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 0x7f15f4075740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16000ae000 tx=0x7f160001ae60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:47.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:47.798+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f160805fd40 con 0x7f161406f110 2024-09-17T05:28:48.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.006+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f161415ee30 con 0x7f161406f110 2024-09-17T05:28:48.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f1611600640 1 -- 172.21.15.43:0/1483048887 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5225 (secure 0 0 0) 0x7f160805f640 con 0x7f161406f110 2024-09-17T05:28:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:48.012 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"btime":"2024-09-17T05:28:47:595804+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:47.012214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:28:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2024-09-17T05:28:48.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 msgr2=0x7f15f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 0x7f15f4075740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16000ae000 tx=0x7f160001ae60 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 msgr2=0x7f1614165ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f160800ea20 tx=0x7f160800eee0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 shutdown_connections 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15f4073230 0x7f15f4075740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1614155d70 0x7f161415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16140b3920 0x7f161409d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 --2- 172.21.15.43:0/1483048887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f161406f110 0x7f1614165ba0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 >> 172.21.15.43:0/1483048887 conn(0x7f16140b23d0 msgr2=0x7f161415e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 shutdown_connections 2024-09-17T05:28:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.010+0000 7f161be00640 1 -- 172.21.15.43:0/1483048887 wait complete. 2024-09-17T05:28:48.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:48.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2810137155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 msgr2=0x7fe588003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2810137155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe588003f60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe594009b60 tx=0x7fe59401d240 comp rx=0 tx=0).stop 2024-09-17T05:28:48.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a5c00640 1 -- 172.21.15.43:0/2810137155 reap_dead start 2024-09-17T05:28:48.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2810137155 shutdown_connections 2024-09-17T05:28:48.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2810137155 >> 172.21.15.43:0/2810137155 conn(0x7fe5a00b23d0 msgr2=0x7fe5a00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2810137155 shutdown_connections 2024-09-17T05:28:48.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.190+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2810137155 wait complete. 2024-09-17T05:28:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 Processor -- start 2024-09-17T05:28:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- start start 2024-09-17T05:28:48.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5a009d580 0x7fe5a0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a0157260 0x7fe5a01f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00add00 con 0x7fe588003af0 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a00adb80 con 0x7fe5a0157260 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a00ade80 con 0x7fe5a009d580 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37220/0 (socket says 172.21.15.43:37220) 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 -- 172.21.15.43:0/2896408189 learned_addr learned my addr 172.21.15.43:0/2896408189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe59fe00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5a009d580 0x7fe5a0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a0157260 0x7fe5a01f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 -- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5a009d580 msgr2=0x7fe5a0156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5a009d580 0x7fe5a0156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 -- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a0157260 msgr2=0x7fe5a01f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a0157260 0x7fe5a01f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 -- 172.21.15.43:0/2896408189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a0157670 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a5200640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5940af040 tx=0x7fe5940acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940b1040 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5940ac400 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a0162ff0 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594009730 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.194+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01637d0 con 0x7fe588003af0 2024-09-17T05:28:48.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.198+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a015c7c0 con 0x7fe588003af0 2024-09-17T05:28:48.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.198+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe5940c0020 con 0x7fe588003af0 2024-09-17T05:28:48.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.198+0000 7fe59cc00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 0x7fe580075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.198+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fe59412f2e0 con 0x7fe588003af0 2024-09-17T05:28:48.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.198+0000 7fe59fe00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 0x7fe580075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.202+0000 7fe59fe00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 0x7fe580075710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001870 tx=0x7fe58c006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.202+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe5a0157670 con 0x7fe588003af0 2024-09-17T05:28:48.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.378+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5a00c3ad0 con 0x7fe588003af0 2024-09-17T05:28:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe59cc00640 1 -- 172.21.15.43:0/2896408189 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16608 (secure 0 0 0) 0x7fe594016fc0 con 0x7fe588003af0 2024-09-17T05:28:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:48.384 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":75,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:28:45.582259+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:41.731155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:42.948479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 msgr2=0x7fe580075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 0x7fe580075710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001870 tx=0x7fe58c006040 comp rx=0 tx=0).stop 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 msgr2=0x7fe5a009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5940af040 tx=0x7fe5940acd20 comp rx=0 tx=0).stop 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 shutdown_connections 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe580073200 0x7fe580075710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a0157260 0x7fe5a01f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe5a009d580 0x7fe5a0156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 --2- 172.21.15.43:0/2896408189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe588003af0 0x7fe5a009cce0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 >> 172.21.15.43:0/2896408189 conn(0x7fe5a00b23d0 msgr2=0x7fe5a015e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 shutdown_connections 2024-09-17T05:28:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.382+0000 7fe5a7c00640 1 -- 172.21.15.43:0/2896408189 wait complete. 2024-09-17T05:28:48.407 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:28:48.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 -- 172.21.15.43:0/1290186232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b0950 msgr2=0x7f26fc0b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 --2- 172.21.15.43:0/1290186232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b0950 0x7f26fc0b0d60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009900 tx=0x7f26e801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 -- 172.21.15.43:0/1290186232 shutdown_connections 2024-09-17T05:28:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 --2- 172.21.15.43:0/1290186232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b5150 0x7f26fc0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 --2- 172.21.15.43:0/1290186232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b0950 0x7f26fc0b0d60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 -- 172.21.15.43:0/1290186232 >> 172.21.15.43:0/1290186232 conn(0x7f26fc0d12f0 msgr2=0x7f26fc0d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.562+0000 7f2704000640 1 -- 172.21.15.43:0/1290186232 shutdown_connections 2024-09-17T05:28:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 -- 172.21.15.43:0/1290186232 wait complete. 2024-09-17T05:28:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 Processor -- start 2024-09-17T05:28:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 -- start start 2024-09-17T05:28:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0b0950 0x7f26fc159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b64d0 0x7f26fc0af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc168d30 con 0x7f26fc0b5150 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc168bb0 con 0x7f26fc0b0950 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.566+0000 7f2704000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc168eb0 con 0x7f26fc0b64d0 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37240/0 (socket says 172.21.15.43:37240) 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 -- 172.21.15.43:0/2599093769 learned_addr learned my addr 172.21.15.43:0/2599093769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2701600640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0b0950 0x7f26fc159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2702000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b64d0 0x7f26fc0af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 -- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b64d0 msgr2=0x7f26fc0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b64d0 0x7f26fc0af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 -- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0b0950 msgr2=0x7f26fc159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0b0950 0x7f26fc159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 -- 172.21.15.43:0/2599093769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc1f6ef0 con 0x7f26fc0b5150 2024-09-17T05:28:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2700c00640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b0040 tx=0x7f26e80a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80b2040 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26e80043b0 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8005290 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc1f71d0 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1f7790 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc0b0d60 con 0x7f26fc0b5150 2024-09-17T05:28:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.570+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26e80be060 con 0x7f26fc0b5150 2024-09-17T05:28:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.574+0000 7f26f9600640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 0x7f26dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.574+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f26e812d5b0 con 0x7f26fc0b5150 2024-09-17T05:28:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.574+0000 7f2701600640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 0x7f26dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.574+0000 7f2701600640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 0x7f26dc075710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007fd0 tx=0x7f26f0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.578+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26e80164d0 con 0x7f26fc0b5150 2024-09-17T05:28:48.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.750+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26fc1659b0 con 0x7f26dc073200 2024-09-17T05:28:48.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.754+0000 7f26f9600640 1 -- 172.21.15.43:0/2599093769 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247127 (secure 0 0 0) 0x7f26fc1659b0 con 0x7f26dc073200 2024-09-17T05:28:48.761 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 msgr2=0x7f26dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 0x7f26dc075710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007fd0 tx=0x7f26f0009040 comp rx=0 tx=0).stop 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 msgr2=0x7f26fc1643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b0040 tx=0x7f26e80a5730 comp rx=0 tx=0).stop 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 shutdown_connections 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26dc073200 0x7f26dc075710 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26fc0b64d0 0x7f26fc0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26fc0b5150 0x7f26fc1643d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 --2- 172.21.15.43:0/2599093769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26fc0b0950 0x7f26fc159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 >> 172.21.15.43:0/2599093769 conn(0x7f26fc0d12f0 msgr2=0x7f26fc15da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 shutdown_connections 2024-09-17T05:28:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.758+0000 7f2704000640 1 -- 172.21.15.43:0/2599093769 wait complete. 2024-09-17T05:28:48.782 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-17T05:28:47.928140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257556,"kb_used_data":8032,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717164,"statfs":{"total":773094113280,"available":772830375936,"internally_reserved":0,"allocated":8224768,"data_stored":4149821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.376536"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564728+0000","last_change":"2024-09-17T05:28:43.564728+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564728+0000","last_undegraded":"2024-09-17T05:28:43.564728+0000","last_fullsized":"2024-09-17T05:28:43.564728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559745+0000","last_change":"2024-09-17T05:28:43.559745+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559745+0000","last_undegraded":"2024-09-17T05:28:43.559745+0000","last_fullsized":"2024-09-17T05:28:43.559745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571536+0000","last_change":"2024-09-17T05:28:43.571536+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571536+0000","last_undegraded":"2024-09-17T05:28:43.571536+0000","last_fullsized":"2024-09-17T05:28:43.571536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571856+0000","last_change":"2024-09-17T05:28:43.571856+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571856+0000","last_undegraded":"2024-09-17T05:28:43.571856+0000","last_fullsized":"2024-09-17T05:28:43.571856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564326+0000","last_change":"2024-09-17T05:28:43.564326+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564326+0000","last_undegraded":"2024-09-17T05:28:43.564326+0000","last_fullsized":"2024-09-17T05:28:43.564326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.576337+0000","last_change":"2024-09-17T05:28:43.576337+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.576337+0000","last_undegraded":"2024-09-17T05:28:43.576337+0000","last_fullsized":"2024-09-17T05:28:43.576337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558152+0000","last_change":"2024-09-17T05:28:43.558152+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558152+0000","last_undegraded":"2024-09-17T05:28:43.558152+0000","last_fullsized":"2024-09-17T05:28:43.558152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569809+0000","last_change":"2024-09-17T05:28:43.569809+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569809+0000","last_undegraded":"2024-09-17T05:28:43.569809+0000","last_fullsized":"2024-09-17T05:28:43.569809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573914+0000","last_change":"2024-09-17T05:28:43.573914+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573914+0000","last_undegraded":"2024-09-17T05:28:43.573914+0000","last_fullsized":"2024-09-17T05:28:43.573914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560035+0000","last_change":"2024-09-17T05:28:43.560035+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560035+0000","last_undegraded":"2024-09-17T05:28:43.560035+0000","last_fullsized":"2024-09-17T05:28:43.560035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566601+0000","last_change":"2024-09-17T05:28:43.566601+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566601+0000","last_undegraded":"2024-09-17T05:28:43.566601+0000","last_fullsized":"2024-09-17T05:28:43.566601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558924+0000","last_change":"2024-09-17T05:28:43.558924+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558924+0000","last_undegraded":"2024-09-17T05:28:43.558924+0000","last_fullsized":"2024-09-17T05:28:43.558924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569546+0000","last_change":"2024-09-17T05:28:43.569546+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569546+0000","last_undegraded":"2024-09-17T05:28:43.569546+0000","last_fullsized":"2024-09-17T05:28:43.569546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558400+0000","last_change":"2024-09-17T05:28:43.558400+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558400+0000","last_undegraded":"2024-09-17T05:28:43.558400+0000","last_fullsized":"2024-09-17T05:28:43.558400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560650+0000","last_change":"2024-09-17T05:28:43.560650+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560650+0000","last_undegraded":"2024-09-17T05:28:43.560650+0000","last_fullsized":"2024-09-17T05:28:43.560650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569216+0000","last_change":"2024-09-17T05:28:43.569216+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569216+0000","last_undegraded":"2024-09-17T05:28:43.569216+0000","last_fullsized":"2024-09-17T05:28:43.569216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564991+0000","last_change":"2024-09-17T05:28:43.564991+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564991+0000","last_undegraded":"2024-09-17T05:28:43.564991+0000","last_fullsized":"2024-09-17T05:28:43.564991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566690+0000","last_change":"2024-09-17T05:28:43.566690+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566690+0000","last_undegraded":"2024-09-17T05:28:43.566690+0000","last_fullsized":"2024-09-17T05:28:43.566690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.567714+0000","last_change":"2024-09-17T05:28:43.567714+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.567714+0000","last_undegraded":"2024-09-17T05:28:43.567714+0000","last_fullsized":"2024-09-17T05:28:43.567714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564898+0000","last_change":"2024-09-17T05:28:43.564898+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564898+0000","last_undegraded":"2024-09-17T05:28:43.564898+0000","last_fullsized":"2024-09-17T05:28:43.564898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559249+0000","last_change":"2024-09-17T05:28:43.559249+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559249+0000","last_undegraded":"2024-09-17T05:28:43.559249+0000","last_fullsized":"2024-09-17T05:28:43.559249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575894+0000","last_change":"2024-09-17T05:28:43.575894+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575894+0000","last_undegraded":"2024-09-17T05:28:43.575894+0000","last_fullsized":"2024-09-17T05:28:43.575894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557101+0000","last_change":"2024-09-17T05:28:43.557101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557101+0000","last_undegraded":"2024-09-17T05:28:43.557101+0000","last_fullsized":"2024-09-17T05:28:43.557101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562470+0000","last_change":"2024-09-17T05:28:43.562470+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562470+0000","last_undegraded":"2024-09-17T05:28:43.562470+0000","last_fullsized":"2024-09-17T05:28:43.562470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558548+0000","last_change":"2024-09-17T05:28:43.558548+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558548+0000","last_undegraded":"2024-09-17T05:28:43.558548+0000","last_fullsized":"2024-09-17T05:28:43.558548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563461+0000","last_change":"2024-09-17T05:28:43.563461+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563461+0000","last_undegraded":"2024-09-17T05:28:43.563461+0000","last_fullsized":"2024-09-17T05:28:43.563461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566497+0000","last_change":"2024-09-17T05:28:43.566497+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566497+0000","last_undegraded":"2024-09-17T05:28:43.566497+0000","last_fullsized":"2024-09-17T05:28:43.566497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561181+0000","last_change":"2024-09-17T05:28:43.561181+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561181+0000","last_undegraded":"2024-09-17T05:28:43.561181+0000","last_fullsized":"2024-09-17T05:28:43.561181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.568780+0000","last_change":"2024-09-17T05:28:43.568780+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.568780+0000","last_undegraded":"2024-09-17T05:28:43.568780+0000","last_fullsized":"2024-09-17T05:28:43.568780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565458+0000","last_change":"2024-09-17T05:28:43.565458+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565458+0000","last_undegraded":"2024-09-17T05:28:43.565458+0000","last_fullsized":"2024-09-17T05:28:43.565458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562373+0000","last_change":"2024-09-17T05:28:43.562373+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562373+0000","last_undegraded":"2024-09-17T05:28:43.562373+0000","last_fullsized":"2024-09-17T05:28:43.562373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560101+0000","last_change":"2024-09-17T05:28:43.560101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560101+0000","last_undegraded":"2024-09-17T05:28:43.560101+0000","last_fullsized":"2024-09-17T05:28:43.560101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562815+0000","last_change":"2024-09-17T05:28:43.562815+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562815+0000","last_undegraded":"2024-09-17T05:28:43.562815+0000","last_fullsized":"2024-09-17T05:28:43.562815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.590450+0000","last_change":"2024-09-17T05:28:43.588925+0000","last_active":"2024-09-17T05:28:43.590450+0000","last_peered":"2024-09-17T05:28:43.590450+0000","last_clean":"2024-09-17T05:28:43.590450+0000","last_became_active":"2024-09-17T05:28:43.588563+0000","last_became_peered":"2024-09-17T05:28:43.588563+0000","last_unstale":"2024-09-17T05:28:43.590450+0000","last_undegraded":"2024-09-17T05:28:43.590450+0000","last_fullsized":"2024-09-17T05:28:43.590450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:46.751852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573432+0000","last_change":"2024-09-17T05:28:43.573432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573432+0000","last_undegraded":"2024-09-17T05:28:43.573432+0000","last_fullsized":"2024-09-17T05:28:43.573432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.581450+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.581046+0000","last_became_peered":"2024-09-17T05:28:43.581046+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:15.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.579261+0000","last_change":"2024-09-17T05:28:43.579261+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.579261+0000","last_undegraded":"2024-09-17T05:28:43.579261+0000","last_fullsized":"2024-09-17T05:28:43.579261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587130+0000","last_change":"2024-09-17T05:28:43.584806+0000","last_active":"2024-09-17T05:28:43.587130+0000","last_peered":"2024-09-17T05:28:43.587130+0000","last_clean":"2024-09-17T05:28:43.587130+0000","last_became_active":"2024-09-17T05:28:43.584417+0000","last_became_peered":"2024-09-17T05:28:43.584417+0000","last_unstale":"2024-09-17T05:28:43.587130+0000","last_undegraded":"2024-09-17T05:28:43.587130+0000","last_fullsized":"2024-09-17T05:28:43.587130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:38.969560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559999+0000","last_change":"2024-09-17T05:28:43.559999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559999+0000","last_undegraded":"2024-09-17T05:28:43.559999+0000","last_fullsized":"2024-09-17T05:28:43.559999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588895+0000","last_change":"2024-09-17T05:28:43.585654+0000","last_active":"2024-09-17T05:28:43.588895+0000","last_peered":"2024-09-17T05:28:43.588895+0000","last_clean":"2024-09-17T05:28:43.588895+0000","last_became_active":"2024-09-17T05:28:43.585273+0000","last_became_peered":"2024-09-17T05:28:43.585273+0000","last_unstale":"2024-09-17T05:28:43.588895+0000","last_undegraded":"2024-09-17T05:28:43.588895+0000","last_fullsized":"2024-09-17T05:28:43.588895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:46.485943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563068+0000","last_change":"2024-09-17T05:28:43.563068+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563068+0000","last_undegraded":"2024-09-17T05:28:43.563068+0000","last_fullsized":"2024-09-17T05:28:43.563068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588565+0000","last_change":"2024-09-17T05:28:43.584092+0000","last_active":"2024-09-17T05:28:43.588565+0000","last_peered":"2024-09-17T05:28:43.588565+0000","last_clean":"2024-09-17T05:28:43.588565+0000","last_became_active":"2024-09-17T05:28:43.583206+0000","last_became_peered":"2024-09-17T05:28:43.583206+0000","last_unstale":"2024-09-17T05:28:43.588565+0000","last_undegraded":"2024-09-17T05:28:43.588565+0000","last_fullsized":"2024-09-17T05:28:43.588565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:05.142867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.578602+0000","last_change":"2024-09-17T05:28:43.578602+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.578602+0000","last_undegraded":"2024-09-17T05:28:43.578602+0000","last_fullsized":"2024-09-17T05:28:43.578602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587843+0000","last_change":"2024-09-17T05:28:43.585492+0000","last_active":"2024-09-17T05:28:43.587843+0000","last_peered":"2024-09-17T05:28:43.587843+0000","last_clean":"2024-09-17T05:28:43.587843+0000","last_became_active":"2024-09-17T05:28:43.585189+0000","last_became_peered":"2024-09-17T05:28:43.585189+0000","last_unstale":"2024-09-17T05:28:43.587843+0000","last_undegraded":"2024-09-17T05:28:43.587843+0000","last_fullsized":"2024-09-17T05:28:43.587843+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:48.503822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811028+0000","last_change":"2024-09-17T05:28:44.589403+0000","last_active":"2024-09-17T05:28:45.811028+0000","last_peered":"2024-09-17T05:28:45.811028+0000","last_clean":"2024-09-17T05:28:45.811028+0000","last_became_active":"2024-09-17T05:28:44.589080+0000","last_became_peered":"2024-09-17T05:28:44.589080+0000","last_unstale":"2024-09-17T05:28:45.811028+0000","last_undegraded":"2024-09-17T05:28:45.811028+0000","last_fullsized":"2024-09-17T05:28:45.811028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:41.124043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811067+0000","last_change":"2024-09-17T05:28:43.582311+0000","last_active":"2024-09-17T05:28:45.811067+0000","last_peered":"2024-09-17T05:28:45.811067+0000","last_clean":"2024-09-17T05:28:45.811067+0000","last_became_active":"2024-09-17T05:28:43.581966+0000","last_became_peered":"2024-09-17T05:28:43.581966+0000","last_unstale":"2024-09-17T05:28:45.811067+0000","last_undegraded":"2024-09-17T05:28:45.811067+0000","last_fullsized":"2024-09-17T05:28:45.811067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:28.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560003+0000","last_change":"2024-09-17T05:28:43.560003+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560003+0000","last_undegraded":"2024-09-17T05:28:43.560003+0000","last_fullsized":"2024-09-17T05:28:43.560003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581405+0000","last_change":"2024-09-17T05:28:43.574709+0000","last_active":"2024-09-17T05:28:43.581405+0000","last_peered":"2024-09-17T05:28:43.581405+0000","last_clean":"2024-09-17T05:28:43.581405+0000","last_became_active":"2024-09-17T05:28:43.574282+0000","last_became_peered":"2024-09-17T05:28:43.574282+0000","last_unstale":"2024-09-17T05:28:43.581405+0000","last_undegraded":"2024-09-17T05:28:43.581405+0000","last_fullsized":"2024-09-17T05:28:43.581405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559486+0000","last_change":"2024-09-17T05:28:43.559486+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559486+0000","last_undegraded":"2024-09-17T05:28:43.559486+0000","last_fullsized":"2024-09-17T05:28:43.559486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587144+0000","last_change":"2024-09-17T05:28:43.583590+0000","last_active":"2024-09-17T05:28:43.587144+0000","last_peered":"2024-09-17T05:28:43.587144+0000","last_clean":"2024-09-17T05:28:43.587144+0000","last_became_active":"2024-09-17T05:28:43.583012+0000","last_became_peered":"2024-09-17T05:28:43.583012+0000","last_unstale":"2024-09-17T05:28:43.587144+0000","last_undegraded":"2024-09-17T05:28:43.587144+0000","last_fullsized":"2024-09-17T05:28:43.587144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:01.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559432+0000","last_change":"2024-09-17T05:28:43.559432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559432+0000","last_undegraded":"2024-09-17T05:28:43.559432+0000","last_fullsized":"2024-09-17T05:28:43.559432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588331+0000","last_change":"2024-09-17T05:28:43.586381+0000","last_active":"2024-09-17T05:28:43.588331+0000","last_peered":"2024-09-17T05:28:43.588331+0000","last_clean":"2024-09-17T05:28:43.588331+0000","last_became_active":"2024-09-17T05:28:43.586051+0000","last_became_peered":"2024-09-17T05:28:43.586051+0000","last_unstale":"2024-09-17T05:28:43.588331+0000","last_undegraded":"2024-09-17T05:28:43.588331+0000","last_fullsized":"2024-09-17T05:28:43.588331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564654+0000","last_change":"2024-09-17T05:28:43.564654+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564654+0000","last_undegraded":"2024-09-17T05:28:43.564654+0000","last_fullsized":"2024-09-17T05:28:43.564654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581371+0000","last_change":"2024-09-17T05:28:43.572513+0000","last_active":"2024-09-17T05:28:43.581371+0000","last_peered":"2024-09-17T05:28:43.581371+0000","last_clean":"2024-09-17T05:28:43.581371+0000","last_became_active":"2024-09-17T05:28:43.571694+0000","last_became_peered":"2024-09-17T05:28:43.571694+0000","last_unstale":"2024-09-17T05:28:43.581371+0000","last_undegraded":"2024-09-17T05:28:43.581371+0000","last_fullsized":"2024-09-17T05:28:43.581371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565864+0000","last_change":"2024-09-17T05:28:43.565864+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565864+0000","last_undegraded":"2024-09-17T05:28:43.565864+0000","last_fullsized":"2024-09-17T05:28:43.565864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811729+0000","last_change":"2024-09-17T05:28:43.581526+0000","last_active":"2024-09-17T05:28:45.811729+0000","last_peered":"2024-09-17T05:28:45.811729+0000","last_clean":"2024-09-17T05:28:45.811729+0000","last_became_active":"2024-09-17T05:28:43.581163+0000","last_became_peered":"2024-09-17T05:28:43.581163+0000","last_unstale":"2024-09-17T05:28:45.811729+0000","last_undegraded":"2024-09-17T05:28:45.811729+0000","last_fullsized":"2024-09-17T05:28:45.811729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:27.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575375+0000","last_change":"2024-09-17T05:28:43.575375+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575375+0000","last_undegraded":"2024-09-17T05:28:43.575375+0000","last_fullsized":"2024-09-17T05:28:43.575375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812166+0000","last_change":"2024-09-17T05:28:43.577962+0000","last_active":"2024-09-17T05:28:45.812166+0000","last_peered":"2024-09-17T05:28:45.812166+0000","last_clean":"2024-09-17T05:28:45.812166+0000","last_became_active":"2024-09-17T05:28:43.577642+0000","last_became_peered":"2024-09-17T05:28:43.577642+0000","last_unstale":"2024-09-17T05:28:45.812166+0000","last_undegraded":"2024-09-17T05:28:45.812166+0000","last_fullsized":"2024-09-17T05:28:45.812166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:55.229169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558340+0000","last_change":"2024-09-17T05:28:43.558340+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558340+0000","last_undegraded":"2024-09-17T05:28:43.558340+0000","last_fullsized":"2024-09-17T05:28:43.558340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.572889+0000","last_change":"2024-09-17T05:28:43.570566+0000","last_active":"2024-09-17T05:28:43.572889+0000","last_peered":"2024-09-17T05:28:43.572889+0000","last_clean":"2024-09-17T05:28:43.572889+0000","last_became_active":"2024-09-17T05:28:43.570239+0000","last_became_peered":"2024-09-17T05:28:43.570239+0000","last_unstale":"2024-09-17T05:28:43.572889+0000","last_undegraded":"2024-09-17T05:28:43.572889+0000","last_fullsized":"2024-09-17T05:28:43.572889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:30.323131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562523+0000","last_change":"2024-09-17T05:28:43.562523+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562523+0000","last_undegraded":"2024-09-17T05:28:43.562523+0000","last_fullsized":"2024-09-17T05:28:43.562523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571701+0000","last_change":"2024-09-17T05:28:43.569060+0000","last_active":"2024-09-17T05:28:43.571701+0000","last_peered":"2024-09-17T05:28:43.571701+0000","last_clean":"2024-09-17T05:28:43.571701+0000","last_became_active":"2024-09-17T05:28:43.568732+0000","last_became_peered":"2024-09-17T05:28:43.568732+0000","last_unstale":"2024-09-17T05:28:43.571701+0000","last_undegraded":"2024-09-17T05:28:43.571701+0000","last_fullsized":"2024-09-17T05:28:43.571701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:41.209993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569463+0000","last_change":"2024-09-17T05:28:43.569463+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569463+0000","last_undegraded":"2024-09-17T05:28:43.569463+0000","last_fullsized":"2024-09-17T05:28:43.569463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586671+0000","last_change":"2024-09-17T05:28:43.581871+0000","last_active":"2024-09-17T05:28:43.586671+0000","last_peered":"2024-09-17T05:28:43.586671+0000","last_clean":"2024-09-17T05:28:43.586671+0000","last_became_active":"2024-09-17T05:28:43.581398+0000","last_became_peered":"2024-09-17T05:28:43.581398+0000","last_unstale":"2024-09-17T05:28:43.586671+0000","last_undegraded":"2024-09-17T05:28:43.586671+0000","last_fullsized":"2024-09-17T05:28:43.586671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:11.127069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558271+0000","last_change":"2024-09-17T05:28:43.558271+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558271+0000","last_undegraded":"2024-09-17T05:28:43.558271+0000","last_fullsized":"2024-09-17T05:28:43.558271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586475+0000","last_change":"2024-09-17T05:28:43.578038+0000","last_active":"2024-09-17T05:28:43.586475+0000","last_peered":"2024-09-17T05:28:43.586475+0000","last_clean":"2024-09-17T05:28:43.586475+0000","last_became_active":"2024-09-17T05:28:43.577245+0000","last_became_peered":"2024-09-17T05:28:43.577245+0000","last_unstale":"2024-09-17T05:28:43.586475+0000","last_undegraded":"2024-09-17T05:28:43.586475+0000","last_fullsized":"2024-09-17T05:28:43.586475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:46.553001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558656+0000","last_change":"2024-09-17T05:28:43.558656+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558656+0000","last_undegraded":"2024-09-17T05:28:43.558656+0000","last_fullsized":"2024-09-17T05:28:43.558656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.576474+0000","last_change":"2024-09-17T05:28:43.566233+0000","last_active":"2024-09-17T05:28:43.576474+0000","last_peered":"2024-09-17T05:28:43.576474+0000","last_clean":"2024-09-17T05:28:43.576474+0000","last_became_active":"2024-09-17T05:28:43.565869+0000","last_became_peered":"2024-09-17T05:28:43.565869+0000","last_unstale":"2024-09-17T05:28:43.576474+0000","last_undegraded":"2024-09-17T05:28:43.576474+0000","last_fullsized":"2024-09-17T05:28:43.576474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571309+0000","last_change":"2024-09-17T05:28:43.571309+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571309+0000","last_undegraded":"2024-09-17T05:28:43.571309+0000","last_fullsized":"2024-09-17T05:28:43.571309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578735+0000","last_change":"2024-09-17T05:28:43.575603+0000","last_active":"2024-09-17T05:28:43.578735+0000","last_peered":"2024-09-17T05:28:43.578735+0000","last_clean":"2024-09-17T05:28:43.578735+0000","last_became_active":"2024-09-17T05:28:43.575283+0000","last_became_peered":"2024-09-17T05:28:43.575283+0000","last_unstale":"2024-09-17T05:28:43.578735+0000","last_undegraded":"2024-09-17T05:28:43.578735+0000","last_fullsized":"2024-09-17T05:28:43.578735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.570892+0000","last_change":"2024-09-17T05:28:43.570892+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.570892+0000","last_undegraded":"2024-09-17T05:28:43.570892+0000","last_fullsized":"2024-09-17T05:28:43.570892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588919+0000","last_change":"2024-09-17T05:28:43.587791+0000","last_active":"2024-09-17T05:28:43.588919+0000","last_peered":"2024-09-17T05:28:43.588919+0000","last_clean":"2024-09-17T05:28:43.588919+0000","last_became_active":"2024-09-17T05:28:43.587450+0000","last_became_peered":"2024-09-17T05:28:43.587450+0000","last_unstale":"2024-09-17T05:28:43.588919+0000","last_undegraded":"2024-09-17T05:28:43.588919+0000","last_fullsized":"2024-09-17T05:28:43.588919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:37.493312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812092+0000","last_change":"2024-09-17T05:28:44.590941+0000","last_active":"2024-09-17T05:28:45.812092+0000","last_peered":"2024-09-17T05:28:45.812092+0000","last_clean":"2024-09-17T05:28:45.812092+0000","last_became_active":"2024-09-17T05:28:44.590589+0000","last_became_peered":"2024-09-17T05:28:44.590589+0000","last_unstale":"2024-09-17T05:28:45.812092+0000","last_undegraded":"2024-09-17T05:28:45.812092+0000","last_fullsized":"2024-09-17T05:28:45.812092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:16.499238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588698+0000","last_change":"2024-09-17T05:28:43.583447+0000","last_active":"2024-09-17T05:28:43.588698+0000","last_peered":"2024-09-17T05:28:43.588698+0000","last_clean":"2024-09-17T05:28:43.588698+0000","last_became_active":"2024-09-17T05:28:43.583145+0000","last_became_peered":"2024-09-17T05:28:43.583145+0000","last_unstale":"2024-09-17T05:28:43.588698+0000","last_undegraded":"2024-09-17T05:28:43.588698+0000","last_fullsized":"2024-09-17T05:28:43.588698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.203097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811078+0000","last_change":"2024-09-17T05:28:44.589236+0000","last_active":"2024-09-17T05:28:45.811078+0000","last_peered":"2024-09-17T05:28:45.811078+0000","last_clean":"2024-09-17T05:28:45.811078+0000","last_became_active":"2024-09-17T05:28:44.588658+0000","last_became_peered":"2024-09-17T05:28:44.588658+0000","last_unstale":"2024-09-17T05:28:45.811078+0000","last_undegraded":"2024-09-17T05:28:45.811078+0000","last_fullsized":"2024-09-17T05:28:45.811078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571752+0000","last_change":"2024-09-17T05:28:43.564033+0000","last_active":"2024-09-17T05:28:43.571752+0000","last_peered":"2024-09-17T05:28:43.571752+0000","last_clean":"2024-09-17T05:28:43.571752+0000","last_became_active":"2024-09-17T05:28:43.563635+0000","last_became_peered":"2024-09-17T05:28:43.563635+0000","last_unstale":"2024-09-17T05:28:43.571752+0000","last_undegraded":"2024-09-17T05:28:43.571752+0000","last_fullsized":"2024-09-17T05:28:43.571752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559428+0000","last_change":"2024-09-17T05:28:43.559428+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559428+0000","last_undegraded":"2024-09-17T05:28:43.559428+0000","last_fullsized":"2024-09-17T05:28:43.559428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.582716+0000","last_change":"2024-09-17T05:28:43.578062+0000","last_active":"2024-09-17T05:28:43.582716+0000","last_peered":"2024-09-17T05:28:43.582716+0000","last_clean":"2024-09-17T05:28:43.582716+0000","last_became_active":"2024-09-17T05:28:43.577664+0000","last_became_peered":"2024-09-17T05:28:43.577664+0000","last_unstale":"2024-09-17T05:28:43.582716+0000","last_undegraded":"2024-09-17T05:28:43.582716+0000","last_fullsized":"2024-09-17T05:28:43.582716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:31.835733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566156+0000","last_change":"2024-09-17T05:28:43.566156+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566156+0000","last_undegraded":"2024-09-17T05:28:43.566156+0000","last_fullsized":"2024-09-17T05:28:43.566156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.577691+0000","last_change":"2024-09-17T05:28:43.571945+0000","last_active":"2024-09-17T05:28:43.577691+0000","last_peered":"2024-09-17T05:28:43.577691+0000","last_clean":"2024-09-17T05:28:43.577691+0000","last_became_active":"2024-09-17T05:28:43.571593+0000","last_became_peered":"2024-09-17T05:28:43.571593+0000","last_unstale":"2024-09-17T05:28:43.577691+0000","last_undegraded":"2024-09-17T05:28:43.577691+0000","last_fullsized":"2024-09-17T05:28:43.577691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:32.061486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557528+0000","last_change":"2024-09-17T05:28:43.557528+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557528+0000","last_undegraded":"2024-09-17T05:28:43.557528+0000","last_fullsized":"2024-09-17T05:28:43.557528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.575109+0000","last_change":"2024-09-17T05:28:43.570857+0000","last_active":"2024-09-17T05:28:43.575109+0000","last_peered":"2024-09-17T05:28:43.575109+0000","last_clean":"2024-09-17T05:28:43.575109+0000","last_became_active":"2024-09-17T05:28:43.570448+0000","last_became_peered":"2024-09-17T05:28:43.570448+0000","last_unstale":"2024-09-17T05:28:43.575109+0000","last_undegraded":"2024-09-17T05:28:43.575109+0000","last_fullsized":"2024-09-17T05:28:43.575109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:21.157484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560566+0000","last_change":"2024-09-17T05:28:43.560566+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560566+0000","last_undegraded":"2024-09-17T05:28:43.560566+0000","last_fullsized":"2024-09-17T05:28:43.560566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578687+0000","last_change":"2024-09-17T05:28:43.574524+0000","last_active":"2024-09-17T05:28:43.578687+0000","last_peered":"2024-09-17T05:28:43.578687+0000","last_clean":"2024-09-17T05:28:43.578687+0000","last_became_active":"2024-09-17T05:28:43.572192+0000","last_became_peered":"2024-09-17T05:28:43.572192+0000","last_unstale":"2024-09-17T05:28:43.578687+0000","last_undegraded":"2024-09-17T05:28:43.578687+0000","last_fullsized":"2024-09-17T05:28:43.578687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557647+0000","last_change":"2024-09-17T05:28:43.557647+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557647+0000","last_undegraded":"2024-09-17T05:28:43.557647+0000","last_fullsized":"2024-09-17T05:28:43.557647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578635+0000","last_change":"2024-09-17T05:28:43.574129+0000","last_active":"2024-09-17T05:28:43.578635+0000","last_peered":"2024-09-17T05:28:43.578635+0000","last_clean":"2024-09-17T05:28:43.578635+0000","last_became_active":"2024-09-17T05:28:43.573723+0000","last_became_peered":"2024-09-17T05:28:43.573723+0000","last_unstale":"2024-09-17T05:28:43.578635+0000","last_undegraded":"2024-09-17T05:28:43.578635+0000","last_fullsized":"2024-09-17T05:28:43.578635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563675+0000","last_change":"2024-09-17T05:28:43.563675+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563675+0000","last_undegraded":"2024-09-17T05:28:43.563675+0000","last_fullsized":"2024-09-17T05:28:43.563675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.582630+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.582246+0000","last_became_peered":"2024-09-17T05:28:43.582246+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562999+0000","last_change":"2024-09-17T05:28:43.562999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562999+0000","last_undegraded":"2024-09-17T05:28:43.562999+0000","last_fullsized":"2024-09-17T05:28:43.562999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588401+0000","last_change":"2024-09-17T05:28:43.583802+0000","last_active":"2024-09-17T05:28:43.588401+0000","last_peered":"2024-09-17T05:28:43.588401+0000","last_clean":"2024-09-17T05:28:43.588401+0000","last_became_active":"2024-09-17T05:28:43.583483+0000","last_became_peered":"2024-09-17T05:28:43.583483+0000","last_unstale":"2024-09-17T05:28:43.588401+0000","last_undegraded":"2024-09-17T05:28:43.588401+0000","last_fullsized":"2024-09-17T05:28:43.588401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:45.524073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811820+0000","last_change":"2024-09-17T05:28:44.596874+0000","last_active":"2024-09-17T05:28:45.811820+0000","last_peered":"2024-09-17T05:28:45.811820+0000","last_clean":"2024-09-17T05:28:45.811820+0000","last_became_active":"2024-09-17T05:28:44.596501+0000","last_became_peered":"2024-09-17T05:28:44.596501+0000","last_unstale":"2024-09-17T05:28:45.811820+0000","last_undegraded":"2024-09-17T05:28:45.811820+0000","last_fullsized":"2024-09-17T05:28:45.811820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:27.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588309+0000","last_change":"2024-09-17T05:28:43.584354+0000","last_active":"2024-09-17T05:28:43.588309+0000","last_peered":"2024-09-17T05:28:43.588309+0000","last_clean":"2024-09-17T05:28:43.588309+0000","last_became_active":"2024-09-17T05:28:43.583900+0000","last_became_peered":"2024-09-17T05:28:43.583900+0000","last_unstale":"2024-09-17T05:28:43.588309+0000","last_undegraded":"2024-09-17T05:28:43.588309+0000","last_fullsized":"2024-09-17T05:28:43.588309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561812+0000","last_change":"2024-09-17T05:28:43.561812+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561812+0000","last_undegraded":"2024-09-17T05:28:43.561812+0000","last_fullsized":"2024-09-17T05:28:43.561812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588633+0000","last_change":"2024-09-17T05:28:43.582802+0000","last_active":"2024-09-17T05:28:43.588633+0000","last_peered":"2024-09-17T05:28:43.588633+0000","last_clean":"2024-09-17T05:28:43.588633+0000","last_became_active":"2024-09-17T05:28:43.582317+0000","last_became_peered":"2024-09-17T05:28:43.582317+0000","last_unstale":"2024-09-17T05:28:43.588633+0000","last_undegraded":"2024-09-17T05:28:43.588633+0000","last_fullsized":"2024-09-17T05:28:43.588633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:55.092115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564277+0000","last_change":"2024-09-17T05:28:43.564277+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564277+0000","last_undegraded":"2024-09-17T05:28:43.564277+0000","last_fullsized":"2024-09-17T05:28:43.564277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.568256+0000","last_change":"2024-09-17T05:28:43.565393+0000","last_active":"2024-09-17T05:28:43.568256+0000","last_peered":"2024-09-17T05:28:43.568256+0000","last_clean":"2024-09-17T05:28:43.568256+0000","last_became_active":"2024-09-17T05:28:43.565048+0000","last_became_peered":"2024-09-17T05:28:43.565048+0000","last_unstale":"2024-09-17T05:28:43.568256+0000","last_undegraded":"2024-09-17T05:28:43.568256+0000","last_fullsized":"2024-09-17T05:28:43.568256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:48.311903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.556862+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:28:43.556862+0000","last_peered":"2024-09-17T05:28:43.556862+0000","last_clean":"2024-09-17T05:28:43.556862+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:28:43.556862+0000","last_undegraded":"2024-09-17T05:28:43.556862+0000","last_fullsized":"2024-09-17T05:28:43.556862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":372429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509509,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":387150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:48.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- 172.21.15.43:0/3245759428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 msgr2=0x7f48980ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 --2- 172.21.15.43:0/3245759428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980ad0b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4890009840 tx=0x7f489001c760 comp rx=0 tx=0).stop 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- 172.21.15.43:0/3245759428 shutdown_connections 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 --2- 172.21.15.43:0/3245759428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980ad0b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 --2- 172.21.15.43:0/3245759428 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48980b3af0 0x7f489815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- 172.21.15.43:0/3245759428 >> 172.21.15.43:0/3245759428 conn(0x7f48980b23d0 msgr2=0x7f48980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- 172.21.15.43:0/3245759428 shutdown_connections 2024-09-17T05:28:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- 172.21.15.43:0/3245759428 wait complete. 2024-09-17T05:28:48.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 Processor -- start 2024-09-17T05:28:48.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.982+0000 7f489ec00640 1 -- start start 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4898155d70 0x7f489809cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4898164770 0x7f4898164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4898168c80 con 0x7f489806f110 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4898168b00 con 0x7f4898155d70 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4898168e00 con 0x7f4898164770 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37244/0 (socket says 172.21.15.43:37244) 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 -- 172.21.15.43:0/2206382517 learned_addr learned my addr 172.21.15.43:0/2206382517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489cc00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4898164770 0x7f4898164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f488f400640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4898155d70 0x7f489809cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 -- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4898164770 msgr2=0x7f4898164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4898164770 0x7f4898164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 -- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4898155d70 msgr2=0x7f489809cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4898155d70 0x7f489809cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 -- 172.21.15.43:0/2206382517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48981650c0 con 0x7f489806f110 2024-09-17T05:28:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4897e00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f488800cb10 tx=0x7f488800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888016020 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488800ce50 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800ade0 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48981fba50 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981fc230 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.986+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48980ad0b0 con 0x7f489806f110 2024-09-17T05:28:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.990+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4888019020 con 0x7f489806f110 2024-09-17T05:28:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.990+0000 7f4894c00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 0x7f4874075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.990+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f48880943c0 con 0x7f489806f110 2024-09-17T05:28:48.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.990+0000 7f488f400640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 0x7f4874075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:48.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.990+0000 7f488f400640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 0x7f4874075740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f489815e730 tx=0x7f488000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:48.994+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4888020020 con 0x7f489806f110 2024-09-17T05:28:49.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.170+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48980ca420 con 0x7f4874073230 2024-09-17T05:28:49.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.174+0000 7f4894c00640 1 -- 172.21.15.43:0/2206382517 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247127 (secure 0 0 0) 0x7f48980ca420 con 0x7f4874073230 2024-09-17T05:28:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 msgr2=0x7f4874075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 0x7f4874075740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f489815e730 tx=0x7f488000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 msgr2=0x7f48980cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f488800cb10 tx=0x7f488800cc00 comp rx=0 tx=0).stop 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 shutdown_connections 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4874073230 0x7f4874075740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4898155d70 0x7f489809cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4898164770 0x7f4898164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 --2- 172.21.15.43:0/2206382517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f489806f110 0x7f48980cb730 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 >> 172.21.15.43:0/2206382517 conn(0x7f48980b23d0 msgr2=0x7f489815db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 shutdown_connections 2024-09-17T05:28:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.178+0000 7f489ec00640 1 -- 172.21.15.43:0/2206382517 wait complete. 2024-09-17T05:28:49.201 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-17T05:28:47.928140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257556,"kb_used_data":8032,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717164,"statfs":{"total":773094113280,"available":772830375936,"internally_reserved":0,"allocated":8224768,"data_stored":4149821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.376536"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564728+0000","last_change":"2024-09-17T05:28:43.564728+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564728+0000","last_undegraded":"2024-09-17T05:28:43.564728+0000","last_fullsized":"2024-09-17T05:28:43.564728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559745+0000","last_change":"2024-09-17T05:28:43.559745+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559745+0000","last_undegraded":"2024-09-17T05:28:43.559745+0000","last_fullsized":"2024-09-17T05:28:43.559745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571536+0000","last_change":"2024-09-17T05:28:43.571536+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571536+0000","last_undegraded":"2024-09-17T05:28:43.571536+0000","last_fullsized":"2024-09-17T05:28:43.571536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571856+0000","last_change":"2024-09-17T05:28:43.571856+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571856+0000","last_undegraded":"2024-09-17T05:28:43.571856+0000","last_fullsized":"2024-09-17T05:28:43.571856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564326+0000","last_change":"2024-09-17T05:28:43.564326+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564326+0000","last_undegraded":"2024-09-17T05:28:43.564326+0000","last_fullsized":"2024-09-17T05:28:43.564326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.576337+0000","last_change":"2024-09-17T05:28:43.576337+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.576337+0000","last_undegraded":"2024-09-17T05:28:43.576337+0000","last_fullsized":"2024-09-17T05:28:43.576337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558152+0000","last_change":"2024-09-17T05:28:43.558152+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558152+0000","last_undegraded":"2024-09-17T05:28:43.558152+0000","last_fullsized":"2024-09-17T05:28:43.558152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569809+0000","last_change":"2024-09-17T05:28:43.569809+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569809+0000","last_undegraded":"2024-09-17T05:28:43.569809+0000","last_fullsized":"2024-09-17T05:28:43.569809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573914+0000","last_change":"2024-09-17T05:28:43.573914+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573914+0000","last_undegraded":"2024-09-17T05:28:43.573914+0000","last_fullsized":"2024-09-17T05:28:43.573914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560035+0000","last_change":"2024-09-17T05:28:43.560035+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560035+0000","last_undegraded":"2024-09-17T05:28:43.560035+0000","last_fullsized":"2024-09-17T05:28:43.560035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566601+0000","last_change":"2024-09-17T05:28:43.566601+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566601+0000","last_undegraded":"2024-09-17T05:28:43.566601+0000","last_fullsized":"2024-09-17T05:28:43.566601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558924+0000","last_change":"2024-09-17T05:28:43.558924+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558924+0000","last_undegraded":"2024-09-17T05:28:43.558924+0000","last_fullsized":"2024-09-17T05:28:43.558924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569546+0000","last_change":"2024-09-17T05:28:43.569546+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569546+0000","last_undegraded":"2024-09-17T05:28:43.569546+0000","last_fullsized":"2024-09-17T05:28:43.569546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558400+0000","last_change":"2024-09-17T05:28:43.558400+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558400+0000","last_undegraded":"2024-09-17T05:28:43.558400+0000","last_fullsized":"2024-09-17T05:28:43.558400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560650+0000","last_change":"2024-09-17T05:28:43.560650+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560650+0000","last_undegraded":"2024-09-17T05:28:43.560650+0000","last_fullsized":"2024-09-17T05:28:43.560650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569216+0000","last_change":"2024-09-17T05:28:43.569216+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569216+0000","last_undegraded":"2024-09-17T05:28:43.569216+0000","last_fullsized":"2024-09-17T05:28:43.569216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564991+0000","last_change":"2024-09-17T05:28:43.564991+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564991+0000","last_undegraded":"2024-09-17T05:28:43.564991+0000","last_fullsized":"2024-09-17T05:28:43.564991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566690+0000","last_change":"2024-09-17T05:28:43.566690+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566690+0000","last_undegraded":"2024-09-17T05:28:43.566690+0000","last_fullsized":"2024-09-17T05:28:43.566690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.567714+0000","last_change":"2024-09-17T05:28:43.567714+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.567714+0000","last_undegraded":"2024-09-17T05:28:43.567714+0000","last_fullsized":"2024-09-17T05:28:43.567714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564898+0000","last_change":"2024-09-17T05:28:43.564898+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564898+0000","last_undegraded":"2024-09-17T05:28:43.564898+0000","last_fullsized":"2024-09-17T05:28:43.564898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559249+0000","last_change":"2024-09-17T05:28:43.559249+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559249+0000","last_undegraded":"2024-09-17T05:28:43.559249+0000","last_fullsized":"2024-09-17T05:28:43.559249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575894+0000","last_change":"2024-09-17T05:28:43.575894+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575894+0000","last_undegraded":"2024-09-17T05:28:43.575894+0000","last_fullsized":"2024-09-17T05:28:43.575894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557101+0000","last_change":"2024-09-17T05:28:43.557101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557101+0000","last_undegraded":"2024-09-17T05:28:43.557101+0000","last_fullsized":"2024-09-17T05:28:43.557101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562470+0000","last_change":"2024-09-17T05:28:43.562470+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562470+0000","last_undegraded":"2024-09-17T05:28:43.562470+0000","last_fullsized":"2024-09-17T05:28:43.562470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558548+0000","last_change":"2024-09-17T05:28:43.558548+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558548+0000","last_undegraded":"2024-09-17T05:28:43.558548+0000","last_fullsized":"2024-09-17T05:28:43.558548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563461+0000","last_change":"2024-09-17T05:28:43.563461+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563461+0000","last_undegraded":"2024-09-17T05:28:43.563461+0000","last_fullsized":"2024-09-17T05:28:43.563461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566497+0000","last_change":"2024-09-17T05:28:43.566497+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566497+0000","last_undegraded":"2024-09-17T05:28:43.566497+0000","last_fullsized":"2024-09-17T05:28:43.566497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561181+0000","last_change":"2024-09-17T05:28:43.561181+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561181+0000","last_undegraded":"2024-09-17T05:28:43.561181+0000","last_fullsized":"2024-09-17T05:28:43.561181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.568780+0000","last_change":"2024-09-17T05:28:43.568780+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.568780+0000","last_undegraded":"2024-09-17T05:28:43.568780+0000","last_fullsized":"2024-09-17T05:28:43.568780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565458+0000","last_change":"2024-09-17T05:28:43.565458+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565458+0000","last_undegraded":"2024-09-17T05:28:43.565458+0000","last_fullsized":"2024-09-17T05:28:43.565458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562373+0000","last_change":"2024-09-17T05:28:43.562373+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562373+0000","last_undegraded":"2024-09-17T05:28:43.562373+0000","last_fullsized":"2024-09-17T05:28:43.562373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560101+0000","last_change":"2024-09-17T05:28:43.560101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560101+0000","last_undegraded":"2024-09-17T05:28:43.560101+0000","last_fullsized":"2024-09-17T05:28:43.560101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562815+0000","last_change":"2024-09-17T05:28:43.562815+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562815+0000","last_undegraded":"2024-09-17T05:28:43.562815+0000","last_fullsized":"2024-09-17T05:28:43.562815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.590450+0000","last_change":"2024-09-17T05:28:43.588925+0000","last_active":"2024-09-17T05:28:43.590450+0000","last_peered":"2024-09-17T05:28:43.590450+0000","last_clean":"2024-09-17T05:28:43.590450+0000","last_became_active":"2024-09-17T05:28:43.588563+0000","last_became_peered":"2024-09-17T05:28:43.588563+0000","last_unstale":"2024-09-17T05:28:43.590450+0000","last_undegraded":"2024-09-17T05:28:43.590450+0000","last_fullsized":"2024-09-17T05:28:43.590450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:46.751852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573432+0000","last_change":"2024-09-17T05:28:43.573432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573432+0000","last_undegraded":"2024-09-17T05:28:43.573432+0000","last_fullsized":"2024-09-17T05:28:43.573432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.581450+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.581046+0000","last_became_peered":"2024-09-17T05:28:43.581046+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:15.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.579261+0000","last_change":"2024-09-17T05:28:43.579261+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.579261+0000","last_undegraded":"2024-09-17T05:28:43.579261+0000","last_fullsized":"2024-09-17T05:28:43.579261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587130+0000","last_change":"2024-09-17T05:28:43.584806+0000","last_active":"2024-09-17T05:28:43.587130+0000","last_peered":"2024-09-17T05:28:43.587130+0000","last_clean":"2024-09-17T05:28:43.587130+0000","last_became_active":"2024-09-17T05:28:43.584417+0000","last_became_peered":"2024-09-17T05:28:43.584417+0000","last_unstale":"2024-09-17T05:28:43.587130+0000","last_undegraded":"2024-09-17T05:28:43.587130+0000","last_fullsized":"2024-09-17T05:28:43.587130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:38.969560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559999+0000","last_change":"2024-09-17T05:28:43.559999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559999+0000","last_undegraded":"2024-09-17T05:28:43.559999+0000","last_fullsized":"2024-09-17T05:28:43.559999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588895+0000","last_change":"2024-09-17T05:28:43.585654+0000","last_active":"2024-09-17T05:28:43.588895+0000","last_peered":"2024-09-17T05:28:43.588895+0000","last_clean":"2024-09-17T05:28:43.588895+0000","last_became_active":"2024-09-17T05:28:43.585273+0000","last_became_peered":"2024-09-17T05:28:43.585273+0000","last_unstale":"2024-09-17T05:28:43.588895+0000","last_undegraded":"2024-09-17T05:28:43.588895+0000","last_fullsized":"2024-09-17T05:28:43.588895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:46.485943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563068+0000","last_change":"2024-09-17T05:28:43.563068+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563068+0000","last_undegraded":"2024-09-17T05:28:43.563068+0000","last_fullsized":"2024-09-17T05:28:43.563068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588565+0000","last_change":"2024-09-17T05:28:43.584092+0000","last_active":"2024-09-17T05:28:43.588565+0000","last_peered":"2024-09-17T05:28:43.588565+0000","last_clean":"2024-09-17T05:28:43.588565+0000","last_became_active":"2024-09-17T05:28:43.583206+0000","last_became_peered":"2024-09-17T05:28:43.583206+0000","last_unstale":"2024-09-17T05:28:43.588565+0000","last_undegraded":"2024-09-17T05:28:43.588565+0000","last_fullsized":"2024-09-17T05:28:43.588565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:05.142867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.578602+0000","last_change":"2024-09-17T05:28:43.578602+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.578602+0000","last_undegraded":"2024-09-17T05:28:43.578602+0000","last_fullsized":"2024-09-17T05:28:43.578602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587843+0000","last_change":"2024-09-17T05:28:43.585492+0000","last_active":"2024-09-17T05:28:43.587843+0000","last_peered":"2024-09-17T05:28:43.587843+0000","last_clean":"2024-09-17T05:28:43.587843+0000","last_became_active":"2024-09-17T05:28:43.585189+0000","last_became_peered":"2024-09-17T05:28:43.585189+0000","last_unstale":"2024-09-17T05:28:43.587843+0000","last_undegraded":"2024-09-17T05:28:43.587843+0000","last_fullsized":"2024-09-17T05:28:43.587843+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:48.503822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811028+0000","last_change":"2024-09-17T05:28:44.589403+0000","last_active":"2024-09-17T05:28:45.811028+0000","last_peered":"2024-09-17T05:28:45.811028+0000","last_clean":"2024-09-17T05:28:45.811028+0000","last_became_active":"2024-09-17T05:28:44.589080+0000","last_became_peered":"2024-09-17T05:28:44.589080+0000","last_unstale":"2024-09-17T05:28:45.811028+0000","last_undegraded":"2024-09-17T05:28:45.811028+0000","last_fullsized":"2024-09-17T05:28:45.811028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:41.124043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811067+0000","last_change":"2024-09-17T05:28:43.582311+0000","last_active":"2024-09-17T05:28:45.811067+0000","last_peered":"2024-09-17T05:28:45.811067+0000","last_clean":"2024-09-17T05:28:45.811067+0000","last_became_active":"2024-09-17T05:28:43.581966+0000","last_became_peered":"2024-09-17T05:28:43.581966+0000","last_unstale":"2024-09-17T05:28:45.811067+0000","last_undegraded":"2024-09-17T05:28:45.811067+0000","last_fullsized":"2024-09-17T05:28:45.811067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:28.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560003+0000","last_change":"2024-09-17T05:28:43.560003+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560003+0000","last_undegraded":"2024-09-17T05:28:43.560003+0000","last_fullsized":"2024-09-17T05:28:43.560003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581405+0000","last_change":"2024-09-17T05:28:43.574709+0000","last_active":"2024-09-17T05:28:43.581405+0000","last_peered":"2024-09-17T05:28:43.581405+0000","last_clean":"2024-09-17T05:28:43.581405+0000","last_became_active":"2024-09-17T05:28:43.574282+0000","last_became_peered":"2024-09-17T05:28:43.574282+0000","last_unstale":"2024-09-17T05:28:43.581405+0000","last_undegraded":"2024-09-17T05:28:43.581405+0000","last_fullsized":"2024-09-17T05:28:43.581405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559486+0000","last_change":"2024-09-17T05:28:43.559486+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559486+0000","last_undegraded":"2024-09-17T05:28:43.559486+0000","last_fullsized":"2024-09-17T05:28:43.559486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587144+0000","last_change":"2024-09-17T05:28:43.583590+0000","last_active":"2024-09-17T05:28:43.587144+0000","last_peered":"2024-09-17T05:28:43.587144+0000","last_clean":"2024-09-17T05:28:43.587144+0000","last_became_active":"2024-09-17T05:28:43.583012+0000","last_became_peered":"2024-09-17T05:28:43.583012+0000","last_unstale":"2024-09-17T05:28:43.587144+0000","last_undegraded":"2024-09-17T05:28:43.587144+0000","last_fullsized":"2024-09-17T05:28:43.587144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:01.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559432+0000","last_change":"2024-09-17T05:28:43.559432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559432+0000","last_undegraded":"2024-09-17T05:28:43.559432+0000","last_fullsized":"2024-09-17T05:28:43.559432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588331+0000","last_change":"2024-09-17T05:28:43.586381+0000","last_active":"2024-09-17T05:28:43.588331+0000","last_peered":"2024-09-17T05:28:43.588331+0000","last_clean":"2024-09-17T05:28:43.588331+0000","last_became_active":"2024-09-17T05:28:43.586051+0000","last_became_peered":"2024-09-17T05:28:43.586051+0000","last_unstale":"2024-09-17T05:28:43.588331+0000","last_undegraded":"2024-09-17T05:28:43.588331+0000","last_fullsized":"2024-09-17T05:28:43.588331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564654+0000","last_change":"2024-09-17T05:28:43.564654+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564654+0000","last_undegraded":"2024-09-17T05:28:43.564654+0000","last_fullsized":"2024-09-17T05:28:43.564654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581371+0000","last_change":"2024-09-17T05:28:43.572513+0000","last_active":"2024-09-17T05:28:43.581371+0000","last_peered":"2024-09-17T05:28:43.581371+0000","last_clean":"2024-09-17T05:28:43.581371+0000","last_became_active":"2024-09-17T05:28:43.571694+0000","last_became_peered":"2024-09-17T05:28:43.571694+0000","last_unstale":"2024-09-17T05:28:43.581371+0000","last_undegraded":"2024-09-17T05:28:43.581371+0000","last_fullsized":"2024-09-17T05:28:43.581371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565864+0000","last_change":"2024-09-17T05:28:43.565864+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565864+0000","last_undegraded":"2024-09-17T05:28:43.565864+0000","last_fullsized":"2024-09-17T05:28:43.565864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811729+0000","last_change":"2024-09-17T05:28:43.581526+0000","last_active":"2024-09-17T05:28:45.811729+0000","last_peered":"2024-09-17T05:28:45.811729+0000","last_clean":"2024-09-17T05:28:45.811729+0000","last_became_active":"2024-09-17T05:28:43.581163+0000","last_became_peered":"2024-09-17T05:28:43.581163+0000","last_unstale":"2024-09-17T05:28:45.811729+0000","last_undegraded":"2024-09-17T05:28:45.811729+0000","last_fullsized":"2024-09-17T05:28:45.811729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:27.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575375+0000","last_change":"2024-09-17T05:28:43.575375+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575375+0000","last_undegraded":"2024-09-17T05:28:43.575375+0000","last_fullsized":"2024-09-17T05:28:43.575375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812166+0000","last_change":"2024-09-17T05:28:43.577962+0000","last_active":"2024-09-17T05:28:45.812166+0000","last_peered":"2024-09-17T05:28:45.812166+0000","last_clean":"2024-09-17T05:28:45.812166+0000","last_became_active":"2024-09-17T05:28:43.577642+0000","last_became_peered":"2024-09-17T05:28:43.577642+0000","last_unstale":"2024-09-17T05:28:45.812166+0000","last_undegraded":"2024-09-17T05:28:45.812166+0000","last_fullsized":"2024-09-17T05:28:45.812166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:55.229169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558340+0000","last_change":"2024-09-17T05:28:43.558340+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558340+0000","last_undegraded":"2024-09-17T05:28:43.558340+0000","last_fullsized":"2024-09-17T05:28:43.558340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.572889+0000","last_change":"2024-09-17T05:28:43.570566+0000","last_active":"2024-09-17T05:28:43.572889+0000","last_peered":"2024-09-17T05:28:43.572889+0000","last_clean":"2024-09-17T05:28:43.572889+0000","last_became_active":"2024-09-17T05:28:43.570239+0000","last_became_peered":"2024-09-17T05:28:43.570239+0000","last_unstale":"2024-09-17T05:28:43.572889+0000","last_undegraded":"2024-09-17T05:28:43.572889+0000","last_fullsized":"2024-09-17T05:28:43.572889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:30.323131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562523+0000","last_change":"2024-09-17T05:28:43.562523+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562523+0000","last_undegraded":"2024-09-17T05:28:43.562523+0000","last_fullsized":"2024-09-17T05:28:43.562523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571701+0000","last_change":"2024-09-17T05:28:43.569060+0000","last_active":"2024-09-17T05:28:43.571701+0000","last_peered":"2024-09-17T05:28:43.571701+0000","last_clean":"2024-09-17T05:28:43.571701+0000","last_became_active":"2024-09-17T05:28:43.568732+0000","last_became_peered":"2024-09-17T05:28:43.568732+0000","last_unstale":"2024-09-17T05:28:43.571701+0000","last_undegraded":"2024-09-17T05:28:43.571701+0000","last_fullsized":"2024-09-17T05:28:43.571701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:41.209993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569463+0000","last_change":"2024-09-17T05:28:43.569463+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569463+0000","last_undegraded":"2024-09-17T05:28:43.569463+0000","last_fullsized":"2024-09-17T05:28:43.569463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586671+0000","last_change":"2024-09-17T05:28:43.581871+0000","last_active":"2024-09-17T05:28:43.586671+0000","last_peered":"2024-09-17T05:28:43.586671+0000","last_clean":"2024-09-17T05:28:43.586671+0000","last_became_active":"2024-09-17T05:28:43.581398+0000","last_became_peered":"2024-09-17T05:28:43.581398+0000","last_unstale":"2024-09-17T05:28:43.586671+0000","last_undegraded":"2024-09-17T05:28:43.586671+0000","last_fullsized":"2024-09-17T05:28:43.586671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:11.127069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558271+0000","last_change":"2024-09-17T05:28:43.558271+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558271+0000","last_undegraded":"2024-09-17T05:28:43.558271+0000","last_fullsized":"2024-09-17T05:28:43.558271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586475+0000","last_change":"2024-09-17T05:28:43.578038+0000","last_active":"2024-09-17T05:28:43.586475+0000","last_peered":"2024-09-17T05:28:43.586475+0000","last_clean":"2024-09-17T05:28:43.586475+0000","last_became_active":"2024-09-17T05:28:43.577245+0000","last_became_peered":"2024-09-17T05:28:43.577245+0000","last_unstale":"2024-09-17T05:28:43.586475+0000","last_undegraded":"2024-09-17T05:28:43.586475+0000","last_fullsized":"2024-09-17T05:28:43.586475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:46.553001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558656+0000","last_change":"2024-09-17T05:28:43.558656+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558656+0000","last_undegraded":"2024-09-17T05:28:43.558656+0000","last_fullsized":"2024-09-17T05:28:43.558656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.576474+0000","last_change":"2024-09-17T05:28:43.566233+0000","last_active":"2024-09-17T05:28:43.576474+0000","last_peered":"2024-09-17T05:28:43.576474+0000","last_clean":"2024-09-17T05:28:43.576474+0000","last_became_active":"2024-09-17T05:28:43.565869+0000","last_became_peered":"2024-09-17T05:28:43.565869+0000","last_unstale":"2024-09-17T05:28:43.576474+0000","last_undegraded":"2024-09-17T05:28:43.576474+0000","last_fullsized":"2024-09-17T05:28:43.576474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571309+0000","last_change":"2024-09-17T05:28:43.571309+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571309+0000","last_undegraded":"2024-09-17T05:28:43.571309+0000","last_fullsized":"2024-09-17T05:28:43.571309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578735+0000","last_change":"2024-09-17T05:28:43.575603+0000","last_active":"2024-09-17T05:28:43.578735+0000","last_peered":"2024-09-17T05:28:43.578735+0000","last_clean":"2024-09-17T05:28:43.578735+0000","last_became_active":"2024-09-17T05:28:43.575283+0000","last_became_peered":"2024-09-17T05:28:43.575283+0000","last_unstale":"2024-09-17T05:28:43.578735+0000","last_undegraded":"2024-09-17T05:28:43.578735+0000","last_fullsized":"2024-09-17T05:28:43.578735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.570892+0000","last_change":"2024-09-17T05:28:43.570892+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.570892+0000","last_undegraded":"2024-09-17T05:28:43.570892+0000","last_fullsized":"2024-09-17T05:28:43.570892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588919+0000","last_change":"2024-09-17T05:28:43.587791+0000","last_active":"2024-09-17T05:28:43.588919+0000","last_peered":"2024-09-17T05:28:43.588919+0000","last_clean":"2024-09-17T05:28:43.588919+0000","last_became_active":"2024-09-17T05:28:43.587450+0000","last_became_peered":"2024-09-17T05:28:43.587450+0000","last_unstale":"2024-09-17T05:28:43.588919+0000","last_undegraded":"2024-09-17T05:28:43.588919+0000","last_fullsized":"2024-09-17T05:28:43.588919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:37.493312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812092+0000","last_change":"2024-09-17T05:28:44.590941+0000","last_active":"2024-09-17T05:28:45.812092+0000","last_peered":"2024-09-17T05:28:45.812092+0000","last_clean":"2024-09-17T05:28:45.812092+0000","last_became_active":"2024-09-17T05:28:44.590589+0000","last_became_peered":"2024-09-17T05:28:44.590589+0000","last_unstale":"2024-09-17T05:28:45.812092+0000","last_undegraded":"2024-09-17T05:28:45.812092+0000","last_fullsized":"2024-09-17T05:28:45.812092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:16.499238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588698+0000","last_change":"2024-09-17T05:28:43.583447+0000","last_active":"2024-09-17T05:28:43.588698+0000","last_peered":"2024-09-17T05:28:43.588698+0000","last_clean":"2024-09-17T05:28:43.588698+0000","last_became_active":"2024-09-17T05:28:43.583145+0000","last_became_peered":"2024-09-17T05:28:43.583145+0000","last_unstale":"2024-09-17T05:28:43.588698+0000","last_undegraded":"2024-09-17T05:28:43.588698+0000","last_fullsized":"2024-09-17T05:28:43.588698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.203097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811078+0000","last_change":"2024-09-17T05:28:44.589236+0000","last_active":"2024-09-17T05:28:45.811078+0000","last_peered":"2024-09-17T05:28:45.811078+0000","last_clean":"2024-09-17T05:28:45.811078+0000","last_became_active":"2024-09-17T05:28:44.588658+0000","last_became_peered":"2024-09-17T05:28:44.588658+0000","last_unstale":"2024-09-17T05:28:45.811078+0000","last_undegraded":"2024-09-17T05:28:45.811078+0000","last_fullsized":"2024-09-17T05:28:45.811078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571752+0000","last_change":"2024-09-17T05:28:43.564033+0000","last_active":"2024-09-17T05:28:43.571752+0000","last_peered":"2024-09-17T05:28:43.571752+0000","last_clean":"2024-09-17T05:28:43.571752+0000","last_became_active":"2024-09-17T05:28:43.563635+0000","last_became_peered":"2024-09-17T05:28:43.563635+0000","last_unstale":"2024-09-17T05:28:43.571752+0000","last_undegraded":"2024-09-17T05:28:43.571752+0000","last_fullsized":"2024-09-17T05:28:43.571752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559428+0000","last_change":"2024-09-17T05:28:43.559428+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559428+0000","last_undegraded":"2024-09-17T05:28:43.559428+0000","last_fullsized":"2024-09-17T05:28:43.559428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.582716+0000","last_change":"2024-09-17T05:28:43.578062+0000","last_active":"2024-09-17T05:28:43.582716+0000","last_peered":"2024-09-17T05:28:43.582716+0000","last_clean":"2024-09-17T05:28:43.582716+0000","last_became_active":"2024-09-17T05:28:43.577664+0000","last_became_peered":"2024-09-17T05:28:43.577664+0000","last_unstale":"2024-09-17T05:28:43.582716+0000","last_undegraded":"2024-09-17T05:28:43.582716+0000","last_fullsized":"2024-09-17T05:28:43.582716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:31.835733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566156+0000","last_change":"2024-09-17T05:28:43.566156+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566156+0000","last_undegraded":"2024-09-17T05:28:43.566156+0000","last_fullsized":"2024-09-17T05:28:43.566156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.577691+0000","last_change":"2024-09-17T05:28:43.571945+0000","last_active":"2024-09-17T05:28:43.577691+0000","last_peered":"2024-09-17T05:28:43.577691+0000","last_clean":"2024-09-17T05:28:43.577691+0000","last_became_active":"2024-09-17T05:28:43.571593+0000","last_became_peered":"2024-09-17T05:28:43.571593+0000","last_unstale":"2024-09-17T05:28:43.577691+0000","last_undegraded":"2024-09-17T05:28:43.577691+0000","last_fullsized":"2024-09-17T05:28:43.577691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:32.061486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557528+0000","last_change":"2024-09-17T05:28:43.557528+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557528+0000","last_undegraded":"2024-09-17T05:28:43.557528+0000","last_fullsized":"2024-09-17T05:28:43.557528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.575109+0000","last_change":"2024-09-17T05:28:43.570857+0000","last_active":"2024-09-17T05:28:43.575109+0000","last_peered":"2024-09-17T05:28:43.575109+0000","last_clean":"2024-09-17T05:28:43.575109+0000","last_became_active":"2024-09-17T05:28:43.570448+0000","last_became_peered":"2024-09-17T05:28:43.570448+0000","last_unstale":"2024-09-17T05:28:43.575109+0000","last_undegraded":"2024-09-17T05:28:43.575109+0000","last_fullsized":"2024-09-17T05:28:43.575109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:21.157484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560566+0000","last_change":"2024-09-17T05:28:43.560566+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560566+0000","last_undegraded":"2024-09-17T05:28:43.560566+0000","last_fullsized":"2024-09-17T05:28:43.560566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578687+0000","last_change":"2024-09-17T05:28:43.574524+0000","last_active":"2024-09-17T05:28:43.578687+0000","last_peered":"2024-09-17T05:28:43.578687+0000","last_clean":"2024-09-17T05:28:43.578687+0000","last_became_active":"2024-09-17T05:28:43.572192+0000","last_became_peered":"2024-09-17T05:28:43.572192+0000","last_unstale":"2024-09-17T05:28:43.578687+0000","last_undegraded":"2024-09-17T05:28:43.578687+0000","last_fullsized":"2024-09-17T05:28:43.578687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557647+0000","last_change":"2024-09-17T05:28:43.557647+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557647+0000","last_undegraded":"2024-09-17T05:28:43.557647+0000","last_fullsized":"2024-09-17T05:28:43.557647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578635+0000","last_change":"2024-09-17T05:28:43.574129+0000","last_active":"2024-09-17T05:28:43.578635+0000","last_peered":"2024-09-17T05:28:43.578635+0000","last_clean":"2024-09-17T05:28:43.578635+0000","last_became_active":"2024-09-17T05:28:43.573723+0000","last_became_peered":"2024-09-17T05:28:43.573723+0000","last_unstale":"2024-09-17T05:28:43.578635+0000","last_undegraded":"2024-09-17T05:28:43.578635+0000","last_fullsized":"2024-09-17T05:28:43.578635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563675+0000","last_change":"2024-09-17T05:28:43.563675+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563675+0000","last_undegraded":"2024-09-17T05:28:43.563675+0000","last_fullsized":"2024-09-17T05:28:43.563675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.582630+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.582246+0000","last_became_peered":"2024-09-17T05:28:43.582246+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562999+0000","last_change":"2024-09-17T05:28:43.562999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562999+0000","last_undegraded":"2024-09-17T05:28:43.562999+0000","last_fullsized":"2024-09-17T05:28:43.562999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588401+0000","last_change":"2024-09-17T05:28:43.583802+0000","last_active":"2024-09-17T05:28:43.588401+0000","last_peered":"2024-09-17T05:28:43.588401+0000","last_clean":"2024-09-17T05:28:43.588401+0000","last_became_active":"2024-09-17T05:28:43.583483+0000","last_became_peered":"2024-09-17T05:28:43.583483+0000","last_unstale":"2024-09-17T05:28:43.588401+0000","last_undegraded":"2024-09-17T05:28:43.588401+0000","last_fullsized":"2024-09-17T05:28:43.588401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:45.524073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811820+0000","last_change":"2024-09-17T05:28:44.596874+0000","last_active":"2024-09-17T05:28:45.811820+0000","last_peered":"2024-09-17T05:28:45.811820+0000","last_clean":"2024-09-17T05:28:45.811820+0000","last_became_active":"2024-09-17T05:28:44.596501+0000","last_became_peered":"2024-09-17T05:28:44.596501+0000","last_unstale":"2024-09-17T05:28:45.811820+0000","last_undegraded":"2024-09-17T05:28:45.811820+0000","last_fullsized":"2024-09-17T05:28:45.811820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:27.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588309+0000","last_change":"2024-09-17T05:28:43.584354+0000","last_active":"2024-09-17T05:28:43.588309+0000","last_peered":"2024-09-17T05:28:43.588309+0000","last_clean":"2024-09-17T05:28:43.588309+0000","last_became_active":"2024-09-17T05:28:43.583900+0000","last_became_peered":"2024-09-17T05:28:43.583900+0000","last_unstale":"2024-09-17T05:28:43.588309+0000","last_undegraded":"2024-09-17T05:28:43.588309+0000","last_fullsized":"2024-09-17T05:28:43.588309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561812+0000","last_change":"2024-09-17T05:28:43.561812+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561812+0000","last_undegraded":"2024-09-17T05:28:43.561812+0000","last_fullsized":"2024-09-17T05:28:43.561812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588633+0000","last_change":"2024-09-17T05:28:43.582802+0000","last_active":"2024-09-17T05:28:43.588633+0000","last_peered":"2024-09-17T05:28:43.588633+0000","last_clean":"2024-09-17T05:28:43.588633+0000","last_became_active":"2024-09-17T05:28:43.582317+0000","last_became_peered":"2024-09-17T05:28:43.582317+0000","last_unstale":"2024-09-17T05:28:43.588633+0000","last_undegraded":"2024-09-17T05:28:43.588633+0000","last_fullsized":"2024-09-17T05:28:43.588633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:55.092115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564277+0000","last_change":"2024-09-17T05:28:43.564277+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564277+0000","last_undegraded":"2024-09-17T05:28:43.564277+0000","last_fullsized":"2024-09-17T05:28:43.564277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.568256+0000","last_change":"2024-09-17T05:28:43.565393+0000","last_active":"2024-09-17T05:28:43.568256+0000","last_peered":"2024-09-17T05:28:43.568256+0000","last_clean":"2024-09-17T05:28:43.568256+0000","last_became_active":"2024-09-17T05:28:43.565048+0000","last_became_peered":"2024-09-17T05:28:43.565048+0000","last_unstale":"2024-09-17T05:28:43.568256+0000","last_undegraded":"2024-09-17T05:28:43.568256+0000","last_fullsized":"2024-09-17T05:28:43.568256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:48.311903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.556862+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:28:43.556862+0000","last_peered":"2024-09-17T05:28:43.556862+0000","last_clean":"2024-09-17T05:28:43.556862+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:28:43.556862+0000","last_undegraded":"2024-09-17T05:28:43.556862+0000","last_fullsized":"2024-09-17T05:28:43.556862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":372429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509509,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":387150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:49.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- 172.21.15.43:0/3915406768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf88155d70 msgr2=0x7faf8815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 --2- 172.21.15.43:0/3915406768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf88155d70 0x7faf8815c7e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009840 tx=0x7faf7c01c760 comp rx=0 tx=0).stop 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- 172.21.15.43:0/3915406768 shutdown_connections 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 --2- 172.21.15.43:0/3915406768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf880b3af0 0x7faf880ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 --2- 172.21.15.43:0/3915406768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf88155d70 0x7faf8815c7e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- 172.21.15.43:0/3915406768 >> 172.21.15.43:0/3915406768 conn(0x7faf880b23d0 msgr2=0x7faf880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- 172.21.15.43:0/3915406768 shutdown_connections 2024-09-17T05:28:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- 172.21.15.43:0/3915406768 wait complete. 2024-09-17T05:28:49.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 Processor -- start 2024-09-17T05:28:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.398+0000 7faf8fa00640 1 -- start start 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf88155d70 0x7faf880c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 0x7faf880c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf88168bc0 con 0x7faf8806f110 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf88168a40 con 0x7faf8809dbf0 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faf88168d40 con 0x7faf88155d70 2024-09-17T05:28:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37258/0 (socket says 172.21.15.43:37258) 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 -- 172.21.15.43:0/2416292519 learned_addr learned my addr 172.21.15.43:0/2416292519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8da00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 0x7faf880c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf87e00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf88155d70 0x7faf880c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 -- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf88155d70 msgr2=0x7faf880c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf88155d70 0x7faf880c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 -- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 msgr2=0x7faf880c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 0x7faf880c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 -- 172.21.15.43:0/2416292519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf881f8610 con 0x7faf8806f110 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8da00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 0x7faf880c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8d000640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009790 tx=0x7faf7c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0bb070 con 0x7faf8806f110 2024-09-17T05:28:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf7c004330 con 0x7faf8806f110 2024-09-17T05:28:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf881f8b10 con 0x7faf8806f110 2024-09-17T05:28:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0b2040 con 0x7faf8806f110 2024-09-17T05:28:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf881f90d0 con 0x7faf8806f110 2024-09-17T05:28:49.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.402+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8815c9d0 con 0x7faf8806f110 2024-09-17T05:28:49.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.406+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf7c0be070 con 0x7faf8806f110 2024-09-17T05:28:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.406+0000 7faf84c00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 0x7faf68075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.406+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7faf7c12e190 con 0x7faf8806f110 2024-09-17T05:28:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.406+0000 7faf87e00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 0x7faf68075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.406+0000 7faf87e00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 0x7faf68075740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf88092210 tx=0x7faf74009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:49.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.410+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf7c0c5020 con 0x7faf8806f110 2024-09-17T05:28:49.544 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:28:49.542+0000 7fac6aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:28:49.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.582+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf8815c6f0 con 0x7faf68073230 2024-09-17T05:28:49.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.586+0000 7faf84c00640 1 -- 172.21.15.43:0/2416292519 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247127 (secure 0 0 0) 0x7faf8815c6f0 con 0x7faf68073230 2024-09-17T05:28:49.593 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:28:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 msgr2=0x7faf68075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 0x7faf68075740 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf88092210 tx=0x7faf74009040 comp rx=0 tx=0).stop 2024-09-17T05:28:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 msgr2=0x7faf880cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009790 tx=0x7faf7c005db0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 shutdown_connections 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf68073230 0x7faf68075740 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf8809dbf0 0x7faf880c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faf88155d70 0x7faf880c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 --2- 172.21.15.43:0/2416292519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf8806f110 0x7faf880cf770 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 >> 172.21.15.43:0/2416292519 conn(0x7faf880b23d0 msgr2=0x7faf8815a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 shutdown_connections 2024-09-17T05:28:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.590+0000 7faf8fa00640 1 -- 172.21.15.43:0/2416292519 wait complete. 2024-09-17T05:28:49.615 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-17T05:28:47.928140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":110,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257556,"kb_used_data":8032,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717164,"statfs":{"total":773094113280,"available":772830375936,"internally_reserved":0,"allocated":8224768,"data_stored":4149821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.376536"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564728+0000","last_change":"2024-09-17T05:28:43.564728+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564728+0000","last_undegraded":"2024-09-17T05:28:43.564728+0000","last_fullsized":"2024-09-17T05:28:43.564728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559745+0000","last_change":"2024-09-17T05:28:43.559745+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559745+0000","last_undegraded":"2024-09-17T05:28:43.559745+0000","last_fullsized":"2024-09-17T05:28:43.559745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571536+0000","last_change":"2024-09-17T05:28:43.571536+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571536+0000","last_undegraded":"2024-09-17T05:28:43.571536+0000","last_fullsized":"2024-09-17T05:28:43.571536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571856+0000","last_change":"2024-09-17T05:28:43.571856+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571856+0000","last_undegraded":"2024-09-17T05:28:43.571856+0000","last_fullsized":"2024-09-17T05:28:43.571856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564326+0000","last_change":"2024-09-17T05:28:43.564326+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564326+0000","last_undegraded":"2024-09-17T05:28:43.564326+0000","last_fullsized":"2024-09-17T05:28:43.564326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.576337+0000","last_change":"2024-09-17T05:28:43.576337+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.576337+0000","last_undegraded":"2024-09-17T05:28:43.576337+0000","last_fullsized":"2024-09-17T05:28:43.576337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558152+0000","last_change":"2024-09-17T05:28:43.558152+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558152+0000","last_undegraded":"2024-09-17T05:28:43.558152+0000","last_fullsized":"2024-09-17T05:28:43.558152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569809+0000","last_change":"2024-09-17T05:28:43.569809+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569809+0000","last_undegraded":"2024-09-17T05:28:43.569809+0000","last_fullsized":"2024-09-17T05:28:43.569809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573914+0000","last_change":"2024-09-17T05:28:43.573914+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573914+0000","last_undegraded":"2024-09-17T05:28:43.573914+0000","last_fullsized":"2024-09-17T05:28:43.573914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560035+0000","last_change":"2024-09-17T05:28:43.560035+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560035+0000","last_undegraded":"2024-09-17T05:28:43.560035+0000","last_fullsized":"2024-09-17T05:28:43.560035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566601+0000","last_change":"2024-09-17T05:28:43.566601+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566601+0000","last_undegraded":"2024-09-17T05:28:43.566601+0000","last_fullsized":"2024-09-17T05:28:43.566601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558924+0000","last_change":"2024-09-17T05:28:43.558924+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558924+0000","last_undegraded":"2024-09-17T05:28:43.558924+0000","last_fullsized":"2024-09-17T05:28:43.558924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569546+0000","last_change":"2024-09-17T05:28:43.569546+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569546+0000","last_undegraded":"2024-09-17T05:28:43.569546+0000","last_fullsized":"2024-09-17T05:28:43.569546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558400+0000","last_change":"2024-09-17T05:28:43.558400+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558400+0000","last_undegraded":"2024-09-17T05:28:43.558400+0000","last_fullsized":"2024-09-17T05:28:43.558400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560650+0000","last_change":"2024-09-17T05:28:43.560650+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560650+0000","last_undegraded":"2024-09-17T05:28:43.560650+0000","last_fullsized":"2024-09-17T05:28:43.560650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569216+0000","last_change":"2024-09-17T05:28:43.569216+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569216+0000","last_undegraded":"2024-09-17T05:28:43.569216+0000","last_fullsized":"2024-09-17T05:28:43.569216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564991+0000","last_change":"2024-09-17T05:28:43.564991+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564991+0000","last_undegraded":"2024-09-17T05:28:43.564991+0000","last_fullsized":"2024-09-17T05:28:43.564991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566690+0000","last_change":"2024-09-17T05:28:43.566690+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566690+0000","last_undegraded":"2024-09-17T05:28:43.566690+0000","last_fullsized":"2024-09-17T05:28:43.566690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.567714+0000","last_change":"2024-09-17T05:28:43.567714+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.567714+0000","last_undegraded":"2024-09-17T05:28:43.567714+0000","last_fullsized":"2024-09-17T05:28:43.567714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564898+0000","last_change":"2024-09-17T05:28:43.564898+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564898+0000","last_undegraded":"2024-09-17T05:28:43.564898+0000","last_fullsized":"2024-09-17T05:28:43.564898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559249+0000","last_change":"2024-09-17T05:28:43.559249+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559249+0000","last_undegraded":"2024-09-17T05:28:43.559249+0000","last_fullsized":"2024-09-17T05:28:43.559249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575894+0000","last_change":"2024-09-17T05:28:43.575894+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575894+0000","last_undegraded":"2024-09-17T05:28:43.575894+0000","last_fullsized":"2024-09-17T05:28:43.575894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557101+0000","last_change":"2024-09-17T05:28:43.557101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557101+0000","last_undegraded":"2024-09-17T05:28:43.557101+0000","last_fullsized":"2024-09-17T05:28:43.557101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562470+0000","last_change":"2024-09-17T05:28:43.562470+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562470+0000","last_undegraded":"2024-09-17T05:28:43.562470+0000","last_fullsized":"2024-09-17T05:28:43.562470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558548+0000","last_change":"2024-09-17T05:28:43.558548+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558548+0000","last_undegraded":"2024-09-17T05:28:43.558548+0000","last_fullsized":"2024-09-17T05:28:43.558548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563461+0000","last_change":"2024-09-17T05:28:43.563461+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563461+0000","last_undegraded":"2024-09-17T05:28:43.563461+0000","last_fullsized":"2024-09-17T05:28:43.563461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566497+0000","last_change":"2024-09-17T05:28:43.566497+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566497+0000","last_undegraded":"2024-09-17T05:28:43.566497+0000","last_fullsized":"2024-09-17T05:28:43.566497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561181+0000","last_change":"2024-09-17T05:28:43.561181+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561181+0000","last_undegraded":"2024-09-17T05:28:43.561181+0000","last_fullsized":"2024-09-17T05:28:43.561181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.568780+0000","last_change":"2024-09-17T05:28:43.568780+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.568780+0000","last_undegraded":"2024-09-17T05:28:43.568780+0000","last_fullsized":"2024-09-17T05:28:43.568780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565458+0000","last_change":"2024-09-17T05:28:43.565458+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565458+0000","last_undegraded":"2024-09-17T05:28:43.565458+0000","last_fullsized":"2024-09-17T05:28:43.565458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562373+0000","last_change":"2024-09-17T05:28:43.562373+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562373+0000","last_undegraded":"2024-09-17T05:28:43.562373+0000","last_fullsized":"2024-09-17T05:28:43.562373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560101+0000","last_change":"2024-09-17T05:28:43.560101+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560101+0000","last_undegraded":"2024-09-17T05:28:43.560101+0000","last_fullsized":"2024-09-17T05:28:43.560101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562815+0000","last_change":"2024-09-17T05:28:43.562815+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562815+0000","last_undegraded":"2024-09-17T05:28:43.562815+0000","last_fullsized":"2024-09-17T05:28:43.562815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.590450+0000","last_change":"2024-09-17T05:28:43.588925+0000","last_active":"2024-09-17T05:28:43.590450+0000","last_peered":"2024-09-17T05:28:43.590450+0000","last_clean":"2024-09-17T05:28:43.590450+0000","last_became_active":"2024-09-17T05:28:43.588563+0000","last_became_peered":"2024-09-17T05:28:43.588563+0000","last_unstale":"2024-09-17T05:28:43.590450+0000","last_undegraded":"2024-09-17T05:28:43.590450+0000","last_fullsized":"2024-09-17T05:28:43.590450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:46.751852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.573432+0000","last_change":"2024-09-17T05:28:43.573432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.573432+0000","last_undegraded":"2024-09-17T05:28:43.573432+0000","last_fullsized":"2024-09-17T05:28:43.573432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.581450+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.581046+0000","last_became_peered":"2024-09-17T05:28:43.581046+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:15.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.579261+0000","last_change":"2024-09-17T05:28:43.579261+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.579261+0000","last_undegraded":"2024-09-17T05:28:43.579261+0000","last_fullsized":"2024-09-17T05:28:43.579261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587130+0000","last_change":"2024-09-17T05:28:43.584806+0000","last_active":"2024-09-17T05:28:43.587130+0000","last_peered":"2024-09-17T05:28:43.587130+0000","last_clean":"2024-09-17T05:28:43.587130+0000","last_became_active":"2024-09-17T05:28:43.584417+0000","last_became_peered":"2024-09-17T05:28:43.584417+0000","last_unstale":"2024-09-17T05:28:43.587130+0000","last_undegraded":"2024-09-17T05:28:43.587130+0000","last_fullsized":"2024-09-17T05:28:43.587130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:38.969560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559999+0000","last_change":"2024-09-17T05:28:43.559999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559999+0000","last_undegraded":"2024-09-17T05:28:43.559999+0000","last_fullsized":"2024-09-17T05:28:43.559999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588895+0000","last_change":"2024-09-17T05:28:43.585654+0000","last_active":"2024-09-17T05:28:43.588895+0000","last_peered":"2024-09-17T05:28:43.588895+0000","last_clean":"2024-09-17T05:28:43.588895+0000","last_became_active":"2024-09-17T05:28:43.585273+0000","last_became_peered":"2024-09-17T05:28:43.585273+0000","last_unstale":"2024-09-17T05:28:43.588895+0000","last_undegraded":"2024-09-17T05:28:43.588895+0000","last_fullsized":"2024-09-17T05:28:43.588895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:46.485943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563068+0000","last_change":"2024-09-17T05:28:43.563068+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563068+0000","last_undegraded":"2024-09-17T05:28:43.563068+0000","last_fullsized":"2024-09-17T05:28:43.563068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588565+0000","last_change":"2024-09-17T05:28:43.584092+0000","last_active":"2024-09-17T05:28:43.588565+0000","last_peered":"2024-09-17T05:28:43.588565+0000","last_clean":"2024-09-17T05:28:43.588565+0000","last_became_active":"2024-09-17T05:28:43.583206+0000","last_became_peered":"2024-09-17T05:28:43.583206+0000","last_unstale":"2024-09-17T05:28:43.588565+0000","last_undegraded":"2024-09-17T05:28:43.588565+0000","last_fullsized":"2024-09-17T05:28:43.588565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:05.142867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.578602+0000","last_change":"2024-09-17T05:28:43.578602+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.578602+0000","last_undegraded":"2024-09-17T05:28:43.578602+0000","last_fullsized":"2024-09-17T05:28:43.578602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587843+0000","last_change":"2024-09-17T05:28:43.585492+0000","last_active":"2024-09-17T05:28:43.587843+0000","last_peered":"2024-09-17T05:28:43.587843+0000","last_clean":"2024-09-17T05:28:43.587843+0000","last_became_active":"2024-09-17T05:28:43.585189+0000","last_became_peered":"2024-09-17T05:28:43.585189+0000","last_unstale":"2024-09-17T05:28:43.587843+0000","last_undegraded":"2024-09-17T05:28:43.587843+0000","last_fullsized":"2024-09-17T05:28:43.587843+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:48.503822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811028+0000","last_change":"2024-09-17T05:28:44.589403+0000","last_active":"2024-09-17T05:28:45.811028+0000","last_peered":"2024-09-17T05:28:45.811028+0000","last_clean":"2024-09-17T05:28:45.811028+0000","last_became_active":"2024-09-17T05:28:44.589080+0000","last_became_peered":"2024-09-17T05:28:44.589080+0000","last_unstale":"2024-09-17T05:28:45.811028+0000","last_undegraded":"2024-09-17T05:28:45.811028+0000","last_fullsized":"2024-09-17T05:28:45.811028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:41.124043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811067+0000","last_change":"2024-09-17T05:28:43.582311+0000","last_active":"2024-09-17T05:28:45.811067+0000","last_peered":"2024-09-17T05:28:45.811067+0000","last_clean":"2024-09-17T05:28:45.811067+0000","last_became_active":"2024-09-17T05:28:43.581966+0000","last_became_peered":"2024-09-17T05:28:43.581966+0000","last_unstale":"2024-09-17T05:28:45.811067+0000","last_undegraded":"2024-09-17T05:28:45.811067+0000","last_fullsized":"2024-09-17T05:28:45.811067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:28.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560003+0000","last_change":"2024-09-17T05:28:43.560003+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560003+0000","last_undegraded":"2024-09-17T05:28:43.560003+0000","last_fullsized":"2024-09-17T05:28:43.560003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581405+0000","last_change":"2024-09-17T05:28:43.574709+0000","last_active":"2024-09-17T05:28:43.581405+0000","last_peered":"2024-09-17T05:28:43.581405+0000","last_clean":"2024-09-17T05:28:43.581405+0000","last_became_active":"2024-09-17T05:28:43.574282+0000","last_became_peered":"2024-09-17T05:28:43.574282+0000","last_unstale":"2024-09-17T05:28:43.581405+0000","last_undegraded":"2024-09-17T05:28:43.581405+0000","last_fullsized":"2024-09-17T05:28:43.581405+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559486+0000","last_change":"2024-09-17T05:28:43.559486+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559486+0000","last_undegraded":"2024-09-17T05:28:43.559486+0000","last_fullsized":"2024-09-17T05:28:43.559486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.587144+0000","last_change":"2024-09-17T05:28:43.583590+0000","last_active":"2024-09-17T05:28:43.587144+0000","last_peered":"2024-09-17T05:28:43.587144+0000","last_clean":"2024-09-17T05:28:43.587144+0000","last_became_active":"2024-09-17T05:28:43.583012+0000","last_became_peered":"2024-09-17T05:28:43.583012+0000","last_unstale":"2024-09-17T05:28:43.587144+0000","last_undegraded":"2024-09-17T05:28:43.587144+0000","last_fullsized":"2024-09-17T05:28:43.587144+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:01.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559432+0000","last_change":"2024-09-17T05:28:43.559432+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559432+0000","last_undegraded":"2024-09-17T05:28:43.559432+0000","last_fullsized":"2024-09-17T05:28:43.559432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588331+0000","last_change":"2024-09-17T05:28:43.586381+0000","last_active":"2024-09-17T05:28:43.588331+0000","last_peered":"2024-09-17T05:28:43.588331+0000","last_clean":"2024-09-17T05:28:43.588331+0000","last_became_active":"2024-09-17T05:28:43.586051+0000","last_became_peered":"2024-09-17T05:28:43.586051+0000","last_unstale":"2024-09-17T05:28:43.588331+0000","last_undegraded":"2024-09-17T05:28:43.588331+0000","last_fullsized":"2024-09-17T05:28:43.588331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564654+0000","last_change":"2024-09-17T05:28:43.564654+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564654+0000","last_undegraded":"2024-09-17T05:28:43.564654+0000","last_fullsized":"2024-09-17T05:28:43.564654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.581371+0000","last_change":"2024-09-17T05:28:43.572513+0000","last_active":"2024-09-17T05:28:43.581371+0000","last_peered":"2024-09-17T05:28:43.581371+0000","last_clean":"2024-09-17T05:28:43.581371+0000","last_became_active":"2024-09-17T05:28:43.571694+0000","last_became_peered":"2024-09-17T05:28:43.571694+0000","last_unstale":"2024-09-17T05:28:43.581371+0000","last_undegraded":"2024-09-17T05:28:43.581371+0000","last_fullsized":"2024-09-17T05:28:43.581371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.565864+0000","last_change":"2024-09-17T05:28:43.565864+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.565864+0000","last_undegraded":"2024-09-17T05:28:43.565864+0000","last_fullsized":"2024-09-17T05:28:43.565864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811729+0000","last_change":"2024-09-17T05:28:43.581526+0000","last_active":"2024-09-17T05:28:45.811729+0000","last_peered":"2024-09-17T05:28:45.811729+0000","last_clean":"2024-09-17T05:28:45.811729+0000","last_became_active":"2024-09-17T05:28:43.581163+0000","last_became_peered":"2024-09-17T05:28:43.581163+0000","last_unstale":"2024-09-17T05:28:45.811729+0000","last_undegraded":"2024-09-17T05:28:45.811729+0000","last_fullsized":"2024-09-17T05:28:45.811729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:27.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.575375+0000","last_change":"2024-09-17T05:28:43.575375+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.575375+0000","last_undegraded":"2024-09-17T05:28:43.575375+0000","last_fullsized":"2024-09-17T05:28:43.575375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812166+0000","last_change":"2024-09-17T05:28:43.577962+0000","last_active":"2024-09-17T05:28:45.812166+0000","last_peered":"2024-09-17T05:28:45.812166+0000","last_clean":"2024-09-17T05:28:45.812166+0000","last_became_active":"2024-09-17T05:28:43.577642+0000","last_became_peered":"2024-09-17T05:28:43.577642+0000","last_unstale":"2024-09-17T05:28:45.812166+0000","last_undegraded":"2024-09-17T05:28:45.812166+0000","last_fullsized":"2024-09-17T05:28:45.812166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:55.229169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558340+0000","last_change":"2024-09-17T05:28:43.558340+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558340+0000","last_undegraded":"2024-09-17T05:28:43.558340+0000","last_fullsized":"2024-09-17T05:28:43.558340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.572889+0000","last_change":"2024-09-17T05:28:43.570566+0000","last_active":"2024-09-17T05:28:43.572889+0000","last_peered":"2024-09-17T05:28:43.572889+0000","last_clean":"2024-09-17T05:28:43.572889+0000","last_became_active":"2024-09-17T05:28:43.570239+0000","last_became_peered":"2024-09-17T05:28:43.570239+0000","last_unstale":"2024-09-17T05:28:43.572889+0000","last_undegraded":"2024-09-17T05:28:43.572889+0000","last_fullsized":"2024-09-17T05:28:43.572889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:30.323131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562523+0000","last_change":"2024-09-17T05:28:43.562523+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562523+0000","last_undegraded":"2024-09-17T05:28:43.562523+0000","last_fullsized":"2024-09-17T05:28:43.562523+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571701+0000","last_change":"2024-09-17T05:28:43.569060+0000","last_active":"2024-09-17T05:28:43.571701+0000","last_peered":"2024-09-17T05:28:43.571701+0000","last_clean":"2024-09-17T05:28:43.571701+0000","last_became_active":"2024-09-17T05:28:43.568732+0000","last_became_peered":"2024-09-17T05:28:43.568732+0000","last_unstale":"2024-09-17T05:28:43.571701+0000","last_undegraded":"2024-09-17T05:28:43.571701+0000","last_fullsized":"2024-09-17T05:28:43.571701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:41.209993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.569463+0000","last_change":"2024-09-17T05:28:43.569463+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.569463+0000","last_undegraded":"2024-09-17T05:28:43.569463+0000","last_fullsized":"2024-09-17T05:28:43.569463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586671+0000","last_change":"2024-09-17T05:28:43.581871+0000","last_active":"2024-09-17T05:28:43.586671+0000","last_peered":"2024-09-17T05:28:43.586671+0000","last_clean":"2024-09-17T05:28:43.586671+0000","last_became_active":"2024-09-17T05:28:43.581398+0000","last_became_peered":"2024-09-17T05:28:43.581398+0000","last_unstale":"2024-09-17T05:28:43.586671+0000","last_undegraded":"2024-09-17T05:28:43.586671+0000","last_fullsized":"2024-09-17T05:28:43.586671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:11.127069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558271+0000","last_change":"2024-09-17T05:28:43.558271+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558271+0000","last_undegraded":"2024-09-17T05:28:43.558271+0000","last_fullsized":"2024-09-17T05:28:43.558271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.586475+0000","last_change":"2024-09-17T05:28:43.578038+0000","last_active":"2024-09-17T05:28:43.586475+0000","last_peered":"2024-09-17T05:28:43.586475+0000","last_clean":"2024-09-17T05:28:43.586475+0000","last_became_active":"2024-09-17T05:28:43.577245+0000","last_became_peered":"2024-09-17T05:28:43.577245+0000","last_unstale":"2024-09-17T05:28:43.586475+0000","last_undegraded":"2024-09-17T05:28:43.586475+0000","last_fullsized":"2024-09-17T05:28:43.586475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:46.553001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.558656+0000","last_change":"2024-09-17T05:28:43.558656+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.558656+0000","last_undegraded":"2024-09-17T05:28:43.558656+0000","last_fullsized":"2024-09-17T05:28:43.558656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.576474+0000","last_change":"2024-09-17T05:28:43.566233+0000","last_active":"2024-09-17T05:28:43.576474+0000","last_peered":"2024-09-17T05:28:43.576474+0000","last_clean":"2024-09-17T05:28:43.576474+0000","last_became_active":"2024-09-17T05:28:43.565869+0000","last_became_peered":"2024-09-17T05:28:43.565869+0000","last_unstale":"2024-09-17T05:28:43.576474+0000","last_undegraded":"2024-09-17T05:28:43.576474+0000","last_fullsized":"2024-09-17T05:28:43.576474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.571309+0000","last_change":"2024-09-17T05:28:43.571309+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.571309+0000","last_undegraded":"2024-09-17T05:28:43.571309+0000","last_fullsized":"2024-09-17T05:28:43.571309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578735+0000","last_change":"2024-09-17T05:28:43.575603+0000","last_active":"2024-09-17T05:28:43.578735+0000","last_peered":"2024-09-17T05:28:43.578735+0000","last_clean":"2024-09-17T05:28:43.578735+0000","last_became_active":"2024-09-17T05:28:43.575283+0000","last_became_peered":"2024-09-17T05:28:43.575283+0000","last_unstale":"2024-09-17T05:28:43.578735+0000","last_undegraded":"2024-09-17T05:28:43.578735+0000","last_fullsized":"2024-09-17T05:28:43.578735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.570892+0000","last_change":"2024-09-17T05:28:43.570892+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.570892+0000","last_undegraded":"2024-09-17T05:28:43.570892+0000","last_fullsized":"2024-09-17T05:28:43.570892+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588919+0000","last_change":"2024-09-17T05:28:43.587791+0000","last_active":"2024-09-17T05:28:43.588919+0000","last_peered":"2024-09-17T05:28:43.588919+0000","last_clean":"2024-09-17T05:28:43.588919+0000","last_became_active":"2024-09-17T05:28:43.587450+0000","last_became_peered":"2024-09-17T05:28:43.587450+0000","last_unstale":"2024-09-17T05:28:43.588919+0000","last_undegraded":"2024-09-17T05:28:43.588919+0000","last_fullsized":"2024-09-17T05:28:43.588919+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:37.493312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812092+0000","last_change":"2024-09-17T05:28:44.590941+0000","last_active":"2024-09-17T05:28:45.812092+0000","last_peered":"2024-09-17T05:28:45.812092+0000","last_clean":"2024-09-17T05:28:45.812092+0000","last_became_active":"2024-09-17T05:28:44.590589+0000","last_became_peered":"2024-09-17T05:28:44.590589+0000","last_unstale":"2024-09-17T05:28:45.812092+0000","last_undegraded":"2024-09-17T05:28:45.812092+0000","last_fullsized":"2024-09-17T05:28:45.812092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:16.499238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588698+0000","last_change":"2024-09-17T05:28:43.583447+0000","last_active":"2024-09-17T05:28:43.588698+0000","last_peered":"2024-09-17T05:28:43.588698+0000","last_clean":"2024-09-17T05:28:43.588698+0000","last_became_active":"2024-09-17T05:28:43.583145+0000","last_became_peered":"2024-09-17T05:28:43.583145+0000","last_unstale":"2024-09-17T05:28:43.588698+0000","last_undegraded":"2024-09-17T05:28:43.588698+0000","last_fullsized":"2024-09-17T05:28:43.588698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.203097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811078+0000","last_change":"2024-09-17T05:28:44.589236+0000","last_active":"2024-09-17T05:28:45.811078+0000","last_peered":"2024-09-17T05:28:45.811078+0000","last_clean":"2024-09-17T05:28:45.811078+0000","last_became_active":"2024-09-17T05:28:44.588658+0000","last_became_peered":"2024-09-17T05:28:44.588658+0000","last_unstale":"2024-09-17T05:28:45.811078+0000","last_undegraded":"2024-09-17T05:28:45.811078+0000","last_fullsized":"2024-09-17T05:28:45.811078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.571752+0000","last_change":"2024-09-17T05:28:43.564033+0000","last_active":"2024-09-17T05:28:43.571752+0000","last_peered":"2024-09-17T05:28:43.571752+0000","last_clean":"2024-09-17T05:28:43.571752+0000","last_became_active":"2024-09-17T05:28:43.563635+0000","last_became_peered":"2024-09-17T05:28:43.563635+0000","last_unstale":"2024-09-17T05:28:43.571752+0000","last_undegraded":"2024-09-17T05:28:43.571752+0000","last_fullsized":"2024-09-17T05:28:43.571752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:29.976027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.559428+0000","last_change":"2024-09-17T05:28:43.559428+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.559428+0000","last_undegraded":"2024-09-17T05:28:43.559428+0000","last_fullsized":"2024-09-17T05:28:43.559428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.582716+0000","last_change":"2024-09-17T05:28:43.578062+0000","last_active":"2024-09-17T05:28:43.582716+0000","last_peered":"2024-09-17T05:28:43.582716+0000","last_clean":"2024-09-17T05:28:43.582716+0000","last_became_active":"2024-09-17T05:28:43.577664+0000","last_became_peered":"2024-09-17T05:28:43.577664+0000","last_unstale":"2024-09-17T05:28:43.582716+0000","last_undegraded":"2024-09-17T05:28:43.582716+0000","last_fullsized":"2024-09-17T05:28:43.582716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:31.835733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.566156+0000","last_change":"2024-09-17T05:28:43.566156+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.566156+0000","last_undegraded":"2024-09-17T05:28:43.566156+0000","last_fullsized":"2024-09-17T05:28:43.566156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.577691+0000","last_change":"2024-09-17T05:28:43.571945+0000","last_active":"2024-09-17T05:28:43.577691+0000","last_peered":"2024-09-17T05:28:43.577691+0000","last_clean":"2024-09-17T05:28:43.577691+0000","last_became_active":"2024-09-17T05:28:43.571593+0000","last_became_peered":"2024-09-17T05:28:43.571593+0000","last_unstale":"2024-09-17T05:28:43.577691+0000","last_undegraded":"2024-09-17T05:28:43.577691+0000","last_fullsized":"2024-09-17T05:28:43.577691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:32.061486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557528+0000","last_change":"2024-09-17T05:28:43.557528+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557528+0000","last_undegraded":"2024-09-17T05:28:43.557528+0000","last_fullsized":"2024-09-17T05:28:43.557528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.575109+0000","last_change":"2024-09-17T05:28:43.570857+0000","last_active":"2024-09-17T05:28:43.575109+0000","last_peered":"2024-09-17T05:28:43.575109+0000","last_clean":"2024-09-17T05:28:43.575109+0000","last_became_active":"2024-09-17T05:28:43.570448+0000","last_became_peered":"2024-09-17T05:28:43.570448+0000","last_unstale":"2024-09-17T05:28:43.575109+0000","last_undegraded":"2024-09-17T05:28:43.575109+0000","last_fullsized":"2024-09-17T05:28:43.575109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:21.157484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.560566+0000","last_change":"2024-09-17T05:28:43.560566+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.560566+0000","last_undegraded":"2024-09-17T05:28:43.560566+0000","last_fullsized":"2024-09-17T05:28:43.560566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578687+0000","last_change":"2024-09-17T05:28:43.574524+0000","last_active":"2024-09-17T05:28:43.578687+0000","last_peered":"2024-09-17T05:28:43.578687+0000","last_clean":"2024-09-17T05:28:43.578687+0000","last_became_active":"2024-09-17T05:28:43.572192+0000","last_became_peered":"2024-09-17T05:28:43.572192+0000","last_unstale":"2024-09-17T05:28:43.578687+0000","last_undegraded":"2024-09-17T05:28:43.578687+0000","last_fullsized":"2024-09-17T05:28:43.578687+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:19.888650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.557647+0000","last_change":"2024-09-17T05:28:43.557647+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.557647+0000","last_undegraded":"2024-09-17T05:28:43.557647+0000","last_fullsized":"2024-09-17T05:28:43.557647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.578635+0000","last_change":"2024-09-17T05:28:43.574129+0000","last_active":"2024-09-17T05:28:43.578635+0000","last_peered":"2024-09-17T05:28:43.578635+0000","last_clean":"2024-09-17T05:28:43.578635+0000","last_became_active":"2024-09-17T05:28:43.573723+0000","last_became_peered":"2024-09-17T05:28:43.573723+0000","last_unstale":"2024-09-17T05:28:43.578635+0000","last_undegraded":"2024-09-17T05:28:43.578635+0000","last_fullsized":"2024-09-17T05:28:43.578635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:41.977874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.563675+0000","last_change":"2024-09-17T05:28:43.563675+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.563675+0000","last_undegraded":"2024-09-17T05:28:43.563675+0000","last_fullsized":"2024-09-17T05:28:43.563675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.585725+0000","last_change":"2024-09-17T05:28:43.582630+0000","last_active":"2024-09-17T05:28:43.585725+0000","last_peered":"2024-09-17T05:28:43.585725+0000","last_clean":"2024-09-17T05:28:43.585725+0000","last_became_active":"2024-09-17T05:28:43.582246+0000","last_became_peered":"2024-09-17T05:28:43.582246+0000","last_unstale":"2024-09-17T05:28:43.585725+0000","last_undegraded":"2024-09-17T05:28:43.585725+0000","last_fullsized":"2024-09-17T05:28:43.585725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.275194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.562999+0000","last_change":"2024-09-17T05:28:43.562999+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.562999+0000","last_undegraded":"2024-09-17T05:28:43.562999+0000","last_fullsized":"2024-09-17T05:28:43.562999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588401+0000","last_change":"2024-09-17T05:28:43.583802+0000","last_active":"2024-09-17T05:28:43.588401+0000","last_peered":"2024-09-17T05:28:43.588401+0000","last_clean":"2024-09-17T05:28:43.588401+0000","last_became_active":"2024-09-17T05:28:43.583483+0000","last_became_peered":"2024-09-17T05:28:43.583483+0000","last_unstale":"2024-09-17T05:28:43.588401+0000","last_undegraded":"2024-09-17T05:28:43.588401+0000","last_fullsized":"2024-09-17T05:28:43.588401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:45.524073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811820+0000","last_change":"2024-09-17T05:28:44.596874+0000","last_active":"2024-09-17T05:28:45.811820+0000","last_peered":"2024-09-17T05:28:45.811820+0000","last_clean":"2024-09-17T05:28:45.811820+0000","last_became_active":"2024-09-17T05:28:44.596501+0000","last_became_peered":"2024-09-17T05:28:44.596501+0000","last_unstale":"2024-09-17T05:28:45.811820+0000","last_undegraded":"2024-09-17T05:28:45.811820+0000","last_fullsized":"2024-09-17T05:28:45.811820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:27.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588309+0000","last_change":"2024-09-17T05:28:43.584354+0000","last_active":"2024-09-17T05:28:43.588309+0000","last_peered":"2024-09-17T05:28:43.588309+0000","last_clean":"2024-09-17T05:28:43.588309+0000","last_became_active":"2024-09-17T05:28:43.583900+0000","last_became_peered":"2024-09-17T05:28:43.583900+0000","last_unstale":"2024-09-17T05:28:43.588309+0000","last_undegraded":"2024-09-17T05:28:43.588309+0000","last_fullsized":"2024-09-17T05:28:43.588309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:54.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.561812+0000","last_change":"2024-09-17T05:28:43.561812+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.561812+0000","last_undegraded":"2024-09-17T05:28:43.561812+0000","last_fullsized":"2024-09-17T05:28:43.561812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.588633+0000","last_change":"2024-09-17T05:28:43.582802+0000","last_active":"2024-09-17T05:28:43.588633+0000","last_peered":"2024-09-17T05:28:43.588633+0000","last_clean":"2024-09-17T05:28:43.588633+0000","last_became_active":"2024-09-17T05:28:43.582317+0000","last_became_peered":"2024-09-17T05:28:43.582317+0000","last_unstale":"2024-09-17T05:28:43.588633+0000","last_undegraded":"2024-09-17T05:28:43.588633+0000","last_fullsized":"2024-09-17T05:28:43.588633+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:55.092115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-09-17T05:28:43.564277+0000","last_change":"2024-09-17T05:28:43.564277+0000","last_active":"2024-09-17T05:28:43.545893+0000","last_peered":"2024-09-17T05:28:43.545893+0000","last_clean":"2024-09-17T05:28:43.545893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:43.564277+0000","last_undegraded":"2024-09-17T05:28:43.564277+0000","last_fullsized":"2024-09-17T05:28:43.564277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.568256+0000","last_change":"2024-09-17T05:28:43.565393+0000","last_active":"2024-09-17T05:28:43.568256+0000","last_peered":"2024-09-17T05:28:43.568256+0000","last_clean":"2024-09-17T05:28:43.568256+0000","last_became_active":"2024-09-17T05:28:43.565048+0000","last_became_peered":"2024-09-17T05:28:43.565048+0000","last_unstale":"2024-09-17T05:28:43.568256+0000","last_undegraded":"2024-09-17T05:28:43.568256+0000","last_fullsized":"2024-09-17T05:28:43.568256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:48.311903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":208,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-17T05:28:43.556862+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:28:43.556862+0000","last_peered":"2024-09-17T05:28:43.556862+0000","last_clean":"2024-09-17T05:28:43.556862+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:28:43.556862+0000","last_undegraded":"2024-09-17T05:28:43.556862+0000","last_fullsized":"2024-09-17T05:28:43.556862+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":372429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509509,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1474560,"data_stored":962797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":366162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":387150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3f is not active+clean 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564728+0000', 'last_change': '2024-09-17T05:28:43.564728+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564728+0000', 'last_undegraded': '2024-09-17T05:28:43.564728+0000', 'last_fullsized': '2024-09-17T05:28:43.564728+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3e is not active+clean 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559745+0000', 'last_change': '2024-09-17T05:28:43.559745+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559745+0000', 'last_undegraded': '2024-09-17T05:28:43.559745+0000', 'last_fullsized': '2024-09-17T05:28:43.559745+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3d is not active+clean 2024-09-17T05:28:49.619 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.571536+0000', 'last_change': '2024-09-17T05:28:43.571536+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.571536+0000', 'last_undegraded': '2024-09-17T05:28:43.571536+0000', 'last_fullsized': '2024-09-17T05:28:43.571536+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.571856+0000', 'last_change': '2024-09-17T05:28:43.571856+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.571856+0000', 'last_undegraded': '2024-09-17T05:28:43.571856+0000', 'last_fullsized': '2024-09-17T05:28:43.571856+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3b is not active+clean 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564326+0000', 'last_change': '2024-09-17T05:28:43.564326+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564326+0000', 'last_undegraded': '2024-09-17T05:28:43.564326+0000', 'last_fullsized': '2024-09-17T05:28:43.564326+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3a is not active+clean 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.576337+0000', 'last_change': '2024-09-17T05:28:43.576337+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.576337+0000', 'last_undegraded': '2024-09-17T05:28:43.576337+0000', 'last_fullsized': '2024-09-17T05:28:43.576337+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.39 is not active+clean 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558152+0000', 'last_change': '2024-09-17T05:28:43.558152+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558152+0000', 'last_undegraded': '2024-09-17T05:28:43.558152+0000', 'last_fullsized': '2024-09-17T05:28:43.558152+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.38 is not active+clean 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.569809+0000', 'last_change': '2024-09-17T05:28:43.569809+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.569809+0000', 'last_undegraded': '2024-09-17T05:28:43.569809+0000', 'last_fullsized': '2024-09-17T05:28:43.569809+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.620 INFO:tasks.ceph.ceph_manager.ceph:PG 11.37 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.573914+0000', 'last_change': '2024-09-17T05:28:43.573914+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.573914+0000', 'last_undegraded': '2024-09-17T05:28:43.573914+0000', 'last_fullsized': '2024-09-17T05:28:43.573914+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:PG 11.36 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.560035+0000', 'last_change': '2024-09-17T05:28:43.560035+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.560035+0000', 'last_undegraded': '2024-09-17T05:28:43.560035+0000', 'last_fullsized': '2024-09-17T05:28:43.560035+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:PG 11.35 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.566601+0000', 'last_change': '2024-09-17T05:28:43.566601+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.566601+0000', 'last_undegraded': '2024-09-17T05:28:43.566601+0000', 'last_fullsized': '2024-09-17T05:28:43.566601+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558924+0000', 'last_change': '2024-09-17T05:28:43.558924+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558924+0000', 'last_undegraded': '2024-09-17T05:28:43.558924+0000', 'last_fullsized': '2024-09-17T05:28:43.558924+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:PG 11.33 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.569546+0000', 'last_change': '2024-09-17T05:28:43.569546+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.569546+0000', 'last_undegraded': '2024-09-17T05:28:43.569546+0000', 'last_fullsized': '2024-09-17T05:28:43.569546+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:PG 11.32 is not active+clean 2024-09-17T05:28:49.621 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558400+0000', 'last_change': '2024-09-17T05:28:43.558400+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558400+0000', 'last_undegraded': '2024-09-17T05:28:43.558400+0000', 'last_fullsized': '2024-09-17T05:28:43.558400+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:PG 11.31 is not active+clean 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.560650+0000', 'last_change': '2024-09-17T05:28:43.560650+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.560650+0000', 'last_undegraded': '2024-09-17T05:28:43.560650+0000', 'last_fullsized': '2024-09-17T05:28:43.560650+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:PG 11.30 is not active+clean 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.569216+0000', 'last_change': '2024-09-17T05:28:43.569216+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.569216+0000', 'last_undegraded': '2024-09-17T05:28:43.569216+0000', 'last_fullsized': '2024-09-17T05:28:43.569216+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2f is not active+clean 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564991+0000', 'last_change': '2024-09-17T05:28:43.564991+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564991+0000', 'last_undegraded': '2024-09-17T05:28:43.564991+0000', 'last_fullsized': '2024-09-17T05:28:43.564991+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.566690+0000', 'last_change': '2024-09-17T05:28:43.566690+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.566690+0000', 'last_undegraded': '2024-09-17T05:28:43.566690+0000', 'last_fullsized': '2024-09-17T05:28:43.566690+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2d is not active+clean 2024-09-17T05:28:49.622 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.567714+0000', 'last_change': '2024-09-17T05:28:43.567714+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.567714+0000', 'last_undegraded': '2024-09-17T05:28:43.567714+0000', 'last_fullsized': '2024-09-17T05:28:43.567714+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2c is not active+clean 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564898+0000', 'last_change': '2024-09-17T05:28:43.564898+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564898+0000', 'last_undegraded': '2024-09-17T05:28:43.564898+0000', 'last_fullsized': '2024-09-17T05:28:43.564898+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2b is not active+clean 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559249+0000', 'last_change': '2024-09-17T05:28:43.559249+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559249+0000', 'last_undegraded': '2024-09-17T05:28:43.559249+0000', 'last_fullsized': '2024-09-17T05:28:43.559249+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2a is not active+clean 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.575894+0000', 'last_change': '2024-09-17T05:28:43.575894+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.575894+0000', 'last_undegraded': '2024-09-17T05:28:43.575894+0000', 'last_fullsized': '2024-09-17T05:28:43.575894+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:PG 11.29 is not active+clean 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.557101+0000', 'last_change': '2024-09-17T05:28:43.557101+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.557101+0000', 'last_undegraded': '2024-09-17T05:28:43.557101+0000', 'last_fullsized': '2024-09-17T05:28:43.557101+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:PG 11.28 is not active+clean 2024-09-17T05:28:49.623 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.562470+0000', 'last_change': '2024-09-17T05:28:43.562470+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.562470+0000', 'last_undegraded': '2024-09-17T05:28:43.562470+0000', 'last_fullsized': '2024-09-17T05:28:43.562470+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:PG 11.27 is not active+clean 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558548+0000', 'last_change': '2024-09-17T05:28:43.558548+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558548+0000', 'last_undegraded': '2024-09-17T05:28:43.558548+0000', 'last_fullsized': '2024-09-17T05:28:43.558548+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.563461+0000', 'last_change': '2024-09-17T05:28:43.563461+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.563461+0000', 'last_undegraded': '2024-09-17T05:28:43.563461+0000', 'last_fullsized': '2024-09-17T05:28:43.563461+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:PG 11.25 is not active+clean 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.566497+0000', 'last_change': '2024-09-17T05:28:43.566497+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.566497+0000', 'last_undegraded': '2024-09-17T05:28:43.566497+0000', 'last_fullsized': '2024-09-17T05:28:43.566497+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:PG 11.24 is not active+clean 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.561181+0000', 'last_change': '2024-09-17T05:28:43.561181+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.561181+0000', 'last_undegraded': '2024-09-17T05:28:43.561181+0000', 'last_fullsized': '2024-09-17T05:28:43.561181+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:PG 11.23 is not active+clean 2024-09-17T05:28:49.624 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.568780+0000', 'last_change': '2024-09-17T05:28:43.568780+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.568780+0000', 'last_undegraded': '2024-09-17T05:28:43.568780+0000', 'last_fullsized': '2024-09-17T05:28:43.568780+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.22 is not active+clean 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.565458+0000', 'last_change': '2024-09-17T05:28:43.565458+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.565458+0000', 'last_undegraded': '2024-09-17T05:28:43.565458+0000', 'last_fullsized': '2024-09-17T05:28:43.565458+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.21 is not active+clean 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.562373+0000', 'last_change': '2024-09-17T05:28:43.562373+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.562373+0000', 'last_undegraded': '2024-09-17T05:28:43.562373+0000', 'last_fullsized': '2024-09-17T05:28:43.562373+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.20 is not active+clean 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.560101+0000', 'last_change': '2024-09-17T05:28:43.560101+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.560101+0000', 'last_undegraded': '2024-09-17T05:28:43.560101+0000', 'last_fullsized': '2024-09-17T05:28:43.560101+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1e is not active+clean 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.562815+0000', 'last_change': '2024-09-17T05:28:43.562815+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.562815+0000', 'last_undegraded': '2024-09-17T05:28:43.562815+0000', 'last_fullsized': '2024-09-17T05:28:43.562815+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1f is not active+clean 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.573432+0000', 'last_change': '2024-09-17T05:28:43.573432+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.573432+0000', 'last_undegraded': '2024-09-17T05:28:43.573432+0000', 'last_fullsized': '2024-09-17T05:28:43.573432+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.625 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1c is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.579261+0000', 'last_change': '2024-09-17T05:28:43.579261+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.579261+0000', 'last_undegraded': '2024-09-17T05:28:43.579261+0000', 'last_fullsized': '2024-09-17T05:28:43.579261+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1d is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559999+0000', 'last_change': '2024-09-17T05:28:43.559999+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559999+0000', 'last_undegraded': '2024-09-17T05:28:43.559999+0000', 'last_fullsized': '2024-09-17T05:28:43.559999+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1a is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.563068+0000', 'last_change': '2024-09-17T05:28:43.563068+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.563068+0000', 'last_undegraded': '2024-09-17T05:28:43.563068+0000', 'last_fullsized': '2024-09-17T05:28:43.563068+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1b is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.578602+0000', 'last_change': '2024-09-17T05:28:43.578602+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.578602+0000', 'last_undegraded': '2024-09-17T05:28:43.578602+0000', 'last_fullsized': '2024-09-17T05:28:43.578602+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.19 is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.560003+0000', 'last_change': '2024-09-17T05:28:43.560003+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.560003+0000', 'last_undegraded': '2024-09-17T05:28:43.560003+0000', 'last_fullsized': '2024-09-17T05:28:43.560003+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.16 is not active+clean 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559486+0000', 'last_change': '2024-09-17T05:28:43.559486+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559486+0000', 'last_undegraded': '2024-09-17T05:28:43.559486+0000', 'last_fullsized': '2024-09-17T05:28:43.559486+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.626 INFO:tasks.ceph.ceph_manager.ceph:PG 11.17 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559432+0000', 'last_change': '2024-09-17T05:28:43.559432+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559432+0000', 'last_undegraded': '2024-09-17T05:28:43.559432+0000', 'last_fullsized': '2024-09-17T05:28:43.559432+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:PG 11.14 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564654+0000', 'last_change': '2024-09-17T05:28:43.564654+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564654+0000', 'last_undegraded': '2024-09-17T05:28:43.564654+0000', 'last_fullsized': '2024-09-17T05:28:43.564654+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:PG 11.15 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.565864+0000', 'last_change': '2024-09-17T05:28:43.565864+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.565864+0000', 'last_undegraded': '2024-09-17T05:28:43.565864+0000', 'last_fullsized': '2024-09-17T05:28:43.565864+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:PG 11.12 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.575375+0000', 'last_change': '2024-09-17T05:28:43.575375+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.575375+0000', 'last_undegraded': '2024-09-17T05:28:43.575375+0000', 'last_fullsized': '2024-09-17T05:28:43.575375+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:PG 11.13 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558340+0000', 'last_change': '2024-09-17T05:28:43.558340+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558340+0000', 'last_undegraded': '2024-09-17T05:28:43.558340+0000', 'last_fullsized': '2024-09-17T05:28:43.558340+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:PG 11.10 is not active+clean 2024-09-17T05:28:49.627 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.562523+0000', 'last_change': '2024-09-17T05:28:43.562523+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.562523+0000', 'last_undegraded': '2024-09-17T05:28:43.562523+0000', 'last_fullsized': '2024-09-17T05:28:43.562523+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.11 is not active+clean 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.569463+0000', 'last_change': '2024-09-17T05:28:43.569463+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.569463+0000', 'last_undegraded': '2024-09-17T05:28:43.569463+0000', 'last_fullsized': '2024-09-17T05:28:43.569463+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.e is not active+clean 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558271+0000', 'last_change': '2024-09-17T05:28:43.558271+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558271+0000', 'last_undegraded': '2024-09-17T05:28:43.558271+0000', 'last_fullsized': '2024-09-17T05:28:43.558271+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.f is not active+clean 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.558656+0000', 'last_change': '2024-09-17T05:28:43.558656+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.558656+0000', 'last_undegraded': '2024-09-17T05:28:43.558656+0000', 'last_fullsized': '2024-09-17T05:28:43.558656+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.571309+0000', 'last_change': '2024-09-17T05:28:43.571309+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.571309+0000', 'last_undegraded': '2024-09-17T05:28:43.571309+0000', 'last_fullsized': '2024-09-17T05:28:43.571309+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.d is not active+clean 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.570892+0000', 'last_change': '2024-09-17T05:28:43.570892+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.570892+0000', 'last_undegraded': '2024-09-17T05:28:43.570892+0000', 'last_fullsized': '2024-09-17T05:28:43.570892+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:49.628 INFO:tasks.ceph.ceph_manager.ceph:PG 11.9 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.559428+0000', 'last_change': '2024-09-17T05:28:43.559428+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.559428+0000', 'last_undegraded': '2024-09-17T05:28:43.559428+0000', 'last_fullsized': '2024-09-17T05:28:43.559428+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:PG 11.6 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.566156+0000', 'last_change': '2024-09-17T05:28:43.566156+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.566156+0000', 'last_undegraded': '2024-09-17T05:28:43.566156+0000', 'last_fullsized': '2024-09-17T05:28:43.566156+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:PG 11.7 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.557528+0000', 'last_change': '2024-09-17T05:28:43.557528+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.557528+0000', 'last_undegraded': '2024-09-17T05:28:43.557528+0000', 'last_fullsized': '2024-09-17T05:28:43.557528+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:PG 11.4 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.560566+0000', 'last_change': '2024-09-17T05:28:43.560566+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.560566+0000', 'last_undegraded': '2024-09-17T05:28:43.560566+0000', 'last_fullsized': '2024-09-17T05:28:43.560566+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:PG 11.5 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.557647+0000', 'last_change': '2024-09-17T05:28:43.557647+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.557647+0000', 'last_undegraded': '2024-09-17T05:28:43.557647+0000', 'last_fullsized': '2024-09-17T05:28:43.557647+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2 is not active+clean 2024-09-17T05:28:49.629 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.563675+0000', 'last_change': '2024-09-17T05:28:43.563675+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.563675+0000', 'last_undegraded': '2024-09-17T05:28:43.563675+0000', 'last_fullsized': '2024-09-17T05:28:43.563675+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3 is not active+clean 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.562999+0000', 'last_change': '2024-09-17T05:28:43.562999+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.562999+0000', 'last_undegraded': '2024-09-17T05:28:43.562999+0000', 'last_fullsized': '2024-09-17T05:28:43.562999+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1 is not active+clean 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.561812+0000', 'last_change': '2024-09-17T05:28:43.561812+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.561812+0000', 'last_undegraded': '2024-09-17T05:28:43.561812+0000', 'last_fullsized': '2024-09-17T05:28:43.561812+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:PG 11.a is not active+clean 2024-09-17T05:28:49.630 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:43.564277+0000', 'last_change': '2024-09-17T05:28:43.564277+0000', 'last_active': '2024-09-17T05:28:43.545893+0000', 'last_peered': '2024-09-17T05:28:43.545893+0000', 'last_clean': '2024-09-17T05:28:43.545893+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:43.564277+0000', 'last_undegraded': '2024-09-17T05:28:43.564277+0000', 'last_fullsized': '2024-09-17T05:28:43.564277+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:43.545893+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:28:49.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:28:49.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 -- 172.21.15.43:0/3237077062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76409ba60 msgr2=0x7fd7640b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 --2- 172.21.15.43:0/3237077062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76409ba60 0x7fd7640b3130 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd754009840 tx=0x7fd75401c740 comp rx=0 tx=0).stop 2024-09-17T05:28:49.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 -- 172.21.15.43:0/3237077062 shutdown_connections 2024-09-17T05:28:49.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 --2- 172.21.15.43:0/3237077062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd76409ba60 0x7fd7640b3130 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 -- 172.21.15.43:0/3237077062 >> 172.21.15.43:0/3237077062 conn(0x7fd7640cee60 msgr2=0x7fd7640cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:49.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 -- 172.21.15.43:0/3237077062 shutdown_connections 2024-09-17T05:28:49.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.814+0000 7fd769800640 1 -- 172.21.15.43:0/3237077062 wait complete. 2024-09-17T05:28:49.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 Processor -- start 2024-09-17T05:28:49.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- start start 2024-09-17T05:28:49.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76409ba60 0x7fd76406f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 0x7fd764149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd76416b330 con 0x7fd7640d0650 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd76416b1b0 con 0x7fd7640d0240 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd76416b4b0 con 0x7fd76409ba60 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37268/0 (socket says 172.21.15.43:37268) 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 -- 172.21.15.43:0/1914376257 learned_addr learned my addr 172.21.15.43:0/1914376257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd762a00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76409ba60 0x7fd76406f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd75be00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 0x7fd764149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 -- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76409ba60 msgr2=0x7fd76406f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76409ba60 0x7fd76406f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 -- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 msgr2=0x7fd764149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 0x7fd764149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 -- 172.21.15.43:0/1914376257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd764149d90 con 0x7fd7640d0650 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd75be00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 0x7fd764149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd763400640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00cbe0 tx=0x7fd75c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:49.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c016020 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd75c00ab60 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76415b140 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012420 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7641fc780 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.818+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640b3130 con 0x7fd7640d0650 2024-09-17T05:28:49.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.822+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd75c019020 con 0x7fd7640d0650 2024-09-17T05:28:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.822+0000 7fd75aa00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 0x7fd740075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.822+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fd75c094740 con 0x7fd7640d0650 2024-09-17T05:28:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.822+0000 7fd762a00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 0x7fd740075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:49.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.822+0000 7fd762a00640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 0x7fd740075740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd7540af000 tx=0x7fd7540b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:49.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:49.826+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd75c020020 con 0x7fd7640d0650 2024-09-17T05:28:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.062+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd76415aed0 con 0x7fd7640d0650 2024-09-17T05:28:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd75aa00640 1 -- 172.21.15.43:0/1914376257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1140 (secure 0 0 0) 0x7fd75c05f640 con 0x7fd7640d0650 2024-09-17T05:28:50.067 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:50.067 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":351,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":75,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550593,"num_in_osds":8,"osd_in_since":1726550582,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":60},{"state_name":"active+clean","count":37}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":592014,"bytes_used":263737344,"bytes_avail":772830375936,"bytes_total":773094113280,"inactive_pgs_ratio":0.6185566782951355,"write_bytes_sec":1904,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":70,"btime":"2024-09-17T05:28:47:595804+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6387}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-17T05:27:49.911542+0000","services":{}},"progress_events":{}} 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 msgr2=0x7fd740075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 0x7fd740075740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd7540af000 tx=0x7fd7540b2040 comp rx=0 tx=0).stop 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 msgr2=0x7fd76415c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00cbe0 tx=0x7fd75c00ccd0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 shutdown_connections 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd740073230 0x7fd740075740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7640d0240 0x7fd764149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7640d0650 0x7fd76415c650 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 --2- 172.21.15.43:0/1914376257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd76409ba60 0x7fd76406f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 >> 172.21.15.43:0/1914376257 conn(0x7fd7640cee60 msgr2=0x7fd7640cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 shutdown_connections 2024-09-17T05:28:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.066+0000 7fd769800640 1 -- 172.21.15.43:0/1914376257 wait complete. 2024-09-17T05:28:50.091 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:28:50.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:50.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.246+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3817470040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde98004810 msgr2=0x7fde98004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.246+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3817470040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde98004810 0x7fde98004c20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009b60 tx=0x7fde9c01d2b0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9000640 1 -- 172.21.15.43:0/3817470040 reap_dead start 2024-09-17T05:28:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3817470040 shutdown_connections 2024-09-17T05:28:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3817470040 >> 172.21.15.43:0/3817470040 conn(0x7fdeb40691b0 msgr2=0x7fdeb40695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3817470040 shutdown_connections 2024-09-17T05:28:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3817470040 wait complete. 2024-09-17T05:28:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 Processor -- start 2024-09-17T05:28:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.250+0000 7fdeb9a00640 1 -- start start 2024-09-17T05:28:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde98004810 0x7fdeb41fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeb406f910 0x7fdeb40b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41531b0 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb4153030 con 0x7fde98004810 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb4153330 con 0x7fdeb406f910 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37278/0 (socket says 172.21.15.43:37278) 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 -- 172.21.15.43:0/3267750914 learned_addr learned my addr 172.21.15.43:0/3267750914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb3e00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde98004810 0x7fdeb41fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9000640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeb406f910 0x7fdeb40b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 -- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeb406f910 msgr2=0x7fdeb40b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeb406f910 0x7fdeb40b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 -- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde98004810 msgr2=0x7fdeb41fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde98004810 0x7fdeb41fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 -- 172.21.15.43:0/3267750914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb41fd740 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeab400640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009a80 tx=0x7fde9c0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0b1040 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde9c004030 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb406fd20 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c004480 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41f9b10 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb4156e70 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.254+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fde9c0bd020 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.258+0000 7fdeb0c00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 0x7fde90075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.258+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fde9c12e9a0 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.258+0000 7fdeb3e00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 0x7fde90075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:50.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.258+0000 7fdeb3e00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 0x7fde90075740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000fe0 tx=0x7fdea4006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:50.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.262+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdeb4156e70 con 0x7fdeb40b7cf0 2024-09-17T05:28:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.434+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdeb4153870 con 0x7fde90073230 2024-09-17T05:28:50.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.438+0000 7fdeb0c00640 1 -- 172.21.15.43:0/3267750914 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252061 (secure 0 0 0) 0x7fdeb4153870 con 0x7fde90073230 2024-09-17T05:28:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:28:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 msgr2=0x7fde90075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 0x7fde90075740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000fe0 tx=0x7fdea4006040 comp rx=0 tx=0).stop 2024-09-17T05:28:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 msgr2=0x7fdeb406f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009a80 tx=0x7fde9c0055d0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 shutdown_connections 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fde90073230 0x7fde90075740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdeb406f910 0x7fdeb40b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdeb40b7cf0 0x7fdeb406f3d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 --2- 172.21.15.43:0/3267750914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde98004810 0x7fdeb41fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 >> 172.21.15.43:0/3267750914 conn(0x7fdeb40691b0 msgr2=0x7fdeb4071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 shutdown_connections 2024-09-17T05:28:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:50.442+0000 7fdeb9a00640 1 -- 172.21.15.43:0/3267750914 wait complete. 2024-09-17T05:28:50.466 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-17T05:28:49.928634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257832,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716888,"statfs":{"total":773094113280,"available":772830093312,"internally_reserved":0,"allocated":8495104,"data_stored":4269588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.374310"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810845+0000","last_change":"2024-09-17T05:28:44.585647+0000","last_active":"2024-09-17T05:28:45.810845+0000","last_peered":"2024-09-17T05:28:45.810845+0000","last_clean":"2024-09-17T05:28:45.810845+0000","last_became_active":"2024-09-17T05:28:44.585156+0000","last_became_peered":"2024-09-17T05:28:44.585156+0000","last_unstale":"2024-09-17T05:28:45.810845+0000","last_undegraded":"2024-09-17T05:28:45.810845+0000","last_fullsized":"2024-09-17T05:28:45.810845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:57.759116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591719+0000","last_change":"2024-09-17T05:28:44.615499+0000","last_active":"2024-09-17T05:28:45.591719+0000","last_peered":"2024-09-17T05:28:45.591719+0000","last_clean":"2024-09-17T05:28:45.591719+0000","last_became_active":"2024-09-17T05:28:44.615166+0000","last_became_peered":"2024-09-17T05:28:44.615166+0000","last_unstale":"2024-09-17T05:28:45.591719+0000","last_undegraded":"2024-09-17T05:28:45.591719+0000","last_fullsized":"2024-09-17T05:28:45.591719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:21.247892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811543+0000","last_change":"2024-09-17T05:28:44.601689+0000","last_active":"2024-09-17T05:28:45.811543+0000","last_peered":"2024-09-17T05:28:45.811543+0000","last_clean":"2024-09-17T05:28:45.811543+0000","last_became_active":"2024-09-17T05:28:44.601350+0000","last_became_peered":"2024-09-17T05:28:44.601350+0000","last_unstale":"2024-09-17T05:28:45.811543+0000","last_undegraded":"2024-09-17T05:28:45.811543+0000","last_fullsized":"2024-09-17T05:28:45.811543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:31.596997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811435+0000","last_change":"2024-09-17T05:28:44.600102+0000","last_active":"2024-09-17T05:28:45.811435+0000","last_peered":"2024-09-17T05:28:45.811435+0000","last_clean":"2024-09-17T05:28:45.811435+0000","last_became_active":"2024-09-17T05:28:44.599793+0000","last_became_peered":"2024-09-17T05:28:44.599793+0000","last_unstale":"2024-09-17T05:28:45.811435+0000","last_undegraded":"2024-09-17T05:28:45.811435+0000","last_fullsized":"2024-09-17T05:28:45.811435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:33.495121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810521+0000","last_change":"2024-09-17T05:28:44.594481+0000","last_active":"2024-09-17T05:28:45.810521+0000","last_peered":"2024-09-17T05:28:45.810521+0000","last_clean":"2024-09-17T05:28:45.810521+0000","last_became_active":"2024-09-17T05:28:44.594116+0000","last_became_peered":"2024-09-17T05:28:44.594116+0000","last_unstale":"2024-09-17T05:28:45.810521+0000","last_undegraded":"2024-09-17T05:28:45.810521+0000","last_fullsized":"2024-09-17T05:28:45.810521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:49:57.919125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809344+0000","last_change":"2024-09-17T05:28:44.608063+0000","last_active":"2024-09-17T05:28:45.809344+0000","last_peered":"2024-09-17T05:28:45.809344+0000","last_clean":"2024-09-17T05:28:45.809344+0000","last_became_active":"2024-09-17T05:28:44.607088+0000","last_became_peered":"2024-09-17T05:28:44.607088+0000","last_unstale":"2024-09-17T05:28:45.809344+0000","last_undegraded":"2024-09-17T05:28:45.809344+0000","last_fullsized":"2024-09-17T05:28:45.809344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:10.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592697+0000","last_change":"2024-09-17T05:28:44.588931+0000","last_active":"2024-09-17T05:28:45.592697+0000","last_peered":"2024-09-17T05:28:45.592697+0000","last_clean":"2024-09-17T05:28:45.592697+0000","last_became_active":"2024-09-17T05:28:44.588583+0000","last_became_peered":"2024-09-17T05:28:44.588583+0000","last_unstale":"2024-09-17T05:28:45.592697+0000","last_undegraded":"2024-09-17T05:28:45.592697+0000","last_fullsized":"2024-09-17T05:28:45.592697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:09.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809448+0000","last_change":"2024-09-17T05:28:44.594090+0000","last_active":"2024-09-17T05:28:45.809448+0000","last_peered":"2024-09-17T05:28:45.809448+0000","last_clean":"2024-09-17T05:28:45.809448+0000","last_became_active":"2024-09-17T05:28:44.593306+0000","last_became_peered":"2024-09-17T05:28:44.593306+0000","last_unstale":"2024-09-17T05:28:45.809448+0000","last_undegraded":"2024-09-17T05:28:45.809448+0000","last_fullsized":"2024-09-17T05:28:45.809448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:07.325509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812113+0000","last_change":"2024-09-17T05:28:44.592878+0000","last_active":"2024-09-17T05:28:45.812113+0000","last_peered":"2024-09-17T05:28:45.812113+0000","last_clean":"2024-09-17T05:28:45.812113+0000","last_became_active":"2024-09-17T05:28:44.592502+0000","last_became_peered":"2024-09-17T05:28:44.592502+0000","last_unstale":"2024-09-17T05:28:45.812113+0000","last_undegraded":"2024-09-17T05:28:45.812113+0000","last_fullsized":"2024-09-17T05:28:45.812113+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:04.179376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593333+0000","last_change":"2024-09-17T05:28:44.582300+0000","last_active":"2024-09-17T05:28:45.593333+0000","last_peered":"2024-09-17T05:28:45.593333+0000","last_clean":"2024-09-17T05:28:45.593333+0000","last_became_active":"2024-09-17T05:28:44.581837+0000","last_became_peered":"2024-09-17T05:28:44.581837+0000","last_unstale":"2024-09-17T05:28:45.593333+0000","last_undegraded":"2024-09-17T05:28:45.593333+0000","last_fullsized":"2024-09-17T05:28:45.593333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:12.838918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810610+0000","last_change":"2024-09-17T05:28:44.601361+0000","last_active":"2024-09-17T05:28:45.810610+0000","last_peered":"2024-09-17T05:28:45.810610+0000","last_clean":"2024-09-17T05:28:45.810610+0000","last_became_active":"2024-09-17T05:28:44.601051+0000","last_became_peered":"2024-09-17T05:28:44.601051+0000","last_unstale":"2024-09-17T05:28:45.810610+0000","last_undegraded":"2024-09-17T05:28:45.810610+0000","last_fullsized":"2024-09-17T05:28:45.810610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:28.681348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593653+0000","last_change":"2024-09-17T05:28:44.600143+0000","last_active":"2024-09-17T05:28:45.593653+0000","last_peered":"2024-09-17T05:28:45.593653+0000","last_clean":"2024-09-17T05:28:45.593653+0000","last_became_active":"2024-09-17T05:28:44.599889+0000","last_became_peered":"2024-09-17T05:28:44.599889+0000","last_unstale":"2024-09-17T05:28:45.593653+0000","last_undegraded":"2024-09-17T05:28:45.593653+0000","last_fullsized":"2024-09-17T05:28:45.593653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:18.480582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811648+0000","last_change":"2024-09-17T05:28:44.597555+0000","last_active":"2024-09-17T05:28:45.811648+0000","last_peered":"2024-09-17T05:28:45.811648+0000","last_clean":"2024-09-17T05:28:45.811648+0000","last_became_active":"2024-09-17T05:28:44.597223+0000","last_became_peered":"2024-09-17T05:28:44.597223+0000","last_unstale":"2024-09-17T05:28:45.811648+0000","last_undegraded":"2024-09-17T05:28:45.811648+0000","last_fullsized":"2024-09-17T05:28:45.811648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:43.205669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593376+0000","last_change":"2024-09-17T05:28:44.601367+0000","last_active":"2024-09-17T05:28:45.593376+0000","last_peered":"2024-09-17T05:28:45.593376+0000","last_clean":"2024-09-17T05:28:45.593376+0000","last_became_active":"2024-09-17T05:28:44.601017+0000","last_became_peered":"2024-09-17T05:28:44.601017+0000","last_unstale":"2024-09-17T05:28:45.593376+0000","last_undegraded":"2024-09-17T05:28:45.593376+0000","last_fullsized":"2024-09-17T05:28:45.593376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:58.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592936+0000","last_change":"2024-09-17T05:28:44.587138+0000","last_active":"2024-09-17T05:28:45.592936+0000","last_peered":"2024-09-17T05:28:45.592936+0000","last_clean":"2024-09-17T05:28:45.592936+0000","last_became_active":"2024-09-17T05:28:44.586743+0000","last_became_peered":"2024-09-17T05:28:44.586743+0000","last_unstale":"2024-09-17T05:28:45.592936+0000","last_undegraded":"2024-09-17T05:28:45.592936+0000","last_fullsized":"2024-09-17T05:28:45.592936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:34.211101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809498+0000","last_change":"2024-09-17T05:28:44.594461+0000","last_active":"2024-09-17T05:28:45.809498+0000","last_peered":"2024-09-17T05:28:45.809498+0000","last_clean":"2024-09-17T05:28:45.809498+0000","last_became_active":"2024-09-17T05:28:44.593769+0000","last_became_peered":"2024-09-17T05:28:44.593769+0000","last_unstale":"2024-09-17T05:28:45.809498+0000","last_undegraded":"2024-09-17T05:28:45.809498+0000","last_fullsized":"2024-09-17T05:28:45.809498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:23.684881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812532+0000","last_change":"2024-09-17T05:28:44.581904+0000","last_active":"2024-09-17T05:28:45.812532+0000","last_peered":"2024-09-17T05:28:45.812532+0000","last_clean":"2024-09-17T05:28:45.812532+0000","last_became_active":"2024-09-17T05:28:44.581483+0000","last_became_peered":"2024-09-17T05:28:44.581483+0000","last_unstale":"2024-09-17T05:28:45.812532+0000","last_undegraded":"2024-09-17T05:28:45.812532+0000","last_fullsized":"2024-09-17T05:28:45.812532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:00.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812514+0000","last_change":"2024-09-17T05:28:44.591284+0000","last_active":"2024-09-17T05:28:45.812514+0000","last_peered":"2024-09-17T05:28:45.812514+0000","last_clean":"2024-09-17T05:28:45.812514+0000","last_became_active":"2024-09-17T05:28:44.590694+0000","last_became_peered":"2024-09-17T05:28:44.590694+0000","last_unstale":"2024-09-17T05:28:45.812514+0000","last_undegraded":"2024-09-17T05:28:45.812514+0000","last_fullsized":"2024-09-17T05:28:45.812514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:24.404477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810375+0000","last_change":"2024-09-17T05:28:44.603832+0000","last_active":"2024-09-17T05:28:45.810375+0000","last_peered":"2024-09-17T05:28:45.810375+0000","last_clean":"2024-09-17T05:28:45.810375+0000","last_became_active":"2024-09-17T05:28:44.603514+0000","last_became_peered":"2024-09-17T05:28:44.603514+0000","last_unstale":"2024-09-17T05:28:45.810375+0000","last_undegraded":"2024-09-17T05:28:45.810375+0000","last_fullsized":"2024-09-17T05:28:45.810375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:40.109889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810906+0000","last_change":"2024-09-17T05:28:44.598317+0000","last_active":"2024-09-17T05:28:45.810906+0000","last_peered":"2024-09-17T05:28:45.810906+0000","last_clean":"2024-09-17T05:28:45.810906+0000","last_became_active":"2024-09-17T05:28:44.597940+0000","last_became_peered":"2024-09-17T05:28:44.597940+0000","last_unstale":"2024-09-17T05:28:45.810906+0000","last_undegraded":"2024-09-17T05:28:45.810906+0000","last_fullsized":"2024-09-17T05:28:45.810906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:46.299125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591414+0000","last_change":"2024-09-17T05:28:44.587053+0000","last_active":"2024-09-17T05:28:45.591414+0000","last_peered":"2024-09-17T05:28:45.591414+0000","last_clean":"2024-09-17T05:28:45.591414+0000","last_became_active":"2024-09-17T05:28:44.586027+0000","last_became_peered":"2024-09-17T05:28:44.586027+0000","last_unstale":"2024-09-17T05:28:45.591414+0000","last_undegraded":"2024-09-17T05:28:45.591414+0000","last_fullsized":"2024-09-17T05:28:45.591414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:45.342193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809354+0000","last_change":"2024-09-17T05:28:44.607483+0000","last_active":"2024-09-17T05:28:45.809354+0000","last_peered":"2024-09-17T05:28:45.809354+0000","last_clean":"2024-09-17T05:28:45.809354+0000","last_became_active":"2024-09-17T05:28:44.606545+0000","last_became_peered":"2024-09-17T05:28:44.606545+0000","last_unstale":"2024-09-17T05:28:45.809354+0000","last_undegraded":"2024-09-17T05:28:45.809354+0000","last_fullsized":"2024-09-17T05:28:45.809354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:45.342193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592891+0000","last_change":"2024-09-17T05:28:44.597731+0000","last_active":"2024-09-17T05:28:45.592891+0000","last_peered":"2024-09-17T05:28:45.592891+0000","last_clean":"2024-09-17T05:28:45.592891+0000","last_became_active":"2024-09-17T05:28:44.594237+0000","last_became_peered":"2024-09-17T05:28:44.594237+0000","last_unstale":"2024-09-17T05:28:45.592891+0000","last_undegraded":"2024-09-17T05:28:45.592891+0000","last_fullsized":"2024-09-17T05:28:45.592891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:00.360443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809704+0000","last_change":"2024-09-17T05:28:44.602937+0000","last_active":"2024-09-17T05:28:45.809704+0000","last_peered":"2024-09-17T05:28:45.809704+0000","last_clean":"2024-09-17T05:28:45.809704+0000","last_became_active":"2024-09-17T05:28:44.600659+0000","last_became_peered":"2024-09-17T05:28:44.600659+0000","last_unstale":"2024-09-17T05:28:45.809704+0000","last_undegraded":"2024-09-17T05:28:45.809704+0000","last_fullsized":"2024-09-17T05:28:45.809704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:41.910571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593235+0000","last_change":"2024-09-17T05:28:44.596922+0000","last_active":"2024-09-17T05:28:45.593235+0000","last_peered":"2024-09-17T05:28:45.593235+0000","last_clean":"2024-09-17T05:28:45.593235+0000","last_became_active":"2024-09-17T05:28:44.596605+0000","last_became_peered":"2024-09-17T05:28:44.596605+0000","last_unstale":"2024-09-17T05:28:45.593235+0000","last_undegraded":"2024-09-17T05:28:45.593235+0000","last_fullsized":"2024-09-17T05:28:45.593235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:45.032755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810639+0000","last_change":"2024-09-17T05:28:44.611062+0000","last_active":"2024-09-17T05:28:45.810639+0000","last_peered":"2024-09-17T05:28:45.810639+0000","last_clean":"2024-09-17T05:28:45.810639+0000","last_became_active":"2024-09-17T05:28:44.610718+0000","last_became_peered":"2024-09-17T05:28:44.610718+0000","last_unstale":"2024-09-17T05:28:45.810639+0000","last_undegraded":"2024-09-17T05:28:45.810639+0000","last_fullsized":"2024-09-17T05:28:45.810639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:41.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810540+0000","last_change":"2024-09-17T05:28:44.592724+0000","last_active":"2024-09-17T05:28:45.810540+0000","last_peered":"2024-09-17T05:28:45.810540+0000","last_clean":"2024-09-17T05:28:45.810540+0000","last_became_active":"2024-09-17T05:28:44.592396+0000","last_became_peered":"2024-09-17T05:28:44.592396+0000","last_unstale":"2024-09-17T05:28:45.810540+0000","last_undegraded":"2024-09-17T05:28:45.810540+0000","last_fullsized":"2024-09-17T05:28:45.810540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:38.774253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591282+0000","last_change":"2024-09-17T05:28:44.605090+0000","last_active":"2024-09-17T05:28:45.591282+0000","last_peered":"2024-09-17T05:28:45.591282+0000","last_clean":"2024-09-17T05:28:45.591282+0000","last_became_active":"2024-09-17T05:28:44.600942+0000","last_became_peered":"2024-09-17T05:28:44.600942+0000","last_unstale":"2024-09-17T05:28:45.591282+0000","last_undegraded":"2024-09-17T05:28:45.591282+0000","last_fullsized":"2024-09-17T05:28:45.591282+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:18.480582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811923+0000","last_change":"2024-09-17T05:28:44.593442+0000","last_active":"2024-09-17T05:28:45.811923+0000","last_peered":"2024-09-17T05:28:45.811923+0000","last_clean":"2024-09-17T05:28:45.811923+0000","last_became_active":"2024-09-17T05:28:44.591992+0000","last_became_peered":"2024-09-17T05:28:44.591992+0000","last_unstale":"2024-09-17T05:28:45.811923+0000","last_undegraded":"2024-09-17T05:28:45.811923+0000","last_fullsized":"2024-09-17T05:28:45.811923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:19.154460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809806+0000","last_change":"2024-09-17T05:28:44.605342+0000","last_active":"2024-09-17T05:28:45.809806+0000","last_peered":"2024-09-17T05:28:45.809806+0000","last_clean":"2024-09-17T05:28:45.809806+0000","last_became_active":"2024-09-17T05:28:44.605005+0000","last_became_peered":"2024-09-17T05:28:44.605005+0000","last_unstale":"2024-09-17T05:28:45.809806+0000","last_undegraded":"2024-09-17T05:28:45.809806+0000","last_fullsized":"2024-09-17T05:28:45.809806+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:26.282603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591998+0000","last_change":"2024-09-17T05:28:44.601760+0000","last_active":"2024-09-17T05:28:45.591998+0000","last_peered":"2024-09-17T05:28:45.591998+0000","last_clean":"2024-09-17T05:28:45.591998+0000","last_became_active":"2024-09-17T05:28:44.592070+0000","last_became_peered":"2024-09-17T05:28:44.592070+0000","last_unstale":"2024-09-17T05:28:45.591998+0000","last_undegraded":"2024-09-17T05:28:45.591998+0000","last_fullsized":"2024-09-17T05:28:45.591998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:32.842755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591115+0000","last_change":"2024-09-17T05:28:44.599429+0000","last_active":"2024-09-17T05:28:45.591115+0000","last_peered":"2024-09-17T05:28:45.591115+0000","last_clean":"2024-09-17T05:28:45.591115+0000","last_became_active":"2024-09-17T05:28:44.599126+0000","last_became_peered":"2024-09-17T05:28:44.599126+0000","last_unstale":"2024-09-17T05:28:45.591115+0000","last_undegraded":"2024-09-17T05:28:45.591115+0000","last_fullsized":"2024-09-17T05:28:45.591115+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:56.568047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810639+0000","last_change":"2024-09-17T05:28:44.593467+0000","last_active":"2024-09-17T05:28:45.810639+0000","last_peered":"2024-09-17T05:28:45.810639+0000","last_clean":"2024-09-17T05:28:45.810639+0000","last_became_active":"2024-09-17T05:28:44.593107+0000","last_became_peered":"2024-09-17T05:28:44.593107+0000","last_unstale":"2024-09-17T05:28:45.810639+0000","last_undegraded":"2024-09-17T05:28:45.810639+0000","last_fullsized":"2024-09-17T05:28:45.810639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:25.470919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810183+0000","last_change":"2024-09-17T05:28:43.588925+0000","last_active":"2024-09-17T05:28:45.810183+0000","last_peered":"2024-09-17T05:28:45.810183+0000","last_clean":"2024-09-17T05:28:45.810183+0000","last_became_active":"2024-09-17T05:28:43.588563+0000","last_became_peered":"2024-09-17T05:28:43.588563+0000","last_unstale":"2024-09-17T05:28:45.810183+0000","last_undegraded":"2024-09-17T05:28:45.810183+0000","last_fullsized":"2024-09-17T05:28:45.810183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:56.307551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812013+0000","last_change":"2024-09-17T05:28:44.597058+0000","last_active":"2024-09-17T05:28:45.812013+0000","last_peered":"2024-09-17T05:28:45.812013+0000","last_clean":"2024-09-17T05:28:45.812013+0000","last_became_active":"2024-09-17T05:28:44.596741+0000","last_became_peered":"2024-09-17T05:28:44.596741+0000","last_unstale":"2024-09-17T05:28:45.812013+0000","last_undegraded":"2024-09-17T05:28:45.812013+0000","last_fullsized":"2024-09-17T05:28:45.812013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:03.433370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812757+0000","last_change":"2024-09-17T05:28:43.581450+0000","last_active":"2024-09-17T05:28:45.812757+0000","last_peered":"2024-09-17T05:28:45.812757+0000","last_clean":"2024-09-17T05:28:45.812757+0000","last_became_active":"2024-09-17T05:28:43.581046+0000","last_became_peered":"2024-09-17T05:28:43.581046+0000","last_unstale":"2024-09-17T05:28:45.812757+0000","last_undegraded":"2024-09-17T05:28:45.812757+0000","last_fullsized":"2024-09-17T05:28:45.812757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:41.396278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810307+0000","last_change":"2024-09-17T05:28:44.596042+0000","last_active":"2024-09-17T05:28:45.810307+0000","last_peered":"2024-09-17T05:28:45.810307+0000","last_clean":"2024-09-17T05:28:45.810307+0000","last_became_active":"2024-09-17T05:28:44.595246+0000","last_became_peered":"2024-09-17T05:28:44.595246+0000","last_unstale":"2024-09-17T05:28:45.810307+0000","last_undegraded":"2024-09-17T05:28:45.810307+0000","last_fullsized":"2024-09-17T05:28:45.810307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.432831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810574+0000","last_change":"2024-09-17T05:28:43.584806+0000","last_active":"2024-09-17T05:28:45.810574+0000","last_peered":"2024-09-17T05:28:45.810574+0000","last_clean":"2024-09-17T05:28:45.810574+0000","last_became_active":"2024-09-17T05:28:43.584417+0000","last_became_peered":"2024-09-17T05:28:43.584417+0000","last_unstale":"2024-09-17T05:28:45.810574+0000","last_undegraded":"2024-09-17T05:28:45.810574+0000","last_fullsized":"2024-09-17T05:28:45.810574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:43.980866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592815+0000","last_change":"2024-09-17T05:28:44.597783+0000","last_active":"2024-09-17T05:28:45.592815+0000","last_peered":"2024-09-17T05:28:45.592815+0000","last_clean":"2024-09-17T05:28:45.592815+0000","last_became_active":"2024-09-17T05:28:44.597414+0000","last_became_peered":"2024-09-17T05:28:44.597414+0000","last_unstale":"2024-09-17T05:28:45.592815+0000","last_undegraded":"2024-09-17T05:28:45.592815+0000","last_fullsized":"2024-09-17T05:28:45.592815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:00.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810187+0000","last_change":"2024-09-17T05:28:43.585654+0000","last_active":"2024-09-17T05:28:45.810187+0000","last_peered":"2024-09-17T05:28:45.810187+0000","last_clean":"2024-09-17T05:28:45.810187+0000","last_became_active":"2024-09-17T05:28:43.585273+0000","last_became_peered":"2024-09-17T05:28:43.585273+0000","last_unstale":"2024-09-17T05:28:45.810187+0000","last_undegraded":"2024-09-17T05:28:45.810187+0000","last_fullsized":"2024-09-17T05:28:45.810187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:40.613619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810053+0000","last_change":"2024-09-17T05:28:44.589045+0000","last_active":"2024-09-17T05:28:45.810053+0000","last_peered":"2024-09-17T05:28:45.810053+0000","last_clean":"2024-09-17T05:28:45.810053+0000","last_became_active":"2024-09-17T05:28:44.588476+0000","last_became_peered":"2024-09-17T05:28:44.588476+0000","last_unstale":"2024-09-17T05:28:45.810053+0000","last_undegraded":"2024-09-17T05:28:45.810053+0000","last_fullsized":"2024-09-17T05:28:45.810053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:02.487585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593360+0000","last_change":"2024-09-17T05:28:43.584092+0000","last_active":"2024-09-17T05:28:45.593360+0000","last_peered":"2024-09-17T05:28:45.593360+0000","last_clean":"2024-09-17T05:28:45.593360+0000","last_became_active":"2024-09-17T05:28:43.583206+0000","last_became_peered":"2024-09-17T05:28:43.583206+0000","last_unstale":"2024-09-17T05:28:45.593360+0000","last_undegraded":"2024-09-17T05:28:45.593360+0000","last_fullsized":"2024-09-17T05:28:45.593360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.203097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809620+0000","last_change":"2024-09-17T05:28:44.608143+0000","last_active":"2024-09-17T05:28:45.809620+0000","last_peered":"2024-09-17T05:28:45.809620+0000","last_clean":"2024-09-17T05:28:45.809620+0000","last_became_active":"2024-09-17T05:28:44.607771+0000","last_became_peered":"2024-09-17T05:28:44.607771+0000","last_unstale":"2024-09-17T05:28:45.809620+0000","last_undegraded":"2024-09-17T05:28:45.809620+0000","last_fullsized":"2024-09-17T05:28:45.809620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:16.202353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592599+0000","last_change":"2024-09-17T05:28:43.585492+0000","last_active":"2024-09-17T05:28:45.592599+0000","last_peered":"2024-09-17T05:28:45.592599+0000","last_clean":"2024-09-17T05:28:45.592599+0000","last_became_active":"2024-09-17T05:28:43.585189+0000","last_became_peered":"2024-09-17T05:28:43.585189+0000","last_unstale":"2024-09-17T05:28:45.592599+0000","last_undegraded":"2024-09-17T05:28:45.592599+0000","last_fullsized":"2024-09-17T05:28:45.592599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:47.513074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811028+0000","last_change":"2024-09-17T05:28:44.589403+0000","last_active":"2024-09-17T05:28:45.811028+0000","last_peered":"2024-09-17T05:28:45.811028+0000","last_clean":"2024-09-17T05:28:45.811028+0000","last_became_active":"2024-09-17T05:28:44.589080+0000","last_became_peered":"2024-09-17T05:28:44.589080+0000","last_unstale":"2024-09-17T05:28:45.811028+0000","last_undegraded":"2024-09-17T05:28:45.811028+0000","last_fullsized":"2024-09-17T05:28:45.811028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:41.124043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811067+0000","last_change":"2024-09-17T05:28:43.582311+0000","last_active":"2024-09-17T05:28:45.811067+0000","last_peered":"2024-09-17T05:28:45.811067+0000","last_clean":"2024-09-17T05:28:45.811067+0000","last_became_active":"2024-09-17T05:28:43.581966+0000","last_became_peered":"2024-09-17T05:28:43.581966+0000","last_unstale":"2024-09-17T05:28:45.811067+0000","last_undegraded":"2024-09-17T05:28:45.811067+0000","last_fullsized":"2024-09-17T05:28:45.811067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:28.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593107+0000","last_change":"2024-09-17T05:28:44.596555+0000","last_active":"2024-09-17T05:28:45.593107+0000","last_peered":"2024-09-17T05:28:45.593107+0000","last_clean":"2024-09-17T05:28:45.593107+0000","last_became_active":"2024-09-17T05:28:44.595102+0000","last_became_peered":"2024-09-17T05:28:44.595102+0000","last_unstale":"2024-09-17T05:28:45.593107+0000","last_undegraded":"2024-09-17T05:28:45.593107+0000","last_fullsized":"2024-09-17T05:28:45.593107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:59.574437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592941+0000","last_change":"2024-09-17T05:28:43.574709+0000","last_active":"2024-09-17T05:28:45.592941+0000","last_peered":"2024-09-17T05:28:45.592941+0000","last_clean":"2024-09-17T05:28:45.592941+0000","last_became_active":"2024-09-17T05:28:43.574282+0000","last_became_peered":"2024-09-17T05:28:43.574282+0000","last_unstale":"2024-09-17T05:28:45.592941+0000","last_undegraded":"2024-09-17T05:28:45.592941+0000","last_fullsized":"2024-09-17T05:28:45.592941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:55.092115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593340+0000","last_change":"2024-09-17T05:28:44.594639+0000","last_active":"2024-09-17T05:28:45.593340+0000","last_peered":"2024-09-17T05:28:45.593340+0000","last_clean":"2024-09-17T05:28:45.593340+0000","last_became_active":"2024-09-17T05:28:44.594326+0000","last_became_peered":"2024-09-17T05:28:44.594326+0000","last_unstale":"2024-09-17T05:28:45.593340+0000","last_undegraded":"2024-09-17T05:28:45.593340+0000","last_fullsized":"2024-09-17T05:28:45.593340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:56.099136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810624+0000","last_change":"2024-09-17T05:28:43.583590+0000","last_active":"2024-09-17T05:28:45.810624+0000","last_peered":"2024-09-17T05:28:45.810624+0000","last_clean":"2024-09-17T05:28:45.810624+0000","last_became_active":"2024-09-17T05:28:43.583012+0000","last_became_peered":"2024-09-17T05:28:43.583012+0000","last_unstale":"2024-09-17T05:28:45.810624+0000","last_undegraded":"2024-09-17T05:28:45.810624+0000","last_fullsized":"2024-09-17T05:28:45.810624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:35.082128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591949+0000","last_change":"2024-09-17T05:28:44.586672+0000","last_active":"2024-09-17T05:28:45.591949+0000","last_peered":"2024-09-17T05:28:45.591949+0000","last_clean":"2024-09-17T05:28:45.591949+0000","last_became_active":"2024-09-17T05:28:44.586286+0000","last_became_peered":"2024-09-17T05:28:44.586286+0000","last_unstale":"2024-09-17T05:28:45.591949+0000","last_undegraded":"2024-09-17T05:28:45.591949+0000","last_fullsized":"2024-09-17T05:28:45.591949+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.432831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592035+0000","last_change":"2024-09-17T05:28:43.586381+0000","last_active":"2024-09-17T05:28:45.592035+0000","last_peered":"2024-09-17T05:28:45.592035+0000","last_clean":"2024-09-17T05:28:45.592035+0000","last_became_active":"2024-09-17T05:28:43.586051+0000","last_became_peered":"2024-09-17T05:28:43.586051+0000","last_unstale":"2024-09-17T05:28:45.592035+0000","last_undegraded":"2024-09-17T05:28:45.592035+0000","last_fullsized":"2024-09-17T05:28:45.592035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:52.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810197+0000","last_change":"2024-09-17T05:28:44.610140+0000","last_active":"2024-09-17T05:28:45.810197+0000","last_peered":"2024-09-17T05:28:45.810197+0000","last_clean":"2024-09-17T05:28:45.810197+0000","last_became_active":"2024-09-17T05:28:44.609822+0000","last_became_peered":"2024-09-17T05:28:44.609822+0000","last_unstale":"2024-09-17T05:28:45.810197+0000","last_undegraded":"2024-09-17T05:28:45.810197+0000","last_fullsized":"2024-09-17T05:28:45.810197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:09.487049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593761+0000","last_change":"2024-09-17T05:28:43.572513+0000","last_active":"2024-09-17T05:28:45.593761+0000","last_peered":"2024-09-17T05:28:45.593761+0000","last_clean":"2024-09-17T05:28:45.593761+0000","last_became_active":"2024-09-17T05:28:43.571694+0000","last_became_peered":"2024-09-17T05:28:43.571694+0000","last_unstale":"2024-09-17T05:28:45.593761+0000","last_undegraded":"2024-09-17T05:28:45.593761+0000","last_fullsized":"2024-09-17T05:28:45.593761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:37.493312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812943+0000","last_change":"2024-09-17T05:28:44.593611+0000","last_active":"2024-09-17T05:28:45.812943+0000","last_peered":"2024-09-17T05:28:45.812943+0000","last_clean":"2024-09-17T05:28:45.812943+0000","last_became_active":"2024-09-17T05:28:44.592236+0000","last_became_peered":"2024-09-17T05:28:44.592236+0000","last_unstale":"2024-09-17T05:28:45.812943+0000","last_undegraded":"2024-09-17T05:28:45.812943+0000","last_fullsized":"2024-09-17T05:28:45.812943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:49.586447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811729+0000","last_change":"2024-09-17T05:28:43.581526+0000","last_active":"2024-09-17T05:28:45.811729+0000","last_peered":"2024-09-17T05:28:45.811729+0000","last_clean":"2024-09-17T05:28:45.811729+0000","last_became_active":"2024-09-17T05:28:43.581163+0000","last_became_peered":"2024-09-17T05:28:43.581163+0000","last_unstale":"2024-09-17T05:28:45.811729+0000","last_undegraded":"2024-09-17T05:28:45.811729+0000","last_fullsized":"2024-09-17T05:28:45.811729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:27.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809610+0000","last_change":"2024-09-17T05:28:44.603138+0000","last_active":"2024-09-17T05:28:45.809610+0000","last_peered":"2024-09-17T05:28:45.809610+0000","last_clean":"2024-09-17T05:28:45.809610+0000","last_became_active":"2024-09-17T05:28:44.600619+0000","last_became_peered":"2024-09-17T05:28:44.600619+0000","last_unstale":"2024-09-17T05:28:45.809610+0000","last_undegraded":"2024-09-17T05:28:45.809610+0000","last_fullsized":"2024-09-17T05:28:45.809610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:49.289264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812166+0000","last_change":"2024-09-17T05:28:43.577962+0000","last_active":"2024-09-17T05:28:45.812166+0000","last_peered":"2024-09-17T05:28:45.812166+0000","last_clean":"2024-09-17T05:28:45.812166+0000","last_became_active":"2024-09-17T05:28:43.577642+0000","last_became_peered":"2024-09-17T05:28:43.577642+0000","last_unstale":"2024-09-17T05:28:45.812166+0000","last_undegraded":"2024-09-17T05:28:45.812166+0000","last_fullsized":"2024-09-17T05:28:45.812166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:55.229169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592767+0000","last_change":"2024-09-17T05:28:44.592697+0000","last_active":"2024-09-17T05:28:45.592767+0000","last_peered":"2024-09-17T05:28:45.592767+0000","last_clean":"2024-09-17T05:28:45.592767+0000","last_became_active":"2024-09-17T05:28:44.592403+0000","last_became_peered":"2024-09-17T05:28:44.592403+0000","last_unstale":"2024-09-17T05:28:45.592767+0000","last_undegraded":"2024-09-17T05:28:45.592767+0000","last_fullsized":"2024-09-17T05:28:45.592767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:05.971624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593059+0000","last_change":"2024-09-17T05:28:43.570566+0000","last_active":"2024-09-17T05:28:45.593059+0000","last_peered":"2024-09-17T05:28:45.593059+0000","last_clean":"2024-09-17T05:28:45.593059+0000","last_became_active":"2024-09-17T05:28:43.570239+0000","last_became_peered":"2024-09-17T05:28:43.570239+0000","last_unstale":"2024-09-17T05:28:45.593059+0000","last_undegraded":"2024-09-17T05:28:45.593059+0000","last_fullsized":"2024-09-17T05:28:45.593059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:06.675935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810295+0000","last_change":"2024-09-17T05:28:44.598400+0000","last_active":"2024-09-17T05:28:45.810295+0000","last_peered":"2024-09-17T05:28:45.810295+0000","last_clean":"2024-09-17T05:28:45.810295+0000","last_became_active":"2024-09-17T05:28:44.598073+0000","last_became_peered":"2024-09-17T05:28:44.598073+0000","last_unstale":"2024-09-17T05:28:45.810295+0000","last_undegraded":"2024-09-17T05:28:45.810295+0000","last_fullsized":"2024-09-17T05:28:45.810295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592890+0000","last_change":"2024-09-17T05:28:43.569060+0000","last_active":"2024-09-17T05:28:45.592890+0000","last_peered":"2024-09-17T05:28:45.592890+0000","last_clean":"2024-09-17T05:28:45.592890+0000","last_became_active":"2024-09-17T05:28:43.568732+0000","last_became_peered":"2024-09-17T05:28:43.568732+0000","last_unstale":"2024-09-17T05:28:45.592890+0000","last_undegraded":"2024-09-17T05:28:45.592890+0000","last_fullsized":"2024-09-17T05:28:45.592890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:52.549218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810612+0000","last_change":"2024-09-17T05:28:44.591498+0000","last_active":"2024-09-17T05:28:45.810612+0000","last_peered":"2024-09-17T05:28:45.810612+0000","last_clean":"2024-09-17T05:28:45.810612+0000","last_became_active":"2024-09-17T05:28:44.591186+0000","last_became_peered":"2024-09-17T05:28:44.591186+0000","last_unstale":"2024-09-17T05:28:45.810612+0000","last_undegraded":"2024-09-17T05:28:45.810612+0000","last_fullsized":"2024-09-17T05:28:45.810612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:57.134332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591906+0000","last_change":"2024-09-17T05:28:43.581871+0000","last_active":"2024-09-17T05:28:45.591906+0000","last_peered":"2024-09-17T05:28:45.591906+0000","last_clean":"2024-09-17T05:28:45.591906+0000","last_became_active":"2024-09-17T05:28:43.581398+0000","last_became_peered":"2024-09-17T05:28:43.581398+0000","last_unstale":"2024-09-17T05:28:45.591906+0000","last_undegraded":"2024-09-17T05:28:45.591906+0000","last_fullsized":"2024-09-17T05:28:45.591906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:44.331139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591947+0000","last_change":"2024-09-17T05:28:44.590902+0000","last_active":"2024-09-17T05:28:45.591947+0000","last_peered":"2024-09-17T05:28:45.591947+0000","last_clean":"2024-09-17T05:28:45.591947+0000","last_became_active":"2024-09-17T05:28:44.589574+0000","last_became_peered":"2024-09-17T05:28:44.589574+0000","last_unstale":"2024-09-17T05:28:45.591947+0000","last_undegraded":"2024-09-17T05:28:45.591947+0000","last_fullsized":"2024-09-17T05:28:45.591947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:58.669142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810631+0000","last_change":"2024-09-17T05:28:43.578038+0000","last_active":"2024-09-17T05:28:45.810631+0000","last_peered":"2024-09-17T05:28:45.810631+0000","last_clean":"2024-09-17T05:28:45.810631+0000","last_became_active":"2024-09-17T05:28:43.577245+0000","last_became_peered":"2024-09-17T05:28:43.577245+0000","last_unstale":"2024-09-17T05:28:45.810631+0000","last_undegraded":"2024-09-17T05:28:45.810631+0000","last_fullsized":"2024-09-17T05:28:45.810631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:21.961572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591758+0000","last_change":"2024-09-17T05:28:44.595876+0000","last_active":"2024-09-17T05:28:45.591758+0000","last_peered":"2024-09-17T05:28:45.591758+0000","last_clean":"2024-09-17T05:28:45.591758+0000","last_became_active":"2024-09-17T05:28:44.591932+0000","last_became_peered":"2024-09-17T05:28:44.591932+0000","last_unstale":"2024-09-17T05:28:45.591758+0000","last_undegraded":"2024-09-17T05:28:45.591758+0000","last_fullsized":"2024-09-17T05:28:45.591758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:59.219871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592143+0000","last_change":"2024-09-17T05:28:43.566233+0000","last_active":"2024-09-17T05:28:45.592143+0000","last_peered":"2024-09-17T05:28:45.592143+0000","last_clean":"2024-09-17T05:28:45.592143+0000","last_became_active":"2024-09-17T05:28:43.565869+0000","last_became_peered":"2024-09-17T05:28:43.565869+0000","last_unstale":"2024-09-17T05:28:45.592143+0000","last_undegraded":"2024-09-17T05:28:45.592143+0000","last_fullsized":"2024-09-17T05:28:45.592143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:45.613915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811776+0000","last_change":"2024-09-17T05:28:44.589460+0000","last_active":"2024-09-17T05:28:45.811776+0000","last_peered":"2024-09-17T05:28:45.811776+0000","last_clean":"2024-09-17T05:28:45.811776+0000","last_became_active":"2024-09-17T05:28:44.589134+0000","last_became_peered":"2024-09-17T05:28:44.589134+0000","last_unstale":"2024-09-17T05:28:45.811776+0000","last_undegraded":"2024-09-17T05:28:45.811776+0000","last_fullsized":"2024-09-17T05:28:45.811776+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:28.107602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591341+0000","last_change":"2024-09-17T05:28:43.575603+0000","last_active":"2024-09-17T05:28:45.591341+0000","last_peered":"2024-09-17T05:28:45.591341+0000","last_clean":"2024-09-17T05:28:45.591341+0000","last_became_active":"2024-09-17T05:28:43.575283+0000","last_became_peered":"2024-09-17T05:28:43.575283+0000","last_unstale":"2024-09-17T05:28:45.591341+0000","last_undegraded":"2024-09-17T05:28:45.591341+0000","last_fullsized":"2024-09-17T05:28:45.591341+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:05.142867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811865+0000","last_change":"2024-09-17T05:28:44.588727+0000","last_active":"2024-09-17T05:28:45.811865+0000","last_peered":"2024-09-17T05:28:45.811865+0000","last_clean":"2024-09-17T05:28:45.811865+0000","last_became_active":"2024-09-17T05:28:44.588397+0000","last_became_peered":"2024-09-17T05:28:44.588397+0000","last_unstale":"2024-09-17T05:28:45.811865+0000","last_undegraded":"2024-09-17T05:28:45.811865+0000","last_fullsized":"2024-09-17T05:28:45.811865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:44.683830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810356+0000","last_change":"2024-09-17T05:28:43.587791+0000","last_active":"2024-09-17T05:28:45.810356+0000","last_peered":"2024-09-17T05:28:45.810356+0000","last_clean":"2024-09-17T05:28:45.810356+0000","last_became_active":"2024-09-17T05:28:43.587450+0000","last_became_peered":"2024-09-17T05:28:43.587450+0000","last_unstale":"2024-09-17T05:28:45.810356+0000","last_undegraded":"2024-09-17T05:28:45.810356+0000","last_fullsized":"2024-09-17T05:28:45.810356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:38.784007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812092+0000","last_change":"2024-09-17T05:28:44.590941+0000","last_active":"2024-09-17T05:28:45.812092+0000","last_peered":"2024-09-17T05:28:45.812092+0000","last_clean":"2024-09-17T05:28:45.812092+0000","last_became_active":"2024-09-17T05:28:44.590589+0000","last_became_peered":"2024-09-17T05:28:44.590589+0000","last_unstale":"2024-09-17T05:28:45.812092+0000","last_undegraded":"2024-09-17T05:28:45.812092+0000","last_fullsized":"2024-09-17T05:28:45.812092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:16.499238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809738+0000","last_change":"2024-09-17T05:28:43.583447+0000","last_active":"2024-09-17T05:28:45.809738+0000","last_peered":"2024-09-17T05:28:45.809738+0000","last_clean":"2024-09-17T05:28:45.809738+0000","last_became_active":"2024-09-17T05:28:43.583145+0000","last_became_peered":"2024-09-17T05:28:43.583145+0000","last_unstale":"2024-09-17T05:28:45.809738+0000","last_undegraded":"2024-09-17T05:28:45.809738+0000","last_fullsized":"2024-09-17T05:28:45.809738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:53.903201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811078+0000","last_change":"2024-09-17T05:28:44.589236+0000","last_active":"2024-09-17T05:28:45.811078+0000","last_peered":"2024-09-17T05:28:45.811078+0000","last_clean":"2024-09-17T05:28:45.811078+0000","last_became_active":"2024-09-17T05:28:44.588658+0000","last_became_peered":"2024-09-17T05:28:44.588658+0000","last_unstale":"2024-09-17T05:28:45.811078+0000","last_undegraded":"2024-09-17T05:28:45.811078+0000","last_fullsized":"2024-09-17T05:28:45.811078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591251+0000","last_change":"2024-09-17T05:28:43.564033+0000","last_active":"2024-09-17T05:28:45.591251+0000","last_peered":"2024-09-17T05:28:45.591251+0000","last_clean":"2024-09-17T05:28:45.591251+0000","last_became_active":"2024-09-17T05:28:43.563635+0000","last_became_peered":"2024-09-17T05:28:43.563635+0000","last_unstale":"2024-09-17T05:28:45.591251+0000","last_undegraded":"2024-09-17T05:28:45.591251+0000","last_fullsized":"2024-09-17T05:28:45.591251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:32.152013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593241+0000","last_change":"2024-09-17T05:28:44.586155+0000","last_active":"2024-09-17T05:28:45.593241+0000","last_peered":"2024-09-17T05:28:45.593241+0000","last_clean":"2024-09-17T05:28:45.593241+0000","last_became_active":"2024-09-17T05:28:44.585601+0000","last_became_peered":"2024-09-17T05:28:44.585601+0000","last_unstale":"2024-09-17T05:28:45.593241+0000","last_undegraded":"2024-09-17T05:28:45.593241+0000","last_fullsized":"2024-09-17T05:28:45.593241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:31.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810105+0000","last_change":"2024-09-17T05:28:43.578062+0000","last_active":"2024-09-17T05:28:45.810105+0000","last_peered":"2024-09-17T05:28:45.810105+0000","last_clean":"2024-09-17T05:28:45.810105+0000","last_became_active":"2024-09-17T05:28:43.577664+0000","last_became_peered":"2024-09-17T05:28:43.577664+0000","last_unstale":"2024-09-17T05:28:45.810105+0000","last_undegraded":"2024-09-17T05:28:45.810105+0000","last_fullsized":"2024-09-17T05:28:45.810105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:13.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812673+0000","last_change":"2024-09-17T05:28:44.591578+0000","last_active":"2024-09-17T05:28:45.812673+0000","last_peered":"2024-09-17T05:28:45.812673+0000","last_clean":"2024-09-17T05:28:45.812673+0000","last_became_active":"2024-09-17T05:28:44.591215+0000","last_became_peered":"2024-09-17T05:28:44.591215+0000","last_unstale":"2024-09-17T05:28:45.812673+0000","last_undegraded":"2024-09-17T05:28:45.812673+0000","last_fullsized":"2024-09-17T05:28:45.812673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:43.026057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810764+0000","last_change":"2024-09-17T05:28:43.571945+0000","last_active":"2024-09-17T05:28:45.810764+0000","last_peered":"2024-09-17T05:28:45.810764+0000","last_clean":"2024-09-17T05:28:45.810764+0000","last_became_active":"2024-09-17T05:28:43.571593+0000","last_became_peered":"2024-09-17T05:28:43.571593+0000","last_unstale":"2024-09-17T05:28:45.810764+0000","last_undegraded":"2024-09-17T05:28:45.810764+0000","last_fullsized":"2024-09-17T05:28:45.810764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:27.481016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593342+0000","last_change":"2024-09-17T05:28:44.592820+0000","last_active":"2024-09-17T05:28:45.593342+0000","last_peered":"2024-09-17T05:28:45.593342+0000","last_clean":"2024-09-17T05:28:45.593342+0000","last_became_active":"2024-09-17T05:28:44.592492+0000","last_became_peered":"2024-09-17T05:28:44.592492+0000","last_unstale":"2024-09-17T05:28:45.593342+0000","last_undegraded":"2024-09-17T05:28:45.593342+0000","last_fullsized":"2024-09-17T05:28:45.593342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:06.149889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810821+0000","last_change":"2024-09-17T05:28:43.570857+0000","last_active":"2024-09-17T05:28:45.810821+0000","last_peered":"2024-09-17T05:28:45.810821+0000","last_clean":"2024-09-17T05:28:45.810821+0000","last_became_active":"2024-09-17T05:28:43.570448+0000","last_became_peered":"2024-09-17T05:28:43.570448+0000","last_unstale":"2024-09-17T05:28:45.810821+0000","last_undegraded":"2024-09-17T05:28:45.810821+0000","last_fullsized":"2024-09-17T05:28:45.810821+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:01.094952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591521+0000","last_change":"2024-09-17T05:28:44.609358+0000","last_active":"2024-09-17T05:28:45.591521+0000","last_peered":"2024-09-17T05:28:45.591521+0000","last_clean":"2024-09-17T05:28:45.591521+0000","last_became_active":"2024-09-17T05:28:44.609063+0000","last_became_peered":"2024-09-17T05:28:44.609063+0000","last_unstale":"2024-09-17T05:28:45.591521+0000","last_undegraded":"2024-09-17T05:28:45.591521+0000","last_fullsized":"2024-09-17T05:28:45.591521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.391505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593798+0000","last_change":"2024-09-17T05:28:43.574524+0000","last_active":"2024-09-17T05:28:45.593798+0000","last_peered":"2024-09-17T05:28:45.593798+0000","last_clean":"2024-09-17T05:28:45.593798+0000","last_became_active":"2024-09-17T05:28:43.572192+0000","last_became_peered":"2024-09-17T05:28:43.572192+0000","last_unstale":"2024-09-17T05:28:45.593798+0000","last_undegraded":"2024-09-17T05:28:45.593798+0000","last_fullsized":"2024-09-17T05:28:45.593798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:46.751852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592860+0000","last_change":"2024-09-17T05:28:44.598664+0000","last_active":"2024-09-17T05:28:45.592860+0000","last_peered":"2024-09-17T05:28:45.592860+0000","last_clean":"2024-09-17T05:28:45.592860+0000","last_became_active":"2024-09-17T05:28:44.598325+0000","last_became_peered":"2024-09-17T05:28:44.598325+0000","last_unstale":"2024-09-17T05:28:45.592860+0000","last_undegraded":"2024-09-17T05:28:45.592860+0000","last_fullsized":"2024-09-17T05:28:45.592860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:35.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593598+0000","last_change":"2024-09-17T05:28:43.574129+0000","last_active":"2024-09-17T05:28:45.593598+0000","last_peered":"2024-09-17T05:28:45.593598+0000","last_clean":"2024-09-17T05:28:45.593598+0000","last_became_active":"2024-09-17T05:28:43.573723+0000","last_became_peered":"2024-09-17T05:28:43.573723+0000","last_unstale":"2024-09-17T05:28:45.593598+0000","last_undegraded":"2024-09-17T05:28:45.593598+0000","last_fullsized":"2024-09-17T05:28:45.593598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:46.485943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810964+0000","last_change":"2024-09-17T05:28:44.584324+0000","last_active":"2024-09-17T05:28:45.810964+0000","last_peered":"2024-09-17T05:28:45.810964+0000","last_clean":"2024-09-17T05:28:45.810964+0000","last_became_active":"2024-09-17T05:28:44.583981+0000","last_became_peered":"2024-09-17T05:28:44.583981+0000","last_unstale":"2024-09-17T05:28:45.810964+0000","last_undegraded":"2024-09-17T05:28:45.810964+0000","last_fullsized":"2024-09-17T05:28:45.810964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:46.461741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593440+0000","last_change":"2024-09-17T05:28:43.582630+0000","last_active":"2024-09-17T05:28:45.593440+0000","last_peered":"2024-09-17T05:28:45.593440+0000","last_clean":"2024-09-17T05:28:45.593440+0000","last_became_active":"2024-09-17T05:28:43.582246+0000","last_became_peered":"2024-09-17T05:28:43.582246+0000","last_unstale":"2024-09-17T05:28:45.593440+0000","last_undegraded":"2024-09-17T05:28:45.593440+0000","last_fullsized":"2024-09-17T05:28:45.593440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:56.307551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810067+0000","last_change":"2024-09-17T05:28:44.606764+0000","last_active":"2024-09-17T05:28:45.810067+0000","last_peered":"2024-09-17T05:28:45.810067+0000","last_clean":"2024-09-17T05:28:45.810067+0000","last_became_active":"2024-09-17T05:28:44.606433+0000","last_became_peered":"2024-09-17T05:28:44.606433+0000","last_unstale":"2024-09-17T05:28:45.810067+0000","last_undegraded":"2024-09-17T05:28:45.810067+0000","last_fullsized":"2024-09-17T05:28:45.810067+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:59.606056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811242+0000","last_change":"2024-09-17T05:28:43.583802+0000","last_active":"2024-09-17T05:28:45.811242+0000","last_peered":"2024-09-17T05:28:45.811242+0000","last_clean":"2024-09-17T05:28:45.811242+0000","last_became_active":"2024-09-17T05:28:43.583483+0000","last_became_peered":"2024-09-17T05:28:43.583483+0000","last_unstale":"2024-09-17T05:28:45.811242+0000","last_undegraded":"2024-09-17T05:28:45.811242+0000","last_fullsized":"2024-09-17T05:28:45.811242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:30.524458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811820+0000","last_change":"2024-09-17T05:28:44.596874+0000","last_active":"2024-09-17T05:28:45.811820+0000","last_peered":"2024-09-17T05:28:45.811820+0000","last_clean":"2024-09-17T05:28:45.811820+0000","last_became_active":"2024-09-17T05:28:44.596501+0000","last_became_peered":"2024-09-17T05:28:44.596501+0000","last_unstale":"2024-09-17T05:28:45.811820+0000","last_undegraded":"2024-09-17T05:28:45.811820+0000","last_fullsized":"2024-09-17T05:28:45.811820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:27.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591364+0000","last_change":"2024-09-17T05:28:43.584354+0000","last_active":"2024-09-17T05:28:45.591364+0000","last_peered":"2024-09-17T05:28:45.591364+0000","last_clean":"2024-09-17T05:28:45.591364+0000","last_became_active":"2024-09-17T05:28:43.583900+0000","last_became_peered":"2024-09-17T05:28:43.583900+0000","last_unstale":"2024-09-17T05:28:45.591364+0000","last_undegraded":"2024-09-17T05:28:45.591364+0000","last_fullsized":"2024-09-17T05:28:45.591364+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:46.826167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591063+0000","last_change":"2024-09-17T05:28:44.602138+0000","last_active":"2024-09-17T05:28:45.591063+0000","last_peered":"2024-09-17T05:28:45.591063+0000","last_clean":"2024-09-17T05:28:45.591063+0000","last_became_active":"2024-09-17T05:28:44.601237+0000","last_became_peered":"2024-09-17T05:28:44.601237+0000","last_unstale":"2024-09-17T05:28:45.591063+0000","last_undegraded":"2024-09-17T05:28:45.591063+0000","last_fullsized":"2024-09-17T05:28:45.591063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:58.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809351+0000","last_change":"2024-09-17T05:28:43.582802+0000","last_active":"2024-09-17T05:28:45.809351+0000","last_peered":"2024-09-17T05:28:45.809351+0000","last_clean":"2024-09-17T05:28:45.809351+0000","last_became_active":"2024-09-17T05:28:43.582317+0000","last_became_peered":"2024-09-17T05:28:43.582317+0000","last_unstale":"2024-09-17T05:28:45.809351+0000","last_undegraded":"2024-09-17T05:28:45.809351+0000","last_fullsized":"2024-09-17T05:28:45.809351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:27.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810681+0000","last_change":"2024-09-17T05:28:44.587771+0000","last_active":"2024-09-17T05:28:45.810681+0000","last_peered":"2024-09-17T05:28:45.810681+0000","last_clean":"2024-09-17T05:28:45.810681+0000","last_became_active":"2024-09-17T05:28:44.587461+0000","last_became_peered":"2024-09-17T05:28:44.587461+0000","last_unstale":"2024-09-17T05:28:45.810681+0000","last_undegraded":"2024-09-17T05:28:45.810681+0000","last_fullsized":"2024-09-17T05:28:45.810681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:17.392846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592943+0000","last_change":"2024-09-17T05:28:43.565393+0000","last_active":"2024-09-17T05:28:45.592943+0000","last_peered":"2024-09-17T05:28:45.592943+0000","last_clean":"2024-09-17T05:28:45.592943+0000","last_became_active":"2024-09-17T05:28:43.565048+0000","last_became_peered":"2024-09-17T05:28:43.565048+0000","last_unstale":"2024-09-17T05:28:45.592943+0000","last_undegraded":"2024-09-17T05:28:45.592943+0000","last_fullsized":"2024-09-17T05:28:45.592943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:59.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592388+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:28:45.592388+0000","last_peered":"2024-09-17T05:28:45.592388+0000","last_clean":"2024-09-17T05:28:45.592388+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:28:45.592388+0000","last_undegraded":"2024-09-17T05:28:45.592388+0000","last_fullsized":"2024-09-17T05:28:45.592388+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":917504,"data_stored":387116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":975962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":386198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":385630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":909312,"data_stored":386086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":975872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":909312,"data_stored":385574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":387150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:51.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:51.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:53.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:53.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 -- 172.21.15.43:0/2687529512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9294003af0 msgr2=0x7f9294003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 --2- 172.21.15.43:0/2687529512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9294003af0 0x7f9294003f60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c470 tx=0x7f92a001cfa0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92af400640 1 -- 172.21.15.43:0/2687529512 reap_dead start 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 -- 172.21.15.43:0/2687529512 shutdown_connections 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 -- 172.21.15.43:0/2687529512 >> 172.21.15.43:0/2687529512 conn(0x7f92b00b23d0 msgr2=0x7f92b00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 -- 172.21.15.43:0/2687529512 shutdown_connections 2024-09-17T05:28:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.642+0000 7f92b5c00640 1 -- 172.21.15.43:0/2687529512 wait complete. 2024-09-17T05:28:53.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 Processor -- start 2024-09-17T05:28:53.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- start start 2024-09-17T05:28:53.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9294003af0 0x7f92b0153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92b0151ed0 0x7f92b01f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0167ec0 con 0x7f92b01f9e10 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0167d40 con 0x7f9294003af0 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92b0168040 con 0x7f92b0151ed0 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37304/0 (socket says 172.21.15.43:37304) 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 -- 172.21.15.43:0/3246483909 learned_addr learned my addr 172.21.15.43:0/3246483909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92aea00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9294003af0 0x7f92b0153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92a6000640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92b0151ed0 0x7f92b01f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 -- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92b0151ed0 msgr2=0x7f92b01f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92b0151ed0 0x7f92b01f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 -- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9294003af0 msgr2=0x7f92b0153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9294003af0 0x7f92b0153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 -- 172.21.15.43:0/3246483909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b0150e30 con 0x7f92b01f9e10 2024-09-17T05:28:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92af400640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f92b00b3c20 tx=0x7f92a8009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a800fce0 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92a800fe80 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8004d50 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b0151110 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.646+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01516d0 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b00700a0 con 0x7f92b01f9e10 2024-09-17T05:28:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92a8020020 con 0x7f92b01f9e10 2024-09-17T05:28:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92a7400640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 0x7f928c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f92a80948e0 con 0x7f92b01f9e10 2024-09-17T05:28:53.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92aea00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 0x7f928c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:53.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.650+0000 7f92aea00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 0x7f928c075710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f92a00ac000 tx=0x7f92a0019de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:53.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.654+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92a805f290 con 0x7f92b01f9e10 2024-09-17T05:28:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.830+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92b01540b0 con 0x7f928c073200 2024-09-17T05:28:53.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.834+0000 7f92a7400640 1 -- 172.21.15.43:0/3246483909 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252061 (secure 0 0 0) 0x7f92b01540b0 con 0x7f928c073200 2024-09-17T05:28:53.839 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:53.841 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:28:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 msgr2=0x7f928c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 0x7f928c075710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f92a00ac000 tx=0x7f92a0019de0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 msgr2=0x7f92b0164790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f92b00b3c20 tx=0x7f92a8009200 comp rx=0 tx=0).stop 2024-09-17T05:28:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 shutdown_connections 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f928c073200 0x7f928c075710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b01f9e10 0x7f92b0164790 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f92b0151ed0 0x7f92b01f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 --2- 172.21.15.43:0/3246483909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9294003af0 0x7f92b0153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 >> 172.21.15.43:0/3246483909 conn(0x7f92b00b23d0 msgr2=0x7f92b015efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 shutdown_connections 2024-09-17T05:28:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:53.838+0000 7f92b5c00640 1 -- 172.21.15.43:0/3246483909 wait complete. 2024-09-17T05:28:53.861 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-17T05:28:51.929350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257832,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716888,"statfs":{"total":773094113280,"available":772830093312,"internally_reserved":0,"allocated":8495104,"data_stored":4269588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002526"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810845+0000","last_change":"2024-09-17T05:28:44.585647+0000","last_active":"2024-09-17T05:28:45.810845+0000","last_peered":"2024-09-17T05:28:45.810845+0000","last_clean":"2024-09-17T05:28:45.810845+0000","last_became_active":"2024-09-17T05:28:44.585156+0000","last_became_peered":"2024-09-17T05:28:44.585156+0000","last_unstale":"2024-09-17T05:28:45.810845+0000","last_undegraded":"2024-09-17T05:28:45.810845+0000","last_fullsized":"2024-09-17T05:28:45.810845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:57.759116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591719+0000","last_change":"2024-09-17T05:28:44.615499+0000","last_active":"2024-09-17T05:28:45.591719+0000","last_peered":"2024-09-17T05:28:45.591719+0000","last_clean":"2024-09-17T05:28:45.591719+0000","last_became_active":"2024-09-17T05:28:44.615166+0000","last_became_peered":"2024-09-17T05:28:44.615166+0000","last_unstale":"2024-09-17T05:28:45.591719+0000","last_undegraded":"2024-09-17T05:28:45.591719+0000","last_fullsized":"2024-09-17T05:28:45.591719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:21.247892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811543+0000","last_change":"2024-09-17T05:28:44.601689+0000","last_active":"2024-09-17T05:28:45.811543+0000","last_peered":"2024-09-17T05:28:45.811543+0000","last_clean":"2024-09-17T05:28:45.811543+0000","last_became_active":"2024-09-17T05:28:44.601350+0000","last_became_peered":"2024-09-17T05:28:44.601350+0000","last_unstale":"2024-09-17T05:28:45.811543+0000","last_undegraded":"2024-09-17T05:28:45.811543+0000","last_fullsized":"2024-09-17T05:28:45.811543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:31.596997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811435+0000","last_change":"2024-09-17T05:28:44.600102+0000","last_active":"2024-09-17T05:28:45.811435+0000","last_peered":"2024-09-17T05:28:45.811435+0000","last_clean":"2024-09-17T05:28:45.811435+0000","last_became_active":"2024-09-17T05:28:44.599793+0000","last_became_peered":"2024-09-17T05:28:44.599793+0000","last_unstale":"2024-09-17T05:28:45.811435+0000","last_undegraded":"2024-09-17T05:28:45.811435+0000","last_fullsized":"2024-09-17T05:28:45.811435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:33.495121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810521+0000","last_change":"2024-09-17T05:28:44.594481+0000","last_active":"2024-09-17T05:28:45.810521+0000","last_peered":"2024-09-17T05:28:45.810521+0000","last_clean":"2024-09-17T05:28:45.810521+0000","last_became_active":"2024-09-17T05:28:44.594116+0000","last_became_peered":"2024-09-17T05:28:44.594116+0000","last_unstale":"2024-09-17T05:28:45.810521+0000","last_undegraded":"2024-09-17T05:28:45.810521+0000","last_fullsized":"2024-09-17T05:28:45.810521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:49:57.919125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809344+0000","last_change":"2024-09-17T05:28:44.608063+0000","last_active":"2024-09-17T05:28:45.809344+0000","last_peered":"2024-09-17T05:28:45.809344+0000","last_clean":"2024-09-17T05:28:45.809344+0000","last_became_active":"2024-09-17T05:28:44.607088+0000","last_became_peered":"2024-09-17T05:28:44.607088+0000","last_unstale":"2024-09-17T05:28:45.809344+0000","last_undegraded":"2024-09-17T05:28:45.809344+0000","last_fullsized":"2024-09-17T05:28:45.809344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:10.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592697+0000","last_change":"2024-09-17T05:28:44.588931+0000","last_active":"2024-09-17T05:28:45.592697+0000","last_peered":"2024-09-17T05:28:45.592697+0000","last_clean":"2024-09-17T05:28:45.592697+0000","last_became_active":"2024-09-17T05:28:44.588583+0000","last_became_peered":"2024-09-17T05:28:44.588583+0000","last_unstale":"2024-09-17T05:28:45.592697+0000","last_undegraded":"2024-09-17T05:28:45.592697+0000","last_fullsized":"2024-09-17T05:28:45.592697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:09.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809448+0000","last_change":"2024-09-17T05:28:44.594090+0000","last_active":"2024-09-17T05:28:45.809448+0000","last_peered":"2024-09-17T05:28:45.809448+0000","last_clean":"2024-09-17T05:28:45.809448+0000","last_became_active":"2024-09-17T05:28:44.593306+0000","last_became_peered":"2024-09-17T05:28:44.593306+0000","last_unstale":"2024-09-17T05:28:45.809448+0000","last_undegraded":"2024-09-17T05:28:45.809448+0000","last_fullsized":"2024-09-17T05:28:45.809448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:07.325509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812113+0000","last_change":"2024-09-17T05:28:44.592878+0000","last_active":"2024-09-17T05:28:45.812113+0000","last_peered":"2024-09-17T05:28:45.812113+0000","last_clean":"2024-09-17T05:28:45.812113+0000","last_became_active":"2024-09-17T05:28:44.592502+0000","last_became_peered":"2024-09-17T05:28:44.592502+0000","last_unstale":"2024-09-17T05:28:45.812113+0000","last_undegraded":"2024-09-17T05:28:45.812113+0000","last_fullsized":"2024-09-17T05:28:45.812113+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:04.179376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593333+0000","last_change":"2024-09-17T05:28:44.582300+0000","last_active":"2024-09-17T05:28:45.593333+0000","last_peered":"2024-09-17T05:28:45.593333+0000","last_clean":"2024-09-17T05:28:45.593333+0000","last_became_active":"2024-09-17T05:28:44.581837+0000","last_became_peered":"2024-09-17T05:28:44.581837+0000","last_unstale":"2024-09-17T05:28:45.593333+0000","last_undegraded":"2024-09-17T05:28:45.593333+0000","last_fullsized":"2024-09-17T05:28:45.593333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:12.838918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810610+0000","last_change":"2024-09-17T05:28:44.601361+0000","last_active":"2024-09-17T05:28:45.810610+0000","last_peered":"2024-09-17T05:28:45.810610+0000","last_clean":"2024-09-17T05:28:45.810610+0000","last_became_active":"2024-09-17T05:28:44.601051+0000","last_became_peered":"2024-09-17T05:28:44.601051+0000","last_unstale":"2024-09-17T05:28:45.810610+0000","last_undegraded":"2024-09-17T05:28:45.810610+0000","last_fullsized":"2024-09-17T05:28:45.810610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:28.681348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593653+0000","last_change":"2024-09-17T05:28:44.600143+0000","last_active":"2024-09-17T05:28:45.593653+0000","last_peered":"2024-09-17T05:28:45.593653+0000","last_clean":"2024-09-17T05:28:45.593653+0000","last_became_active":"2024-09-17T05:28:44.599889+0000","last_became_peered":"2024-09-17T05:28:44.599889+0000","last_unstale":"2024-09-17T05:28:45.593653+0000","last_undegraded":"2024-09-17T05:28:45.593653+0000","last_fullsized":"2024-09-17T05:28:45.593653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:18.480582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811648+0000","last_change":"2024-09-17T05:28:44.597555+0000","last_active":"2024-09-17T05:28:45.811648+0000","last_peered":"2024-09-17T05:28:45.811648+0000","last_clean":"2024-09-17T05:28:45.811648+0000","last_became_active":"2024-09-17T05:28:44.597223+0000","last_became_peered":"2024-09-17T05:28:44.597223+0000","last_unstale":"2024-09-17T05:28:45.811648+0000","last_undegraded":"2024-09-17T05:28:45.811648+0000","last_fullsized":"2024-09-17T05:28:45.811648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:43.205669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593376+0000","last_change":"2024-09-17T05:28:44.601367+0000","last_active":"2024-09-17T05:28:45.593376+0000","last_peered":"2024-09-17T05:28:45.593376+0000","last_clean":"2024-09-17T05:28:45.593376+0000","last_became_active":"2024-09-17T05:28:44.601017+0000","last_became_peered":"2024-09-17T05:28:44.601017+0000","last_unstale":"2024-09-17T05:28:45.593376+0000","last_undegraded":"2024-09-17T05:28:45.593376+0000","last_fullsized":"2024-09-17T05:28:45.593376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:58.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592936+0000","last_change":"2024-09-17T05:28:44.587138+0000","last_active":"2024-09-17T05:28:45.592936+0000","last_peered":"2024-09-17T05:28:45.592936+0000","last_clean":"2024-09-17T05:28:45.592936+0000","last_became_active":"2024-09-17T05:28:44.586743+0000","last_became_peered":"2024-09-17T05:28:44.586743+0000","last_unstale":"2024-09-17T05:28:45.592936+0000","last_undegraded":"2024-09-17T05:28:45.592936+0000","last_fullsized":"2024-09-17T05:28:45.592936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:34.211101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809498+0000","last_change":"2024-09-17T05:28:44.594461+0000","last_active":"2024-09-17T05:28:45.809498+0000","last_peered":"2024-09-17T05:28:45.809498+0000","last_clean":"2024-09-17T05:28:45.809498+0000","last_became_active":"2024-09-17T05:28:44.593769+0000","last_became_peered":"2024-09-17T05:28:44.593769+0000","last_unstale":"2024-09-17T05:28:45.809498+0000","last_undegraded":"2024-09-17T05:28:45.809498+0000","last_fullsized":"2024-09-17T05:28:45.809498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:23.684881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812532+0000","last_change":"2024-09-17T05:28:44.581904+0000","last_active":"2024-09-17T05:28:45.812532+0000","last_peered":"2024-09-17T05:28:45.812532+0000","last_clean":"2024-09-17T05:28:45.812532+0000","last_became_active":"2024-09-17T05:28:44.581483+0000","last_became_peered":"2024-09-17T05:28:44.581483+0000","last_unstale":"2024-09-17T05:28:45.812532+0000","last_undegraded":"2024-09-17T05:28:45.812532+0000","last_fullsized":"2024-09-17T05:28:45.812532+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:00.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812514+0000","last_change":"2024-09-17T05:28:44.591284+0000","last_active":"2024-09-17T05:28:45.812514+0000","last_peered":"2024-09-17T05:28:45.812514+0000","last_clean":"2024-09-17T05:28:45.812514+0000","last_became_active":"2024-09-17T05:28:44.590694+0000","last_became_peered":"2024-09-17T05:28:44.590694+0000","last_unstale":"2024-09-17T05:28:45.812514+0000","last_undegraded":"2024-09-17T05:28:45.812514+0000","last_fullsized":"2024-09-17T05:28:45.812514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:24.404477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810375+0000","last_change":"2024-09-17T05:28:44.603832+0000","last_active":"2024-09-17T05:28:45.810375+0000","last_peered":"2024-09-17T05:28:45.810375+0000","last_clean":"2024-09-17T05:28:45.810375+0000","last_became_active":"2024-09-17T05:28:44.603514+0000","last_became_peered":"2024-09-17T05:28:44.603514+0000","last_unstale":"2024-09-17T05:28:45.810375+0000","last_undegraded":"2024-09-17T05:28:45.810375+0000","last_fullsized":"2024-09-17T05:28:45.810375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:40.109889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810906+0000","last_change":"2024-09-17T05:28:44.598317+0000","last_active":"2024-09-17T05:28:45.810906+0000","last_peered":"2024-09-17T05:28:45.810906+0000","last_clean":"2024-09-17T05:28:45.810906+0000","last_became_active":"2024-09-17T05:28:44.597940+0000","last_became_peered":"2024-09-17T05:28:44.597940+0000","last_unstale":"2024-09-17T05:28:45.810906+0000","last_undegraded":"2024-09-17T05:28:45.810906+0000","last_fullsized":"2024-09-17T05:28:45.810906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:46.299125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591414+0000","last_change":"2024-09-17T05:28:44.587053+0000","last_active":"2024-09-17T05:28:45.591414+0000","last_peered":"2024-09-17T05:28:45.591414+0000","last_clean":"2024-09-17T05:28:45.591414+0000","last_became_active":"2024-09-17T05:28:44.586027+0000","last_became_peered":"2024-09-17T05:28:44.586027+0000","last_unstale":"2024-09-17T05:28:45.591414+0000","last_undegraded":"2024-09-17T05:28:45.591414+0000","last_fullsized":"2024-09-17T05:28:45.591414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:45.342193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809354+0000","last_change":"2024-09-17T05:28:44.607483+0000","last_active":"2024-09-17T05:28:45.809354+0000","last_peered":"2024-09-17T05:28:45.809354+0000","last_clean":"2024-09-17T05:28:45.809354+0000","last_became_active":"2024-09-17T05:28:44.606545+0000","last_became_peered":"2024-09-17T05:28:44.606545+0000","last_unstale":"2024-09-17T05:28:45.809354+0000","last_undegraded":"2024-09-17T05:28:45.809354+0000","last_fullsized":"2024-09-17T05:28:45.809354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:45.342193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592891+0000","last_change":"2024-09-17T05:28:44.597731+0000","last_active":"2024-09-17T05:28:45.592891+0000","last_peered":"2024-09-17T05:28:45.592891+0000","last_clean":"2024-09-17T05:28:45.592891+0000","last_became_active":"2024-09-17T05:28:44.594237+0000","last_became_peered":"2024-09-17T05:28:44.594237+0000","last_unstale":"2024-09-17T05:28:45.592891+0000","last_undegraded":"2024-09-17T05:28:45.592891+0000","last_fullsized":"2024-09-17T05:28:45.592891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:00.360443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809704+0000","last_change":"2024-09-17T05:28:44.602937+0000","last_active":"2024-09-17T05:28:45.809704+0000","last_peered":"2024-09-17T05:28:45.809704+0000","last_clean":"2024-09-17T05:28:45.809704+0000","last_became_active":"2024-09-17T05:28:44.600659+0000","last_became_peered":"2024-09-17T05:28:44.600659+0000","last_unstale":"2024-09-17T05:28:45.809704+0000","last_undegraded":"2024-09-17T05:28:45.809704+0000","last_fullsized":"2024-09-17T05:28:45.809704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:41.910571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593235+0000","last_change":"2024-09-17T05:28:44.596922+0000","last_active":"2024-09-17T05:28:45.593235+0000","last_peered":"2024-09-17T05:28:45.593235+0000","last_clean":"2024-09-17T05:28:45.593235+0000","last_became_active":"2024-09-17T05:28:44.596605+0000","last_became_peered":"2024-09-17T05:28:44.596605+0000","last_unstale":"2024-09-17T05:28:45.593235+0000","last_undegraded":"2024-09-17T05:28:45.593235+0000","last_fullsized":"2024-09-17T05:28:45.593235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:45.032755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810639+0000","last_change":"2024-09-17T05:28:44.611062+0000","last_active":"2024-09-17T05:28:45.810639+0000","last_peered":"2024-09-17T05:28:45.810639+0000","last_clean":"2024-09-17T05:28:45.810639+0000","last_became_active":"2024-09-17T05:28:44.610718+0000","last_became_peered":"2024-09-17T05:28:44.610718+0000","last_unstale":"2024-09-17T05:28:45.810639+0000","last_undegraded":"2024-09-17T05:28:45.810639+0000","last_fullsized":"2024-09-17T05:28:45.810639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:41.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810540+0000","last_change":"2024-09-17T05:28:44.592724+0000","last_active":"2024-09-17T05:28:45.810540+0000","last_peered":"2024-09-17T05:28:45.810540+0000","last_clean":"2024-09-17T05:28:45.810540+0000","last_became_active":"2024-09-17T05:28:44.592396+0000","last_became_peered":"2024-09-17T05:28:44.592396+0000","last_unstale":"2024-09-17T05:28:45.810540+0000","last_undegraded":"2024-09-17T05:28:45.810540+0000","last_fullsized":"2024-09-17T05:28:45.810540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:38.774253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591282+0000","last_change":"2024-09-17T05:28:44.605090+0000","last_active":"2024-09-17T05:28:45.591282+0000","last_peered":"2024-09-17T05:28:45.591282+0000","last_clean":"2024-09-17T05:28:45.591282+0000","last_became_active":"2024-09-17T05:28:44.600942+0000","last_became_peered":"2024-09-17T05:28:44.600942+0000","last_unstale":"2024-09-17T05:28:45.591282+0000","last_undegraded":"2024-09-17T05:28:45.591282+0000","last_fullsized":"2024-09-17T05:28:45.591282+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:18.480582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811923+0000","last_change":"2024-09-17T05:28:44.593442+0000","last_active":"2024-09-17T05:28:45.811923+0000","last_peered":"2024-09-17T05:28:45.811923+0000","last_clean":"2024-09-17T05:28:45.811923+0000","last_became_active":"2024-09-17T05:28:44.591992+0000","last_became_peered":"2024-09-17T05:28:44.591992+0000","last_unstale":"2024-09-17T05:28:45.811923+0000","last_undegraded":"2024-09-17T05:28:45.811923+0000","last_fullsized":"2024-09-17T05:28:45.811923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:19.154460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809806+0000","last_change":"2024-09-17T05:28:44.605342+0000","last_active":"2024-09-17T05:28:45.809806+0000","last_peered":"2024-09-17T05:28:45.809806+0000","last_clean":"2024-09-17T05:28:45.809806+0000","last_became_active":"2024-09-17T05:28:44.605005+0000","last_became_peered":"2024-09-17T05:28:44.605005+0000","last_unstale":"2024-09-17T05:28:45.809806+0000","last_undegraded":"2024-09-17T05:28:45.809806+0000","last_fullsized":"2024-09-17T05:28:45.809806+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:26.282603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591998+0000","last_change":"2024-09-17T05:28:44.601760+0000","last_active":"2024-09-17T05:28:45.591998+0000","last_peered":"2024-09-17T05:28:45.591998+0000","last_clean":"2024-09-17T05:28:45.591998+0000","last_became_active":"2024-09-17T05:28:44.592070+0000","last_became_peered":"2024-09-17T05:28:44.592070+0000","last_unstale":"2024-09-17T05:28:45.591998+0000","last_undegraded":"2024-09-17T05:28:45.591998+0000","last_fullsized":"2024-09-17T05:28:45.591998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:32.842755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591115+0000","last_change":"2024-09-17T05:28:44.599429+0000","last_active":"2024-09-17T05:28:45.591115+0000","last_peered":"2024-09-17T05:28:45.591115+0000","last_clean":"2024-09-17T05:28:45.591115+0000","last_became_active":"2024-09-17T05:28:44.599126+0000","last_became_peered":"2024-09-17T05:28:44.599126+0000","last_unstale":"2024-09-17T05:28:45.591115+0000","last_undegraded":"2024-09-17T05:28:45.591115+0000","last_fullsized":"2024-09-17T05:28:45.591115+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:56.568047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810639+0000","last_change":"2024-09-17T05:28:44.593467+0000","last_active":"2024-09-17T05:28:45.810639+0000","last_peered":"2024-09-17T05:28:45.810639+0000","last_clean":"2024-09-17T05:28:45.810639+0000","last_became_active":"2024-09-17T05:28:44.593107+0000","last_became_peered":"2024-09-17T05:28:44.593107+0000","last_unstale":"2024-09-17T05:28:45.810639+0000","last_undegraded":"2024-09-17T05:28:45.810639+0000","last_fullsized":"2024-09-17T05:28:45.810639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:25.470919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810183+0000","last_change":"2024-09-17T05:28:43.588925+0000","last_active":"2024-09-17T05:28:45.810183+0000","last_peered":"2024-09-17T05:28:45.810183+0000","last_clean":"2024-09-17T05:28:45.810183+0000","last_became_active":"2024-09-17T05:28:43.588563+0000","last_became_peered":"2024-09-17T05:28:43.588563+0000","last_unstale":"2024-09-17T05:28:45.810183+0000","last_undegraded":"2024-09-17T05:28:45.810183+0000","last_fullsized":"2024-09-17T05:28:45.810183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:56.307551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812013+0000","last_change":"2024-09-17T05:28:44.597058+0000","last_active":"2024-09-17T05:28:45.812013+0000","last_peered":"2024-09-17T05:28:45.812013+0000","last_clean":"2024-09-17T05:28:45.812013+0000","last_became_active":"2024-09-17T05:28:44.596741+0000","last_became_peered":"2024-09-17T05:28:44.596741+0000","last_unstale":"2024-09-17T05:28:45.812013+0000","last_undegraded":"2024-09-17T05:28:45.812013+0000","last_fullsized":"2024-09-17T05:28:45.812013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:03.433370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812757+0000","last_change":"2024-09-17T05:28:43.581450+0000","last_active":"2024-09-17T05:28:45.812757+0000","last_peered":"2024-09-17T05:28:45.812757+0000","last_clean":"2024-09-17T05:28:45.812757+0000","last_became_active":"2024-09-17T05:28:43.581046+0000","last_became_peered":"2024-09-17T05:28:43.581046+0000","last_unstale":"2024-09-17T05:28:45.812757+0000","last_undegraded":"2024-09-17T05:28:45.812757+0000","last_fullsized":"2024-09-17T05:28:45.812757+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:41.396278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810307+0000","last_change":"2024-09-17T05:28:44.596042+0000","last_active":"2024-09-17T05:28:45.810307+0000","last_peered":"2024-09-17T05:28:45.810307+0000","last_clean":"2024-09-17T05:28:45.810307+0000","last_became_active":"2024-09-17T05:28:44.595246+0000","last_became_peered":"2024-09-17T05:28:44.595246+0000","last_unstale":"2024-09-17T05:28:45.810307+0000","last_undegraded":"2024-09-17T05:28:45.810307+0000","last_fullsized":"2024-09-17T05:28:45.810307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.432831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810574+0000","last_change":"2024-09-17T05:28:43.584806+0000","last_active":"2024-09-17T05:28:45.810574+0000","last_peered":"2024-09-17T05:28:45.810574+0000","last_clean":"2024-09-17T05:28:45.810574+0000","last_became_active":"2024-09-17T05:28:43.584417+0000","last_became_peered":"2024-09-17T05:28:43.584417+0000","last_unstale":"2024-09-17T05:28:45.810574+0000","last_undegraded":"2024-09-17T05:28:45.810574+0000","last_fullsized":"2024-09-17T05:28:45.810574+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:43.980866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592815+0000","last_change":"2024-09-17T05:28:44.597783+0000","last_active":"2024-09-17T05:28:45.592815+0000","last_peered":"2024-09-17T05:28:45.592815+0000","last_clean":"2024-09-17T05:28:45.592815+0000","last_became_active":"2024-09-17T05:28:44.597414+0000","last_became_peered":"2024-09-17T05:28:44.597414+0000","last_unstale":"2024-09-17T05:28:45.592815+0000","last_undegraded":"2024-09-17T05:28:45.592815+0000","last_fullsized":"2024-09-17T05:28:45.592815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:00.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810187+0000","last_change":"2024-09-17T05:28:43.585654+0000","last_active":"2024-09-17T05:28:45.810187+0000","last_peered":"2024-09-17T05:28:45.810187+0000","last_clean":"2024-09-17T05:28:45.810187+0000","last_became_active":"2024-09-17T05:28:43.585273+0000","last_became_peered":"2024-09-17T05:28:43.585273+0000","last_unstale":"2024-09-17T05:28:45.810187+0000","last_undegraded":"2024-09-17T05:28:45.810187+0000","last_fullsized":"2024-09-17T05:28:45.810187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:40.613619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810053+0000","last_change":"2024-09-17T05:28:44.589045+0000","last_active":"2024-09-17T05:28:45.810053+0000","last_peered":"2024-09-17T05:28:45.810053+0000","last_clean":"2024-09-17T05:28:45.810053+0000","last_became_active":"2024-09-17T05:28:44.588476+0000","last_became_peered":"2024-09-17T05:28:44.588476+0000","last_unstale":"2024-09-17T05:28:45.810053+0000","last_undegraded":"2024-09-17T05:28:45.810053+0000","last_fullsized":"2024-09-17T05:28:45.810053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:02.487585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593360+0000","last_change":"2024-09-17T05:28:43.584092+0000","last_active":"2024-09-17T05:28:45.593360+0000","last_peered":"2024-09-17T05:28:45.593360+0000","last_clean":"2024-09-17T05:28:45.593360+0000","last_became_active":"2024-09-17T05:28:43.583206+0000","last_became_peered":"2024-09-17T05:28:43.583206+0000","last_unstale":"2024-09-17T05:28:45.593360+0000","last_undegraded":"2024-09-17T05:28:45.593360+0000","last_fullsized":"2024-09-17T05:28:45.593360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.203097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809620+0000","last_change":"2024-09-17T05:28:44.608143+0000","last_active":"2024-09-17T05:28:45.809620+0000","last_peered":"2024-09-17T05:28:45.809620+0000","last_clean":"2024-09-17T05:28:45.809620+0000","last_became_active":"2024-09-17T05:28:44.607771+0000","last_became_peered":"2024-09-17T05:28:44.607771+0000","last_unstale":"2024-09-17T05:28:45.809620+0000","last_undegraded":"2024-09-17T05:28:45.809620+0000","last_fullsized":"2024-09-17T05:28:45.809620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:16.202353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592599+0000","last_change":"2024-09-17T05:28:43.585492+0000","last_active":"2024-09-17T05:28:45.592599+0000","last_peered":"2024-09-17T05:28:45.592599+0000","last_clean":"2024-09-17T05:28:45.592599+0000","last_became_active":"2024-09-17T05:28:43.585189+0000","last_became_peered":"2024-09-17T05:28:43.585189+0000","last_unstale":"2024-09-17T05:28:45.592599+0000","last_undegraded":"2024-09-17T05:28:45.592599+0000","last_fullsized":"2024-09-17T05:28:45.592599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:47.513074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811028+0000","last_change":"2024-09-17T05:28:44.589403+0000","last_active":"2024-09-17T05:28:45.811028+0000","last_peered":"2024-09-17T05:28:45.811028+0000","last_clean":"2024-09-17T05:28:45.811028+0000","last_became_active":"2024-09-17T05:28:44.589080+0000","last_became_peered":"2024-09-17T05:28:44.589080+0000","last_unstale":"2024-09-17T05:28:45.811028+0000","last_undegraded":"2024-09-17T05:28:45.811028+0000","last_fullsized":"2024-09-17T05:28:45.811028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:41.124043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811067+0000","last_change":"2024-09-17T05:28:43.582311+0000","last_active":"2024-09-17T05:28:45.811067+0000","last_peered":"2024-09-17T05:28:45.811067+0000","last_clean":"2024-09-17T05:28:45.811067+0000","last_became_active":"2024-09-17T05:28:43.581966+0000","last_became_peered":"2024-09-17T05:28:43.581966+0000","last_unstale":"2024-09-17T05:28:45.811067+0000","last_undegraded":"2024-09-17T05:28:45.811067+0000","last_fullsized":"2024-09-17T05:28:45.811067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:28.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593107+0000","last_change":"2024-09-17T05:28:44.596555+0000","last_active":"2024-09-17T05:28:45.593107+0000","last_peered":"2024-09-17T05:28:45.593107+0000","last_clean":"2024-09-17T05:28:45.593107+0000","last_became_active":"2024-09-17T05:28:44.595102+0000","last_became_peered":"2024-09-17T05:28:44.595102+0000","last_unstale":"2024-09-17T05:28:45.593107+0000","last_undegraded":"2024-09-17T05:28:45.593107+0000","last_fullsized":"2024-09-17T05:28:45.593107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:59.574437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592941+0000","last_change":"2024-09-17T05:28:43.574709+0000","last_active":"2024-09-17T05:28:45.592941+0000","last_peered":"2024-09-17T05:28:45.592941+0000","last_clean":"2024-09-17T05:28:45.592941+0000","last_became_active":"2024-09-17T05:28:43.574282+0000","last_became_peered":"2024-09-17T05:28:43.574282+0000","last_unstale":"2024-09-17T05:28:45.592941+0000","last_undegraded":"2024-09-17T05:28:45.592941+0000","last_fullsized":"2024-09-17T05:28:45.592941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:55.092115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593340+0000","last_change":"2024-09-17T05:28:44.594639+0000","last_active":"2024-09-17T05:28:45.593340+0000","last_peered":"2024-09-17T05:28:45.593340+0000","last_clean":"2024-09-17T05:28:45.593340+0000","last_became_active":"2024-09-17T05:28:44.594326+0000","last_became_peered":"2024-09-17T05:28:44.594326+0000","last_unstale":"2024-09-17T05:28:45.593340+0000","last_undegraded":"2024-09-17T05:28:45.593340+0000","last_fullsized":"2024-09-17T05:28:45.593340+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:56.099136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810624+0000","last_change":"2024-09-17T05:28:43.583590+0000","last_active":"2024-09-17T05:28:45.810624+0000","last_peered":"2024-09-17T05:28:45.810624+0000","last_clean":"2024-09-17T05:28:45.810624+0000","last_became_active":"2024-09-17T05:28:43.583012+0000","last_became_peered":"2024-09-17T05:28:43.583012+0000","last_unstale":"2024-09-17T05:28:45.810624+0000","last_undegraded":"2024-09-17T05:28:45.810624+0000","last_fullsized":"2024-09-17T05:28:45.810624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:35.082128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591949+0000","last_change":"2024-09-17T05:28:44.586672+0000","last_active":"2024-09-17T05:28:45.591949+0000","last_peered":"2024-09-17T05:28:45.591949+0000","last_clean":"2024-09-17T05:28:45.591949+0000","last_became_active":"2024-09-17T05:28:44.586286+0000","last_became_peered":"2024-09-17T05:28:44.586286+0000","last_unstale":"2024-09-17T05:28:45.591949+0000","last_undegraded":"2024-09-17T05:28:45.591949+0000","last_fullsized":"2024-09-17T05:28:45.591949+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.432831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592035+0000","last_change":"2024-09-17T05:28:43.586381+0000","last_active":"2024-09-17T05:28:45.592035+0000","last_peered":"2024-09-17T05:28:45.592035+0000","last_clean":"2024-09-17T05:28:45.592035+0000","last_became_active":"2024-09-17T05:28:43.586051+0000","last_became_peered":"2024-09-17T05:28:43.586051+0000","last_unstale":"2024-09-17T05:28:45.592035+0000","last_undegraded":"2024-09-17T05:28:45.592035+0000","last_fullsized":"2024-09-17T05:28:45.592035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:52.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810197+0000","last_change":"2024-09-17T05:28:44.610140+0000","last_active":"2024-09-17T05:28:45.810197+0000","last_peered":"2024-09-17T05:28:45.810197+0000","last_clean":"2024-09-17T05:28:45.810197+0000","last_became_active":"2024-09-17T05:28:44.609822+0000","last_became_peered":"2024-09-17T05:28:44.609822+0000","last_unstale":"2024-09-17T05:28:45.810197+0000","last_undegraded":"2024-09-17T05:28:45.810197+0000","last_fullsized":"2024-09-17T05:28:45.810197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:09.487049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593761+0000","last_change":"2024-09-17T05:28:43.572513+0000","last_active":"2024-09-17T05:28:45.593761+0000","last_peered":"2024-09-17T05:28:45.593761+0000","last_clean":"2024-09-17T05:28:45.593761+0000","last_became_active":"2024-09-17T05:28:43.571694+0000","last_became_peered":"2024-09-17T05:28:43.571694+0000","last_unstale":"2024-09-17T05:28:45.593761+0000","last_undegraded":"2024-09-17T05:28:45.593761+0000","last_fullsized":"2024-09-17T05:28:45.593761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:37.493312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812943+0000","last_change":"2024-09-17T05:28:44.593611+0000","last_active":"2024-09-17T05:28:45.812943+0000","last_peered":"2024-09-17T05:28:45.812943+0000","last_clean":"2024-09-17T05:28:45.812943+0000","last_became_active":"2024-09-17T05:28:44.592236+0000","last_became_peered":"2024-09-17T05:28:44.592236+0000","last_unstale":"2024-09-17T05:28:45.812943+0000","last_undegraded":"2024-09-17T05:28:45.812943+0000","last_fullsized":"2024-09-17T05:28:45.812943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:49.586447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811729+0000","last_change":"2024-09-17T05:28:43.581526+0000","last_active":"2024-09-17T05:28:45.811729+0000","last_peered":"2024-09-17T05:28:45.811729+0000","last_clean":"2024-09-17T05:28:45.811729+0000","last_became_active":"2024-09-17T05:28:43.581163+0000","last_became_peered":"2024-09-17T05:28:43.581163+0000","last_unstale":"2024-09-17T05:28:45.811729+0000","last_undegraded":"2024-09-17T05:28:45.811729+0000","last_fullsized":"2024-09-17T05:28:45.811729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:27.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809610+0000","last_change":"2024-09-17T05:28:44.603138+0000","last_active":"2024-09-17T05:28:45.809610+0000","last_peered":"2024-09-17T05:28:45.809610+0000","last_clean":"2024-09-17T05:28:45.809610+0000","last_became_active":"2024-09-17T05:28:44.600619+0000","last_became_peered":"2024-09-17T05:28:44.600619+0000","last_unstale":"2024-09-17T05:28:45.809610+0000","last_undegraded":"2024-09-17T05:28:45.809610+0000","last_fullsized":"2024-09-17T05:28:45.809610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:49.289264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812166+0000","last_change":"2024-09-17T05:28:43.577962+0000","last_active":"2024-09-17T05:28:45.812166+0000","last_peered":"2024-09-17T05:28:45.812166+0000","last_clean":"2024-09-17T05:28:45.812166+0000","last_became_active":"2024-09-17T05:28:43.577642+0000","last_became_peered":"2024-09-17T05:28:43.577642+0000","last_unstale":"2024-09-17T05:28:45.812166+0000","last_undegraded":"2024-09-17T05:28:45.812166+0000","last_fullsized":"2024-09-17T05:28:45.812166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:55.229169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592767+0000","last_change":"2024-09-17T05:28:44.592697+0000","last_active":"2024-09-17T05:28:45.592767+0000","last_peered":"2024-09-17T05:28:45.592767+0000","last_clean":"2024-09-17T05:28:45.592767+0000","last_became_active":"2024-09-17T05:28:44.592403+0000","last_became_peered":"2024-09-17T05:28:44.592403+0000","last_unstale":"2024-09-17T05:28:45.592767+0000","last_undegraded":"2024-09-17T05:28:45.592767+0000","last_fullsized":"2024-09-17T05:28:45.592767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:05.971624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593059+0000","last_change":"2024-09-17T05:28:43.570566+0000","last_active":"2024-09-17T05:28:45.593059+0000","last_peered":"2024-09-17T05:28:45.593059+0000","last_clean":"2024-09-17T05:28:45.593059+0000","last_became_active":"2024-09-17T05:28:43.570239+0000","last_became_peered":"2024-09-17T05:28:43.570239+0000","last_unstale":"2024-09-17T05:28:45.593059+0000","last_undegraded":"2024-09-17T05:28:45.593059+0000","last_fullsized":"2024-09-17T05:28:45.593059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:06.675935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810295+0000","last_change":"2024-09-17T05:28:44.598400+0000","last_active":"2024-09-17T05:28:45.810295+0000","last_peered":"2024-09-17T05:28:45.810295+0000","last_clean":"2024-09-17T05:28:45.810295+0000","last_became_active":"2024-09-17T05:28:44.598073+0000","last_became_peered":"2024-09-17T05:28:44.598073+0000","last_unstale":"2024-09-17T05:28:45.810295+0000","last_undegraded":"2024-09-17T05:28:45.810295+0000","last_fullsized":"2024-09-17T05:28:45.810295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.710176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592890+0000","last_change":"2024-09-17T05:28:43.569060+0000","last_active":"2024-09-17T05:28:45.592890+0000","last_peered":"2024-09-17T05:28:45.592890+0000","last_clean":"2024-09-17T05:28:45.592890+0000","last_became_active":"2024-09-17T05:28:43.568732+0000","last_became_peered":"2024-09-17T05:28:43.568732+0000","last_unstale":"2024-09-17T05:28:45.592890+0000","last_undegraded":"2024-09-17T05:28:45.592890+0000","last_fullsized":"2024-09-17T05:28:45.592890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:52.549218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810612+0000","last_change":"2024-09-17T05:28:44.591498+0000","last_active":"2024-09-17T05:28:45.810612+0000","last_peered":"2024-09-17T05:28:45.810612+0000","last_clean":"2024-09-17T05:28:45.810612+0000","last_became_active":"2024-09-17T05:28:44.591186+0000","last_became_peered":"2024-09-17T05:28:44.591186+0000","last_unstale":"2024-09-17T05:28:45.810612+0000","last_undegraded":"2024-09-17T05:28:45.810612+0000","last_fullsized":"2024-09-17T05:28:45.810612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:57.134332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591906+0000","last_change":"2024-09-17T05:28:43.581871+0000","last_active":"2024-09-17T05:28:45.591906+0000","last_peered":"2024-09-17T05:28:45.591906+0000","last_clean":"2024-09-17T05:28:45.591906+0000","last_became_active":"2024-09-17T05:28:43.581398+0000","last_became_peered":"2024-09-17T05:28:43.581398+0000","last_unstale":"2024-09-17T05:28:45.591906+0000","last_undegraded":"2024-09-17T05:28:45.591906+0000","last_fullsized":"2024-09-17T05:28:45.591906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:44.331139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591947+0000","last_change":"2024-09-17T05:28:44.590902+0000","last_active":"2024-09-17T05:28:45.591947+0000","last_peered":"2024-09-17T05:28:45.591947+0000","last_clean":"2024-09-17T05:28:45.591947+0000","last_became_active":"2024-09-17T05:28:44.589574+0000","last_became_peered":"2024-09-17T05:28:44.589574+0000","last_unstale":"2024-09-17T05:28:45.591947+0000","last_undegraded":"2024-09-17T05:28:45.591947+0000","last_fullsized":"2024-09-17T05:28:45.591947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:58.669142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810631+0000","last_change":"2024-09-17T05:28:43.578038+0000","last_active":"2024-09-17T05:28:45.810631+0000","last_peered":"2024-09-17T05:28:45.810631+0000","last_clean":"2024-09-17T05:28:45.810631+0000","last_became_active":"2024-09-17T05:28:43.577245+0000","last_became_peered":"2024-09-17T05:28:43.577245+0000","last_unstale":"2024-09-17T05:28:45.810631+0000","last_undegraded":"2024-09-17T05:28:45.810631+0000","last_fullsized":"2024-09-17T05:28:45.810631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:21.961572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591758+0000","last_change":"2024-09-17T05:28:44.595876+0000","last_active":"2024-09-17T05:28:45.591758+0000","last_peered":"2024-09-17T05:28:45.591758+0000","last_clean":"2024-09-17T05:28:45.591758+0000","last_became_active":"2024-09-17T05:28:44.591932+0000","last_became_peered":"2024-09-17T05:28:44.591932+0000","last_unstale":"2024-09-17T05:28:45.591758+0000","last_undegraded":"2024-09-17T05:28:45.591758+0000","last_fullsized":"2024-09-17T05:28:45.591758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:59.219871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592143+0000","last_change":"2024-09-17T05:28:43.566233+0000","last_active":"2024-09-17T05:28:45.592143+0000","last_peered":"2024-09-17T05:28:45.592143+0000","last_clean":"2024-09-17T05:28:45.592143+0000","last_became_active":"2024-09-17T05:28:43.565869+0000","last_became_peered":"2024-09-17T05:28:43.565869+0000","last_unstale":"2024-09-17T05:28:45.592143+0000","last_undegraded":"2024-09-17T05:28:45.592143+0000","last_fullsized":"2024-09-17T05:28:45.592143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:45.613915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811776+0000","last_change":"2024-09-17T05:28:44.589460+0000","last_active":"2024-09-17T05:28:45.811776+0000","last_peered":"2024-09-17T05:28:45.811776+0000","last_clean":"2024-09-17T05:28:45.811776+0000","last_became_active":"2024-09-17T05:28:44.589134+0000","last_became_peered":"2024-09-17T05:28:44.589134+0000","last_unstale":"2024-09-17T05:28:45.811776+0000","last_undegraded":"2024-09-17T05:28:45.811776+0000","last_fullsized":"2024-09-17T05:28:45.811776+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:28.107602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591341+0000","last_change":"2024-09-17T05:28:43.575603+0000","last_active":"2024-09-17T05:28:45.591341+0000","last_peered":"2024-09-17T05:28:45.591341+0000","last_clean":"2024-09-17T05:28:45.591341+0000","last_became_active":"2024-09-17T05:28:43.575283+0000","last_became_peered":"2024-09-17T05:28:43.575283+0000","last_unstale":"2024-09-17T05:28:45.591341+0000","last_undegraded":"2024-09-17T05:28:45.591341+0000","last_fullsized":"2024-09-17T05:28:45.591341+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:05.142867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811865+0000","last_change":"2024-09-17T05:28:44.588727+0000","last_active":"2024-09-17T05:28:45.811865+0000","last_peered":"2024-09-17T05:28:45.811865+0000","last_clean":"2024-09-17T05:28:45.811865+0000","last_became_active":"2024-09-17T05:28:44.588397+0000","last_became_peered":"2024-09-17T05:28:44.588397+0000","last_unstale":"2024-09-17T05:28:45.811865+0000","last_undegraded":"2024-09-17T05:28:45.811865+0000","last_fullsized":"2024-09-17T05:28:45.811865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:44.683830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810356+0000","last_change":"2024-09-17T05:28:43.587791+0000","last_active":"2024-09-17T05:28:45.810356+0000","last_peered":"2024-09-17T05:28:45.810356+0000","last_clean":"2024-09-17T05:28:45.810356+0000","last_became_active":"2024-09-17T05:28:43.587450+0000","last_became_peered":"2024-09-17T05:28:43.587450+0000","last_unstale":"2024-09-17T05:28:45.810356+0000","last_undegraded":"2024-09-17T05:28:45.810356+0000","last_fullsized":"2024-09-17T05:28:45.810356+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:38.784007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812092+0000","last_change":"2024-09-17T05:28:44.590941+0000","last_active":"2024-09-17T05:28:45.812092+0000","last_peered":"2024-09-17T05:28:45.812092+0000","last_clean":"2024-09-17T05:28:45.812092+0000","last_became_active":"2024-09-17T05:28:44.590589+0000","last_became_peered":"2024-09-17T05:28:44.590589+0000","last_unstale":"2024-09-17T05:28:45.812092+0000","last_undegraded":"2024-09-17T05:28:45.812092+0000","last_fullsized":"2024-09-17T05:28:45.812092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:16.499238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809738+0000","last_change":"2024-09-17T05:28:43.583447+0000","last_active":"2024-09-17T05:28:45.809738+0000","last_peered":"2024-09-17T05:28:45.809738+0000","last_clean":"2024-09-17T05:28:45.809738+0000","last_became_active":"2024-09-17T05:28:43.583145+0000","last_became_peered":"2024-09-17T05:28:43.583145+0000","last_unstale":"2024-09-17T05:28:45.809738+0000","last_undegraded":"2024-09-17T05:28:45.809738+0000","last_fullsized":"2024-09-17T05:28:45.809738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:53.903201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811078+0000","last_change":"2024-09-17T05:28:44.589236+0000","last_active":"2024-09-17T05:28:45.811078+0000","last_peered":"2024-09-17T05:28:45.811078+0000","last_clean":"2024-09-17T05:28:45.811078+0000","last_became_active":"2024-09-17T05:28:44.588658+0000","last_became_peered":"2024-09-17T05:28:44.588658+0000","last_unstale":"2024-09-17T05:28:45.811078+0000","last_undegraded":"2024-09-17T05:28:45.811078+0000","last_fullsized":"2024-09-17T05:28:45.811078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.776592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591251+0000","last_change":"2024-09-17T05:28:43.564033+0000","last_active":"2024-09-17T05:28:45.591251+0000","last_peered":"2024-09-17T05:28:45.591251+0000","last_clean":"2024-09-17T05:28:45.591251+0000","last_became_active":"2024-09-17T05:28:43.563635+0000","last_became_peered":"2024-09-17T05:28:43.563635+0000","last_unstale":"2024-09-17T05:28:45.591251+0000","last_undegraded":"2024-09-17T05:28:45.591251+0000","last_fullsized":"2024-09-17T05:28:45.591251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:32.152013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593241+0000","last_change":"2024-09-17T05:28:44.586155+0000","last_active":"2024-09-17T05:28:45.593241+0000","last_peered":"2024-09-17T05:28:45.593241+0000","last_clean":"2024-09-17T05:28:45.593241+0000","last_became_active":"2024-09-17T05:28:44.585601+0000","last_became_peered":"2024-09-17T05:28:44.585601+0000","last_unstale":"2024-09-17T05:28:45.593241+0000","last_undegraded":"2024-09-17T05:28:45.593241+0000","last_fullsized":"2024-09-17T05:28:45.593241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:31.730157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810105+0000","last_change":"2024-09-17T05:28:43.578062+0000","last_active":"2024-09-17T05:28:45.810105+0000","last_peered":"2024-09-17T05:28:45.810105+0000","last_clean":"2024-09-17T05:28:45.810105+0000","last_became_active":"2024-09-17T05:28:43.577664+0000","last_became_peered":"2024-09-17T05:28:43.577664+0000","last_unstale":"2024-09-17T05:28:45.810105+0000","last_undegraded":"2024-09-17T05:28:45.810105+0000","last_fullsized":"2024-09-17T05:28:45.810105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:13.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.812673+0000","last_change":"2024-09-17T05:28:44.591578+0000","last_active":"2024-09-17T05:28:45.812673+0000","last_peered":"2024-09-17T05:28:45.812673+0000","last_clean":"2024-09-17T05:28:45.812673+0000","last_became_active":"2024-09-17T05:28:44.591215+0000","last_became_peered":"2024-09-17T05:28:44.591215+0000","last_unstale":"2024-09-17T05:28:45.812673+0000","last_undegraded":"2024-09-17T05:28:45.812673+0000","last_fullsized":"2024-09-17T05:28:45.812673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:43.026057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810764+0000","last_change":"2024-09-17T05:28:43.571945+0000","last_active":"2024-09-17T05:28:45.810764+0000","last_peered":"2024-09-17T05:28:45.810764+0000","last_clean":"2024-09-17T05:28:45.810764+0000","last_became_active":"2024-09-17T05:28:43.571593+0000","last_became_peered":"2024-09-17T05:28:43.571593+0000","last_unstale":"2024-09-17T05:28:45.810764+0000","last_undegraded":"2024-09-17T05:28:45.810764+0000","last_fullsized":"2024-09-17T05:28:45.810764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:27.481016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593342+0000","last_change":"2024-09-17T05:28:44.592820+0000","last_active":"2024-09-17T05:28:45.593342+0000","last_peered":"2024-09-17T05:28:45.593342+0000","last_clean":"2024-09-17T05:28:45.593342+0000","last_became_active":"2024-09-17T05:28:44.592492+0000","last_became_peered":"2024-09-17T05:28:44.592492+0000","last_unstale":"2024-09-17T05:28:45.593342+0000","last_undegraded":"2024-09-17T05:28:45.593342+0000","last_fullsized":"2024-09-17T05:28:45.593342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:06.149889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810821+0000","last_change":"2024-09-17T05:28:43.570857+0000","last_active":"2024-09-17T05:28:45.810821+0000","last_peered":"2024-09-17T05:28:45.810821+0000","last_clean":"2024-09-17T05:28:45.810821+0000","last_became_active":"2024-09-17T05:28:43.570448+0000","last_became_peered":"2024-09-17T05:28:43.570448+0000","last_unstale":"2024-09-17T05:28:45.810821+0000","last_undegraded":"2024-09-17T05:28:45.810821+0000","last_fullsized":"2024-09-17T05:28:45.810821+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:01.094952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591521+0000","last_change":"2024-09-17T05:28:44.609358+0000","last_active":"2024-09-17T05:28:45.591521+0000","last_peered":"2024-09-17T05:28:45.591521+0000","last_clean":"2024-09-17T05:28:45.591521+0000","last_became_active":"2024-09-17T05:28:44.609063+0000","last_became_peered":"2024-09-17T05:28:44.609063+0000","last_unstale":"2024-09-17T05:28:45.591521+0000","last_undegraded":"2024-09-17T05:28:45.591521+0000","last_fullsized":"2024-09-17T05:28:45.591521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.391505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593798+0000","last_change":"2024-09-17T05:28:43.574524+0000","last_active":"2024-09-17T05:28:45.593798+0000","last_peered":"2024-09-17T05:28:45.593798+0000","last_clean":"2024-09-17T05:28:45.593798+0000","last_became_active":"2024-09-17T05:28:43.572192+0000","last_became_peered":"2024-09-17T05:28:43.572192+0000","last_unstale":"2024-09-17T05:28:45.593798+0000","last_undegraded":"2024-09-17T05:28:45.593798+0000","last_fullsized":"2024-09-17T05:28:45.593798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:46.751852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592860+0000","last_change":"2024-09-17T05:28:44.598664+0000","last_active":"2024-09-17T05:28:45.592860+0000","last_peered":"2024-09-17T05:28:45.592860+0000","last_clean":"2024-09-17T05:28:45.592860+0000","last_became_active":"2024-09-17T05:28:44.598325+0000","last_became_peered":"2024-09-17T05:28:44.598325+0000","last_unstale":"2024-09-17T05:28:45.592860+0000","last_undegraded":"2024-09-17T05:28:45.592860+0000","last_fullsized":"2024-09-17T05:28:45.592860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:35.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593598+0000","last_change":"2024-09-17T05:28:43.574129+0000","last_active":"2024-09-17T05:28:45.593598+0000","last_peered":"2024-09-17T05:28:45.593598+0000","last_clean":"2024-09-17T05:28:45.593598+0000","last_became_active":"2024-09-17T05:28:43.573723+0000","last_became_peered":"2024-09-17T05:28:43.573723+0000","last_unstale":"2024-09-17T05:28:45.593598+0000","last_undegraded":"2024-09-17T05:28:45.593598+0000","last_fullsized":"2024-09-17T05:28:45.593598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:46.485943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810964+0000","last_change":"2024-09-17T05:28:44.584324+0000","last_active":"2024-09-17T05:28:45.810964+0000","last_peered":"2024-09-17T05:28:45.810964+0000","last_clean":"2024-09-17T05:28:45.810964+0000","last_became_active":"2024-09-17T05:28:44.583981+0000","last_became_peered":"2024-09-17T05:28:44.583981+0000","last_unstale":"2024-09-17T05:28:45.810964+0000","last_undegraded":"2024-09-17T05:28:45.810964+0000","last_fullsized":"2024-09-17T05:28:45.810964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:46.461741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.593440+0000","last_change":"2024-09-17T05:28:43.582630+0000","last_active":"2024-09-17T05:28:45.593440+0000","last_peered":"2024-09-17T05:28:45.593440+0000","last_clean":"2024-09-17T05:28:45.593440+0000","last_became_active":"2024-09-17T05:28:43.582246+0000","last_became_peered":"2024-09-17T05:28:43.582246+0000","last_unstale":"2024-09-17T05:28:45.593440+0000","last_undegraded":"2024-09-17T05:28:45.593440+0000","last_fullsized":"2024-09-17T05:28:45.593440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:56.307551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810067+0000","last_change":"2024-09-17T05:28:44.606764+0000","last_active":"2024-09-17T05:28:45.810067+0000","last_peered":"2024-09-17T05:28:45.810067+0000","last_clean":"2024-09-17T05:28:45.810067+0000","last_became_active":"2024-09-17T05:28:44.606433+0000","last_became_peered":"2024-09-17T05:28:44.606433+0000","last_unstale":"2024-09-17T05:28:45.810067+0000","last_undegraded":"2024-09-17T05:28:45.810067+0000","last_fullsized":"2024-09-17T05:28:45.810067+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:59.606056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811242+0000","last_change":"2024-09-17T05:28:43.583802+0000","last_active":"2024-09-17T05:28:45.811242+0000","last_peered":"2024-09-17T05:28:45.811242+0000","last_clean":"2024-09-17T05:28:45.811242+0000","last_became_active":"2024-09-17T05:28:43.583483+0000","last_became_peered":"2024-09-17T05:28:43.583483+0000","last_unstale":"2024-09-17T05:28:45.811242+0000","last_undegraded":"2024-09-17T05:28:45.811242+0000","last_fullsized":"2024-09-17T05:28:45.811242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:30.524458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.811820+0000","last_change":"2024-09-17T05:28:44.596874+0000","last_active":"2024-09-17T05:28:45.811820+0000","last_peered":"2024-09-17T05:28:45.811820+0000","last_clean":"2024-09-17T05:28:45.811820+0000","last_became_active":"2024-09-17T05:28:44.596501+0000","last_became_peered":"2024-09-17T05:28:44.596501+0000","last_unstale":"2024-09-17T05:28:45.811820+0000","last_undegraded":"2024-09-17T05:28:45.811820+0000","last_fullsized":"2024-09-17T05:28:45.811820+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:27.780146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591364+0000","last_change":"2024-09-17T05:28:43.584354+0000","last_active":"2024-09-17T05:28:45.591364+0000","last_peered":"2024-09-17T05:28:45.591364+0000","last_clean":"2024-09-17T05:28:45.591364+0000","last_became_active":"2024-09-17T05:28:43.583900+0000","last_became_peered":"2024-09-17T05:28:43.583900+0000","last_unstale":"2024-09-17T05:28:45.591364+0000","last_undegraded":"2024-09-17T05:28:45.591364+0000","last_fullsized":"2024-09-17T05:28:45.591364+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:46.826167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.591063+0000","last_change":"2024-09-17T05:28:44.602138+0000","last_active":"2024-09-17T05:28:45.591063+0000","last_peered":"2024-09-17T05:28:45.591063+0000","last_clean":"2024-09-17T05:28:45.591063+0000","last_became_active":"2024-09-17T05:28:44.601237+0000","last_became_peered":"2024-09-17T05:28:44.601237+0000","last_unstale":"2024-09-17T05:28:45.591063+0000","last_undegraded":"2024-09-17T05:28:45.591063+0000","last_fullsized":"2024-09-17T05:28:45.591063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:58.301938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.809351+0000","last_change":"2024-09-17T05:28:43.582802+0000","last_active":"2024-09-17T05:28:45.809351+0000","last_peered":"2024-09-17T05:28:45.809351+0000","last_clean":"2024-09-17T05:28:45.809351+0000","last_became_active":"2024-09-17T05:28:43.582317+0000","last_became_peered":"2024-09-17T05:28:43.582317+0000","last_unstale":"2024-09-17T05:28:45.809351+0000","last_undegraded":"2024-09-17T05:28:45.809351+0000","last_fullsized":"2024-09-17T05:28:45.809351+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:27.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.810681+0000","last_change":"2024-09-17T05:28:44.587771+0000","last_active":"2024-09-17T05:28:45.810681+0000","last_peered":"2024-09-17T05:28:45.810681+0000","last_clean":"2024-09-17T05:28:45.810681+0000","last_became_active":"2024-09-17T05:28:44.587461+0000","last_became_peered":"2024-09-17T05:28:44.587461+0000","last_unstale":"2024-09-17T05:28:45.810681+0000","last_undegraded":"2024-09-17T05:28:45.810681+0000","last_fullsized":"2024-09-17T05:28:45.810681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:43.545893+0000","last_clean_scrub_stamp":"2024-09-17T05:28:43.545893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:17.392846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592943+0000","last_change":"2024-09-17T05:28:43.565393+0000","last_active":"2024-09-17T05:28:45.592943+0000","last_peered":"2024-09-17T05:28:45.592943+0000","last_clean":"2024-09-17T05:28:45.592943+0000","last_became_active":"2024-09-17T05:28:43.565048+0000","last_became_peered":"2024-09-17T05:28:43.565048+0000","last_unstale":"2024-09-17T05:28:45.592943+0000","last_undegraded":"2024-09-17T05:28:45.592943+0000","last_fullsized":"2024-09-17T05:28:45.592943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:42.538872+0000","last_clean_scrub_stamp":"2024-09-17T05:28:42.538872+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:59.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:28:45.592388+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:28:45.592388+0000","last_peered":"2024-09-17T05:28:45.592388+0000","last_clean":"2024-09-17T05:28:45.592388+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:28:45.592388+0000","last_undegraded":"2024-09-17T05:28:45.592388+0000","last_fullsized":"2024-09-17T05:28:45.592388+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":917504,"data_stored":387116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":975962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":386198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":385630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":909312,"data_stored":386086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":975872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":909312,"data_stored":385574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":387150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:53.866 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:28:53.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 -- 172.21.15.43:0/2752942996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c0737c0 msgr2=0x7fad5c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 --2- 172.21.15.43:0/2752942996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c0737c0 0x7fad5c0700a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fad54009bc0 tx=0x7fad5401ca40 comp rx=0 tx=0).stop 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 -- 172.21.15.43:0/2752942996 shutdown_connections 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 --2- 172.21.15.43:0/2752942996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c0737c0 0x7fad5c0700a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 --2- 172.21.15.43:0/2752942996 >> v2:172.21.15.62:3300/0 conn(0x7fad5c15fe10 0x7fad5c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 -- 172.21.15.43:0/2752942996 >> 172.21.15.43:0/2752942996 conn(0x7fad5c0b23d0 msgr2=0x7fad5c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 -- 172.21.15.43:0/2752942996 shutdown_connections 2024-09-17T05:28:54.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 -- 172.21.15.43:0/2752942996 wait complete. 2024-09-17T05:28:54.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.066+0000 7fad64200640 1 Processor -- start 2024-09-17T05:28:54.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- start start 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c06fc90 0x7fad5c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 0x7fad5c15c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c169f20 con 0x7fad5c15e3b0 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c169da0 con 0x7fad5c06fc90 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c16a0a0 con 0x7fad5c15fe10 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60734/0 (socket says 172.21.15.43:60734) 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 -- 172.21.15.43:0/4173651954 learned_addr learned my addr 172.21.15.43:0/4173651954 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad61800640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c06fc90 0x7fad5c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 -- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 msgr2=0x7fad5c15c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad60e00640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 0x7fad5c15c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 0x7fad5c15c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 -- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c06fc90 msgr2=0x7fad5c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c06fc90 0x7fad5c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 -- 172.21.15.43:0/4173651954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c15ed00 con 0x7fad5c15e3b0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad60e00640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 0x7fad5c15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad62200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fad5401c4b0 tx=0x7fad54003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad540b2040 con 0x7fad5c15e3b0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad540bb070 con 0x7fad5c15e3b0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54004370 con 0x7fad5c15e3b0 2024-09-17T05:28:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c157880 con 0x7fad5c15e3b0 2024-09-17T05:28:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c157e40 con 0x7fad5c15e3b0 2024-09-17T05:28:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.070+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c0700a0 con 0x7fad5c15e3b0 2024-09-17T05:28:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.074+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad54004510 con 0x7fad5c15e3b0 2024-09-17T05:28:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.074+0000 7fad59600640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 0x7fad3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.074+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fad5412ec80 con 0x7fad5c15e3b0 2024-09-17T05:28:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.074+0000 7fad61800640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 0x7fad3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.074+0000 7fad61800640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 0x7fad3c075740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad500058d0 tx=0x7fad50005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.078+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad5c15ed00 con 0x7fad5c15e3b0 2024-09-17T05:28:54.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.314+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fad5c0ad0b0 con 0x7fad5c15e3b0 2024-09-17T05:28:54.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad59600640 1 -- 172.21.15.43:0/4173651954 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7fad54016750 con 0x7fad5c15e3b0 2024-09-17T05:28:54.320 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:28:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:28:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:28:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:28:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:28:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:28:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 msgr2=0x7fad3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 0x7fad3c075740 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad500058d0 tx=0x7fad50005860 comp rx=0 tx=0).stop 2024-09-17T05:28:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 msgr2=0x7fad5c15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fad5401c4b0 tx=0x7fad54003f40 comp rx=0 tx=0).stop 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 shutdown_connections 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad3c073230 0x7fad3c075740 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad5c15e3b0 0x7fad5c15e7c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad5c15fe10 0x7fad5c15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 --2- 172.21.15.43:0/4173651954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c06fc90 0x7fad5c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 >> 172.21.15.43:0/4173651954 conn(0x7fad5c0b23d0 msgr2=0x7fad5c0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 shutdown_connections 2024-09-17T05:28:54.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.318+0000 7fad64200640 1 -- 172.21.15.43:0/4173651954 wait complete. 2024-09-17T05:28:54.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 -- 172.21.15.43:0/3576708623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0650 msgr2=0x7f7c540cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/3576708623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0650 0x7f7c540cd750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009bc0 tx=0x7f7c4c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 -- 172.21.15.43:0/3576708623 shutdown_connections 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/3576708623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0650 0x7f7c540cd750 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/3576708623 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c5409ba60 0x7f7c540b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 -- 172.21.15.43:0/3576708623 >> 172.21.15.43:0/3576708623 conn(0x7f7c540cee60 msgr2=0x7f7c540cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 -- 172.21.15.43:0/3576708623 shutdown_connections 2024-09-17T05:28:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.498+0000 7f7c5aa00640 1 -- 172.21.15.43:0/3576708623 wait complete. 2024-09-17T05:28:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 Processor -- start 2024-09-17T05:28:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 -- start start 2024-09-17T05:28:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c5409ba60 0x7f7c541fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c540d0650 0x7f7c540b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540b2b80 con 0x7f7c540d0240 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540b2a00 con 0x7f7c5409ba60 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c5aa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7c54169e30 con 0x7f7c540d0650 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60744/0 (socket says 172.21.15.43:60744) 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.502+0000 7f7c4be00640 1 -- 172.21.15.43:0/2508282568 learned_addr learned my addr 172.21.15.43:0/2508282568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c58a00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c540d0650 0x7f7c540b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c53e00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c5409ba60 0x7f7c541fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 -- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c540d0650 msgr2=0x7f7c540b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c540d0650 0x7f7c540b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 -- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c5409ba60 msgr2=0x7f7c541fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c5409ba60 0x7f7c541fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 -- 172.21.15.43:0/2508282568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c541fc3d0 con 0x7f7c540d0240 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c4be00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c420 tx=0x7f7c3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c017070 con 0x7f7c540d0240 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c3c003600 con 0x7f7c540d0240 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c003a50 con 0x7f7c540d0240 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c541fc6b0 con 0x7f7c540d0240 2024-09-17T05:28:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541fcc70 con 0x7f7c540d0240 2024-09-17T05:28:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c540b3130 con 0x7f7c540d0240 2024-09-17T05:28:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.506+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c3c01a070 con 0x7f7c540d0240 2024-09-17T05:28:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.510+0000 7f7c50c00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 0x7f7c30075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.510+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f7c3c093440 con 0x7f7c540d0240 2024-09-17T05:28:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.510+0000 7f7c53e00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 0x7f7c30075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.510+0000 7f7c53e00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 0x7f7c30075740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005fd0 tx=0x7f7c44005e40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:54.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.514+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c3c05e4e0 con 0x7f7c540d0240 2024-09-17T05:28:54.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.722+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c541543e0 con 0x7f7c540d0240 2024-09-17T05:28:54.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.722+0000 7f7c50c00640 1 -- 172.21.15.43:0/2508282568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5225 (secure 0 0 0) 0x7f7c3c05dde0 con 0x7f7c540d0240 2024-09-17T05:28:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:54.726 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"btime":"2024-09-17T05:28:47:595804+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:47.012214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:28:54.726 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2024-09-17T05:28:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 msgr2=0x7f7c30075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 0x7f7c30075740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005fd0 tx=0x7f7c44005e40 comp rx=0 tx=0).stop 2024-09-17T05:28:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 msgr2=0x7f7c540b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c420 tx=0x7f7c3c00c8e0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 shutdown_connections 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c30073230 0x7f7c30075740 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c540d0650 0x7f7c540b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c540d0240 0x7f7c540b62d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 --2- 172.21.15.43:0/2508282568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c5409ba60 0x7f7c541fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 >> 172.21.15.43:0/2508282568 conn(0x7f7c540cee60 msgr2=0x7f7c540cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 shutdown_connections 2024-09-17T05:28:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:54.726+0000 7f7c5aa00640 1 -- 172.21.15.43:0/2508282568 wait complete. 2024-09-17T05:28:55.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:28:43.992520+0000', 'modified': '2024-09-17T05:28:47.012214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6387': {'gid': 6387, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/3375674498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 3375674498}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 3375674498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6387, 'qdb_cluster': [6387]} 2024-09-17T05:28:55.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:28:55.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T05:28:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 -- 172.21.15.43:0/4053137492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f00b3af0 msgr2=0x7f51f00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 --2- 172.21.15.43:0/4053137492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f00b3af0 0x7f51f00ad0b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009c80 tx=0x7f51e801cce0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 -- 172.21.15.43:0/4053137492 shutdown_connections 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 --2- 172.21.15.43:0/4053137492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f00b3af0 0x7f51f00ad0b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 --2- 172.21.15.43:0/4053137492 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f0155d70 0x7f51f015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 -- 172.21.15.43:0/4053137492 >> 172.21.15.43:0/4053137492 conn(0x7f51f00b23d0 msgr2=0x7f51f00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 -- 172.21.15.43:0/4053137492 shutdown_connections 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.310+0000 7f51f7a00640 1 -- 172.21.15.43:0/4053137492 wait complete. 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 Processor -- start 2024-09-17T05:28:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 -- start start 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f006f110 0x7f51f0162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 0x7f51f009bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0168c80 con 0x7f51f0155d70 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0168b00 con 0x7f51f006f110 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f7a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f51f0168e00 con 0x7f51f01627a0 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60768/0 (socket says 172.21.15.43:60768) 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 -- 172.21.15.43:0/1070705997 learned_addr learned my addr 172.21.15.43:0/1070705997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f5000640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f006f110 0x7f51f0162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f5a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 0x7f51f009bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 -- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 msgr2=0x7f51f009bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 0x7f51f009bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 -- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f006f110 msgr2=0x7f51f0162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f006f110 0x7f51f0162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51efe00640 1 -- 172.21.15.43:0/1070705997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f0160ab0 con 0x7f51f0155d70 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.314+0000 7f51f5a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 0x7f51f009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51efe00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c3f0 tx=0x7f51dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51ecc00640 1 -- 172.21.15.43:0/1070705997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51f0155d70 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51ecc00640 1 -- 172.21.15.43:0/1070705997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc0053c0 con 0x7f51f0155d70 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51ecc00640 1 -- 172.21.15.43:0/1070705997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003860 con 0x7f51f0155d70 2024-09-17T05:28:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f0160d90 con 0x7f51f0155d70 2024-09-17T05:28:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f0161350 con 0x7f51f0155d70 2024-09-17T05:28:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51ecc00640 1 -- 172.21.15.43:0/1070705997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f51dc01a070 con 0x7f51f0155d70 2024-09-17T05:28:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.318+0000 7f51ecc00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 0x7f51d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51ecc00640 1 -- 172.21.15.43:0/1070705997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f51dc093610 con 0x7f51f0155d70 2024-09-17T05:28:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 Processor -- start 2024-09-17T05:28:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 -- start start 2024-09-17T05:28:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 0x7f51d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 0x7f51d0075710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f51f00715c0 tx=0x7f51e40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 0x55f8969e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f8969e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8968b9410 con 0x55f8964a3460 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8967f6b40 con 0x55f896893950 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f8967e5f70 con 0x55f89687cca0 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60784/0 (socket says 172.21.15.43:60784) 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 -- 172.21.15.43:0/2400086519 learned_addr learned my addr 172.21.15.43:0/2400086519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51efe00640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 0x55f8969e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5a00640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f8969e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 -- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 msgr2=0x55f8969e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f8969e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 -- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 msgr2=0x55f8969e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 0x55f8969e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 -- 172.21.15.43:0/2400086519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8967ad5b0 con 0x55f8964a3460 2024-09-17T05:28:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51efe00640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 0x55f8969e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51f5000640 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f51e40058d0 tx=0x7f51e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7645d2efeb245d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51e1600640 1 -- 172.21.15.43:0/2400086519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e400eb30 con 0x55f8964a3460 2024-09-17T05:28:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51e1600640 1 -- 172.21.15.43:0/2400086519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51e400ecd0 con 0x55f8964a3460 2024-09-17T05:28:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51e1600640 1 -- 172.21.15.43:0/2400086519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4026630 con 0x55f8964a3460 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 -- 172.21.15.43:0/2400086519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 msgr2=0x55f8969e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.322+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f51e40058d0 tx=0x7f51e4014000 comp rx=0 tx=0).stop 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- 172.21.15.43:0/2400086519 shutdown_connections 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f896893950 0x55f8969e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f8969e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2400086519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969e46a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- 172.21.15.43:0/2400086519 >> 172.21.15.43:0/2400086519 conn(0x55f896969d90 msgr2=0x55f896999690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- 172.21.15.43:0/2400086519 shutdown_connections 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- 172.21.15.43:0/2400086519 wait complete. 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- start start 2024-09-17T05:28:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 Processor -- start 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f8969ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f896893950 0x55f8969ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8967805f0 con 0x55f8964a3460 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8967e9d80 con 0x55f89687cca0 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f89678c520 con 0x55f896893950 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51fefb1000 10 client.0 _ll_register_callbacks cb 0x55f896790830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60796/0 (socket says 172.21.15.43:60796) 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 -- 172.21.15.43:0/2615924506 learned_addr learned my addr 172.21.15.43:0/2615924506 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5a00640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f896893950 0x55f8969ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51efe00640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f8969ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 -- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f896893950 msgr2=0x55f8969ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f896893950 0x55f8969ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 -- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 msgr2=0x55f8969ebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f8969ebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 -- 172.21.15.43:0/2615924506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f896784190 con 0x55f8964a3460 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51f5000640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005860 tx=0x7f51e40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725d495bcb830626 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.326+0000 7f51d6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4005020 con 0x55f8964a3460 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f89677e110 con 0x55f8964a3460 2024-09-17T05:28:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f51e40051c0 con 0x55f8964a3460 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f51e403b050 con 0x55f8964a3460 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 10 client.6534 fetch_fsmap learned FSMap version 70 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55f89686af10 con 0x55f8964a3460 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f89677e110 con 0x55f8964a3460 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f51e403e020 con 0x55f8964a3460 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 10 client.6534 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 10 client.6534 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 20 client.6534 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f8964887d0 0x55f8969f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 4 client.6534 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 --> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8967f8a90 con 0x55f8964887d0 2024-09-17T05:28:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51efe00640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f8964887d0 0x55f8969f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 10 client.6534 ms_handle_connect on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51efe00640 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f8964887d0 0x55f8969f5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829889c0ce97a4a5 server_cookie=4a3f78aaf119b530 in_seq=0 out_seq=0 2024-09-17T05:28:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 1 -- 172.21.15.43:0/2615924506 <== mds.0 v2:172.21.15.43:6834/3375674498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8967f8a90 con 0x55f8964887d0 2024-09-17T05:28:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 10 client.6534 handle_command_reply: tid=0 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 client.6534 shutdown 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 msgr2=0x55f8969eb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005860 tx=0x7f51e40098e0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 shutdown_connections 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f8969ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f8964887d0 0x55f8969f5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f896893950 0x55f8969ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2615924506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f8964a3460 0x55f8969eb920 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51d6000640 0 client.6534 ms_handle_reset on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 >> 172.21.15.43:0/2615924506 conn(0x55f896969d90 msgr2=0x55f8969a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 shutdown_connections 2024-09-17T05:28:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 1 -- 172.21.15.43:0/2615924506 wait complete. 2024-09-17T05:28:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.330+0000 7f51fefb1000 20 client.6534 trim_cache size 0 max 16384 2024-09-17T05:28:55.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 Processor -- start 2024-09-17T05:28:55.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 -- start start 2024-09-17T05:28:55.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8964a3460 0x55f896a5aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f896a5af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51f02031d0 con 0x55f896893950 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0202210 con 0x55f8964a3460 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.350+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f51f0164e30 con 0x55f89687cca0 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60812/0 (socket says 172.21.15.43:60812) 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 -- 172.21.15.43:0/3565641473 learned_addr learned my addr 172.21.15.43:0/3565641473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51efe00640 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f896a5af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5000640 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8964a3460 0x55f896a5aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 -- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 msgr2=0x55f896a5af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f896a5af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 -- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8964a3460 msgr2=0x55f896a5aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8964a3460 0x55f896a5aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 -- 172.21.15.43:0/3565641473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f896841db0 con 0x55f896893950 2024-09-17T05:28:55.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51f5a00640 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f51e80b0040 tx=0x7f51e8019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76aea953af5190c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51e0c00640 1 -- 172.21.15.43:0/3565641473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80b2040 con 0x55f896893950 2024-09-17T05:28:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51e0c00640 1 -- 172.21.15.43:0/3565641473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51e8004d40 con 0x55f896893950 2024-09-17T05:28:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51e0c00640 1 -- 172.21.15.43:0/3565641473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8003bc0 con 0x55f896893950 2024-09-17T05:28:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- 172.21.15.43:0/3565641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 msgr2=0x55f896a5b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f51e80b0040 tx=0x7f51e8019a80 comp rx=0 tx=0).stop 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- 172.21.15.43:0/3565641473 shutdown_connections 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5b4b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f89687cca0 0x55f896a5af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 --2- 172.21.15.43:0/3565641473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8964a3460 0x55f896a5aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- 172.21.15.43:0/3565641473 >> 172.21.15.43:0/3565641473 conn(0x55f896969d90 msgr2=0x55f8969e2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- 172.21.15.43:0/3565641473 shutdown_connections 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- 172.21.15.43:0/3565641473 wait complete. 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 -- start start 2024-09-17T05:28:55.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.354+0000 7f51fefb1000 1 Processor -- start 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8964a3460 0x55f896a5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f8967b1ee0 con 0x55f896893950 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8967f5910 con 0x55f89687cca0 2024-09-17T05:28:55.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f8967e9090 con 0x55f8964a3460 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 10 client.0 _ll_register_callbacks cb 0x7f51e80b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:37886/0 (socket says 172.21.15.43:37886) 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8964a3460 0x55f896a5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51efe00640 1 -- 172.21.15.43:0/2181234757 learned_addr learned my addr 172.21.15.43:0/2181234757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 -- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8964a3460 msgr2=0x55f896a5e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8964a3460 0x55f896a5e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 -- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 msgr2=0x55f896a5ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 -- 172.21.15.43:0/2181234757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8967acab0 con 0x55f896893950 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51efe00640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5a00640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5f220 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f51e801b390 tx=0x7f51e8003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891034bce9ae4dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80c3070 con 0x55f896893950 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e8004d40 con 0x55f896893950 2024-09-17T05:28:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f51e8003f80 con 0x55f896893950 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f51e80ce050 con 0x55f896893950 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 10 client.6540 fetch_fsmap learned FSMap version 70 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55f89678b070 con 0x55f896893950 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51e8004d40 con 0x55f896893950 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f51e80d1020 con 0x55f896893950 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 10 client.6540 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 10 client.6540 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 20 client.6540 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f896476830 0x55f896a1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 4 client.6540 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 --> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] -- command(tid 0: {"prefix": "status"}) -- 0x55f8967f4540 con 0x55f896476830 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5000640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f896476830 0x55f896a1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51f5000640 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f896476830 0x55f896a1aa80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04601d4e72f9aa3 server_cookie=30a6ac275a46af1f in_seq=0 out_seq=0 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.358+0000 7f51d6000640 10 client.6540 ms_handle_connect on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51d6000640 1 -- 172.21.15.43:0/2181234757 <== mds.0 v2:172.21.15.43:6834/3375674498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f8967f4540 con 0x55f896476830 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51d6000640 10 client.6540 handle_command_reply: tid=0 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 client.6540 shutdown 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 msgr2=0x55f896a5f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5f220 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f51e801b390 tx=0x7f51e8003d90 comp rx=0 tx=0).stop 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 shutdown_connections 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f89687cca0 0x55f896a5ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x55f896476830 0x55f896a1aa80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f896893950 0x55f896a5f220 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 --2- 172.21.15.43:0/2181234757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f8964a3460 0x55f896a5e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51d6000640 0 client.6540 ms_handle_reset on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 >> 172.21.15.43:0/2181234757 conn(0x55f896969d90 msgr2=0x55f8969e2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 shutdown_connections 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 1 -- 172.21.15.43:0/2181234757 wait complete. 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51fefb1000 20 client.6540 trim_cache size 0 max 16384 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6387, 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:28:55.367 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 10.798168198000001, 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 70, 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 75, 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 75, 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 16.284293559000002 2024-09-17T05:28:55.368 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 msgr2=0x7f51d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 0x7f51d0075710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f51f00715c0 tx=0x7f51e40091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 msgr2=0x7f51f009b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c3f0 tx=0x7f51dc00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 shutdown_connections 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51d0073200 0x7f51d0075710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51f01627a0 0x7f51f009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f0155d70 0x7f51f009b9e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 --2- 172.21.15.43:0/1070705997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51f006f110 0x7f51f0162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 >> 172.21.15.43:0/1070705997 conn(0x7f51f00b23d0 msgr2=0x7f51f015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 shutdown_connections 2024-09-17T05:28:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.362+0000 7f51f7a00640 1 -- 172.21.15.43:0/1070705997 wait complete. 2024-09-17T05:28:55.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- 172.21.15.43:0/4289007449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4340003af0 msgr2=0x7f4340003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 --2- 172.21.15.43:0/4289007449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4340003af0 0x7f4340003f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f434c009b10 tx=0x7f434c01d220 comp rx=0 tx=0).stop 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f435be00640 1 -- 172.21.15.43:0/4289007449 reap_dead start 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- 172.21.15.43:0/4289007449 shutdown_connections 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- 172.21.15.43:0/4289007449 >> 172.21.15.43:0/4289007449 conn(0x7f435c0b23d0 msgr2=0x7f435c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- 172.21.15.43:0/4289007449 shutdown_connections 2024-09-17T05:28:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- 172.21.15.43:0/4289007449 wait complete. 2024-09-17T05:28:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 Processor -- start 2024-09-17T05:28:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.542+0000 7f4362200640 1 -- start start 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 0x7f435c09d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 0x7f435c1655d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1667f0 con 0x7f435c165b10 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f435c166670 con 0x7f435c09d980 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f435c166970 con 0x7f4340003af0 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60848/0 (socket says 172.21.15.43:60848) 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 -- 172.21.15.43:0/186581978 learned_addr learned my addr 172.21.15.43:0/186581978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435b400640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 0x7f435c09d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4353e00640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 0x7f435c1655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 -- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 msgr2=0x7f435c09d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 0x7f435c09d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 -- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 msgr2=0x7f435c1655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 0x7f435c1655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 -- 172.21.15.43:0/186581978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c165f20 con 0x7f435c165b10 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4353e00640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 0x7f435c1655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435b400640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 0x7f435c09d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f435be00640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4354001340 tx=0x7f4354006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400fa00 con 0x7f435c165b10 2024-09-17T05:28:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4354019070 con 0x7f435c165b10 2024-09-17T05:28:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400fe80 con 0x7f435c165b10 2024-09-17T05:28:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1f9420 con 0x7f435c165b10 2024-09-17T05:28:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1f99e0 con 0x7f435c165b10 2024-09-17T05:28:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.546+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c0ca240 con 0x7f435c165b10 2024-09-17T05:28:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.550+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f435401e460 con 0x7f435c165b10 2024-09-17T05:28:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.550+0000 7f4353400640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 0x7f4338075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.550+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f4354094840 con 0x7f435c165b10 2024-09-17T05:28:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.550+0000 7f435b400640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 0x7f4338075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.550+0000 7f435b400640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 0x7f4338075740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f434c009b10 tx=0x7f434c019fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:55.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.554+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f435c165f20 con 0x7f435c165b10 2024-09-17T05:28:55.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f435c0b33e0 con 0x7f435c165b10 2024-09-17T05:28:55.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4353400640 1 -- 172.21.15.43:0/186581978 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7f435c0b3d30 con 0x7f435c165b10 2024-09-17T05:28:55.768 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 msgr2=0x7f4338075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 0x7f4338075740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f434c009b10 tx=0x7f434c019fc0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 msgr2=0x7f435c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4354001340 tx=0x7f4354006720 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 shutdown_connections 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4338073230 0x7f4338075740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c09d980 0x7f435c1655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f435c165b10 0x7f435c0cadb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 --2- 172.21.15.43:0/186581978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4340003af0 0x7f435c09d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 >> 172.21.15.43:0/186581978 conn(0x7f435c0b23d0 msgr2=0x7f435c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 shutdown_connections 2024-09-17T05:28:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:55.766+0000 7f4362200640 1 -- 172.21.15.43:0/186581978 wait complete. 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:55.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:28:55.791 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:28:55.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:28:55.791 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:55.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:55.791 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:28:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42108sec preferred_lft 42108sec 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:28:55.802 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.803 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:55.804 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:28:55.804 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:28:55.804 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:55.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:55.805 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:28:55.864 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:28:55.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:55.867 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:28:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:28:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:28:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:28:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:28:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:28:55.921 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:28:55.921 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:28:56.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:28:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:56.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:56.064 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:28:56.072 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:28:56.072 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:28:56.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:56.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:56.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:28:56.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:56.139 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:56.194 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:56.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:56.195 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:56.246 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:28:56.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:56.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:56.348 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:56.348 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:56.348 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:56.366 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:56.366 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:56.366 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:56.366 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:28:56.366 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:56.377 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:28:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:28:56.382 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:28:56.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:56.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:56.402 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:28:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:28:56.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:56.513 INFO:teuthology.orchestra.run.smithi043.stdout:client6304 2024-09-17T05:28:56.514 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:28:56.514 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:56.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 -- 172.21.15.43:0/2149541917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe7c003ad0 msgr2=0x7fbe7c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2149541917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe7c003ad0 0x7fbe7c003f00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009b30 tx=0x7fbe8801d250 comp rx=0 tx=0).stop 2024-09-17T05:28:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe97e00640 1 -- 172.21.15.43:0/2149541917 reap_dead start 2024-09-17T05:28:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 -- 172.21.15.43:0/2149541917 shutdown_connections 2024-09-17T05:28:56.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 -- 172.21.15.43:0/2149541917 >> 172.21.15.43:0/2149541917 conn(0x7fbe980b23d0 msgr2=0x7fbe980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:56.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 -- 172.21.15.43:0/2149541917 shutdown_connections 2024-09-17T05:28:56.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.722+0000 7fbe9e000640 1 -- 172.21.15.43:0/2149541917 wait complete. 2024-09-17T05:28:56.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 Processor -- start 2024-09-17T05:28:56.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 -- start start 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe7c003ad0 0x7fbe9809c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe98073090 0x7fbe980734a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98167ad0 con 0x7fbe981edf20 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98167950 con 0x7fbe98073090 2024-09-17T05:28:56.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe9e000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbe98167c50 con 0x7fbe7c003ad0 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe7c003ad0 0x7fbe9809c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60860/0 (socket says 172.21.15.43:60860) 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 -- 172.21.15.43:0/2647594877 learned_addr learned my addr 172.21.15.43:0/2647594877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe98073090 0x7fbe980734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 -- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe7c003ad0 msgr2=0x7fbe9809c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe7c003ad0 0x7fbe9809c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 -- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe98073090 msgr2=0x7fbe980734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe98073090 0x7fbe980734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.726+0000 7fbe97e00640 1 -- 172.21.15.43:0/2647594877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe981ee870 con 0x7fbe981edf20 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe97e00640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbe980c7280 tx=0x7fbe90001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90010890 con 0x7fbe981edf20 2024-09-17T05:28:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe90017070 con 0x7fbe981edf20 2024-09-17T05:28:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90010dc0 con 0x7fbe981edf20 2024-09-17T05:28:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe98073c50 con 0x7fbe981edf20 2024-09-17T05:28:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe980c7500 con 0x7fbe981edf20 2024-09-17T05:28:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe980ad0b0 con 0x7fbe981edf20 2024-09-17T05:28:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe9001c460 con 0x7fbe981edf20 2024-09-17T05:28:56.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 0x7fbe74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:56.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.730+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fbe90094b30 con 0x7fbe981edf20 2024-09-17T05:28:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.734+0000 7fbe97400640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 0x7fbe74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.734+0000 7fbe97400640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 0x7fbe74075740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe880ae000 tx=0x7fbe88019ff0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:56.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.738+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe981ee870 con 0x7fbe981edf20 2024-09-17T05:28:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe980739e0 con 0x7fbe981edf20 2024-09-17T05:28:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe8f400640 1 -- 172.21.15.43:0/2647594877 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5225 (secure 0 0 0) 0x7fbe9005f460 con 0x7fbe981edf20 2024-09-17T05:28:56.952 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:28:56.952 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"btime":"2024-09-17T05:28:47:595804+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:47.012214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:28:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 msgr2=0x7fbe74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 0x7fbe74075740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe880ae000 tx=0x7fbe88019ff0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 msgr2=0x7fbe981ee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbe980c7280 tx=0x7fbe90001ac0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 shutdown_connections 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe74073230 0x7fbe74075740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe98073090 0x7fbe980734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe981edf20 0x7fbe981ee330 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 --2- 172.21.15.43:0/2647594877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe7c003ad0 0x7fbe9809c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 >> 172.21.15.43:0/2647594877 conn(0x7fbe980b23d0 msgr2=0x7fbe980ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 shutdown_connections 2024-09-17T05:28:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:56.950+0000 7fbe9e000640 1 -- 172.21.15.43:0/2647594877 wait complete. 2024-09-17T05:28:56.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 -- 172.21.15.43:0/2870627194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 msgr2=0x7f9cb4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/2870627194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9cb4004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009d10 tx=0x7f9cb801d3f0 comp rx=0 tx=0).stop 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 -- 172.21.15.43:0/2870627194 shutdown_connections 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/2870627194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9cb4004c40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 -- 172.21.15.43:0/2870627194 >> 172.21.15.43:0/2870627194 conn(0x7f9ccc0691b0 msgr2=0x7f9ccc0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 -- 172.21.15.43:0/2870627194 shutdown_connections 2024-09-17T05:28:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.134+0000 7f9cd1e00640 1 -- 172.21.15.43:0/2870627194 wait complete. 2024-09-17T05:28:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 Processor -- start 2024-09-17T05:28:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 -- start start 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ccc1f7680 0x7f9ccc1f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ccc1558c0 0x7f9ccc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc160740 con 0x7f9cb4004810 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1605c0 con 0x7f9ccc1f7680 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc0d21a0 con 0x7f9ccc1558c0 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60886/0 (socket says 172.21.15.43:60886) 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 -- 172.21.15.43:0/1577346076 learned_addr learned my addr 172.21.15.43:0/1577346076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1400640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ccc1558c0 0x7f9ccc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 -- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ccc1558c0 msgr2=0x7f9ccc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ccc1558c0 0x7f9ccc155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 -- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ccc1f7680 msgr2=0x7f9ccc1f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ccc1f7680 0x7f9ccc1f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 -- 172.21.15.43:0/1577346076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc156210 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd0a00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005e40 tx=0x7f9cc0011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0011e90 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cc0018020 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003600 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.138+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1f3af0 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1f4070 con 0x7f9cb4004810 2024-09-17T05:28:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc0d2700 con 0x7f9cb4004810 2024-09-17T05:28:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9cc00037a0 con 0x7f9cb4004810 2024-09-17T05:28:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9cc8c00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 0x7f9cac0758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f9cc00979e0 con 0x7f9cb4004810 2024-09-17T05:28:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.142+0000 7f9ccbe00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 0x7f9cac0758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.146+0000 7f9ccbe00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 0x7f9cac0758e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b0000 tx=0x7f9cb4005840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.146+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9cc0022020 con 0x7f9cb4004810 2024-09-17T05:28:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:57.386+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f9ccc06a1e0 con 0x7f9cb4004810 2024-09-17T05:28:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.002+0000 7f9cc8c00640 1 -- 172.21.15.43:0/1577346076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6387 v71) ==== 112+0+0 (secure 0 0 0) 0x7f9ccc156210 con 0x7f9cb4004810 2024-09-17T05:28:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:freezing mds.6387 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 msgr2=0x7f9cac0758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 0x7f9cac0758e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b0000 tx=0x7f9cb4005840 comp rx=0 tx=0).stop 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 msgr2=0x7f9ccc0b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0005e40 tx=0x7f9cc0011740 comp rx=0 tx=0).stop 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 shutdown_connections 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cac0733d0 0x7f9cac0758e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ccc1f7680 0x7f9ccc1f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9ccc1558c0 0x7f9ccc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 --2- 172.21.15.43:0/1577346076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb4004810 0x7f9ccc0b7770 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 >> 172.21.15.43:0/1577346076 conn(0x7f9ccc0691b0 msgr2=0x7f9ccc0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 shutdown_connections 2024-09-17T05:28:58.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.006+0000 7f9cd1e00640 1 -- 172.21.15.43:0/1577346076 wait complete. 2024-09-17T05:28:58.034 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-09-17T05:28:58.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:28:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:58.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 -- 172.21.15.43:0/3231810768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 msgr2=0x7faa800700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 --2- 172.21.15.43:0/3231810768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa800700f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa78009c80 tx=0x7faa7801cc80 comp rx=0 tx=0).stop 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 -- 172.21.15.43:0/3231810768 shutdown_connections 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 --2- 172.21.15.43:0/3231810768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa800700f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 --2- 172.21.15.43:0/3231810768 >> v2:172.21.15.62:3300/0 conn(0x7faa8015f470 0x7faa800d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 -- 172.21.15.43:0/3231810768 >> 172.21.15.43:0/3231810768 conn(0x7faa800b2810 msgr2=0x7faa800b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 -- 172.21.15.43:0/3231810768 shutdown_connections 2024-09-17T05:28:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.266+0000 7faa85c00640 1 -- 172.21.15.43:0/3231810768 wait complete. 2024-09-17T05:28:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 Processor -- start 2024-09-17T05:28:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 -- start start 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa80142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800b5610 0x7faa80155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa801695d0 con 0x7faa8015f470 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa80169450 con 0x7faa800b5610 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa85c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faa80169750 con 0x7faa8006fce0 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.270+0000 7faa7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60894/0 (socket says 172.21.15.43:60894) 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 -- 172.21.15.43:0/1534919057 learned_addr learned my addr 172.21.15.43:0/1534919057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa80142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa76000640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800b5610 0x7faa80155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 -- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 msgr2=0x7faa80142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa80142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 -- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800b5610 msgr2=0x7faa80155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800b5610 0x7faa80155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 -- 172.21.15.43:0/1534919057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa801f8ed0 con 0x7faa8015f470 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa7f400640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7faa78009bd0 tx=0x7faa78005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa77400640 1 -- 172.21.15.43:0/1534919057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa780bb070 con 0x7faa8015f470 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa77400640 1 -- 172.21.15.43:0/1534919057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa78004b40 con 0x7faa8015f470 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801f91c0 con 0x7faa8015f470 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa77400640 1 -- 172.21.15.43:0/1534919057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa780b2040 con 0x7faa8015f470 2024-09-17T05:28:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.274+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801f9780 con 0x7faa8015f470 2024-09-17T05:28:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa77400640 1 -- 172.21.15.43:0/1534919057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa78004000 con 0x7faa8015f470 2024-09-17T05:28:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa77400640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 0x7faa5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 Processor -- start 2024-09-17T05:28:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa77400640 1 -- 172.21.15.43:0/1534919057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7faa7812e130 con 0x7faa8015f470 2024-09-17T05:28:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 -- start start 2024-09-17T05:28:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 0x7faa5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 0x7faa5c075710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faa70009600 tx=0x7faa700091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af2849f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af23f8aa0 0x563af284a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563af26e8cd0 con 0x563af26dd6d0 2024-09-17T05:28:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563af25fe330 con 0x563af23f8aa0 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563af265bbb0 con 0x563af2334890 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60896/0 (socket says 172.21.15.43:60896) 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 -- 172.21.15.43:0/1716991678 learned_addr learned my addr 172.21.15.43:0/1716991678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa76000640 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af2849f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7f400640 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af23f8aa0 0x563af284a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 -- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 msgr2=0x563af2849f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af2849f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 -- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af23f8aa0 msgr2=0x563af284a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af23f8aa0 0x563af284a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.278+0000 7faa7ea00640 1 -- 172.21.15.43:0/1716991678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af25ffc80 con 0x563af26dd6d0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa7ea00640 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa70011e80 tx=0x7faa70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742cf99b18bc3737 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa6f400640 1 -- 172.21.15.43:0/1716991678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70003d80 con 0x563af26dd6d0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa6f400640 1 -- 172.21.15.43:0/1716991678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa700044d0 con 0x563af26dd6d0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa6f400640 1 -- 172.21.15.43:0/1716991678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7001e660 con 0x563af26dd6d0 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- 172.21.15.43:0/1716991678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 msgr2=0x563af28499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa70011e80 tx=0x7faa70014000 comp rx=0 tx=0).stop 2024-09-17T05:28:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- 172.21.15.43:0/1716991678 shutdown_connections 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af23f8aa0 0x563af284a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af2849f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/1716991678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af26dd6d0 0x563af28499f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- 172.21.15.43:0/1716991678 >> 172.21.15.43:0/1716991678 conn(0x563af23085c0 msgr2=0x563af2806ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- 172.21.15.43:0/1716991678 shutdown_connections 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- 172.21.15.43:0/1716991678 wait complete. 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 -- start start 2024-09-17T05:28:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.282+0000 7faa8d2ae000 1 Processor -- start 2024-09-17T05:28:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af23f8aa0 0x563af2805ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af280c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563af264edf0 con 0x563af2334890 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563af264afe0 con 0x563af26dd6d0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563af25f04d0 con 0x563af23f8aa0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 10 client.0 _ll_register_callbacks cb 0x563af25ce7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60910/0 (socket says 172.21.15.43:60910) 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 -- 172.21.15.43:0/3280815599 learned_addr learned my addr 172.21.15.43:0/3280815599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa76000640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af23f8aa0 0x563af2805ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7f400640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af280c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 -- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af23f8aa0 msgr2=0x563af2805ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af23f8aa0 0x563af2805ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 -- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 msgr2=0x563af280c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af280c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 -- 172.21.15.43:0/3280815599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af25e9210 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa7ea00640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x563af26fe760 tx=0x7faa70026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd43152aa947cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70034060 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7faa700044d0 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563af25e3350 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7faa7003a050 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 10 client.6567 fetch_fsmap learned FSMap version 71 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x563af25cffc0 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563af25e3350 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7faa7003d020 con 0x563af2334890 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 10 client.6567 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T05:28:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 10 client.6567 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-09-17T05:28:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 20 client.6567 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22ed930 0x563af2850e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 4 client.6567 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:28:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 --> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af265db00 con 0x563af22ed930 2024-09-17T05:28:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa76000640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22ed930 0x563af2850e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 10 client.6567 ms_handle_connect on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa76000640 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22ed930 0x563af2850e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5bddc7d10b86a55 server_cookie=fd8c2e0486157103 in_seq=0 out_seq=0 2024-09-17T05:28:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 1 -- 172.21.15.43:0/3280815599 <== mds.0 v2:172.21.15.43:6834/3375674498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563af265db00 con 0x563af22ed930 2024-09-17T05:28:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa63e00640 10 client.6567 handle_command_reply: tid=0 2024-09-17T05:28:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.286+0000 7faa8d2ae000 1 client.6567 shutdown 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 msgr2=0x563af28059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x563af26fe760 tx=0x7faa70026630 comp rx=0 tx=0).stop 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 shutdown_connections 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af280c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22ed930 0x563af2850e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af23f8aa0 0x563af2805ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/3280815599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af2334890 0x563af28059b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa63e00640 0 client.6567 ms_handle_reset on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 >> 172.21.15.43:0/3280815599 conn(0x563af23085c0 msgr2=0x563af27fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 shutdown_connections 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 1 -- 172.21.15.43:0/3280815599 wait complete. 2024-09-17T05:28:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.290+0000 7faa8d2ae000 20 client.6567 trim_cache size 0 max 16384 2024-09-17T05:28:58.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 Processor -- start 2024-09-17T05:28:58.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 -- start start 2024-09-17T05:28:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563af2804b20 con 0x563af23f8aa0 2024-09-17T05:28:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa80205d80 con 0x563af26dd6d0 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faa80204da0 con 0x563af2334890 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60922/0 (socket says 172.21.15.43:60922) 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 -- 172.21.15.43:0/930348737 learned_addr learned my addr 172.21.15.43:0/930348737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa7ea00640 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa7f400640 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 -- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 msgr2=0x563af28bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 -- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 msgr2=0x563af28bbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28bbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 -- 172.21.15.43:0/930348737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af26a8680 con 0x563af23f8aa0 2024-09-17T05:28:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa76000640 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7faa68007fd0 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250a73c10e41e215 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa6ea00640 1 -- 172.21.15.43:0/930348737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68025070 con 0x563af23f8aa0 2024-09-17T05:28:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa6ea00640 1 -- 172.21.15.43:0/930348737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa680050d0 con 0x563af23f8aa0 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa6ea00640 1 -- 172.21.15.43:0/930348737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68016a50 con 0x563af23f8aa0 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 -- 172.21.15.43:0/930348737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 msgr2=0x563af28bb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.306+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7faa68007fd0 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- 172.21.15.43:0/930348737 shutdown_connections 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28bbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bb940 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/930348737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- 172.21.15.43:0/930348737 >> 172.21.15.43:0/930348737 conn(0x563af23085c0 msgr2=0x563af285bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- 172.21.15.43:0/930348737 shutdown_connections 2024-09-17T05:28:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- 172.21.15.43:0/930348737 wait complete. 2024-09-17T05:28:58.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- start start 2024-09-17T05:28:58.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 Processor -- start 2024-09-17T05:28:58.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa80204880 con 0x563af23f8aa0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa801fce00 con 0x563af26dd6d0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563af2310680 con 0x563af2334890 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa8d2ae000 10 client.0 _ll_register_callbacks cb 0x7faa70009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60932/0 (socket says 172.21.15.43:60932) 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 -- 172.21.15.43:0/525676858 learned_addr learned my addr 172.21.15.43:0/525676858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa7ea00640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa7f400640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 -- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 msgr2=0x563af28bf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 -- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 msgr2=0x563af28c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 -- 172.21.15.43:0/525676858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af26150b0 con 0x563af23f8aa0 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa7f400640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa76000640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faa68005040 tx=0x7faa68005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65e8af05871efb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa63e00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.310+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68025070 con 0x563af23f8aa0 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7faa68007030 con 0x563af23f8aa0 2024-09-17T05:28:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa680050d0 con 0x563af23f8aa0 2024-09-17T05:28:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7faa68032050 con 0x563af23f8aa0 2024-09-17T05:28:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 10 client.6573 fetch_fsmap learned FSMap version 71 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x563af25e1610 con 0x563af23f8aa0 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa680050d0 con 0x563af23f8aa0 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7faa68035020 con 0x563af23f8aa0 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 10 client.6573 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 10 client.6573 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 20 client.6573 populate_metadata read hostname 'smithi043' 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22db990 0x563af2800a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 4 client.6573 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 --> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563af26595b0 con 0x563af22db990 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa7ea00640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22db990 0x563af2800a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa63e00640 10 client.6573 ms_handle_connect on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.314+0000 7faa7ea00640 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22db990 0x563af2800a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9592d5c15fd63367 server_cookie=c37cb5c8d3157e77 in_seq=0 out_seq=0 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa63e00640 1 -- 172.21.15.43:0/525676858 <== mds.0 v2:172.21.15.43:6834/3375674498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563af26595b0 con 0x563af22db990 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa63e00640 10 client.6573 handle_command_reply: tid=0 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 client.6573 shutdown 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 msgr2=0x563af28bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faa68005040 tx=0x7faa68005490 comp rx=0 tx=0).stop 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 shutdown_connections 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563af26dd6d0 0x563af28c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:6834/3375674498,v1:172.21.15.43:6835/3375674498] conn(0x563af22db990 0x563af2800a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563af23f8aa0 0x563af28bfb90 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 --2- 172.21.15.43:0/525676858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563af2334890 0x563af28bf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 >> 172.21.15.43:0/525676858 conn(0x563af23085c0 msgr2=0x563af280b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa63e00640 0 client.6573 ms_handle_reset on v2:172.21.15.43:6834/3375674498 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 shutdown_connections 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 1 -- 172.21.15.43:0/525676858 wait complete. 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa8d2ae000 20 client.6573 trim_cache size 0 max 16384 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:28:58.337 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 msgr2=0x7faa5c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 0x7faa5c075710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faa70009600 tx=0x7faa700091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 msgr2=0x7faa800b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7faa78009bd0 tx=0x7faa78005b00 comp rx=0 tx=0).stop 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 shutdown_connections 2024-09-17T05:28:58.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa5c073200 0x7faa5c075710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa800b5610 0x7faa80155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa8015f470 0x7faa800b1800 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 --2- 172.21.15.43:0/1534919057 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa8006fce0 0x7faa80142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 >> 172.21.15.43:0/1534919057 conn(0x7faa800b2810 msgr2=0x7faa80152250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 shutdown_connections 2024-09-17T05:28:58.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.334+0000 7faa85c00640 1 -- 172.21.15.43:0/1534919057 wait complete. 2024-09-17T05:28:58.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-17T05:28:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 -- 172.21.15.43:0/3759253039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 msgr2=0x7f076015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 --2- 172.21.15.43:0/3759253039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015c7e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0750009840 tx=0x7f075001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 -- 172.21.15.43:0/3759253039 shutdown_connections 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 --2- 172.21.15.43:0/3759253039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015c7e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 -- 172.21.15.43:0/3759253039 >> 172.21.15.43:0/3759253039 conn(0x7f07600b23d0 msgr2=0x7f07600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 -- 172.21.15.43:0/3759253039 shutdown_connections 2024-09-17T05:28:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.514+0000 7f0767200640 1 -- 172.21.15.43:0/3759253039 wait complete. 2024-09-17T05:28:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 Processor -- start 2024-09-17T05:28:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 -- start start 2024-09-17T05:28:58.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07600cba20 0x7f07600cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0760169dc0 con 0x7f076006f110 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0760169c40 con 0x7f0760155d70 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0767200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0760169f40 con 0x7f07600cba20 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60944/0 (socket says 172.21.15.43:60944) 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 -- 172.21.15.43:0/3128917084 learned_addr learned my addr 172.21.15.43:0/3128917084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0757400640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0765200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07600cba20 0x7f07600cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 -- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07600cba20 msgr2=0x7f07600cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07600cba20 0x7f07600cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 -- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 msgr2=0x7f076015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f075fe00640 1 -- 172.21.15.43:0/3128917084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076015e410 con 0x7f076006f110 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.518+0000 7f0757400640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075fe00640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0750009790 tx=0x7f0750005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500ac070 con 0x7f076006f110 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0750004b40 con 0x7f076006f110 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500a66b0 con 0x7f076006f110 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076015e6f0 con 0x7f076006f110 2024-09-17T05:28:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07600c70c0 con 0x7f076006f110 2024-09-17T05:28:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07600cb2f0 con 0x7f076006f110 2024-09-17T05:28:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f07500af070 con 0x7f076006f110 2024-09-17T05:28:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 0x7f073c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:58.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.522+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f075012e310 con 0x7f076006f110 2024-09-17T05:28:58.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.526+0000 7f0757400640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 0x7f073c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:58.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.526+0000 7f0757400640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 0x7f073c075710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f076015e150 tx=0x7f0748009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.530+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0750016330 con 0x7f076006f110 2024-09-17T05:28:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.702+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f076015c7e0 con 0x7f076006f110 2024-09-17T05:28:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.706+0000 7f075cc00640 1 -- 172.21.15.43:0/3128917084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v44)=0 v44) ==== 138+0+0 (secure 0 0 0) 0x7f07500f9290 con 0x7f076006f110 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 msgr2=0x7f073c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 0x7f073c075710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f076015e150 tx=0x7f0748009040 comp rx=0 tx=0).stop 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 msgr2=0x7f07600c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0750009790 tx=0x7f0750005d80 comp rx=0 tx=0).stop 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 shutdown_connections 2024-09-17T05:28:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f073c073200 0x7f073c075710 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0760155d70 0x7f076015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f07600cba20 0x7f07600cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 --2- 172.21.15.43:0/3128917084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076006f110 0x7f07600c6b30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 >> 172.21.15.43:0/3128917084 conn(0x7f07600b23d0 msgr2=0x7f07600ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 shutdown_connections 2024-09-17T05:28:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:28:58.710+0000 7f0767200640 1 -- 172.21.15.43:0/3128917084 wait complete. 2024-09-17T05:29:03.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-17T05:29:03.738 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-09-17T05:29:03.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-17T05:29:03.801 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] with next_snap:2 2024-09-17T05:29:03.801 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] 2024-09-17T05:29:03.801 INFO:tasks.ceph.mds.a.smithi043.stderr:./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fac6dc00640 time 2024-09-17T05:29:03.801774+0000 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7fac74e4330e] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.802 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr:*** Caught signal (Aborted) ** 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr: in thread 7fac6dc00640 thread_name:ms_dispatch 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fac6dc00640 time 2024-09-17T05:29:03.801774+0000 2024-09-17T05:29:03.803 INFO:tasks.ceph.mds.a.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7fac74e4330e] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.804 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.805 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fac74442520] 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: raise() 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: abort() 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7fac74e43480] 2024-09-17T05:29:03.806 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.807 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.808 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.808 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.808 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.808 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 24: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:29:03.802+0000 7fac6dc00640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: in thread 7fac6dc00640 thread_name:ms_dispatch 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fac74442520] 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:29:03.809 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: raise() 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: abort() 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7fac74e43480] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.810 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 24: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:29:03.811 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.816 INFO:tasks.ceph.mds.a.smithi043.stderr: -5095> 2024-09-17T05:28:49.542+0000 7fac6aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr: -9> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] with next_snap:2 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr: -7> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr: -1> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fac6dc00640 time 2024-09-17T05:29:03.801774+0000 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.824 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7fac74e4330e] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.825 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 0> 2024-09-17T05:29:03.802+0000 7fac6dc00640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: in thread 7fac6dc00640 thread_name:ms_dispatch 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.826 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fac74442520] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: raise() 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: abort() 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7fac74e43480] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.827 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 24: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.828 INFO:tasks.ceph.mds.a.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.829 INFO:tasks.ceph.mds.a.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:29:03.829 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.829 INFO:tasks.ceph.mds.a.smithi043.stderr: -5095> 2024-09-17T05:28:49.542+0000 7fac6aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: -9> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] with next_snap:2 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: -7> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: -1> 2024-09-17T05:29:03.798+0000 7fac6dc00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fac6dc00640 time 2024-09-17T05:29:03.801774+0000 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr:./src/mds/MDSRank.cc: 913: ceph_abort_msg("abort() called") 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7fac74e4330e] 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.833 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.834 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: 0> 2024-09-17T05:29:03.802+0000 7fac6dc00640 -1 *** Caught signal (Aborted) ** 2024-09-17T05:29:03.835 INFO:tasks.ceph.mds.a.smithi043.stderr: in thread 7fac6dc00640 thread_name:ms_dispatch 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev) 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fac74442520] 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 2: pthread_kill() 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 3: raise() 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 4: abort() 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7fac74e43480] 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55aca656009b] 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 7: (CDentry::check_corruption(bool)+0x54e) [0x55aca68367d0] 2024-09-17T05:29:03.836 INFO:tasks.ceph.mds.a.smithi043.stderr: 8: ceph-mds(+0x178c40) [0x55aca65c8c40] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 9: ceph-mds(+0x179040) [0x55aca65c9040] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x2571) [0x55aca666cfc9] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2ec1) [0x55aca66707bd] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xd27) [0x55aca6635045] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x55aca6635cf2] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x55aca661c74a] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x55aca656a696] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55aca656e4cf] 2024-09-17T05:29:03.837 INFO:tasks.ceph.mds.a.smithi043.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55aca656edf6] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55aca6551ef6] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 19: /usr/lib/ceph/libceph-common.so.2(+0x390157) [0x7fac74f90157] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 20: (DispatchQueue::entry()+0x641) [0x7fac74f90a7b] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 21: /usr/lib/ceph/libceph-common.so.2(+0x47e7a9) [0x7fac7507e7a9] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7fac74e0ce65] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7fac74e0ce81] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 24: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fac74494b43] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 25: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fac74526a00] 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-17T05:29:03.838 INFO:tasks.ceph.mds.a.smithi043.stderr: 2024-09-17T05:29:03.921 INFO:teuthology.orchestra.run.smithi043.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-17T05:29:03.922 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-17T05:29:03.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:03.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1956481729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 msgr2=0x7f5ecc15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 --2- 172.21.15.43:0/1956481729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc15ff20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0cb8a0 tx=0x7f5eb401c800 comp rx=0 tx=0).stop 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1956481729 shutdown_connections 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 --2- 172.21.15.43:0/1956481729 >> v2:172.21.15.62:3300/0 conn(0x7f5ecc0b0980 0x7f5ecc0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 --2- 172.21.15.43:0/1956481729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc15ff20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1956481729 >> 172.21.15.43:0/1956481729 conn(0x7f5ecc09aa50 msgr2=0x7f5ecc0d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1956481729 shutdown_connections 2024-09-17T05:29:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1956481729 wait complete. 2024-09-17T05:29:03.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 Processor -- start 2024-09-17T05:29:03.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.942+0000 7f5ed0c00640 1 -- start start 2024-09-17T05:29:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ecc0b0980 0x7f5ecc1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ecc0b27a0 0x7f5ecc1f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc1fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b1c60 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b1ae0 con 0x7f5ecc0b0980 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b1de0 con 0x7f5ecc0b27a0 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc1fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc1fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44714/0 (socket says 172.21.15.43:44714) 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 -- 172.21.15.43:0/1891375261 learned_addr learned my addr 172.21.15.43:0/1891375261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec9e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ecc0b0980 0x7f5ecc1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec3e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ecc0b27a0 0x7f5ecc1f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 -- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ecc0b27a0 msgr2=0x7f5ecc1f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ecc0b27a0 0x7f5ecc1f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 -- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ecc0b0980 msgr2=0x7f5ecc1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ecc0b0980 0x7f5ecc1570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 -- 172.21.15.43:0/1891375261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc1faaf0 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec9e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ecc0b0980 0x7f5ecc1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec3e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ecc0b27a0 0x7f5ecc1f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5eca800640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ecc0b5de0 0x7f5ecc1fa2e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400e970 tx=0x7f5ec400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4004d30 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec400cb20 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1891375261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc09b420 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4005020 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1891375261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc0c8ef0 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.946+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1891375261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5ecc0b0d90 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ec40051c0 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec2000640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ea8073200 0x7f5ea8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7f5ec4095130 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec9e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ea8073200 0x7f5ea8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+509 (secure 0 0 0) 0x7f5ec4016030 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ed0c00640 1 -- 172.21.15.43:0/1891375261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f5ecc1fa880 con 0x7f5ecc0b5de0 2024-09-17T05:29:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:03.950+0000 7f5ec9e00640 1 --2- 172.21.15.43:0/1891375261 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ea8073200 0x7f5ea8075710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0016b0 tx=0x7f5ebc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:04.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.022+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 7 ==== log(1 entries from seq 1 at 2024-09-17T05:29:03.801732+0000) ==== 468+0+0 (secure 0 0 0) 0x7f5ec40936d0 con 0x7f5ecc0b5de0 2024-09-17T05:29:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: cluster: 2024-09-17T05:29:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: id: 1cac0027-7a1b-472a-9fc8-e0027668aa6d 2024-09-17T05:29:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: health: HEALTH_OK 2024-09-17T05:29:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: services: 2024-09-17T05:29:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-17T05:29:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: mgr: x(active, since 6m), standbys: y 2024-09-17T05:29:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds: 1/1 daemons up, 3 standby 2024-09-17T05:29:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: osd: 8 osds: 8 up (since 5m), 8 in (since 6m) 2024-09-17T05:29:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: data: 2024-09-17T05:29:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: volumes: 1/1 healthy 2024-09-17T05:29:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: pools: 3 pools, 97 pgs 2024-09-17T05:29:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: objects: 29 objects, 579 KiB 2024-09-17T05:29:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: usage: 252 MiB used, 720 GiB / 720 GiB avail 2024-09-17T05:29:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: pgs: 97 active+clean 2024-09-17T05:29:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: io: 2024-09-17T05:29:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: client: 0 B/s rd, 682 B/s wr, 0 op/s rd, 2 op/s wr 2024-09-17T05:29:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.029 INFO:teuthology.orchestra.run.smithi043.stdout:2024-09-17T05:29:03.801732+0000 mds.a [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6304:47 nref=7 cr=0x55acb0229880)) (dversion lock w=1 last_client=6304) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x55acb03d5680] 2024-09-17T05:29:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 -- 172.21.15.43:0/647957032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 msgr2=0x7fe6e80b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 --2- 172.21.15.43:0/647957032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b2740 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009900 tx=0x7fe6d401c810 comp rx=0 tx=0).stop 2024-09-17T05:29:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 -- 172.21.15.43:0/647957032 shutdown_connections 2024-09-17T05:29:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 --2- 172.21.15.43:0/647957032 >> v2:172.21.15.62:3300/0 conn(0x7fe6e80cd9a0 0x7fe6e80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 --2- 172.21.15.43:0/647957032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b2740 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 -- 172.21.15.43:0/647957032 >> 172.21.15.43:0/647957032 conn(0x7fe6e80b2e80 msgr2=0x7fe6e80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 -- 172.21.15.43:0/647957032 shutdown_connections 2024-09-17T05:29:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.082+0000 7fe6f0600640 1 -- 172.21.15.43:0/647957032 wait complete. 2024-09-17T05:29:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 Processor -- start 2024-09-17T05:29:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 -- start start 2024-09-17T05:29:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6e80b6b10 0x7fe6e81fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6e80cd9a0 0x7fe6e81f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e816a110 con 0x7fe6e80b2330 2024-09-17T05:29:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8169f90 con 0x7fe6e80cd9a0 2024-09-17T05:29:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6f0600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e816a290 con 0x7fe6e80b6b10 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6edc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6edc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44738/0 (socket says 172.21.15.43:44738) 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6edc00640 1 -- 172.21.15.43:0/3053049181 learned_addr learned my addr 172.21.15.43:0/3053049181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6ed200640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6e80b6b10 0x7fe6e81fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.086+0000 7fe6ee600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6e80cd9a0 0x7fe6e81f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 -- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6e80b6b10 msgr2=0x7fe6e81fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:04.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6e80b6b10 0x7fe6e81fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 -- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6e80cd9a0 msgr2=0x7fe6e81f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:04.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6e80cd9a0 0x7fe6e81f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 -- 172.21.15.43:0/3053049181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e81fd350 con 0x7fe6e80b2330 2024-09-17T05:29:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6edc00640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c3c0 tx=0x7fe6dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc017070 con 0x7fe6e80b2330 2024-09-17T05:29:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6dc0053c0 con 0x7fe6e80b2330 2024-09-17T05:29:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc003860 con 0x7fe6e80b2330 2024-09-17T05:29:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e815ba90 con 0x7fe6e80b2330 2024-09-17T05:29:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e81f9030 con 0x7fe6e80b2330 2024-09-17T05:29:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e81f8e40 con 0x7fe6e80b2330 2024-09-17T05:29:04.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.090+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6dc01a070 con 0x7fe6e80b2330 2024-09-17T05:29:04.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.094+0000 7fe6e5600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 0x7fe6c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:04.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.094+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5709+0+0 (secure 0 0 0) 0x7fe6dc093580 con 0x7fe6e80b2330 2024-09-17T05:29:04.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.094+0000 7fe6ed200640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 0x7fe6c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.094+0000 7fe6ed200640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 0x7fe6c8075710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40af000 tx=0x7fe6d401b050 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.098+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe6e80b2c80 con 0x7fe6e80b2330 2024-09-17T05:29:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.306+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6e80b2740 con 0x7fe6e80b2330 2024-09-17T05:29:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.306+0000 7fe6e5600640 1 -- 172.21.15.43:0/3053049181 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fe6dc05deb0 con 0x7fe6e80b2330 2024-09-17T05:29:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:04.311 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"btime":"2024-09-17T05:28:57:996909+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:57.390603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:29:04.311 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 71 2024-09-17T05:29:04.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 msgr2=0x7fe6c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:04.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 0x7fe6c8075710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40af000 tx=0x7fe6d401b050 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 msgr2=0x7fe6e80b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c3c0 tx=0x7fe6dc00c880 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 shutdown_connections 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6c8073200 0x7fe6c8075710 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6e80cd9a0 0x7fe6e81f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6e80b6b10 0x7fe6e81fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 --2- 172.21.15.43:0/3053049181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e80b2330 0x7fe6e80b5da0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 >> 172.21.15.43:0/3053049181 conn(0x7fe6e80b2e80 msgr2=0x7fe6e806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 shutdown_connections 2024-09-17T05:29:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:04.310+0000 7fe6f0600640 1 -- 172.21.15.43:0/3053049181 wait complete. 2024-09-17T05:29:04.334 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:29:04.661 INFO:tasks.ceph.mds.a.smithi043.stderr:daemon-helper: command crashed with signal 6 2024-09-17T05:29:05.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:05.034+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 8 ==== log(4 entries from seq 709 at 2024-09-17T05:29:04.666552+0000) ==== 1118+0+0 (secure 0 0 0) 0x7f5ec405bd40 con 0x7f5ecc0b5de0 2024-09-17T05:29:06.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:06.034+0000 7f5ec2000640 1 -- 172.21.15.43:0/1891375261 <== mon.0 v2:172.21.15.43:3300/0 9 ==== log(2 entries from seq 711 at 2024-09-17T05:29:05.026605+0000) ==== 559+0+0 (secure 0 0 0) 0x7f5ec405fd30 con 0x7f5ecc0b5de0 2024-09-17T05:29:09.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- 172.21.15.43:0/2214407467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a4004810 msgr2=0x7fa3a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 --2- 172.21.15.43:0/2214407467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009b40 tx=0x7fa3ac01d270 comp rx=0 tx=0).stop 2024-09-17T05:29:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c2800640 1 -- 172.21.15.43:0/2214407467 reap_dead start 2024-09-17T05:29:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- 172.21.15.43:0/2214407467 shutdown_connections 2024-09-17T05:29:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- 172.21.15.43:0/2214407467 >> 172.21.15.43:0/2214407467 conn(0x7fa3bc0cee60 msgr2=0x7fa3bc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- 172.21.15.43:0/2214407467 shutdown_connections 2024-09-17T05:29:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- 172.21.15.43:0/2214407467 wait complete. 2024-09-17T05:29:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 Processor -- start 2024-09-17T05:29:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.506+0000 7fa3c4800640 1 -- start start 2024-09-17T05:29:09.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3a4004810 0x7fa3bc15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:09.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3bc15dd00 0x7fa3bc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:09.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:09.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cd1c0 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cd040 con 0x7fa3bc15dd00 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc0cd340 con 0x7fa3a4004810 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44752/0 (socket says 172.21.15.43:44752) 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 -- 172.21.15.43:0/85878894 learned_addr learned my addr 172.21.15.43:0/85878894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c1400640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3bc15dd00 0x7fa3bc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c1e00640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3a4004810 0x7fa3bc15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 -- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3a4004810 msgr2=0x7fa3bc15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3a4004810 0x7fa3bc15d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 -- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3bc15dd00 msgr2=0x7fa3bc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3bc15dd00 0x7fa3bc06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 -- 172.21.15.43:0/85878894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc06ec60 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c2800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0b1c60 tx=0x7fa3b80119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8013920 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3b8013bb0 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc06ef40 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8012440 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc06f500 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.510+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3bc0b3130 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.514+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3b80125e0 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.514+0000 7fa3b6000640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 0x7fa3a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.514+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5767+0+0 (secure 0 0 0) 0x7fa3b8097f70 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.514+0000 7fa3c1e00640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 0x7fa3a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:09.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.514+0000 7fa3c1e00640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 0x7fa3a0075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d80 tx=0x7fa3b00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:09.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.518+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa3b8062e30 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3bc0b5740 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3b6000640 1 -- 172.21.15.43:0/85878894 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fa3b8062730 con 0x7fa3bc06e4b0 2024-09-17T05:29:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:09.733 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"btime":"2024-09-17T05:28:57:996909+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:57.390603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:29:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 71 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 msgr2=0x7fa3a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 0x7fa3a0075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009d80 tx=0x7fa3b00091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 msgr2=0x7fa3bc1547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0b1c60 tx=0x7fa3b80119e0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 shutdown_connections 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3a0073200 0x7fa3a0075710 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3bc15dd00 0x7fa3bc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc06e4b0 0x7fa3bc1547c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 --2- 172.21.15.43:0/85878894 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3a4004810 0x7fa3bc15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:09.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 >> 172.21.15.43:0/85878894 conn(0x7fa3bc0cee60 msgr2=0x7fa3bc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:09.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 shutdown_connections 2024-09-17T05:29:09.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:09.730+0000 7fa3c4800640 1 -- 172.21.15.43:0/85878894 wait complete. 2024-09-17T05:29:09.754 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:29:11.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T05:29:14.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 -- 172.21.15.43:0/2808205520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c155d70 msgr2=0x7fbe2c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 --2- 172.21.15.43:0/2808205520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c155d70 0x7fbe2c15c7e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c071050 tx=0x7fbe1c01c800 comp rx=0 tx=0).stop 2024-09-17T05:29:14.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 -- 172.21.15.43:0/2808205520 shutdown_connections 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 --2- 172.21.15.43:0/2808205520 >> v2:172.21.15.62:3300/0 conn(0x7fbe2c0b3920 0x7fbe2c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 --2- 172.21.15.43:0/2808205520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 0x7fbe2c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 --2- 172.21.15.43:0/2808205520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c155d70 0x7fbe2c15c7e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 -- 172.21.15.43:0/2808205520 >> 172.21.15.43:0/2808205520 conn(0x7fbe2c0b23d0 msgr2=0x7fbe2c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 -- 172.21.15.43:0/2808205520 shutdown_connections 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.922+0000 7fbe30e00640 1 -- 172.21.15.43:0/2808205520 wait complete. 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 Processor -- start 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- start start 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 0x7fbe2c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe2c155d70 0x7fbe2c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:14.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c15f8d0 con 0x7fbe2c0b3920 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c15f750 con 0x7fbe2c155d70 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c15fa50 con 0x7fbe2c06f110 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45204/0 (socket says 172.21.15.43:45204) 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 -- 172.21.15.43:0/248172907 learned_addr learned my addr 172.21.15.43:0/248172907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe2a800640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe2c155d70 0x7fbe2c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe29e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 0x7fbe2c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 -- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 msgr2=0x7fbe2c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 0x7fbe2c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 -- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe2c155d70 msgr2=0x7fbe2c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe2c155d70 0x7fbe2c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 -- 172.21.15.43:0/248172907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1f72c0 con 0x7fbe2c0b3920 2024-09-17T05:29:14.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe23e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e770 tx=0x7fbe1400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400ccc0 con 0x7fbe2c0b3920 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1400ce60 con 0x7fbe2c0b3920 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c1f77c0 con 0x7fbe2c0b3920 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005280 con 0x7fbe2c0b3920 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.926+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c0c8680 con 0x7fbe2c0b3920 2024-09-17T05:29:14.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c15ff40 con 0x7fbe2c0b3920 2024-09-17T05:29:14.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe14005420 con 0x7fbe2c0b3920 2024-09-17T05:29:14.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe22000640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 0x7fbe08075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:14.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5767+0+0 (secure 0 0 0) 0x7fbe14094cd0 con 0x7fbe2c0b3920 2024-09-17T05:29:14.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe29e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 0x7fbe08075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:14.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.930+0000 7fbe29e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 0x7fbe08075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0af000 tx=0x7fbe1c01b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:14.934+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe1405fc20 con 0x7fbe2c0b3920 2024-09-17T05:29:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.146+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe2c1f7550 con 0x7fbe2c0b3920 2024-09-17T05:29:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.146+0000 7fbe22000640 1 -- 172.21.15.43:0/248172907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fbe1405f520 con 0x7fbe2c0b3920 2024-09-17T05:29:15.151 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:15.151 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"btime":"2024-09-17T05:28:57:996909+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:28:57.390603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:29:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 71 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 msgr2=0x7fbe08075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 0x7fbe08075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0af000 tx=0x7fbe1c01b040 comp rx=0 tx=0).stop 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 msgr2=0x7fbe2c1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e770 tx=0x7fbe1400ec30 comp rx=0 tx=0).stop 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 shutdown_connections 2024-09-17T05:29:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe08073230 0x7fbe08075740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe2c155d70 0x7fbe2c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2c0b3920 0x7fbe2c1544a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 --2- 172.21.15.43:0/248172907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe2c06f110 0x7fbe2c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 >> 172.21.15.43:0/248172907 conn(0x7fbe2c0b23d0 msgr2=0x7fbe2c15cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 shutdown_connections 2024-09-17T05:29:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:15.150+0000 7fbe30e00640 1 -- 172.21.15.43:0/248172907 wait complete. 2024-09-17T05:29:15.173 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-17T05:29:18.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-17T05:29:20.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- 172.21.15.43:0/1498579173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f768c004be0 msgr2=0x7f768c005010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 --2- 172.21.15.43:0/1498579173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f768c004be0 0x7f768c005010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7698006fb0 tx=0x7f769801cf00 comp rx=0 tx=0).stop 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76a7e00640 1 -- 172.21.15.43:0/1498579173 reap_dead start 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- 172.21.15.43:0/1498579173 shutdown_connections 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- 172.21.15.43:0/1498579173 >> 172.21.15.43:0/1498579173 conn(0x7f76a80b23d0 msgr2=0x7f76a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- 172.21.15.43:0/1498579173 shutdown_connections 2024-09-17T05:29:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- 172.21.15.43:0/1498579173 wait complete. 2024-09-17T05:29:20.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 Processor -- start 2024-09-17T05:29:20.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.345+0000 7f76ae800640 1 -- start start 2024-09-17T05:29:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f768c004be0 0x7f76a81fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a809d2c0 0x7f76a80ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76a80cc2e0 con 0x7f76a809d6d0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76a80cc160 con 0x7f768c004be0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76a80cc460 con 0x7f76a809d2c0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45226/0 (socket says 172.21.15.43:45226) 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 -- 172.21.15.43:0/726323626 learned_addr learned my addr 172.21.15.43:0/726323626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f769ea00640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a809d2c0 0x7f76a80ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7400640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f768c004be0 0x7f76a81fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 -- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a809d2c0 msgr2=0x7f76a80ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a809d2c0 0x7f76a80ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 -- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f768c004be0 msgr2=0x7f76a81fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f768c004be0 0x7f76a81fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 -- 172.21.15.43:0/726323626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a81fa940 con 0x7f76a809d6d0 2024-09-17T05:29:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76a7e00640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001530 tx=0x7f76a0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011900 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76a0018070 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011d80 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a81613f0 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81f6a30 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.349+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8161160 con 0x7f76a809d6d0 2024-09-17T05:29:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.353+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76a001d460 con 0x7f76a809d6d0 2024-09-17T05:29:20.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.353+0000 7f769fe00640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 0x7f7684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.353+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5767+0+0 (secure 0 0 0) 0x7f76a0094960 con 0x7f76a809d6d0 2024-09-17T05:29:20.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.353+0000 7f76a7400640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 0x7f7684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.353+0000 7f76a7400640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 0x7f7684075710 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76980ab000 tx=0x7f7698019d40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:20.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.357+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76a81fa940 con 0x7f76a809d6d0 2024-09-17T05:29:20.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76a806ce60 con 0x7f76a809d6d0 2024-09-17T05:29:20.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f769fe00640 1 -- 172.21.15.43:0/726323626 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5273 (secure 0 0 0) 0x7f76a005f8a0 con 0x7f76a809d6d0 2024-09-17T05:29:20.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:20.572 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"btime":"2024-09-17T05:29:18:204532+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:18.204464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3375674498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3375674498},{"type":"v1","addr":"172.21.15.43:6835","nonce":3375674498}]},"join_fscid":-1,"laggy_since":"2024-09-17T05:29:18.204464+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-09-17T05:29:20.573 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 72 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 msgr2=0x7f7684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 0x7f7684075710 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76980ab000 tx=0x7f7698019d40 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 msgr2=0x7f76a80cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001530 tx=0x7f76a0006430 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 shutdown_connections 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7684073200 0x7f7684075710 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a809d6d0 0x7f76a80cd150 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76a809d2c0 0x7f76a80ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 --2- 172.21.15.43:0/726323626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f768c004be0 0x7f76a81fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 >> 172.21.15.43:0/726323626 conn(0x7f76a80b23d0 msgr2=0x7f76a80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 shutdown_connections 2024-09-17T05:29:20.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.569+0000 7f76ae800640 1 -- 172.21.15.43:0/726323626 wait complete. 2024-09-17T05:29:20.595 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-17T05:29:20.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 -- 172.21.15.43:0/2484701755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df4155d70 msgr2=0x7f5df415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 --2- 172.21.15.43:0/2484701755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df4155d70 0x7f5df415c7e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009900 tx=0x7f5de401c820 comp rx=0 tx=0).stop 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 -- 172.21.15.43:0/2484701755 shutdown_connections 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 --2- 172.21.15.43:0/2484701755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df406f110 0x7f5df40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 --2- 172.21.15.43:0/2484701755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df4155d70 0x7f5df415c7e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 -- 172.21.15.43:0/2484701755 >> 172.21.15.43:0/2484701755 conn(0x7f5df40b23d0 msgr2=0x7f5df40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 -- 172.21.15.43:0/2484701755 shutdown_connections 2024-09-17T05:29:20.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.853+0000 7f5dfae00640 1 -- 172.21.15.43:0/2484701755 wait complete. 2024-09-17T05:29:20.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 Processor -- start 2024-09-17T05:29:20.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 -- start start 2024-09-17T05:29:20.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df406f110 0x7f5df4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df4155d70 0x7f5df40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4168ab0 con 0x7f5df40b3920 2024-09-17T05:29:20.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4168930 con 0x7f5df406f110 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5dfae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5df4168c30 con 0x7f5df4155d70 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5debe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5debe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45238/0 (socket says 172.21.15.43:45238) 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5debe00640 1 -- 172.21.15.43:0/3889650544 learned_addr learned my addr 172.21.15.43:0/3889650544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5df8e00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df4155d70 0x7f5df40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.857+0000 7f5df3e00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df406f110 0x7f5df4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 -- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df4155d70 msgr2=0x7f5df40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df4155d70 0x7f5df40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 -- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df406f110 msgr2=0x7f5df4073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:20.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df406f110 0x7f5df4073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:20.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 -- 172.21.15.43:0/3889650544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df4074070 con 0x7f5df40b3920 2024-09-17T05:29:20.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5debe00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d950 tx=0x7f5ddc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc017070 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ddc009d20 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df40c7900 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0106c0 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df40c7ec0 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40b33e0 con 0x7f5df40b3920 2024-09-17T05:29:20.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.861+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ddc01a070 con 0x7f5df40b3920 2024-09-17T05:29:20.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.865+0000 7f5df0c00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 0x7f5dd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:20.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.865+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5767+0+0 (secure 0 0 0) 0x7f5ddc094430 con 0x7f5df40b3920 2024-09-17T05:29:20.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.865+0000 7f5df3e00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 0x7f5dd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:20.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.865+0000 7f5df3e00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 0x7f5dd0075710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5de40af000 tx=0x7f5de401b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:20.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:20.869+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ddc05ed20 con 0x7f5df40b3920 2024-09-17T05:29:21.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.045+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f5df40c7650 con 0x7f5df40b3920 2024-09-17T05:29:21.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.049+0000 7f5df0c00640 1 -- 172.21.15.43:0/3889650544 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v45) ==== 121+0+0 (secure 0 0 0) 0x7f5ddc063750 con 0x7f5df40b3920 2024-09-17T05:29:21.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 msgr2=0x7f5dd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:21.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 0x7f5dd0075710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5de40af000 tx=0x7f5de401b060 comp rx=0 tx=0).stop 2024-09-17T05:29:21.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 msgr2=0x7f5df415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:21.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00d950 tx=0x7f5ddc00de10 comp rx=0 tx=0).stop 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 shutdown_connections 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dd0073200 0x7f5dd0075710 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5df4155d70 0x7f5df40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5df40b3920 0x7f5df415a9a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 --2- 172.21.15.43:0/3889650544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df406f110 0x7f5df4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 >> 172.21.15.43:0/3889650544 conn(0x7f5df40b23d0 msgr2=0x7f5df415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 shutdown_connections 2024-09-17T05:29:21.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.053+0000 7f5dfae00640 1 -- 172.21.15.43:0/3889650544 wait complete. 2024-09-17T05:29:21.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-09-17T05:29:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 -- 172.21.15.43:0/1441434840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f110 msgr2=0x7f50dc0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 --2- 172.21.15.43:0/1441434840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f110 0x7f50dc0b33e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009900 tx=0x7f50c801c820 comp rx=0 tx=0).stop 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 -- 172.21.15.43:0/1441434840 shutdown_connections 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 --2- 172.21.15.43:0/1441434840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f110 0x7f50dc0b33e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 --2- 172.21.15.43:0/1441434840 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc155d70 0x7f50dc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 -- 172.21.15.43:0/1441434840 >> 172.21.15.43:0/1441434840 conn(0x7f50dc0b23d0 msgr2=0x7f50dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 -- 172.21.15.43:0/1441434840 shutdown_connections 2024-09-17T05:29:21.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.233+0000 7f50e3800640 1 -- 172.21.15.43:0/1441434840 wait complete. 2024-09-17T05:29:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 Processor -- start 2024-09-17T05:29:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- start start 2024-09-17T05:29:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc06f110 0x7f50dc09cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:21.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50dc155d70 0x7f50dc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc168b10 con 0x7f50dc0b3920 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc168990 con 0x7f50dc155d70 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f50dc168c90 con 0x7f50dc06f110 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45258/0 (socket says 172.21.15.43:45258) 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 -- 172.21.15.43:0/312014237 learned_addr learned my addr 172.21.15.43:0/312014237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e0e00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc06f110 0x7f50dc09cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e1800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50dc155d70 0x7f50dc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 -- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc06f110 msgr2=0x7f50dc09cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc06f110 0x7f50dc09cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 -- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50dc155d70 msgr2=0x7f50dc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50dc155d70 0x7f50dc15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 -- 172.21.15.43:0/312014237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc15e9d0 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50dbe00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f50c801aed0 tx=0x7f50c80a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80b2040 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50c80042c0 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc0caf20 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8005030 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.237+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc073bb0 con 0x7f50dc0b3920 2024-09-17T05:29:21.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50dc0b33e0 con 0x7f50dc0b3920 2024-09-17T05:29:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50c80051f0 con 0x7f50dc0b3920 2024-09-17T05:29:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50d8c00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 0x7f50bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5767+0+0 (secure 0 0 0) 0x7f50c812df90 con 0x7f50dc0b3920 2024-09-17T05:29:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50e0e00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 0x7f50bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:21.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.241+0000 7f50e0e00640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 0x7f50bc075710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0715c0 tx=0x7f50d00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:21.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.245+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50c80164d0 con 0x7f50dc0b3920 2024-09-17T05:29:21.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:21.485+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7f50dc0cb120 con 0x7f50dc0b3920 2024-09-17T05:29:21.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:21.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:22.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.217+0000 7f50d8c00640 1 -- 172.21.15.43:0/312014237 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6387 v73) ==== 115+0+0 (secure 0 0 0) 0x7f50c80f8ff0 con 0x7f50dc0b3920 2024-09-17T05:29:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:unfreezing mds.6387 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 msgr2=0x7f50bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 0x7f50bc075710 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0715c0 tx=0x7f50d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 msgr2=0x7f50dc15a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f50c801aed0 tx=0x7f50c80a5720 comp rx=0 tx=0).stop 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 shutdown_connections 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f50bc073200 0x7f50bc075710 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50dc155d70 0x7f50dc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc0b3920 0x7f50dc15a710 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 --2- 172.21.15.43:0/312014237 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50dc06f110 0x7f50dc09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 >> 172.21.15.43:0/312014237 conn(0x7f50dc0b23d0 msgr2=0x7f50dc15df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:22.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 shutdown_connections 2024-09-17T05:29:22.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.221+0000 7f50e3800640 1 -- 172.21.15.43:0/312014237 wait complete. 2024-09-17T05:29:22.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -n kernel.core_pattern 2024-09-17T05:29:22.275 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:29:22.277 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:29:22.278 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-17T05:29:22.344 INFO:teuthology.orchestra.run.smithi043.stdout:1726550943.41594.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-mds', platform: 'x86_64' 2024-09-17T05:29:22.348 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726550943.41594.core'] 2024-09-17T05:29:22.348 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726550943.41594.core, deleting it 2024-09-17T05:29:22.348 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726550943.41594.core 2024-09-17T05:29:22.450 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:29:22.450 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:29:22.450 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:29:22.450 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:29:22.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:29:22.450 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi043 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-17T05:29:22.451 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:29:22.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:29:22.454 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:29:22.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:22.552 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:29:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- 172.21.15.43:0/2060805253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41006fc90 msgr2=0x7fa4100700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 --2- 172.21.15.43:0/2060805253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41006fc90 0x7fa4100700a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009900 tx=0x7fa3f801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- 172.21.15.43:0/2060805253 shutdown_connections 2024-09-17T05:29:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 --2- 172.21.15.43:0/2060805253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4100730b0 0x7fa4100734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 --2- 172.21.15.43:0/2060805253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41006fc90 0x7fa4100700a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- 172.21.15.43:0/2060805253 >> 172.21.15.43:0/2060805253 conn(0x7fa4100b23d0 msgr2=0x7fa4100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- 172.21.15.43:0/2060805253 shutdown_connections 2024-09-17T05:29:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- 172.21.15.43:0/2060805253 wait complete. 2024-09-17T05:29:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 Processor -- start 2024-09-17T05:29:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.613+0000 7fa416000640 1 -- start start 2024-09-17T05:29:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa41006fc90 0x7fa4101536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41015fe10 0x7fa4101f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa41016a060 con 0x7fa4100730b0 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa410169ee0 con 0x7fa41015fe10 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa41016a1e0 con 0x7fa41006fc90 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45276/0 (socket says 172.21.15.43:45276) 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 -- 172.21.15.43:0/1375785468 learned_addr learned my addr 172.21.15.43:0/1375785468 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa40f400640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa41006fc90 0x7fa4101536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa40fe00640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41015fe10 0x7fa4101f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 -- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa41006fc90 msgr2=0x7fa4101536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa41006fc90 0x7fa4101536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 -- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41015fe10 msgr2=0x7fa4101f9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41015fe10 0x7fa4101f9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 -- 172.21.15.43:0/1375785468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4101fa230 con 0x7fa4100730b0 2024-09-17T05:29:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407e00640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009850 tx=0x7fa3f8005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80b2040 con 0x7fa4100730b0 2024-09-17T05:29:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3f8004240 con 0x7fa4100730b0 2024-09-17T05:29:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4100c0f80 con 0x7fa4100730b0 2024-09-17T05:29:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8005030 con 0x7fa4100730b0 2024-09-17T05:29:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4100c1540 con 0x7fa4100730b0 2024-09-17T05:29:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.617+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4100700a0 con 0x7fa4100730b0 2024-09-17T05:29:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.621+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3f80051d0 con 0x7fa4100730b0 2024-09-17T05:29:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.621+0000 7fa407400640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 0x7fa3ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.621+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7fa3f812db90 con 0x7fa4100730b0 2024-09-17T05:29:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.621+0000 7fa40f400640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 0x7fa3ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:22.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.621+0000 7fa40f400640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 0x7fa3ec075710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa400007fd0 tx=0x7fa400009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.625+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa3f80164d0 con 0x7fa4100730b0 2024-09-17T05:29:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.837+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4100b17f0 con 0x7fa4100730b0 2024-09-17T05:29:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa407400640 1 -- 172.21.15.43:0/1375785468 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+4394 (secure 0 0 0) 0x7fa3f80f84c0 con 0x7fa4100730b0 2024-09-17T05:29:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:22.844 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":74,"btime":"2024-09-17T05:29:22:227478+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:22.227470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-17T05:29:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 74 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 msgr2=0x7fa3ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 0x7fa3ec075710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa400007fd0 tx=0x7fa400009040 comp rx=0 tx=0).stop 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 msgr2=0x7fa4101f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009850 tx=0x7fa3f8005d60 comp rx=0 tx=0).stop 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 shutdown_connections 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3ec073200 0x7fa3ec075710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa41015fe10 0x7fa4101f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100730b0 0x7fa4101f6860 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 --2- 172.21.15.43:0/1375785468 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa41006fc90 0x7fa4101536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 >> 172.21.15.43:0/1375785468 conn(0x7fa4100b23d0 msgr2=0x7fa4100adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 shutdown_connections 2024-09-17T05:29:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:22.841+0000 7fa416000640 1 -- 172.21.15.43:0/1375785468 wait complete. 2024-09-17T05:29:23.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:28:43.992520+0000', 'modified': '2024-09-17T05:29:22.227470+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/1855028227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 1855028227}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 1855028227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:29:23.269 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6172:up:replay 2024-09-17T05:29:24.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- 172.21.15.43:0/253534826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57700b3af0 msgr2=0x7f57700ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 --2- 172.21.15.43:0/253534826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57700b3af0 0x7f57700ad0b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5768009c80 tx=0x7f576801cce0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- 172.21.15.43:0/253534826 shutdown_connections 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 --2- 172.21.15.43:0/253534826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57700b3af0 0x7f57700ad0b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 --2- 172.21.15.43:0/253534826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f577015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- 172.21.15.43:0/253534826 >> 172.21.15.43:0/253534826 conn(0x7f57700b23d0 msgr2=0x7f57700d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- 172.21.15.43:0/253534826 shutdown_connections 2024-09-17T05:29:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- 172.21.15.43:0/253534826 wait complete. 2024-09-17T05:29:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 Processor -- start 2024-09-17T05:29:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.437+0000 7f5778400640 1 -- start start 2024-09-17T05:29:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f5770154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 0x7f5770155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5770168c10 con 0x7f577006f110 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5770168a90 con 0x7f577009db00 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5770168d90 con 0x7f5770155d70 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57900/0 (socket says 172.21.15.43:57900) 2024-09-17T05:29:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 -- 172.21.15.43:0/807842644 learned_addr learned my addr 172.21.15.43:0/807842644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5776400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 0x7f5770155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775000640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f5770154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 -- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 msgr2=0x7f5770154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f5770154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 -- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 msgr2=0x7f5770155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 0x7f5770155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 -- 172.21.15.43:0/807842644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57700cf2e0 con 0x7f577006f110 2024-09-17T05:29:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5776400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 0x7f5770155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:24.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775000640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f5770154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:24.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5775a00640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f576400d6d0 tx=0x7f576400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:24.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764017070 con 0x7f577006f110 2024-09-17T05:29:24.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764009b10 con 0x7f577006f110 2024-09-17T05:29:24.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764010460 con 0x7f577006f110 2024-09-17T05:29:24.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5770151800 con 0x7f577006f110 2024-09-17T05:29:24.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5770151dc0 con 0x7f577006f110 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57700ad0b0 con 0x7f577006f110 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.441+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f576401a070 con 0x7f577006f110 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.445+0000 7f576d600640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 0x7f5750075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.445+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f5764094740 con 0x7f577006f110 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.445+0000 7f5775000640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 0x7f5750075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.445+0000 7f5775000640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 0x7f5750075740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57700b1b90 tx=0x7f575c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:24.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.449+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f57700cf2e0 con 0x7f577006f110 2024-09-17T05:29:24.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f577015ea90 con 0x7f577006f110 2024-09-17T05:29:24.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f576d600640 1 -- 172.21.15.43:0/807842644 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5220 (secure 0 0 0) 0x7f576405f630 con 0x7f577006f110 2024-09-17T05:29:24.665 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:24.665 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"btime":"2024-09-17T05:29:24:231205+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:23.234910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-17T05:29:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 msgr2=0x7f5750075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 0x7f5750075740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f57700b1b90 tx=0x7f575c008040 comp rx=0 tx=0).stop 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 msgr2=0x7f577009d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f576400d6d0 tx=0x7f576400db90 comp rx=0 tx=0).stop 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 shutdown_connections 2024-09-17T05:29:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5750073230 0x7f5750075740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577009db00 0x7f5770155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5770155d70 0x7f5770154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 --2- 172.21.15.43:0/807842644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f577006f110 0x7f577009d340 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 >> 172.21.15.43:0/807842644 conn(0x7f57700b23d0 msgr2=0x7f57700ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.661+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 shutdown_connections 2024-09-17T05:29:24.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:24.665+0000 7f5778400640 1 -- 172.21.15.43:0/807842644 wait complete. 2024-09-17T05:29:24.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T05:29:25.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:28:43.992520+0000', 'modified': '2024-09-17T05:29:23.234910+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:rejoin', 'state_seq': 15, 'addr': '172.21.15.62:6837/1855028227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 1855028227}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 1855028227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:29:25.091 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6172:up:rejoin 2024-09-17T05:29:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:29:26.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 -- 172.21.15.43:0/3515830712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c155d70 msgr2=0x7efe3c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 --2- 172.21.15.43:0/3515830712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c155d70 0x7efe3c15c7e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7efe30009840 tx=0x7efe3001c790 comp rx=0 tx=0).stop 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 -- 172.21.15.43:0/3515830712 shutdown_connections 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 --2- 172.21.15.43:0/3515830712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c155d70 0x7efe3c15c7e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 -- 172.21.15.43:0/3515830712 >> 172.21.15.43:0/3515830712 conn(0x7efe3c0b23d0 msgr2=0x7efe3c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 -- 172.21.15.43:0/3515830712 shutdown_connections 2024-09-17T05:29:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.261+0000 7efe44200640 1 -- 172.21.15.43:0/3515830712 wait complete. 2024-09-17T05:29:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 Processor -- start 2024-09-17T05:29:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- start start 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 0x7efe3c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe3c155d70 0x7efe3c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c169e10 con 0x7efe3c0c62c0 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c169c90 con 0x7efe3c06f110 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c169f90 con 0x7efe3c155d70 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57914/0 (socket says 172.21.15.43:57914) 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 -- 172.21.15.43:0/2458410456 learned_addr learned my addr 172.21.15.43:0/2458410456 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe41800640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 0x7efe3c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe40e00640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe3c155d70 0x7efe3c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 -- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe3c155d70 msgr2=0x7efe3c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe3c155d70 0x7efe3c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 -- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 msgr2=0x7efe3c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 0x7efe3c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 -- 172.21.15.43:0/2458410456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c09df50 con 0x7efe3c0c62c0 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe41800640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 0x7efe3c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe42200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b9e0 tx=0x7efe3400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34019070 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe3400cc10 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34014440 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c0c6c10 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.265+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c0c9dc0 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c15c7e0 con 0x7efe3c0c62c0 2024-09-17T05:29:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efe3401d3f0 con 0x7efe3c0c62c0 2024-09-17T05:29:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe39600640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 0x7efe1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7efe34093ef0 con 0x7efe3c0c62c0 2024-09-17T05:29:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe41800640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 0x7efe1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.269+0000 7efe41800640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 0x7efe1c075740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe300af000 tx=0x7efe3001afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:26.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.273+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efe3c0c6c10 con 0x7efe3c0c62c0 2024-09-17T05:29:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.481+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3c0d04c0 con 0x7efe3c0c62c0 2024-09-17T05:29:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.481+0000 7efe39600640 1 -- 172.21.15.43:0/2458410456 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5227 (secure 0 0 0) 0x7efe3c09df50 con 0x7efe3c0c62c0 2024-09-17T05:29:26.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:26.487 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"btime":"2024-09-17T05:29:25:238079+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:25.238070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6172,"qdb_cluster":[6172]},"id":5}]} 2024-09-17T05:29:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 77 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 msgr2=0x7efe1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 0x7efe1c075740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe300af000 tx=0x7efe3001afd0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 msgr2=0x7efe3c0c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b9e0 tx=0x7efe3400bea0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 shutdown_connections 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efe1c073230 0x7efe1c075740 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe3c0c62c0 0x7efe3c0c66d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efe3c155d70 0x7efe3c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 --2- 172.21.15.43:0/2458410456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c06f110 0x7efe3c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 >> 172.21.15.43:0/2458410456 conn(0x7efe3c0b23d0 msgr2=0x7efe3c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 shutdown_connections 2024-09-17T05:29:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:26.485+0000 7efe44200640 1 -- 172.21.15.43:0/2458410456 wait complete. 2024-09-17T05:29:26.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:28:43.992520+0000', 'modified': '2024-09-17T05:29:25.238070+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6172}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6172': {'gid': 6172, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.62:6837/1855028227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6835', 'nonce': 1855028227}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 1855028227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6172, 'qdb_cluster': [6172]} 2024-09-17T05:29:26.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:29:26.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:29:27.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- 172.21.15.43:0/2525833256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0b2f60 msgr2=0x7ff63c15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 --2- 172.21.15.43:0/2525833256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0b2f60 0x7ff63c15b310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff63c072c40 tx=0x7ff62c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- 172.21.15.43:0/2525833256 shutdown_connections 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 --2- 172.21.15.43:0/2525833256 >> v2:172.21.15.62:3300/0 conn(0x7ff63c06fe40 0x7ff63c070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 --2- 172.21.15.43:0/2525833256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0adf80 0x7ff63c0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 --2- 172.21.15.43:0/2525833256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0b2f60 0x7ff63c15b310 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- 172.21.15.43:0/2525833256 >> 172.21.15.43:0/2525833256 conn(0x7ff63c0d0010 msgr2=0x7ff63c0d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- 172.21.15.43:0/2525833256 shutdown_connections 2024-09-17T05:29:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- 172.21.15.43:0/2525833256 wait complete. 2024-09-17T05:29:27.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 Processor -- start 2024-09-17T05:29:27.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.073+0000 7ff640c00640 1 -- start start 2024-09-17T05:29:27.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff63c06fe40 0x7ff63c140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0b2f60 0x7ff63c1f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c074970 con 0x7ff63c0adf80 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0747f0 con 0x7ff63c06fe40 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c074af0 con 0x7ff63c0b2f60 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57922/0 (socket says 172.21.15.43:57922) 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 -- 172.21.15.43:0/1930912718 learned_addr learned my addr 172.21.15.43:0/1930912718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff63b400640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff63c06fe40 0x7ff63c140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff63be00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0b2f60 0x7ff63c1f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 -- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0b2f60 msgr2=0x7ff63c1f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0b2f60 0x7ff63c1f9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 -- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff63c06fe40 msgr2=0x7ff63c140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff63c06fe40 0x7ff63c140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 -- 172.21.15.43:0/1930912718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1f72b0 con 0x7ff63c0adf80 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff632a00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b3f0 tx=0x7ff62400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff633e00640 1 -- 172.21.15.43:0/1930912718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624015020 con 0x7ff63c0adf80 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff633e00640 1 -- 172.21.15.43:0/1930912718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff624011920 con 0x7ff63c0adf80 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff633e00640 1 -- 172.21.15.43:0/1930912718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624011d70 con 0x7ff63c0adf80 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1f7590 con 0x7ff63c0adf80 2024-09-17T05:29:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.077+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1fa7a0 con 0x7ff63c0adf80 2024-09-17T05:29:27.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff633e00640 1 -- 172.21.15.43:0/1930912718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff624018020 con 0x7ff63c0adf80 2024-09-17T05:29:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff633e00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 0x7ff618075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff633e00640 1 -- 172.21.15.43:0/1930912718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7ff624094100 con 0x7ff63c0adf80 2024-09-17T05:29:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff649910000 1 Processor -- start 2024-09-17T05:29:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff63b400640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 0x7ff618075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff649910000 1 -- start start 2024-09-17T05:29:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.081+0000 7ff63b400640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 0x7ff618075710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0af000 tx=0x7ff62c01b530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c782d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c782e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5558c7702410 con 0x5558c76dc950 2024-09-17T05:29:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5558c763fb40 con 0x5558c72ec460 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5558c762ef70 con 0x5558c76c5ca0 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57928/0 (socket says 172.21.15.43:57928) 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 -- 172.21.15.43:0/1358979631 learned_addr learned my addr 172.21.15.43:0/1358979631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff63b400640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c782d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff63be00640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c782e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 -- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 msgr2=0x5558c782e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c782e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 -- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 msgr2=0x5558c782d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c782d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 -- 172.21.15.43:0/1358979631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c75f65b0 con 0x5558c76dc950 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff63b400640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c782d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff632a00640 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff62405e9c0 tx=0x7ff62405e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca91cd7aaa4e752f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff62b400640 1 -- 172.21.15.43:0/1358979631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624084070 con 0x5558c76dc950 2024-09-17T05:29:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff62b400640 1 -- 172.21.15.43:0/1358979631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff624068310 con 0x5558c76dc950 2024-09-17T05:29:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff62b400640 1 -- 172.21.15.43:0/1358979631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62405dc90 con 0x5558c76dc950 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- 172.21.15.43:0/1358979631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 msgr2=0x5558c782dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff62405e9c0 tx=0x7ff62405e2a0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- 172.21.15.43:0/1358979631 shutdown_connections 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c782e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782dbe0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 --2- 172.21.15.43:0/1358979631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c782d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- 172.21.15.43:0/1358979631 >> 172.21.15.43:0/1358979631 conn(0x5558c77b2d90 msgr2=0x5558c77e2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- 172.21.15.43:0/1358979631 shutdown_connections 2024-09-17T05:29:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- 172.21.15.43:0/1358979631 wait complete. 2024-09-17T05:29:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 -- start start 2024-09-17T05:29:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.085+0000 7ff649910000 1 Processor -- start 2024-09-17T05:29:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c7834e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76dc950 0x5558c78353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5558c75c95f0 con 0x5558c72ec460 2024-09-17T05:29:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5558c7632d80 con 0x5558c76dc950 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5558c75d5520 con 0x5558c76c5ca0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 10 client.0 _ll_register_callbacks cb 0x5558c75d9830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57936/0 (socket says 172.21.15.43:57936) 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 -- 172.21.15.43:0/2443395014 learned_addr learned my addr 172.21.15.43:0/2443395014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff632a00640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c7834e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63be00640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76dc950 0x5558c78353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 -- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 msgr2=0x5558c7834e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c7834e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 -- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76dc950 msgr2=0x5558c78353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76dc950 0x5558c78353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 -- 172.21.15.43:0/2443395014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c75cd190 con 0x5558c72ec460 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff63b400640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff62c000f90 tx=0x7ff62c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15bb63fb2c8f690 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0b2040 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558c75c7110 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7ff62c004040 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff62c0a5ae0 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 10 client.6663 fetch_fsmap learned FSMap version 77 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5558c76b3f10 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5558c75c7110 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7ff62c004240 con 0x5558c72ec460 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 10 client.6663 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 10 client.6663 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 20 client.6663 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72d17d0 0x5558c783ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 4 client.6663 mds_command: new command op to 6172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:29:27.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 --> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558c7641a90 con 0x5558c72d17d0 2024-09-17T05:29:27.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.089+0000 7ff632a00640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72d17d0 0x5558c783ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff61fe00640 10 client.6663 ms_handle_connect on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff632a00640 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72d17d0 0x5558c783ed00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e754ebb3533ca21 server_cookie=2c236ce862f92548 in_seq=0 out_seq=0 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff61fe00640 1 -- 172.21.15.43:0/2443395014 <== mds.0 v2:172.21.15.62:6835/1855028227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558c7641a90 con 0x5558c72d17d0 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff61fe00640 10 client.6663 handle_command_reply: tid=0 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 client.6663 shutdown 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 msgr2=0x5558c7834920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff62c000f90 tx=0x7ff62c0095c0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 shutdown_connections 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72d17d0 0x5558c783ed00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76dc950 0x5558c78353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c7834e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff61fe00640 0 client.6663 ms_handle_reset on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 --2- 172.21.15.43:0/2443395014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c72ec460 0x5558c7834920 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 >> 172.21.15.43:0/2443395014 conn(0x5558c77b2d90 msgr2=0x5558c77e9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 shutdown_connections 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 1 -- 172.21.15.43:0/2443395014 wait complete. 2024-09-17T05:29:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.093+0000 7ff649910000 20 client.6663 trim_cache size 0 max 16384 2024-09-17T05:29:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 Processor -- start 2024-09-17T05:29:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 -- start start 2024-09-17T05:29:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c783ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c783b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c203530 con 0x5558c76dc950 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c202560 con 0x5558c72ec460 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c155040 con 0x5558c76c5ca0 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57938/0 (socket says 172.21.15.43:57938) 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 -- 172.21.15.43:0/2726393261 learned_addr learned my addr 172.21.15.43:0/2726393261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff632a00640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c783b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63b400640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c783ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 -- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 msgr2=0x5558c783b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c783b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 -- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 msgr2=0x5558c783ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c783ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 -- 172.21.15.43:0/2726393261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c768adb0 con 0x5558c76dc950 2024-09-17T05:29:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63b400640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c783ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff63be00640 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ab0 tx=0x7ff634006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a6c3ba465d668e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.113+0000 7ff62aa00640 1 -- 172.21.15.43:0/2726393261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634014bd0 con 0x5558c76dc950 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff62aa00640 1 -- 172.21.15.43:0/2726393261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff634014e80 con 0x5558c76dc950 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff62aa00640 1 -- 172.21.15.43:0/2726393261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63401f6e0 con 0x5558c76dc950 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- 172.21.15.43:0/2726393261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 msgr2=0x5558c782b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ab0 tx=0x7ff634006f90 comp rx=0 tx=0).stop 2024-09-17T05:29:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- 172.21.15.43:0/2726393261 shutdown_connections 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c782b620 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c76c5ca0 0x5558c783b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- 172.21.15.43:0/2726393261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c72ec460 0x5558c783ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- 172.21.15.43:0/2726393261 >> 172.21.15.43:0/2726393261 conn(0x5558c77b2d90 msgr2=0x5558c77ef0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- 172.21.15.43:0/2726393261 shutdown_connections 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- 172.21.15.43:0/2726393261 wait complete. 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- start start 2024-09-17T05:29:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 Processor -- start 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c72ec460 0x5558c77e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76c5ca0 0x5558c77e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5558c75faee0 con 0x5558c76dc950 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5558c76b3a80 con 0x5558c76c5ca0 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5558c7632090 con 0x5558c72ec460 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff649910000 10 client.0 _ll_register_callbacks cb 0x7ff62c009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff63be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff63be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57954/0 (socket says 172.21.15.43:57954) 2024-09-17T05:29:27.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff63be00640 1 -- 172.21.15.43:0/651512009 learned_addr learned my addr 172.21.15.43:0/651512009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.117+0000 7ff63b400640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c72ec460 0x5558c77e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff632a00640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76c5ca0 0x5558c77e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 -- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c72ec460 msgr2=0x5558c77e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c72ec460 0x5558c77e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 -- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76c5ca0 msgr2=0x5558c77e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76c5ca0 0x5558c77e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 -- 172.21.15.43:0/651512009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c75f5ab0 con 0x5558c76dc950 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63be00640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x5558c77dbaf0 tx=0x7ff634014e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93ef235624a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63402f060 con 0x5558c76dc950 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7ff63401fa90 con 0x5558c76dc950 2024-09-17T05:29:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558c763d540 con 0x5558c76dc950 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff63401fcf0 con 0x5558c76dc950 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 10 client.6669 fetch_fsmap learned FSMap version 77 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x5558c75d4070 con 0x5558c76dc950 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5558c763d540 con 0x5558c76dc950 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7ff63402a470 con 0x5558c76dc950 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 10 client.6669 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 10 client.6669 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 20 client.6669 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72bf830 0x5558c78afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 4 client.6669 mds_command: new command op to 6172 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:29:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 --> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] -- command(tid 0: {"prefix": "status"}) -- 0x5558c763a970 con 0x5558c72bf830 2024-09-17T05:29:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63b400640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72bf830 0x5558c78afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff63b400640 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72bf830 0x5558c78afa90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f54249a3d0f1cf4 server_cookie=b30dafc0c2e755a5 in_seq=0 out_seq=0 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 10 client.6669 ms_handle_connect on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 1 -- 172.21.15.43:0/651512009 <== mds.0 v2:172.21.15.62:6835/1855028227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5558c763a970 con 0x5558c72bf830 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 10 client.6669 handle_command_reply: tid=0 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 client.6669 shutdown 2024-09-17T05:29:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 msgr2=0x5558c77e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x5558c77dbaf0 tx=0x7ff634014e10 comp rx=0 tx=0).stop 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 shutdown_connections 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x5558c72bf830 0x5558c78afa90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5558c76c5ca0 0x5558c77e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5558c76dc950 0x5558c77e2750 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff61fe00640 0 client.6669 ms_handle_reset on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 --2- 172.21.15.43:0/651512009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5558c72ec460 0x5558c77e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 >> 172.21.15.43:0/651512009 conn(0x5558c77b2d90 msgr2=0x5558c7837510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 shutdown_connections 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 1 -- 172.21.15.43:0/651512009 wait complete. 2024-09-17T05:29:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.121+0000 7ff649910000 20 client.6669 trim_cache size 0 max 16384 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6172, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.8965361730000003, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 77, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 78, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 78, 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 53.995212838999997 2024-09-17T05:29:27.131 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:29:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 msgr2=0x7ff618075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 0x7ff618075710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0af000 tx=0x7ff62c01b530 comp rx=0 tx=0).stop 2024-09-17T05:29:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 msgr2=0x7ff63c1434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b3f0 tx=0x7ff62400b8b0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 shutdown_connections 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff618073200 0x7ff618075710 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff63c0b2f60 0x7ff63c1f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0adf80 0x7ff63c1434a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 --2- 172.21.15.43:0/1930912718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff63c06fe40 0x7ff63c140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 >> 172.21.15.43:0/1930912718 conn(0x7ff63c0d0010 msgr2=0x7ff63c0b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 shutdown_connections 2024-09-17T05:29:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.125+0000 7ff640c00640 1 -- 172.21.15.43:0/1930912718 wait complete. 2024-09-17T05:29:27.147 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-17T05:29:27.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:29:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: File: a/ 2024-09-17T05:29:27.200 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-17T05:29:27.200 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 2ch/44d Inode: 1099511627776 Links: 4 2024-09-17T05:29:27.200 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2024-09-17T05:29:27.201 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-17 05:28:58.090312067 +0000 2024-09-17T05:29:27.201 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-17 05:29:03.798215332 +0000 2024-09-17T05:29:27.201 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-17 05:29:03.798215332 +0000 2024-09-17T05:29:27.201 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-17 05:28:58.090312067 +0000 2024-09-17T05:29:27.210 INFO:teuthology.orchestra.run.smithi043.stdout:a/ 2024-09-17T05:29:27.211 INFO:teuthology.orchestra.run.smithi043.stdout:a/z 2024-09-17T05:29:27.211 INFO:teuthology.orchestra.run.smithi043.stdout:a/z/d 2024-09-17T05:29:27.211 INFO:teuthology.orchestra.run.smithi043.stdout:a/c 2024-09-17T05:29:27.211 INFO:teuthology.orchestra.run.smithi043.stdout:a/c/d 2024-09-17T05:29:27.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:29:27.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-17T05:29:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- 172.21.15.43:0/1116070944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 msgr2=0x7f7cf80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1116070944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf80cce70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7cf809bbe0 tx=0x7f7ce001c820 comp rx=0 tx=0).stop 2024-09-17T05:29:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- 172.21.15.43:0/1116070944 shutdown_connections 2024-09-17T05:29:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1116070944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf80cce70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- 172.21.15.43:0/1116070944 >> 172.21.15.43:0/1116070944 conn(0x7f7cf80cee80 msgr2=0x7f7cf80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- 172.21.15.43:0/1116070944 shutdown_connections 2024-09-17T05:29:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- 172.21.15.43:0/1116070944 wait complete. 2024-09-17T05:29:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 Processor -- start 2024-09-17T05:29:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- start start 2024-09-17T05:29:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf80b7ce0 0x7f7cf80c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80cd3b0 0x7f7cf814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf816b030 con 0x7f7cf80d0f50 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf816aeb0 con 0x7f7cf80cd3b0 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.377+0000 7f7cfe600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf816b1b0 con 0x7f7cf80b7ce0 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57980/0 (socket says 172.21.15.43:57980) 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 -- 172.21.15.43:0/1026560676 learned_addr learned my addr 172.21.15.43:0/1026560676 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7400640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf80b7ce0 0x7f7cf80c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cefe00640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80cd3b0 0x7f7cf814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 -- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf80b7ce0 msgr2=0x7f7cf80c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf80b7ce0 0x7f7cf80c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 -- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80cd3b0 msgr2=0x7f7cf814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80cd3b0 0x7f7cf814d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 -- 172.21.15.43:0/1026560676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf80c6f30 con 0x7f7cf80d0f50 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c760 tx=0x7f7cf000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cef400640 1 -- 172.21.15.43:0/1026560676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003db0 con 0x7f7cf80d0f50 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cef400640 1 -- 172.21.15.43:0/1026560676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf000ce10 con 0x7f7cf80d0f50 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cef400640 1 -- 172.21.15.43:0/1026560676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00052d0 con 0x7f7cf80d0f50 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf815e540 con 0x7f7cf80d0f50 2024-09-17T05:29:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81600a0 con 0x7f7cf80d0f50 2024-09-17T05:29:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.381+0000 7f7cef400640 1 -- 172.21.15.43:0/1026560676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7cf0005470 con 0x7f7cf80d0f50 2024-09-17T05:29:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cef400640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 0x7f7cd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cef400640 1 -- 172.21.15.43:0/1026560676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f7cf00940b0 con 0x7f7cf80d0f50 2024-09-17T05:29:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 Processor -- start 2024-09-17T05:29:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 0x7f7cd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 -- start start 2024-09-17T05:29:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 0x7f7cd4075740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8007fd0 tx=0x7f7ce8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899e55f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899e56470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c899cf4cd0 con 0x55c899ce96d0 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c899c0a330 con 0x55c899a04aa0 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c899c67bb0 con 0x55c899940890 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57996/0 (socket says 172.21.15.43:57996) 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 -- 172.21.15.43:0/132012646 learned_addr learned my addr 172.21.15.43:0/132012646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cefe00640 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899e55f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899e56470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 -- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 msgr2=0x55c899e55f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899e55f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 -- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 msgr2=0x55c899e56470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899e56470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 -- 172.21.15.43:0/132012646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c899c0bc80 con 0x55c899ce96d0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7cf7400640 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80052a0 tx=0x7f7ce8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30890266240c796 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7ce7e00640 1 -- 172.21.15.43:0/132012646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80054d0 con 0x55c899ce96d0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7ce7e00640 1 -- 172.21.15.43:0/132012646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ce800e7b0 con 0x55c899ce96d0 2024-09-17T05:29:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.385+0000 7f7ce7e00640 1 -- 172.21.15.43:0/132012646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800eb60 con 0x55c899ce96d0 2024-09-17T05:29:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- 172.21.15.43:0/132012646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 msgr2=0x55c899e559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80052a0 tx=0x7f7ce8011000 comp rx=0 tx=0).stop 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- 172.21.15.43:0/132012646 shutdown_connections 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899e56470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899e55f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- 172.21.15.43:0/132012646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899e559f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- 172.21.15.43:0/132012646 >> 172.21.15.43:0/132012646 conn(0x55c8999145c0 msgr2=0x55c899e12ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- 172.21.15.43:0/132012646 shutdown_connections 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- 172.21.15.43:0/132012646 wait complete. 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- start start 2024-09-17T05:29:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 Processor -- start 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899e11a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899ce96d0 0x55c899e185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c899c5adf0 con 0x55c899a04aa0 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c899c56fe0 con 0x55c899940890 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c899bfc4d0 con 0x55c899ce96d0 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7d05e10000 10 client.0 _ll_register_callbacks cb 0x55c899bda7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7cefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7cefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58012/0 (socket says 172.21.15.43:58012) 2024-09-17T05:29:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.389+0000 7f7cefe00640 1 -- 172.21.15.43:0/341543412 learned_addr learned my addr 172.21.15.43:0/341543412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899ce96d0 0x55c899e185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cf7400640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899e11a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 -- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899ce96d0 msgr2=0x55c899e185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899ce96d0 0x55c899e185f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 -- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 msgr2=0x55c899e11a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899e11a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 -- 172.21.15.43:0/341543412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c899bf5210 con 0x55c899a04aa0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cefe00640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0019430 tx=0x7f7ce00a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634b317c2e106ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0004300 con 0x55c899a04aa0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c899bef350 con 0x55c899a04aa0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f7ce00bcb70 con 0x55c899a04aa0 2024-09-17T05:29:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7ce00044a0 con 0x55c899a04aa0 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 10 client.6684 fetch_fsmap learned FSMap version 77 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55c899bdbfc0 con 0x55c899a04aa0 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c899bef350 con 0x55c899a04aa0 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f7ce00a68a0 con 0x55c899a04aa0 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 10 client.6684 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 10 client.6684 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 20 client.6684 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998f9930 0x55c899e5ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 4 client.6684 mds_command: new command op to 6172 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:29:27.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 --> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c899c69b00 con 0x55c8998f9930 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cf7400640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998f9930 0x55c899e5ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 10 client.6684 ms_handle_connect on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7cf7400640 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998f9930 0x55c899e5ce80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2bd714bdbc00ca0 server_cookie=8050fecb09cdd6d8 in_seq=0 out_seq=0 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.393+0000 7f7ce4c00640 1 -- 172.21.15.43:0/341543412 <== mds.0 v2:172.21.15.62:6835/1855028227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c899c69b00 con 0x55c8998f9930 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7ce4c00640 10 client.6684 handle_command_reply: tid=0 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 client.6684 shutdown 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 msgr2=0x55c899e11f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0019430 tx=0x7f7ce00a65d0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 shutdown_connections 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998f9930 0x55c899e5ce80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899ce96d0 0x55c899e185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899a04aa0 0x55c899e11f40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7ce4c00640 0 client.6684 ms_handle_reset on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 --2- 172.21.15.43:0/341543412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899e11a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 >> 172.21.15.43:0/341543412 conn(0x55c8999145c0 msgr2=0x55c899e0ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 shutdown_connections 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 1 -- 172.21.15.43:0/341543412 wait complete. 2024-09-17T05:29:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.397+0000 7f7d05e10000 20 client.6684 trim_cache size 0 max 16384 2024-09-17T05:29:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 Processor -- start 2024-09-17T05:29:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- start start 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899ec6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 0x55c899ec7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c899e10b20 con 0x55c899ce96d0 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8205620 con 0x55c899940890 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf8204640 con 0x55c899a04aa0 2024-09-17T05:29:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58026/0 (socket says 172.21.15.43:58026) 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 -- 172.21.15.43:0/717729942 learned_addr learned my addr 172.21.15.43:0/717729942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cefe00640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 0x55c899ec7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 -- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 msgr2=0x55c899ec7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7400640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899ec6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 0x55c899ec7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 -- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 msgr2=0x55c899ec6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899ec6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 -- 172.21.15.43:0/717729942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c899cb4680 con 0x55c899ce96d0 2024-09-17T05:29:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cefe00640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 0x55c899ec7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c730 tx=0x7f7cf001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc4f0d3ce2a776b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7ce7400640 1 -- 172.21.15.43:0/717729942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf005f030 con 0x55c899ce96d0 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7ce7400640 1 -- 172.21.15.43:0/717729942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf005e930 con 0x55c899ce96d0 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7ce7400640 1 -- 172.21.15.43:0/717729942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0063410 con 0x55c899ce96d0 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- 172.21.15.43:0/717729942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 msgr2=0x55c899ec7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c730 tx=0x7f7cf001d040 comp rx=0 tx=0).stop 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- 172.21.15.43:0/717729942 shutdown_connections 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ec7860 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899a04aa0 0x55c899ec7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 --2- 172.21.15.43:0/717729942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899940890 0x55c899ec6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- 172.21.15.43:0/717729942 >> 172.21.15.43:0/717729942 conn(0x55c8999145c0 msgr2=0x55c899e66d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- 172.21.15.43:0/717729942 shutdown_connections 2024-09-17T05:29:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.413+0000 7f7d05e10000 1 -- 172.21.15.43:0/717729942 wait complete. 2024-09-17T05:29:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- start start 2024-09-17T05:29:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 Processor -- start 2024-09-17T05:29:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899ecb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899ecb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8204020 con 0x55c899ce96d0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf815fb70 con 0x55c899a04aa0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c89991c680 con 0x55c899940890 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 10 client.0 _ll_register_callbacks cb 0x7f7ce00096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58030/0 (socket says 172.21.15.43:58030) 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 -- 172.21.15.43:0/2734787381 learned_addr learned my addr 172.21.15.43:0/2734787381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7400640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899ecb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cefe00640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899ecb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 -- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 msgr2=0x55c899ecb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899ecb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 -- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 msgr2=0x55c899ecb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899ecb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 -- 172.21.15.43:0/2734787381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c899c210b0 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7cf7e00640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7cf005bba0 tx=0x7f7cf005f030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dceeda3016d8788 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0063410 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f7cf0067e30 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf005e930 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7cf0080150 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 10 client.6690 fetch_fsmap learned FSMap version 77 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55c899bed610 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.417+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf005e930 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f7cf005db90 con 0x55c899ce96d0 2024-09-17T05:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 10 client.6690 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 10 client.6690 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 20 client.6690 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998e7990 0x55c899e86860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 4 client.6690 mds_command: new command op to 6172 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 --> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c899c655b0 con 0x55c8998e7990 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7cf7400640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998e7990 0x55c899e86860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7cf7400640 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998e7990 0x55c899e86860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2377a6e07afbc613 server_cookie=a66d1b337077ef4a in_seq=0 out_seq=0 2024-09-17T05:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.421+0000 7f7ce4c00640 10 client.6690 ms_handle_connect on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7ce4c00640 1 -- 172.21.15.43:0/2734787381 <== mds.0 v2:172.21.15.62:6835/1855028227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c899c655b0 con 0x55c8998e7990 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7ce4c00640 10 client.6690 handle_command_reply: tid=0 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 client.6690 shutdown 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 msgr2=0x55c899ecbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7cf005bba0 tx=0x7f7cf005f030 comp rx=0 tx=0).stop 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 shutdown_connections 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:6835/1855028227,v1:172.21.15.62:6837/1855028227] conn(0x55c8998e7990 0x55c899e86860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c899a04aa0 0x55c899ecb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7ce4c00640 0 client.6690 ms_handle_reset on v2:172.21.15.62:6835/1855028227 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c899ce96d0 0x55c899ecbc00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 --2- 172.21.15.43:0/2734787381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c899940890 0x55c899ecb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 >> 172.21.15.43:0/2734787381 conn(0x55c8999145c0 msgr2=0x55c899e175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 shutdown_connections 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 1 -- 172.21.15.43:0/2734787381 wait complete. 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7d05e10000 20 client.6690 trim_cache size 0 max 16384 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "message": "", 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-09-17T05:29:27.439 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:29:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 msgr2=0x7f7cd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 0x7f7cd4075740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8007fd0 tx=0x7f7ce8009040 comp rx=0 tx=0).stop 2024-09-17T05:29:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 msgr2=0x7f7cf815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c760 tx=0x7f7cf000cc20 comp rx=0 tx=0).stop 2024-09-17T05:29:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 shutdown_connections 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cd4073230 0x7f7cd4075740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80cd3b0 0x7f7cf814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf80d0f50 0x7f7cf815c4a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 --2- 172.21.15.43:0/1026560676 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7cf80b7ce0 0x7f7cf80c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 >> 172.21.15.43:0/1026560676 conn(0x7f7cf80cee80 msgr2=0x7f7cf80af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 shutdown_connections 2024-09-17T05:29:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.433+0000 7f7cfe600640 1 -- 172.21.15.43:0/1026560676 wait complete. 2024-09-17T05:29:27.458 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:29:27.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:29:27.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:29:27.634 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:29:27.635 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:29:27.635 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:27.635 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:27.644 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:27.644 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:27.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- 172.21.15.43:0/3877773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc0d08b0 msgr2=0x7f4bfc0d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 --2- 172.21.15.43:0/3877773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc0d0cc0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0d02c0 tx=0x7f4bf401c700 comp rx=0 tx=0).stop 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- 172.21.15.43:0/3877773739 shutdown_connections 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 --2- 172.21.15.43:0/3877773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc0d0cc0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 --2- 172.21.15.43:0/3877773739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 0x7f4bfc0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- 172.21.15.43:0/3877773739 >> 172.21.15.43:0/3877773739 conn(0x7f4bfc0691b0 msgr2=0x7f4bfc0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- 172.21.15.43:0/3877773739 shutdown_connections 2024-09-17T05:29:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- 172.21.15.43:0/3877773739 wait complete. 2024-09-17T05:29:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 Processor -- start 2024-09-17T05:29:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.897+0000 7f4c00c00640 1 -- start start 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 0x7f4bfc154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc1fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc169d50 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4bfc169bd0 con 0x7f4bfc0d08b0 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4bfc169ed0 con 0x7f4bfc0b5070 2024-09-17T05:29:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58046/0 (socket says 172.21.15.43:58046) 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 -- 172.21.15.43:0/2092401178 learned_addr learned my addr 172.21.15.43:0/2092401178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfb400640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 0x7f4bfc154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bf2a00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc1fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 -- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 msgr2=0x7f4bfc154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 0x7f4bfc154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 -- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bfc0d08b0 msgr2=0x7f4bfc1fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc1fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 -- 172.21.15.43:0/2092401178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc1f9cb0 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bfbe00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009790 tx=0x7f4bf40a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40042c0 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bf40bc070 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bfc06ea50 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4005290 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bfc06f010 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.901+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bfc06a1e0 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4bf4005430 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bf3e00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 0x7f4bd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:27.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f4bf412f960 con 0x7f4bfc15c4e0 2024-09-17T05:29:27.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bfb400640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 0x7f4bd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:27.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bfb400640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 0x7f4bd8075710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4bec001640 tx=0x7f4bec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:27.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:27.905+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4bf4016410 con 0x7f4bfc15c4e0 2024-09-17T05:29:28.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.093+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4bfc0b7f30 con 0x7f4bfc15c4e0 2024-09-17T05:29:28.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4bf3e00640 1 -- 172.21.15.43:0/2092401178 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) ==== 80+0+122 (secure 0 0 0) 0x7f4bf40fa9f0 con 0x7f4bfc15c4e0 2024-09-17T05:29:28.100 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3375674498 2024-09-18T05:29:22.227124+0000 2024-09-17T05:29:28.100 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/3375674498 2024-09-18T05:29:22.227124+0000 2024-09-17T05:29:28.100 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 msgr2=0x7f4bd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 0x7f4bd8075710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4bec001640 tx=0x7f4bec009040 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 msgr2=0x7f4bfc06f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009790 tx=0x7f4bf40a66d0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 shutdown_connections 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4bd8073200 0x7f4bd8075710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bfc0d08b0 0x7f4bfc1fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bfc15c4e0 0x7f4bfc06f810 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 --2- 172.21.15.43:0/2092401178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4bfc0b5070 0x7f4bfc154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 >> 172.21.15.43:0/2092401178 conn(0x7f4bfc0691b0 msgr2=0x7f4bfc15b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:28.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 shutdown_connections 2024-09-17T05:29:28.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.097+0000 7f4c00c00640 1 -- 172.21.15.43:0/2092401178 wait complete. 2024-09-17T05:29:28.122 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:29:28.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:28.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:28.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:29:28.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:28.149 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:29:28.200 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:29:28.200 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:29:28.200 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:28.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:29:28.210 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:29:28.210 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:28.210 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:28.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:29:28.255 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:29:28.255 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:29:28.255 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:28.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:29:28.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- 172.21.15.43:0/2802043533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 msgr2=0x7fbf2c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 --2- 172.21.15.43:0/2802043533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c0700a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0b3bb0 tx=0x7fbf1401c5c0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- 172.21.15.43:0/2802043533 shutdown_connections 2024-09-17T05:29:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 --2- 172.21.15.43:0/2802043533 >> v2:172.21.15.62:3300/0 conn(0x7fbf2c0730b0 0x7fbf2c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 --2- 172.21.15.43:0/2802043533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c0700a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- 172.21.15.43:0/2802043533 >> 172.21.15.43:0/2802043533 conn(0x7fbf2c0b23d0 msgr2=0x7fbf2c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- 172.21.15.43:0/2802043533 shutdown_connections 2024-09-17T05:29:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- 172.21.15.43:0/2802043533 wait complete. 2024-09-17T05:29:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 Processor -- start 2024-09-17T05:29:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.453+0000 7fbf33000640 1 -- start start 2024-09-17T05:29:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 0x7fbf2c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf2c15fe10 0x7fbf2c15f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c16a000 con 0x7fbf2c06fc90 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c169e80 con 0x7fbf2c0730b0 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c16a180 con 0x7fbf2c15fe10 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58060/0 (socket says 172.21.15.43:58060) 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 -- 172.21.15.43:0/623110104 learned_addr learned my addr 172.21.15.43:0/623110104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf31000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf2c15fe10 0x7fbf2c15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf23400640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 0x7fbf2c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 -- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf2c15fe10 msgr2=0x7fbf2c15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf2c15fe10 0x7fbf2c15f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 -- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 msgr2=0x7fbf2c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 0x7fbf2c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 -- 172.21.15.43:0/623110104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0aeb90 con 0x7fbf2c06fc90 2024-09-17T05:29:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf23400640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 0x7fbf2c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf2be00640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b410 tx=0x7fbf1c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c015020 con 0x7fbf2c06fc90 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf1c011920 con 0x7fbf2c06fc90 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0aee70 con 0x7fbf2c06fc90 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c011d70 con 0x7fbf2c06fc90 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c154710 con 0x7fbf2c06fc90 2024-09-17T05:29:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.457+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c0700a0 con 0x7fbf2c06fc90 2024-09-17T05:29:28.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.461+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf1c018020 con 0x7fbf2c06fc90 2024-09-17T05:29:28.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.461+0000 7fbf28c00640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 0x7fbf08075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.461+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7fbf1c0941e0 con 0x7fbf2c06fc90 2024-09-17T05:29:28.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.461+0000 7fbf23400640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 0x7fbf08075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.461+0000 7fbf23400640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 0x7fbf08075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf140af000 tx=0x7fbf1401ae00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:28.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.465+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf1c01f020 con 0x7fbf2c06fc90 2024-09-17T05:29:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.673+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf2c159dd0 con 0x7fbf2c06fc90 2024-09-17T05:29:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.673+0000 7fbf28c00640 1 -- 172.21.15.43:0/623110104 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5227 (secure 0 0 0) 0x7fbf1c05f0d0 con 0x7fbf2c06fc90 2024-09-17T05:29:28.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:28.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"btime":"2024-09-17T05:29:25:238079+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:25.238070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6172,"qdb_cluster":[6172]},"id":5}]} 2024-09-17T05:29:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 77 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 msgr2=0x7fbf08075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 0x7fbf08075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbf140af000 tx=0x7fbf1401ae00 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 msgr2=0x7fbf2c1541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00b410 tx=0x7fbf1c00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 shutdown_connections 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf08073230 0x7fbf08075740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf2c0730b0 0x7fbf2c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf2c15fe10 0x7fbf2c15f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 --2- 172.21.15.43:0/623110104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06fc90 0x7fbf2c1541d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 >> 172.21.15.43:0/623110104 conn(0x7fbf2c0b23d0 msgr2=0x7fbf2c1593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 shutdown_connections 2024-09-17T05:29:28.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.677+0000 7fbf33000640 1 -- 172.21.15.43:0/623110104 wait complete. 2024-09-17T05:29:28.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:28.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 -- 172.21.15.43:0/1771535728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 msgr2=0x7f23cc073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 --2- 172.21.15.43:0/1771535728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc073ec0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0ce8c0 tx=0x7f23c401cb50 comp rx=0 tx=0).stop 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 -- 172.21.15.43:0/1771535728 shutdown_connections 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 --2- 172.21.15.43:0/1771535728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc073ec0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 --2- 172.21.15.43:0/1771535728 >> v2:172.21.15.62:3300/0 conn(0x7f23cc15ec90 0x7f23cc0ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.853+0000 7f23cb200640 1 -- 172.21.15.43:0/1771535728 >> 172.21.15.43:0/1771535728 conn(0x7f23cc0d1100 msgr2=0x7f23cc0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 -- 172.21.15.43:0/1771535728 shutdown_connections 2024-09-17T05:29:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 -- 172.21.15.43:0/1771535728 wait complete. 2024-09-17T05:29:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 Processor -- start 2024-09-17T05:29:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 -- start start 2024-09-17T05:29:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23cc15e880 0x7f23cc0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.857+0000 7f23cb200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23cc15ec90 0x7f23cc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc168fb0 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc168e30 con 0x7f23cc15e880 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f23cc169130 con 0x7f23cc15ec90 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58070/0 (socket says 172.21.15.43:58070) 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 -- 172.21.15.43:0/2628078050 learned_addr learned my addr 172.21.15.43:0/2628078050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c3e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23cc15e880 0x7f23cc0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23ca800640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23cc15ec90 0x7f23cc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 -- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23cc15ec90 msgr2=0x7f23cc159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23cc15ec90 0x7f23cc159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 -- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23cc15e880 msgr2=0x7f23cc0cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23cc15e880 0x7f23cc0cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 -- 172.21.15.43:0/2628078050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc1f9910 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c9e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00c400 tx=0x7f23bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc017070 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc0ca4b0 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23bc003600 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003a50 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc157ca0 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23cc0ca680 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.861+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23bc01a070 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.865+0000 7f23c2000640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 0x7f23a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.865+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f23bc093750 con 0x7f23cc0cf3e0 2024-09-17T05:29:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.865+0000 7f23c3e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 0x7f23a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.865+0000 7f23c3e00640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 0x7f23a8075710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0cd490 tx=0x7f23b40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:28.869+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23cc1f9910 con 0x7f23cc0cf3e0 2024-09-17T05:29:29.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.081+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23cc073ec0 con 0x7f23cc0cf3e0 2024-09-17T05:29:29.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.081+0000 7f23c2000640 1 -- 172.21.15.43:0/2628078050 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5227 (secure 0 0 0) 0x7f23bc05e5c0 con 0x7f23cc0cf3e0 2024-09-17T05:29:29.086 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:29.086 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"btime":"2024-09-17T05:29:25:238079+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:25.238070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6172,"qdb_cluster":[6172]},"id":5}]} 2024-09-17T05:29:29.086 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 77 2024-09-17T05:29:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 msgr2=0x7f23a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 0x7f23a8075710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0cd490 tx=0x7f23b40091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 msgr2=0x7f23cc1f93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00c400 tx=0x7f23bc00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 shutdown_connections 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f23a8073200 0x7f23a8075710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23cc15e880 0x7f23cc0cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f23cc15ec90 0x7f23cc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 --2- 172.21.15.43:0/2628078050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23cc0cf3e0 0x7f23cc1f93d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 >> 172.21.15.43:0/2628078050 conn(0x7f23cc0d1100 msgr2=0x7f23cc06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 shutdown_connections 2024-09-17T05:29:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.085+0000 7f23cb200640 1 -- 172.21.15.43:0/2628078050 wait complete. 2024-09-17T05:29:29.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:29.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- 172.21.15.43:0/2535513945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc0d1cb0 msgr2=0x7ff3cc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/2535513945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc0ccc40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009840 tx=0x7ff3bc01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- 172.21.15.43:0/2535513945 shutdown_connections 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/2535513945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc0ccc40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- 172.21.15.43:0/2535513945 >> 172.21.15.43:0/2535513945 conn(0x7ff3cc068fe0 msgr2=0x7ff3cc069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- 172.21.15.43:0/2535513945 shutdown_connections 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- 172.21.15.43:0/2535513945 wait complete. 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 Processor -- start 2024-09-17T05:29:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- start start 2024-09-17T05:29:29.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 0x7ff3cc162d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc15ce90 con 0x7ff3cc15ca80 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc153960 con 0x7ff3cc0d1cb0 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.265+0000 7ff3d1c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc15d010 con 0x7ff3cc160cf0 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58100/0 (socket says 172.21.15.43:58100) 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 -- 172.21.15.43:0/112716053 learned_addr learned my addr 172.21.15.43:0/112716053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cbe00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3d1200640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 0x7ff3cc162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 -- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 msgr2=0x7ff3cc162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 0x7ff3cc162d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 -- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3cc0d1cb0 msgr2=0x7ff3cc162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 -- 172.21.15.43:0/112716053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc0b5820 con 0x7ff3cc15ca80 2024-09-17T05:29:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3d1200640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 0x7ff3cc162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3cb400640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c3c0 tx=0x7ff3b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011790 con 0x7ff3cc15ca80 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8017070 con 0x7ff3cc15ca80 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc0b5b00 con 0x7ff3cc15ca80 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011cc0 con 0x7ff3cc15ca80 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc0b60c0 con 0x7ff3cc15ca80 2024-09-17T05:29:29.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc0b5590 con 0x7ff3cc15ca80 2024-09-17T05:29:29.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.269+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3b801c460 con 0x7ff3cc15ca80 2024-09-17T05:29:29.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.273+0000 7ff3c3e00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 0x7ff3ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.273+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7ff3b8093f70 con 0x7ff3cc15ca80 2024-09-17T05:29:29.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.273+0000 7ff3cbe00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 0x7ff3ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.273+0000 7ff3cbe00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 0x7ff3ac075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0af000 tx=0x7ff3bc01b030 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:29.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.277+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff3b805ef00 con 0x7ff3cc15ca80 2024-09-17T05:29:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.453+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3cc15e260 con 0x7ff3cc15ca80 2024-09-17T05:29:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.453+0000 7ff3c3e00640 1 -- 172.21.15.43:0/112716053 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16834 (secure 0 0 0) 0x7ff3b805e800 con 0x7ff3cc15ca80 2024-09-17T05:29:29.458 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:29.458 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:22.227202+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:41.731155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:42.948479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1","mappings":[{"from":2,"to":7}]},{"pgid":"11.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/3375674498":"2024-09-18T05:29:22.227124+0000","172.21.15.43:6834/3375674498":"2024-09-18T05:29:22.227124+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.453+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 msgr2=0x7ff3ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.453+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 0x7ff3ac075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0af000 tx=0x7ff3bc01b030 comp rx=0 tx=0).stop 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.453+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 msgr2=0x7ff3cc150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c3c0 tx=0x7ff3b800c880 comp rx=0 tx=0).stop 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 shutdown_connections 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff3ac073200 0x7ff3ac075710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff3cc160cf0 0x7ff3cc162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3cc15ca80 0x7ff3cc150f50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 --2- 172.21.15.43:0/112716053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3cc0d1cb0 0x7ff3cc162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 >> 172.21.15.43:0/112716053 conn(0x7ff3cc068fe0 msgr2=0x7ff3cc0cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 shutdown_connections 2024-09-17T05:29:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.457+0000 7ff3d1c00640 1 -- 172.21.15.43:0/112716053 wait complete. 2024-09-17T05:29:29.481 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:29:29.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:29.482 INFO:tasks.ceph.mds.b.smithi062.stderr:2024-09-17T05:29:29.478+0000 7fe8c4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:29.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 -- 172.21.15.43:0/8237667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc98003af0 msgr2=0x7fcc98003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 --2- 172.21.15.43:0/8237667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc98003af0 0x7fcc98003f60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcca400c310 tx=0x7fcca401cfc0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccb9600640 1 -- 172.21.15.43:0/8237667 reap_dead start 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 -- 172.21.15.43:0/8237667 shutdown_connections 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 -- 172.21.15.43:0/8237667 >> 172.21.15.43:0/8237667 conn(0x7fccb40b23d0 msgr2=0x7fccb40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 -- 172.21.15.43:0/8237667 shutdown_connections 2024-09-17T05:29:29.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.653+0000 7fccbb600640 1 -- 172.21.15.43:0/8237667 wait complete. 2024-09-17T05:29:29.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 Processor -- start 2024-09-17T05:29:29.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 -- start start 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 0x7fccb4158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 0x7fccb40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4165fc0 con 0x7fccb40c7470 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4165e40 con 0x7fcc98003af0 2024-09-17T05:29:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccbb600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fccb4166140 con 0x7fccb41fcf10 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58106/0 (socket says 172.21.15.43:58106) 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 -- 172.21.15.43:0/2587290898 learned_addr learned my addr 172.21.15.43:0/2587290898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccabe00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 0x7fccb40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb8c00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 0x7fccb4158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 -- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 msgr2=0x7fccb40cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 0x7fccb40cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 -- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 msgr2=0x7fccb4158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 0x7fccb4158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb9600640 1 -- 172.21.15.43:0/2587290898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb41588b0 con 0x7fccb40c7470 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccabe00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 0x7fccb40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.657+0000 7fccb8c00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 0x7fccb4158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:29.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb9600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fccb40d0f70 tx=0x7fccac010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:29.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac017070 con 0x7fccb40c7470 2024-09-17T05:29:29.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccac015070 con 0x7fccb40c7470 2024-09-17T05:29:29.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac004d90 con 0x7fccb40c7470 2024-09-17T05:29:29.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb415b4b0 con 0x7fccb40c7470 2024-09-17T05:29:29.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb41fc710 con 0x7fccb40c7470 2024-09-17T05:29:29.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb41fc240 con 0x7fccb40c7470 2024-09-17T05:29:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fccac00e910 con 0x7fccb40c7470 2024-09-17T05:29:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 0x7fcc90075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.661+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7fccac094800 con 0x7fccb40c7470 2024-09-17T05:29:29.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.665+0000 7fccb8c00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 0x7fcc90075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:29.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.665+0000 7fccb8c00640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 0x7fcc90075740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcca40ac000 tx=0x7fcca4019e00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:29.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.669+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fccac05f780 con 0x7fccb40c7470 2024-09-17T05:29:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.873+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccb406f1a0 con 0x7fccb40c7470 2024-09-17T05:29:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccb1600640 1 -- 172.21.15.43:0/2587290898 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5227 (secure 0 0 0) 0x7fccac05f080 con 0x7fccb40c7470 2024-09-17T05:29:29.880 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:29.880 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"btime":"2024-09-17T05:29:25:238079+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:25.238070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6172,"qdb_cluster":[6172]},"id":5}]} 2024-09-17T05:29:29.880 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 77 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 msgr2=0x7fcc90075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 0x7fcc90075740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcca40ac000 tx=0x7fcca4019e00 comp rx=0 tx=0).stop 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 msgr2=0x7fccb40cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fccb40d0f70 tx=0x7fccac010e70 comp rx=0 tx=0).stop 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 shutdown_connections 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc90073230 0x7fcc90075740 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb40c7470 0x7fccb40cba00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fccb41fcf10 0x7fccb40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 --2- 172.21.15.43:0/2587290898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc98003af0 0x7fccb4158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:29.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 >> 172.21.15.43:0/2587290898 conn(0x7fccb40b23d0 msgr2=0x7fccb4162ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:29.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 shutdown_connections 2024-09-17T05:29:29.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:29.877+0000 7fccbb600640 1 -- 172.21.15.43:0/2587290898 wait complete. 2024-09-17T05:29:29.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 -- 172.21.15.43:0/1268683276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 msgr2=0x7fe49006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 --2- 172.21.15.43:0/1268683276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe49006f110 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe488009bc0 tx=0x7fe48801cb10 comp rx=0 tx=0).stop 2024-09-17T05:29:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 -- 172.21.15.43:0/1268683276 shutdown_connections 2024-09-17T05:29:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 --2- 172.21.15.43:0/1268683276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe49006f110 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 -- 172.21.15.43:0/1268683276 >> 172.21.15.43:0/1268683276 conn(0x7fe4900b23d0 msgr2=0x7fe4900d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 -- 172.21.15.43:0/1268683276 shutdown_connections 2024-09-17T05:29:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.057+0000 7fe496600640 1 -- 172.21.15.43:0/1268683276 wait complete. 2024-09-17T05:29:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 Processor -- start 2024-09-17T05:29:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- start start 2024-09-17T05:29:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe490155d70 0x7fe49009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490151930 0x7fe490151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe490169dc0 con 0x7fe49015c7e0 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe490169c40 con 0x7fe490155d70 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe490169f40 con 0x7fe490151930 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58138/0 (socket says 172.21.15.43:58138) 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 -- 172.21.15.43:0/1951927754 learned_addr learned my addr 172.21.15.43:0/1951927754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48fe00640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490151930 0x7fe490151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48f400640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe490155d70 0x7fe49009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 -- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490151930 msgr2=0x7fe490151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490151930 0x7fe490151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 -- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe490155d70 msgr2=0x7fe49009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe490155d70 0x7fe49009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 -- 172.21.15.43:0/1951927754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4900731c0 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe48ea00640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c3c0 tx=0x7fe47c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c017070 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe47c0053c0 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4900734a0 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c003860 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.061+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe490073a60 con 0x7fe49015c7e0 2024-09-17T05:29:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49009dcb0 con 0x7fe49015c7e0 2024-09-17T05:29:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe47c01a070 con 0x7fe49015c7e0 2024-09-17T05:29:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe487400640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 0x7fe470075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7fe47c0935e0 con 0x7fe49015c7e0 2024-09-17T05:29:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe48f400640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 0x7fe470075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.065+0000 7fe48f400640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 0x7fe470075770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe4800046c0 tx=0x7fe480009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.069+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe47c05e560 con 0x7fe49015c7e0 2024-09-17T05:29:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.281+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe49006f110 con 0x7fe49015c7e0 2024-09-17T05:29:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.281+0000 7fe487400640 1 -- 172.21.15.43:0/1951927754 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5227 (secure 0 0 0) 0x7fe47c001090 con 0x7fe49015c7e0 2024-09-17T05:29:30.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:30.287 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"btime":"2024-09-17T05:29:25:238079+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:43.992520+0000","modified":"2024-09-17T05:29:25.238070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6172},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6172":{"gid":6172,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6837/1855028227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":1855028227},{"type":"v1","addr":"172.21.15.62:6837","nonce":1855028227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6172,"qdb_cluster":[6172]},"id":5}]} 2024-09-17T05:29:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 77 2024-09-17T05:29:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 msgr2=0x7fe470075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 0x7fe470075770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe4800046c0 tx=0x7fe480009040 comp rx=0 tx=0).stop 2024-09-17T05:29:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 msgr2=0x7fe490155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c3c0 tx=0x7fe47c00c880 comp rx=0 tx=0).stop 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 shutdown_connections 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe470073260 0x7fe470075770 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe490151930 0x7fe490151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe49015c7e0 0x7fe490155400 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 --2- 172.21.15.43:0/1951927754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe490155d70 0x7fe49009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 >> 172.21.15.43:0/1951927754 conn(0x7fe4900b23d0 msgr2=0x7fe4901664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 shutdown_connections 2024-09-17T05:29:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.285+0000 7fe496600640 1 -- 172.21.15.43:0/1951927754 wait complete. 2024-09-17T05:29:30.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- 172.21.15.43:0/3023959776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37280cf3e0 msgr2=0x7f3728073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 --2- 172.21.15.43:0/3023959776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37280cf3e0 0x7f3728073ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f37280ce8c0 tx=0x7f372001cb50 comp rx=0 tx=0).stop 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- 172.21.15.43:0/3023959776 shutdown_connections 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 --2- 172.21.15.43:0/3023959776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37280cf3e0 0x7f3728073ec0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 --2- 172.21.15.43:0/3023959776 >> v2:172.21.15.62:3300/0 conn(0x7f372815ec90 0x7f37280ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- 172.21.15.43:0/3023959776 >> 172.21.15.43:0/3023959776 conn(0x7f37280d1100 msgr2=0x7f37280d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- 172.21.15.43:0/3023959776 shutdown_connections 2024-09-17T05:29:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- 172.21.15.43:0/3023959776 wait complete. 2024-09-17T05:29:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 Processor -- start 2024-09-17T05:29:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.465+0000 7f372ec00640 1 -- start start 2024-09-17T05:29:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 0x7f372814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f372815e880 0x7f3728159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3728168f10 con 0x7f372815ec90 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3728168d90 con 0x7f37280cf3e0 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3728169090 con 0x7f372815e880 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58160/0 (socket says 172.21.15.43:58160) 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 -- 172.21.15.43:0/1106427980 learned_addr learned my addr 172.21.15.43:0/1106427980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 -- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f372815e880 msgr2=0x7f3728159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f372815e880 0x7f3728159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 -- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 msgr2=0x7f372814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372d800640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 0x7f372814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 0x7f372814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 -- 172.21.15.43:0/1106427980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281505f0 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372d800640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 0x7f372814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372e200640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f372001c5c0 tx=0x7f3720005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200b2040 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37200bb070 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720004530 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281508d0 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3728155540 con 0x7f372815ec90 2024-09-17T05:29:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.469+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728073ec0 con 0x7f372815ec90 2024-09-17T05:29:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.473+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37200046d0 con 0x7f372815ec90 2024-09-17T05:29:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.473+0000 7f3725600640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 0x7f3708075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.473+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7f372012eca0 con 0x7f372815ec90 2024-09-17T05:29:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.473+0000 7f372d800640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 0x7f3708075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.473+0000 7f372d800640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 0x7f3708075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f371c001230 tx=0x7f371c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.477+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3720016770 con 0x7f372815ec90 2024-09-17T05:29:30.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.653+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37280b29a0 con 0x7f372815ec90 2024-09-17T05:29:30.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.653+0000 7f3725600640 1 -- 172.21.15.43:0/1106427980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16834 (secure 0 0 0) 0x7f37200f9c20 con 0x7f372815ec90 2024-09-17T05:29:30.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:22.227202+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:41.731155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:42.948479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1","mappings":[{"from":2,"to":7}]},{"pgid":"11.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/3375674498":"2024-09-18T05:29:22.227124+0000","172.21.15.43:6834/3375674498":"2024-09-18T05:29:22.227124+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 msgr2=0x7f3708075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 0x7f3708075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f371c001230 tx=0x7f371c009040 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 msgr2=0x7f372815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f372001c5c0 tx=0x7f3720005a60 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 shutdown_connections 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3708073200 0x7f3708075710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f372815ec90 0x7f372815a050 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f372815e880 0x7f3728159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 --2- 172.21.15.43:0/1106427980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37280cf3e0 0x7f372814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 >> 172.21.15.43:0/1106427980 conn(0x7f37280d1100 msgr2=0x7f3728071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 shutdown_connections 2024-09-17T05:29:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.657+0000 7f372ec00640 1 -- 172.21.15.43:0/1106427980 wait complete. 2024-09-17T05:29:30.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:29:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 -- 172.21.15.43:0/2150172763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 msgr2=0x7fb2d00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2150172763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d00ad0b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009bc0 tx=0x7fb2c801cae0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 -- 172.21.15.43:0/2150172763 shutdown_connections 2024-09-17T05:29:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2150172763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d00ad0b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2150172763 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d00b3af0 0x7fb2d015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 -- 172.21.15.43:0/2150172763 >> 172.21.15.43:0/2150172763 conn(0x7fb2d00b23d0 msgr2=0x7fb2d00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 -- 172.21.15.43:0/2150172763 shutdown_connections 2024-09-17T05:29:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.837+0000 7fb2d7200640 1 -- 172.21.15.43:0/2150172763 wait complete. 2024-09-17T05:29:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 Processor -- start 2024-09-17T05:29:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 -- start start 2024-09-17T05:29:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d0162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d01627a0 0x7fb2d009bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0168c80 con 0x7fb2d0155d70 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0168b00 con 0x7fb2d006f110 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d7200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d0168e00 con 0x7fb2d01627a0 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58166/0 (socket says 172.21.15.43:58166) 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 -- 172.21.15.43:0/2351030427 learned_addr learned my addr 172.21.15.43:0/2351030427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2d5200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d01627a0 0x7fb2d009bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2cfe00640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d0162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 -- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d01627a0 msgr2=0x7fb2d009bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d01627a0 0x7fb2d009bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 -- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 msgr2=0x7fb2d0162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d0162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:30.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 -- 172.21.15.43:0/2351030427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d0160ab0 con 0x7fb2d0155d70 2024-09-17T05:29:30.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2c7400640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e7d0 tx=0x7fb2b800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b800ccc0 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b8010430 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.841+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8010860 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d0160d90 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d0161350 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d00ad0b0 con 0x7fb2d0155d70 2024-09-17T05:29:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb2b8010a00 con 0x7fb2d0155d70 2024-09-17T05:29:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2ccc00640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5853+0+0 (secure 0 0 0) 0x7fb2b8094de0 con 0x7fb2d0155d70 2024-09-17T05:29:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.845+0000 7fb2cfe00640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.849+0000 7fb2cfe00640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 0x7fb2ac075710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007fd0 tx=0x7fb2c0007ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:30.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:30.849+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb2b805f600 con 0x7fb2d0155d70 2024-09-17T05:29:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.065+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb2d00ca2f0 con 0x7fb2d0155d70 2024-09-17T05:29:31.070 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:29:31.065+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:29:31.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2ccc00640 1 -- 172.21.15.43:0/2351030427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7fb2b8064030 con 0x7fb2d0155d70 2024-09-17T05:29:31.072 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:29:31.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 msgr2=0x7fb2ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 0x7fb2ac075710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007fd0 tx=0x7fb2c0007ea0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 msgr2=0x7fb2d009b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e7d0 tx=0x7fb2b800ec90 comp rx=0 tx=0).stop 2024-09-17T05:29:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 shutdown_connections 2024-09-17T05:29:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb2d01627a0 0x7fb2d009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2d0155d70 0x7fb2d009b9e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 --2- 172.21.15.43:0/2351030427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d006f110 0x7fb2d0162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.069+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 >> 172.21.15.43:0/2351030427 conn(0x7fb2d00b23d0 msgr2=0x7fb2d015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.073+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 shutdown_connections 2024-09-17T05:29:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.073+0000 7fb2d7200640 1 -- 172.21.15.43:0/2351030427 wait complete. 2024-09-17T05:29:31.089 INFO:tasks.ceph.mds.b.smithi062.stderr: -42> 2024-09-17T05:29:29.478+0000 7fe8c4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:31.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:29:31.207 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:29:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 -- 172.21.15.43:0/768843158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2415fe10 msgr2=0x7f4e240b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 --2- 172.21.15.43:0/768843158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2415fe10 0x7f4e240b4b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009870 tx=0x7f4e1401c740 comp rx=0 tx=0).stop 2024-09-17T05:29:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 -- 172.21.15.43:0/768843158 shutdown_connections 2024-09-17T05:29:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 --2- 172.21.15.43:0/768843158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2415fe10 0x7f4e240b4b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 -- 172.21.15.43:0/768843158 >> 172.21.15.43:0/768843158 conn(0x7f4e240b23d0 msgr2=0x7f4e240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 -- 172.21.15.43:0/768843158 shutdown_connections 2024-09-17T05:29:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.253+0000 7f4e2b000640 1 -- 172.21.15.43:0/768843158 wait complete. 2024-09-17T05:29:31.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 Processor -- start 2024-09-17T05:29:31.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 -- start start 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 0x7f4e24154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e240cc610 0x7f4e2415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2416aff0 con 0x7f4e2415fe10 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2416ae70 con 0x7f4e240cc610 2024-09-17T05:29:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e2b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2416b170 con 0x7f4e2406fc90 2024-09-17T05:29:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58168/0 (socket says 172.21.15.43:58168) 2024-09-17T05:29:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 -- 172.21.15.43:0/2781351354 learned_addr learned my addr 172.21.15.43:0/2781351354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23e00640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 0x7f4e24154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e29000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e240cc610 0x7f4e2415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 -- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 msgr2=0x7f4e24154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 0x7f4e24154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 -- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e240cc610 msgr2=0x7f4e2415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e240cc610 0x7f4e2415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23400640 1 -- 172.21.15.43:0/2781351354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241554a0 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.257+0000 7f4e23e00640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 0x7f4e24154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e23400640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c430 tx=0x7f4e1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10017070 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e100053c0 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2415c410 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10003860 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2415c9d0 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24154020 con 0x7f4e2415fe10 2024-09-17T05:29:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4e1001a070 con 0x7f4e2415fe10 2024-09-17T05:29:31.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 0x7f4e04075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:31.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5939+0+0 (secure 0 0 0) 0x7f4e10093790 con 0x7f4e2415fe10 2024-09-17T05:29:31.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.261+0000 7f4e23e00640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 0x7f4e04075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:31.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.265+0000 7f4e23e00640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 0x7f4e04075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4e140af000 tx=0x7f4e1401af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.269+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4e1005dfe0 con 0x7f4e2415fe10 2024-09-17T05:29:31.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:31.477+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4e2415df30 con 0x7f4e2415fe10 2024-09-17T05:29:32.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.069+0000 7f4e1be00640 1 -- 172.21.15.43:0/2781351354 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7f4e10062a10 con 0x7f4e2415fe10 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 msgr2=0x7f4e04075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 0x7f4e04075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4e140af000 tx=0x7f4e1401af80 comp rx=0 tx=0).stop 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 msgr2=0x7f4e2415ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c430 tx=0x7f4e1000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 shutdown_connections 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4e04073230 0x7f4e04075740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e240cc610 0x7f4e2415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2415fe10 0x7f4e2415ec10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 --2- 172.21.15.43:0/2781351354 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4e2406fc90 0x7f4e24154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 >> 172.21.15.43:0/2781351354 conn(0x7f4e240b23d0 msgr2=0x7f4e240ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 shutdown_connections 2024-09-17T05:29:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.073+0000 7f4e2b000640 1 -- 172.21.15.43:0/2781351354 wait complete. 2024-09-17T05:29:32.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 -- 172.21.15.43:0/2374535693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440cd9a0 msgr2=0x7fdc440ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 --2- 172.21.15.43:0/2374535693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440cd9a0 0x7fdc440ce160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009900 tx=0x7fdc3401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 -- 172.21.15.43:0/2374535693 shutdown_connections 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 --2- 172.21.15.43:0/2374535693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440ccce0 0x7fdc4406fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 --2- 172.21.15.43:0/2374535693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440cd9a0 0x7fdc440ce160 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 -- 172.21.15.43:0/2374535693 >> 172.21.15.43:0/2374535693 conn(0x7fdc4406ce60 msgr2=0x7fdc4406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 -- 172.21.15.43:0/2374535693 shutdown_connections 2024-09-17T05:29:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.269+0000 7fdc4b800640 1 -- 172.21.15.43:0/2374535693 wait complete. 2024-09-17T05:29:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 Processor -- start 2024-09-17T05:29:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 -- start start 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 0x7fdc4414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc440cd9a0 0x7fdc44074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44168d60 con 0x7fdc440ccce0 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4415c250 con 0x7fdc440cd9a0 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc4b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdc44168ee0 con 0x7fdc440cd0f0 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58176/0 (socket says 172.21.15.43:58176) 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc48e00640 1 -- 172.21.15.43:0/1802253264 learned_addr learned my addr 172.21.15.43:0/1802253264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc3be00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 0x7fdc4414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.273+0000 7fdc49800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc440cd9a0 0x7fdc44074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 -- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 msgr2=0x7fdc4414c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 0x7fdc4414c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 -- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc440cd9a0 msgr2=0x7fdc44074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc440cd9a0 0x7fdc44074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 -- 172.21.15.43:0/1802253264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4414ce50 con 0x7fdc440ccce0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc3be00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 0x7fdc4414c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc48e00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009850 tx=0x7fdc340a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340b2040 con 0x7fdc440ccce0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc340043b0 con 0x7fdc440ccce0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34003860 con 0x7fdc440ccce0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc441546d0 con 0x7fdc440ccce0 2024-09-17T05:29:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc440aec30 con 0x7fdc440ccce0 2024-09-17T05:29:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc440ce160 con 0x7fdc440ccce0 2024-09-17T05:29:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc34003a00 con 0x7fdc440ccce0 2024-09-17T05:29:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.277+0000 7fdc41600640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 0x7fdc20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.281+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5939+0+0 (secure 0 0 0) 0x7fdc3412e280 con 0x7fdc440ccce0 2024-09-17T05:29:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.281+0000 7fdc3be00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 0x7fdc20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.281+0000 7fdc3be00640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 0x7fdc20075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdc440af490 tx=0x7fdc2c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:32.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.285+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc340164d0 con 0x7fdc440ccce0 2024-09-17T05:29:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:32.457+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdc44150a60 con 0x7fdc440ccce0 2024-09-17T05:29:33.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.273+0000 7fdc41600640 1 -- 172.21.15.43:0/1802253264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7fdc340f92b0 con 0x7fdc440ccce0 2024-09-17T05:29:33.278 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:29:33.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 msgr2=0x7fdc20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:33.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 0x7fdc20075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdc440af490 tx=0x7fdc2c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 msgr2=0x7fdc4415c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:33.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009850 tx=0x7fdc340a5690 comp rx=0 tx=0).stop 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 shutdown_connections 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc20073200 0x7fdc20075710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc440cd9a0 0x7fdc44074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdc440cd0f0 0x7fdc4414c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 --2- 172.21.15.43:0/1802253264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440ccce0 0x7fdc4415c8e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 >> 172.21.15.43:0/1802253264 conn(0x7fdc4406ce60 msgr2=0x7fdc4414d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:33.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 shutdown_connections 2024-09-17T05:29:33.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.281+0000 7fdc4b800640 1 -- 172.21.15.43:0/1802253264 wait complete. 2024-09-17T05:29:33.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 -- 172.21.15.43:0/1224103040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe8003d20 msgr2=0x7f2fe8004170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 --2- 172.21.15.43:0/1224103040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe8003d20 0x7f2fe8004170 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009b40 tx=0x7f2ff401d250 comp rx=0 tx=0).stop 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 -- 172.21.15.43:0/1224103040 shutdown_connections 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 --2- 172.21.15.43:0/1224103040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe8003d20 0x7f2fe8004170 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 -- 172.21.15.43:0/1224103040 >> 172.21.15.43:0/1224103040 conn(0x7f300406ce60 msgr2=0x7f300406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 -- 172.21.15.43:0/1224103040 shutdown_connections 2024-09-17T05:29:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.473+0000 7f3009000640 1 -- 172.21.15.43:0/1224103040 wait complete. 2024-09-17T05:29:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 Processor -- start 2024-09-17T05:29:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 -- start start 2024-09-17T05:29:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe8003d20 0x7f300409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 0x7f30041fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f300415fc70 con 0x7f30041fa640 2024-09-17T05:29:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f300415faf0 con 0x7f3004145fd0 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3009000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f300415fdf0 con 0x7f2fe8003d20 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.477+0000 7f3002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58196/0 (socket says 172.21.15.43:58196) 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 -- 172.21.15.43:0/774479533 learned_addr learned my addr 172.21.15.43:0/774479533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3001e00640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe8003d20 0x7f300409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3001400640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 0x7f30041fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 -- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe8003d20 msgr2=0x7f300409a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe8003d20 0x7f300409a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 -- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 msgr2=0x7f30041fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 0x7f30041fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 -- 172.21.15.43:0/774479533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3004075340 con 0x7f30041fa640 2024-09-17T05:29:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3001400640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 0x7f30041fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3002800640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc002030 tx=0x7f2ffc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc010950 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ffc017070 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc010e40 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30041faa50 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3004155140 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3004073c80 con 0x7f30041fa640 2024-09-17T05:29:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.481+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ffc01c460 con 0x7f30041fa640 2024-09-17T05:29:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.485+0000 7f2ffa000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 0x7f2fe4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.485+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5472+0+0 (secure 0 0 0) 0x7f2ffc094420 con 0x7f30041fa640 2024-09-17T05:29:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.485+0000 7f3001e00640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 0x7f2fe4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.485+0000 7f3001e00640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 0x7f2fe4075710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40ad000 tx=0x7f2ff4019ff0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.489+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30041faa50 con 0x7f30041fa640 2024-09-17T05:29:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:33.661+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f300409a890 con 0x7f30041fa640 2024-09-17T05:29:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.281+0000 7f2ffa000640 1 -- 172.21.15.43:0/774479533 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f2ffc05f4c0 con 0x7f30041fa640 2024-09-17T05:29:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:29:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 msgr2=0x7f2fe4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 0x7f2fe4075710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40ad000 tx=0x7f2ff4019ff0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 msgr2=0x7f3004155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc002030 tx=0x7f2ffc00ea10 comp rx=0 tx=0).stop 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 shutdown_connections 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fe4073200 0x7f2fe4075710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3004145fd0 0x7f30041fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30041fa640 0x7f3004155c20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 --2- 172.21.15.43:0/774479533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe8003d20 0x7f300409a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 >> 172.21.15.43:0/774479533 conn(0x7f300406ce60 msgr2=0x7f3004074480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.285+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 shutdown_connections 2024-09-17T05:29:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.289+0000 7f3009000640 1 -- 172.21.15.43:0/774479533 wait complete. 2024-09-17T05:29:34.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:29:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 -- 172.21.15.43:0/3358354343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8003b40 msgr2=0x7f2bd8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 --2- 172.21.15.43:0/3358354343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8003b40 0x7f2bd8003fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009b10 tx=0x7f2be401d230 comp rx=0 tx=0).stop 2024-09-17T05:29:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 -- 172.21.15.43:0/3358354343 shutdown_connections 2024-09-17T05:29:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 --2- 172.21.15.43:0/3358354343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bd8003b40 0x7f2bd8003fb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 -- 172.21.15.43:0/3358354343 >> 172.21.15.43:0/3358354343 conn(0x7f2bf40b2810 msgr2=0x7f2bf40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 -- 172.21.15.43:0/3358354343 shutdown_connections 2024-09-17T05:29:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.477+0000 7f2bf9200640 1 -- 172.21.15.43:0/3358354343 wait complete. 2024-09-17T05:29:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 Processor -- start 2024-09-17T05:29:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- start start 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd8003b40 0x7f2bf40afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bf40b0960 0x7f2bf40b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf40b46b0 con 0x7f2bf4150c30 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf40b4530 con 0x7f2bd8003b40 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf40b4830 con 0x7f2bf40b0960 2024-09-17T05:29:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd8003b40 0x7f2bf40afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bf40b0960 0x7f2bf40b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45856/0 (socket says 172.21.15.43:45856) 2024-09-17T05:29:34.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 -- 172.21.15.43:0/1785741983 learned_addr learned my addr 172.21.15.43:0/1785741983 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 -- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bf40b0960 msgr2=0x7f2bf40b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bf40b0960 0x7f2bf40b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 -- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd8003b40 msgr2=0x7f2bf40afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd8003b40 0x7f2bf40afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 -- 172.21.15.43:0/1785741983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf41f6c30 con 0x7f2bf4150c30 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf2800640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4074d50 tx=0x7f2bec006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec011900 con 0x7f2bf4150c30 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bec018070 con 0x7f2bf4150c30 2024-09-17T05:29:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41f6f10 con 0x7f2bf4150c30 2024-09-17T05:29:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec011d60 con 0x7f2bf4150c30 2024-09-17T05:29:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.481+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41f74d0 con 0x7f2bf4150c30 2024-09-17T05:29:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf40702b0 con 0x7f2bf4150c30 2024-09-17T05:29:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2bec01d460 con 0x7f2bf4150c30 2024-09-17T05:29:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bea000640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 0x7f2bd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7f2bec095090 con 0x7f2bf4150c30 2024-09-17T05:29:34.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bf1e00640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 0x7f2bd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.485+0000 7f2bf1e00640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 0x7f2bd0075710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2be40ae000 tx=0x7f2be4019fd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.489+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2bec05fce0 con 0x7f2bf4150c30 2024-09-17T05:29:34.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.673+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f2bf40b0160 con 0x7f2bf4150c30 2024-09-17T05:29:34.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.677+0000 7f2bea000640 1 -- 172.21.15.43:0/1785741983 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7f2bec064710 con 0x7f2bf4150c30 2024-09-17T05:29:34.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 msgr2=0x7f2bd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 0x7f2bd0075710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2be40ae000 tx=0x7f2be4019fd0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 msgr2=0x7f2bf4151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4074d50 tx=0x7f2bec006430 comp rx=0 tx=0).stop 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 shutdown_connections 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2bd0073200 0x7f2bd0075710 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bf4150c30 0x7f2bf4151040 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2bf40b0960 0x7f2bf40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 --2- 172.21.15.43:0/1785741983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd8003b40 0x7f2bf40afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 >> 172.21.15.43:0/1785741983 conn(0x7f2bf40b2810 msgr2=0x7f2bf40b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:34.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 shutdown_connections 2024-09-17T05:29:34.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.681+0000 7f2bf9200640 1 -- 172.21.15.43:0/1785741983 wait complete. 2024-09-17T05:29:34.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-17T05:29:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 -- 172.21.15.43:0/2838111075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 msgr2=0x7fcd88003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 --2- 172.21.15.43:0/2838111075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcd88003f60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcda4072880 tx=0x7fcd9401d280 comp rx=0 tx=0).stop 2024-09-17T05:29:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcda3e00640 1 -- 172.21.15.43:0/2838111075 reap_dead start 2024-09-17T05:29:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 -- 172.21.15.43:0/2838111075 shutdown_connections 2024-09-17T05:29:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 -- 172.21.15.43:0/2838111075 >> 172.21.15.43:0/2838111075 conn(0x7fcda40b23d0 msgr2=0x7fcda40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 -- 172.21.15.43:0/2838111075 shutdown_connections 2024-09-17T05:29:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.869+0000 7fcdaa000640 1 -- 172.21.15.43:0/2838111075 wait complete. 2024-09-17T05:29:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 Processor -- start 2024-09-17T05:29:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 -- start start 2024-09-17T05:29:34.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcda40c99d0 0x7fcda40c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcda40c5700 0x7fcda41fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcda41667f0 con 0x7fcd88003af0 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4166670 con 0x7fcda40c99d0 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcdaa000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcda4166970 con 0x7fcda40c5700 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45866/0 (socket says 172.21.15.43:45866) 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 -- 172.21.15.43:0/1539821324 learned_addr learned my addr 172.21.15.43:0/1539821324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3e00640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcda40c5700 0x7fcda41fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcd9be00640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcda40c99d0 0x7fcda40c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 -- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcda40c5700 msgr2=0x7fcda41fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcda40c5700 0x7fcda41fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 -- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcda40c99d0 msgr2=0x7fcda40c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcda40c99d0 0x7fcda40c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 -- 172.21.15.43:0/1539821324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda40c5b10 con 0x7fcd88003af0 2024-09-17T05:29:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.873+0000 7fcda3400640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcd94001ec0 tx=0x7fcd94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:34.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940bb030 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd94004110 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda40c39d0 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94004a00 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda40c3f50 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda40b33e0 con 0x7fcd88003af0 2024-09-17T05:29:34.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd940b5040 con 0x7fcd88003af0 2024-09-17T05:29:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.877+0000 7fcd9b400640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 0x7fcd80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.881+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fcd9412f780 con 0x7fcd88003af0 2024-09-17T05:29:34.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.881+0000 7fcd9be00640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 0x7fcd80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.881+0000 7fcd9be00640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 0x7fcd80075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c005ec0 tx=0x7fcd8c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:34.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:34.885+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcda40c5b10 con 0x7fcd88003af0 2024-09-17T05:29:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.117+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7fcda415e170 con 0x7fcd88003af0 2024-09-17T05:29:35.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.281+0000 7fcd9b400640 1 -- 172.21.15.43:0/1539821324 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v305) ==== 141+0+0 (secure 0 0 0) 0x7fcd94016fc0 con 0x7fcd88003af0 2024-09-17T05:29:35.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 msgr2=0x7fcd80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 0x7fcd80075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c005ec0 tx=0x7fcd8c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 msgr2=0x7fcda4165d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcd94001ec0 tx=0x7fcd94004750 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 shutdown_connections 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd80073200 0x7fcd80075710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcda40c99d0 0x7fcda40c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcda40c5700 0x7fcda41fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 --2- 172.21.15.43:0/1539821324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd88003af0 0x7fcda4165d10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 >> 172.21.15.43:0/1539821324 conn(0x7fcda40b23d0 msgr2=0x7fcda415fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 shutdown_connections 2024-09-17T05:29:35.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.285+0000 7fcdaa000640 1 -- 172.21.15.43:0/1539821324 wait complete. 2024-09-17T05:29:35.309 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-17T05:29:35.309 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-09-17T05:29:35.309 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:29:35.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- 172.21.15.43:0/21401873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c15adc0 msgr2=0x7fc85c06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 --2- 172.21.15.43:0/21401873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c15adc0 0x7fc85c06ce60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0cac70 tx=0x7fc85001c800 comp rx=0 tx=0).stop 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- 172.21.15.43:0/21401873 shutdown_connections 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 --2- 172.21.15.43:0/21401873 >> v2:172.21.15.62:3300/0 conn(0x7fc85c0cc6d0 0x7fc85c06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 --2- 172.21.15.43:0/21401873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c15adc0 0x7fc85c06ce60 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- 172.21.15.43:0/21401873 >> 172.21.15.43:0/21401873 conn(0x7fc85c0b23d0 msgr2=0x7fc85c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- 172.21.15.43:0/21401873 shutdown_connections 2024-09-17T05:29:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- 172.21.15.43:0/21401873 wait complete. 2024-09-17T05:29:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 Processor -- start 2024-09-17T05:29:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.465+0000 7fc864200640 1 -- start start 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc85c0cc6d0 0x7fc85c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc85c15adc0 0x7fc85c09d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1661d0 con 0x7fc85c0cc160 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c166050 con 0x7fc85c0cc6d0 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c166350 con 0x7fc85c15adc0 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45876/0 (socket says 172.21.15.43:45876) 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 -- 172.21.15.43:0/3457685433 learned_addr learned my addr 172.21.15.43:0/3457685433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc860e00640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc85c0cc6d0 0x7fc85c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc862200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc85c15adc0 0x7fc85c09d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 -- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc85c15adc0 msgr2=0x7fc85c09d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc85c15adc0 0x7fc85c09d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 -- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc85c0cc6d0 msgr2=0x7fc85c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc85c0cc6d0 0x7fc85c157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 -- 172.21.15.43:0/3457685433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c1fa360 con 0x7fc85c0cc160 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc861800640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc85001cce0 tx=0x7fc850005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500b2040 con 0x7fc85c0cc160 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8500a5980 con 0x7fc85c0cc160 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c165230 con 0x7fc85c0cc160 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500a5d90 con 0x7fc85c0cc160 2024-09-17T05:29:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c072fa0 con 0x7fc85c0cc160 2024-09-17T05:29:35.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.469+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c0ce100 con 0x7fc85c0cc160 2024-09-17T05:29:35.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.473+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8500bd030 con 0x7fc85c0cc160 2024-09-17T05:29:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.473+0000 7fc859600640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 0x7fc83c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.473+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fc85012dc20 con 0x7fc85c0cc160 2024-09-17T05:29:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.473+0000 7fc860e00640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 0x7fc83c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.473+0000 7fc860e00640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 0x7fc83c075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0ad5c0 tx=0x7fc84800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:35.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.477+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc8500c4020 con 0x7fc85c0cc160 2024-09-17T05:29:35.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.653+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc85c06f490 con 0x7fc85c0cc160 2024-09-17T05:29:35.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.653+0000 7fc859600640 1 -- 172.21.15.43:0/3457685433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13103 (secure 0 0 0) 0x7fc85c1fa360 con 0x7fc85c0cc160 2024-09-17T05:29:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:35.659 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":81,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:34.279301+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6837/1855028227":"2024-09-18T05:29:31.070615+0000","172.21.15.62:6835/1855028227":"2024-09-18T05:29:31.070615+0000","172.21.15.43:6835/3375674498":"2024-09-18T05:29:22.227124+0000","172.21.15.43:6834/3375674498":"2024-09-18T05:29:22.227124+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:35.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 msgr2=0x7fc83c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 0x7fc83c075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0ad5c0 tx=0x7fc84800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 msgr2=0x7fc85c1f9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc85001cce0 tx=0x7fc850005d20 comp rx=0 tx=0).stop 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 shutdown_connections 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc83c073200 0x7fc83c075710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc85c0cc6d0 0x7fc85c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc85c15adc0 0x7fc85c09d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 --2- 172.21.15.43:0/3457685433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc85c0cc160 0x7fc85c1f9e20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 >> 172.21.15.43:0/3457685433 conn(0x7fc85c0b23d0 msgr2=0x7fc85c06fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 shutdown_connections 2024-09-17T05:29:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.657+0000 7fc864200640 1 -- 172.21.15.43:0/3457685433 wait complete. 2024-09-17T05:29:35.681 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:29:35.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:29:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- 172.21.15.43:0/697029304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae380b3af0 msgr2=0x7fae380ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 --2- 172.21.15.43:0/697029304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae380b3af0 0x7fae380ad0b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae30009c80 tx=0x7fae3001cce0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- 172.21.15.43:0/697029304 shutdown_connections 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 --2- 172.21.15.43:0/697029304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae380b3af0 0x7fae380ad0b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 --2- 172.21.15.43:0/697029304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 0x7fae3815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- 172.21.15.43:0/697029304 >> 172.21.15.43:0/697029304 conn(0x7fae380b23d0 msgr2=0x7fae380d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- 172.21.15.43:0/697029304 shutdown_connections 2024-09-17T05:29:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- 172.21.15.43:0/697029304 wait complete. 2024-09-17T05:29:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 Processor -- start 2024-09-17T05:29:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 -- start start 2024-09-17T05:29:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3806f110 0x7fae3815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.841+0000 7fae3d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 0x7fae3815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae38168be0 con 0x7fae3808f110 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae38168a60 con 0x7fae3806f110 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fae38168d60 con 0x7fae38155d70 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45902/0 (socket says 172.21.15.43:45902) 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 -- 172.21.15.43:0/680943698 learned_addr learned my addr 172.21.15.43:0/680943698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae35e00640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3806f110 0x7fae3815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae2fe00640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 0x7fae3815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 -- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 msgr2=0x7fae3815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 0x7fae3815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 -- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3806f110 msgr2=0x7fae3815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3806f110 0x7fae3815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 -- 172.21.15.43:0/680943698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38160ac0 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae36800640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae30009bd0 tx=0x7fae30005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300bc070 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae30004130 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300b2040 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae38160500 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3808f5c0 con 0x7fae3808f110 2024-09-17T05:29:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3808f820 con 0x7fae3808f110 2024-09-17T05:29:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.845+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae300bf070 con 0x7fae3808f110 2024-09-17T05:29:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.849+0000 7fae2e000640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 0x7fae14075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.849+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fae3012dfd0 con 0x7fae3808f110 2024-09-17T05:29:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.849+0000 7fae35e00640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 0x7fae14075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.849+0000 7fae35e00640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 0x7fae14075740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae380715c0 tx=0x7fae280091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:35.853+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fae30016990 con 0x7fae3808f110 2024-09-17T05:29:36.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae380ad0b0 con 0x7fae3808f110 2024-09-17T05:29:36.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae2e000640 1 -- 172.21.15.43:0/680943698 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fae300f8ba0 con 0x7fae3808f110 2024-09-17T05:29:36.037 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 msgr2=0x7fae14075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 0x7fae14075740 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae380715c0 tx=0x7fae280091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 msgr2=0x7fae380cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae30009bd0 tx=0x7fae30005ae0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 shutdown_connections 2024-09-17T05:29:36.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae14073230 0x7fae14075740 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3808f110 0x7fae380cbf20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae38155d70 0x7fae3815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 --2- 172.21.15.43:0/680943698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3806f110 0x7fae3815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 >> 172.21.15.43:0/680943698 conn(0x7fae380b23d0 msgr2=0x7fae3806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 shutdown_connections 2024-09-17T05:29:36.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.033+0000 7fae3d200640 1 -- 172.21.15.43:0/680943698 wait complete. 2024-09-17T05:29:36.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 -- 172.21.15.43:0/682963379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 msgr2=0x7fc0f806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 --2- 172.21.15.43:0/682963379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f806ce60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009840 tx=0x7fc0e801c820 comp rx=0 tx=0).stop 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 -- 172.21.15.43:0/682963379 shutdown_connections 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 --2- 172.21.15.43:0/682963379 >> v2:172.21.15.62:3300/0 conn(0x7fc0f80cc160 0x7fc0f806d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 --2- 172.21.15.43:0/682963379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f806ce60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 -- 172.21.15.43:0/682963379 >> 172.21.15.43:0/682963379 conn(0x7fc0f80b23d0 msgr2=0x7fc0f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 -- 172.21.15.43:0/682963379 shutdown_connections 2024-09-17T05:29:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.213+0000 7fc0fec00640 1 -- 172.21.15.43:0/682963379 wait complete. 2024-09-17T05:29:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 Processor -- start 2024-09-17T05:29:36.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 -- start start 2024-09-17T05:29:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 0x7fc0f8163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f8164040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81681a0 con 0x7fc0f80cc6d0 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8168020 con 0x7fc0f80cc160 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f8168320 con 0x7fc0f815adc0 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0f7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 0x7fc0f8163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:35630/0 (socket says 172.21.15.43:35630) 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0f7e00640 1 -- 172.21.15.43:0/764156155 learned_addr learned my addr 172.21.15.43:0/764156155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.217+0000 7fc0fcc00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f8164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 -- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 msgr2=0x7fc0f8164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f8164040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 -- 172.21.15.43:0/764156155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 msgr2=0x7fc0f8163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 0x7fc0f8163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 -- 172.21.15.43:0/764156155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f80c7500 con 0x7fc0f80cc160 2024-09-17T05:29:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7400640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 0x7fc0f8163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0f7e00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009790 tx=0x7fc0e8005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80b2040 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0e8004130 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f80c77e0 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8003600 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f8159760 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f80ce100 con 0x7fc0f80cc160 2024-09-17T05:29:36.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.221+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc0e80bd060 con 0x7fc0f80cc160 2024-09-17T05:29:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.225+0000 7fc0efe00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 0x7fc0d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.225+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fc0e812d7a0 con 0x7fc0f80cc160 2024-09-17T05:29:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.225+0000 7fc0f7400640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 0x7fc0d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.225+0000 7fc0f7400640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 0x7fc0d8075740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80cf210 tx=0x7fc0e4001790 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.229+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc0e8016410 con 0x7fc0f80cc160 2024-09-17T05:29:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:36.465+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7fc0f80af0c0 con 0x7fc0f80cc160 2024-09-17T05:29:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0efe00640 1 -- 172.21.15.43:0/764156155 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v307) ==== 138+0+0 (secure 0 0 0) 0x7fc0e80f8480 con 0x7fc0f80cc160 2024-09-17T05:29:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 msgr2=0x7fc0d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 0x7fc0d8075740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80cf210 tx=0x7fc0e4001790 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 msgr2=0x7fc0f809d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009790 tx=0x7fc0e8005760 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 shutdown_connections 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc0d8073230 0x7fc0d8075740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc0f815adc0 0x7fc0f8164040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0f80cc6d0 0x7fc0f8163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 --2- 172.21.15.43:0/764156155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0f80cc160 0x7fc0f809d4f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 >> 172.21.15.43:0/764156155 conn(0x7fc0f80b23d0 msgr2=0x7fc0f80afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 shutdown_connections 2024-09-17T05:29:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.297+0000 7fc0fec00640 1 -- 172.21.15.43:0/764156155 wait complete. 2024-09-17T05:29:37.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:29:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 -- 172.21.15.43:0/4197846480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 msgr2=0x7f1d9015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 --2- 172.21.15.43:0/4197846480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d9015c810 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009900 tx=0x7f1d8001c7e0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 -- 172.21.15.43:0/4197846480 shutdown_connections 2024-09-17T05:29:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 --2- 172.21.15.43:0/4197846480 >> v2:172.21.15.62:3300/0 conn(0x7f1d900b58e0 0x7f1d900b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 --2- 172.21.15.43:0/4197846480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d9015c810 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 -- 172.21.15.43:0/4197846480 >> 172.21.15.43:0/4197846480 conn(0x7f1d900b2e80 msgr2=0x7f1d900b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 -- 172.21.15.43:0/4197846480 shutdown_connections 2024-09-17T05:29:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.485+0000 7f1d95400640 1 -- 172.21.15.43:0/4197846480 wait complete. 2024-09-17T05:29:37.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 Processor -- start 2024-09-17T05:29:37.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 -- start start 2024-09-17T05:29:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9006ce60 0x7f1d900cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 0x7f1d9015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d90169dc0 con 0x7f1d900b4180 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d90169c40 con 0x7f1d9006ce60 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d95400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1d90169f40 con 0x7f1d900b58e0 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45928/0 (socket says 172.21.15.43:45928) 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d85e00640 1 -- 172.21.15.43:0/3675646490 learned_addr learned my addr 172.21.15.43:0/3675646490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d8e800640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9006ce60 0x7f1d900cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.489+0000 7f1d8f200640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 0x7f1d9015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 -- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 msgr2=0x7f1d9015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 0x7f1d9015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 -- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9006ce60 msgr2=0x7f1d900cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9006ce60 0x7f1d900cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 -- 172.21.15.43:0/3675646490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d900ca380 con 0x7f1d900b4180 2024-09-17T05:29:37.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d8f200640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 0x7f1d9015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d85e00640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c3c0 tx=0x7f1d7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d780052d0 con 0x7f1d900b4180 2024-09-17T05:29:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d78007020 con 0x7f1d900b4180 2024-09-17T05:29:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78007470 con 0x7f1d900b4180 2024-09-17T05:29:37.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d900c8540 con 0x7f1d900b4180 2024-09-17T05:29:37.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d900c8b00 con 0x7f1d900b4180 2024-09-17T05:29:37.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d900cff00 con 0x7f1d900b4180 2024-09-17T05:29:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.493+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d780067e0 con 0x7f1d900b4180 2024-09-17T05:29:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.497+0000 7f1d87400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 0x7f1d6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.497+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7f1d780933b0 con 0x7f1d900b4180 2024-09-17T05:29:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.497+0000 7f1d8e800640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 0x7f1d6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.497+0000 7f1d8e800640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 0x7f1d6c075710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d800af000 tx=0x7f1d8001b020 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:37.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.501+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d9015cd50 con 0x7f1d900b4180 2024-09-17T05:29:37.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1d9015c810 con 0x7f1d900b4180 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d87400640 1 -- 172.21.15.43:0/3675646490 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) ==== 59+0+2294 (secure 0 0 0) 0x7f1d7805df80 con 0x7f1d900b4180 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:29:37.684 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:29:37.685 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:29:37.686 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:29:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 msgr2=0x7f1d6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 0x7f1d6c075710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d800af000 tx=0x7f1d8001b020 comp rx=0 tx=0).stop 2024-09-17T05:29:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 msgr2=0x7f1d90152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c3c0 tx=0x7f1d7800c880 comp rx=0 tx=0).stop 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 shutdown_connections 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1d6c073200 0x7f1d6c075710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1d900b58e0 0x7f1d9015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d900b4180 0x7f1d90152460 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 --2- 172.21.15.43:0/3675646490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9006ce60 0x7f1d900cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 >> 172.21.15.43:0/3675646490 conn(0x7f1d900b2e80 msgr2=0x7f1d900b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 shutdown_connections 2024-09-17T05:29:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.681+0000 7f1d95400640 1 -- 172.21.15.43:0/3675646490 wait complete. 2024-09-17T05:29:37.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:29:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- 172.21.15.43:0/1930297378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 msgr2=0x7fddfc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 --2- 172.21.15.43:0/1930297378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0ad0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009bc0 tx=0x7fddf401cae0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- 172.21.15.43:0/1930297378 shutdown_connections 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 --2- 172.21.15.43:0/1930297378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0ad0b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 --2- 172.21.15.43:0/1930297378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc0b3af0 0x7fddfc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- 172.21.15.43:0/1930297378 >> 172.21.15.43:0/1930297378 conn(0x7fddfc0b23d0 msgr2=0x7fddfc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- 172.21.15.43:0/1930297378 shutdown_connections 2024-09-17T05:29:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- 172.21.15.43:0/1930297378 wait complete. 2024-09-17T05:29:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 Processor -- start 2024-09-17T05:29:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.865+0000 7fde02200640 1 -- start start 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 0x7fddfc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc168be0 con 0x7fddfc155d70 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc168a60 con 0x7fddfc06f110 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc168d60 con 0x7fddfc09cb90 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45938/0 (socket says 172.21.15.43:45938) 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 -- 172.21.15.43:0/3082648232 learned_addr learned my addr 172.21.15.43:0/3082648232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddfbe00640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 0x7fddfc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddfb400640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 -- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 msgr2=0x7fddfc15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 0x7fddfc15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 -- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 msgr2=0x7fddfc0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 -- 172.21.15.43:0/3082648232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc09c4b0 con 0x7fddfc155d70 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddfb400640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddfbe00640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 0x7fddfc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3e00640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e7a0 tx=0x7fdde400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde400ccc0 con 0x7fddfc155d70 2024-09-17T05:29:37.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde4010430 con 0x7fddfc155d70 2024-09-17T05:29:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4010860 con 0x7fddfc155d70 2024-09-17T05:29:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc0cced0 con 0x7fddfc155d70 2024-09-17T05:29:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1fbd70 con 0x7fddfc155d70 2024-09-17T05:29:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.869+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc0ad0b0 con 0x7fddfc155d70 2024-09-17T05:29:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.873+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdde4010a00 con 0x7fddfc155d70 2024-09-17T05:29:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.873+0000 7fddf3400640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 0x7fddd8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.873+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fdde4094900 con 0x7fddfc155d70 2024-09-17T05:29:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.873+0000 7fddfb400640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 0x7fddd8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:37.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.873+0000 7fddfb400640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 0x7fddd8075740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fddec001230 tx=0x7fddec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:37.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:37.877+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fddfc09c4b0 con 0x7fddfc155d70 2024-09-17T05:29:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.053+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fddfc0ccc60 con 0x7fddfc155d70 2024-09-17T05:29:38.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fddf3400640 1 -- 172.21.15.43:0/3082648232 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7fdde4066020 con 0x7fddfc155d70 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 msgr2=0x7fddd8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 0x7fddd8075740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fddec001230 tx=0x7fddec009040 comp rx=0 tx=0).stop 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 msgr2=0x7fddfc09bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdde400e7a0 tx=0x7fdde400ec60 comp rx=0 tx=0).stop 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 shutdown_connections 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fddd8073230 0x7fddd8075740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fddfc09cb90 0x7fddfc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddfc155d70 0x7fddfc09bf70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 --2- 172.21.15.43:0/3082648232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc06f110 0x7fddfc0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 >> 172.21.15.43:0/3082648232 conn(0x7fddfc0b23d0 msgr2=0x7fddfc09d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:38.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 shutdown_connections 2024-09-17T05:29:38.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.057+0000 7fde02200640 1 -- 172.21.15.43:0/3082648232 wait complete. 2024-09-17T05:29:38.082 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:29:38.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:29:38.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:29:38.310 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:29:38.311 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:38.311 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:38.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:29:38.320 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:29:38.320 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:38.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:29:38.329 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:38.329 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:38.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:29:38.371 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:29:38.371 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:38.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:29:38.380 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:29:38.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 -- 172.21.15.43:0/158628304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a80b3af0 msgr2=0x7fa0a80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 --2- 172.21.15.43:0/158628304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a80b3af0 0x7fa0a80ad0b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8072b60 tx=0x7fa0a001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 -- 172.21.15.43:0/158628304 shutdown_connections 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 --2- 172.21.15.43:0/158628304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a80b3af0 0x7fa0a80ad0b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 --2- 172.21.15.43:0/158628304 >> v2:172.21.15.62:3300/0 conn(0x7fa0a8155d70 0x7fa0a815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 -- 172.21.15.43:0/158628304 >> 172.21.15.43:0/158628304 conn(0x7fa0a80b23d0 msgr2=0x7fa0a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 -- 172.21.15.43:0/158628304 shutdown_connections 2024-09-17T05:29:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.569+0000 7fa0af200640 1 -- 172.21.15.43:0/158628304 wait complete. 2024-09-17T05:29:38.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 Processor -- start 2024-09-17T05:29:38.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- start start 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a8155d70 0x7fa0a80cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 0x7fa0a81f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8168c60 con 0x7fa0a806f110 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8168ae0 con 0x7fa0a8155d70 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a8168de0 con 0x7fa0a809cea0 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45958/0 (socket says 172.21.15.43:45958) 2024-09-17T05:29:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 -- 172.21.15.43:0/2667091012 learned_addr learned my addr 172.21.15.43:0/2667091012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa09f400640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a8155d70 0x7fa0a80cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0ad200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 0x7fa0a81f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 -- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 msgr2=0x7fa0a81f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 0x7fa0a81f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 -- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a8155d70 msgr2=0x7fa0a80cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a8155d70 0x7fa0a80cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 -- 172.21.15.43:0/2667091012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a80cc820 con 0x7fa0a806f110 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0ad200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 0x7fa0a81f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a7e00640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c970 tx=0x7fa09800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098007bb0 con 0x7fa0a806f110 2024-09-17T05:29:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa098007e40 con 0x7fa0a806f110 2024-09-17T05:29:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a80cd770 con 0x7fa0a806f110 2024-09-17T05:29:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09800f6b0 con 0x7fa0a806f110 2024-09-17T05:29:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.573+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a81fa340 con 0x7fa0a806f110 2024-09-17T05:29:38.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a80ad0b0 con 0x7fa0a806f110 2024-09-17T05:29:38.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa09800f850 con 0x7fa0a806f110 2024-09-17T05:29:38.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa0a4c00640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 0x7fa084075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:38.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fa098094630 con 0x7fa0a806f110 2024-09-17T05:29:38.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa09f400640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 0x7fa084075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:38.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.577+0000 7fa09f400640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 0x7fa084075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8051150 tx=0x7fa090009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.581+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa098014030 con 0x7fa0a806f110 2024-09-17T05:29:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.789+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0a80b3d30 con 0x7fa0a806f110 2024-09-17T05:29:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0a4c00640 1 -- 172.21.15.43:0/2667091012 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3911 (secure 0 0 0) 0x7fa09805f980 con 0x7fa0a806f110 2024-09-17T05:29:38.796 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:38.796 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":79,"btime":"2024-09-17T05:29:32:073179+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/2353446907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2353446907},{"type":"v1","addr":"172.21.15.62:6836","nonce":2353446907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/4002554588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":4002554588},{"type":"v1","addr":"172.21.15.62:6837","nonce":4002554588}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1536877764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1536877764},{"type":"v1","addr":"172.21.15.43:6833","nonce":1536877764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2273925294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2273925294},{"type":"v1","addr":"172.21.15.43:6835","nonce":2273925294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-09-17T05:29:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 79 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 msgr2=0x7fa084075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 0x7fa084075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8051150 tx=0x7fa090009060 comp rx=0 tx=0).stop 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 msgr2=0x7fa0a80b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c970 tx=0x7fa09800ce30 comp rx=0 tx=0).stop 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 shutdown_connections 2024-09-17T05:29:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa084073200 0x7fa084075710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0a8155d70 0x7fa0a80cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa0a809cea0 0x7fa0a81f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 --2- 172.21.15.43:0/2667091012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0a806f110 0x7fa0a80b0500 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 >> 172.21.15.43:0/2667091012 conn(0x7fa0a80b23d0 msgr2=0x7fa0a815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 shutdown_connections 2024-09-17T05:29:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:38.793+0000 7fa0af200640 1 -- 172.21.15.43:0/2667091012 wait complete. 2024-09-17T05:29:38.818 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:29:38.818 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:29:38.819 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:29:38.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:29:38.819 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:29:38.819 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:29:38.819 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:29:38.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:29:38.820 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:29:38.820 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:29:38.820 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:29:38.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:29:38.820 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:29:38.820 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:29:38.820 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:29:38.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:29:38.833 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:29:38.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:29:38.834 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:29:38.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:29:38.835 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:29:38.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:29:38.841 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:29:38.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:29:38.881 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:29:38.882 DEBUG:teuthology.parallel:result is None 2024-09-17T05:29:38.883 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:29:38.883 DEBUG:teuthology.parallel:result is None 2024-09-17T05:29:38.886 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:29:38.886 DEBUG:teuthology.parallel:result is None 2024-09-17T05:29:38.888 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:29:38.888 DEBUG:teuthology.parallel:result is None 2024-09-17T05:29:38.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:29:38.981 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:29:38.983 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:29:38.984 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-09-17T05:29:38.981+0000 7f8b31600640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6833/0: (98) Address already in use 2024-09-17T05:29:38.992 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:29:38.995 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:29:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- 172.21.15.43:0/272963389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca540ce3c0 msgr2=0x7fca54073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 --2- 172.21.15.43:0/272963389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca540ce3c0 0x7fca54073f60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009c80 tx=0x7fca4c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- 172.21.15.43:0/272963389 shutdown_connections 2024-09-17T05:29:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 --2- 172.21.15.43:0/272963389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca540ce3c0 0x7fca54073f60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 --2- 172.21.15.43:0/272963389 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cda80 0x7fca54073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- 172.21.15.43:0/272963389 >> 172.21.15.43:0/272963389 conn(0x7fca540b2810 msgr2=0x7fca540b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- 172.21.15.43:0/272963389 shutdown_connections 2024-09-17T05:29:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- 172.21.15.43:0/272963389 wait complete. 2024-09-17T05:29:39.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 Processor -- start 2024-09-17T05:29:39.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.053+0000 7fca59200640 1 -- start start 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 0x7fca5414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5414fc90 0x7fca541fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca54169d20 con 0x7fca540ce3c0 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fca54169ba0 con 0x7fca5414fc90 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fca54169ea0 con 0x7fca540cdfb0 2024-09-17T05:29:39.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca4be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 0x7fca5414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:54092/0 (socket says 172.21.15.43:54092) 2024-09-17T05:29:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 -- 172.21.15.43:0/1486800193 learned_addr learned my addr 172.21.15.43:0/1486800193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca52800640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5414fc90 0x7fca541fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 -- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5414fc90 msgr2=0x7fca541fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5414fc90 0x7fca541fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 -- 172.21.15.43:0/1486800193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 msgr2=0x7fca5414b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 0x7fca5414b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 -- 172.21.15.43:0/1486800193 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5414e590 con 0x7fca540cdfb0 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca4be00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 0x7fca5414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca51e00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c3f0 tx=0x7fca4400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44017070 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca440053c0 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44003860 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca541fb7f0 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca540b5ad0 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.057+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca54073a20 con 0x7fca540cdfb0 2024-09-17T05:29:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.061+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fca4401a070 con 0x7fca540cdfb0 2024-09-17T05:29:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.061+0000 7fca4a000640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 0x7fca30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.061+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(81..81 src has 1..81) ==== 5029+0+0 (secure 0 0 0) 0x7fca44093360 con 0x7fca540cdfb0 2024-09-17T05:29:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.061+0000 7fca4be00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 0x7fca30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.061+0000 7fca4be00640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 0x7fca30075710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca5415fd70 tx=0x7fca3c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.065+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fca4405df30 con 0x7fca540cdfb0 2024-09-17T05:29:39.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.249+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fca54071270 con 0x7fca540cdfb0 2024-09-17T05:29:39.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.313+0000 7fca4a000640 1 -- 172.21.15.43:0/1486800193 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7fca44062960 con 0x7fca540cdfb0 2024-09-17T05:29:39.319 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:29:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 msgr2=0x7fca30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 0x7fca30075710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca5415fd70 tx=0x7fca3c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 msgr2=0x7fca5414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c3f0 tx=0x7fca4400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 shutdown_connections 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca30073200 0x7fca30075710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca5414fc90 0x7fca541fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca540ce3c0 0x7fca5414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 --2- 172.21.15.43:0/1486800193 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fca540cdfb0 0x7fca5414e050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 >> 172.21.15.43:0/1486800193 conn(0x7fca540b2810 msgr2=0x7fca5415aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 shutdown_connections 2024-09-17T05:29:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.317+0000 7fca59200640 1 -- 172.21.15.43:0/1486800193 wait complete. 2024-09-17T05:29:39.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:29:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 -- 172.21.15.43:0/1019276987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc5015f470 msgr2=0x7fcc500d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 --2- 172.21.15.43:0/1019276987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc5015f470 0x7fcc500d0390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009900 tx=0x7fcc4401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 -- 172.21.15.43:0/1019276987 shutdown_connections 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 --2- 172.21.15.43:0/1019276987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5006fce0 0x7fcc500700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 --2- 172.21.15.43:0/1019276987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc5015f470 0x7fcc500d0390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 -- 172.21.15.43:0/1019276987 >> 172.21.15.43:0/1019276987 conn(0x7fcc500b2810 msgr2=0x7fcc500b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 -- 172.21.15.43:0/1019276987 shutdown_connections 2024-09-17T05:29:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.497+0000 7fcc57c00640 1 -- 172.21.15.43:0/1019276987 wait complete. 2024-09-17T05:29:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 Processor -- start 2024-09-17T05:29:39.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 -- start start 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc500b5610 0x7fcc5015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5015f470 0x7fcc5015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc501695b0 con 0x7fcc5006fce0 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50169430 con 0x7fcc500b5610 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc57c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcc50169730 con 0x7fcc5015f470 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46012/0 (socket says 172.21.15.43:46012) 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 -- 172.21.15.43:0/2888899171 learned_addr learned my addr 172.21.15.43:0/2888899171 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc4fe00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc500b5610 0x7fcc5015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5015f470 0x7fcc5015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 -- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5015f470 msgr2=0x7fcc5015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5015f470 0x7fcc5015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 -- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc500b5610 msgr2=0x7fcc5015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc500b5610 0x7fcc5015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 -- 172.21.15.43:0/2888899171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc50155b80 con 0x7fcc5006fce0 2024-09-17T05:29:39.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.501+0000 7fcc55200640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcc4401a3f0 tx=0x7fcc440a6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440ac070 con 0x7fcc5006fce0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc440050c0 con 0x7fcc5006fce0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc50155e20 con 0x7fcc5006fce0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440054f0 con 0x7fcc5006fce0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5014cfe0 con 0x7fcc5006fce0 2024-09-17T05:29:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc500700f0 con 0x7fcc5006fce0 2024-09-17T05:29:39.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc440af070 con 0x7fcc5006fce0 2024-09-17T05:29:39.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 0x7fcc30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4857+0+0 (secure 0 0 0) 0x7fcc4412e490 con 0x7fcc5006fce0 2024-09-17T05:29:39.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.505+0000 7fcc4fe00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 0x7fcc30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.509+0000 7fcc4fe00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 0x7fcc30075710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc501599b0 tx=0x7fcc3c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.509+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcc440bf020 con 0x7fcc5006fce0 2024-09-17T05:29:39.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.745+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcc50076770 con 0x7fcc5006fce0 2024-09-17T05:29:39.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.745+0000 7fcc4cc00640 1 -- 172.21.15.43:0/2888899171 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7fcc440b6070 con 0x7fcc5006fce0 2024-09-17T05:29:39.749 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.750 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:29:39.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:29:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:29:39.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:29:39.762 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:29:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 msgr2=0x7fcc30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 0x7fcc30075710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc501599b0 tx=0x7fcc3c009040 comp rx=0 tx=0).stop 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 msgr2=0x7fcc5014d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcc4401a3f0 tx=0x7fcc440a6490 comp rx=0 tx=0).stop 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 shutdown_connections 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcc30073200 0x7fcc30075710 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc500b5610 0x7fcc5015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc5015f470 0x7fcc5015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 --2- 172.21.15.43:0/2888899171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc5006fce0 0x7fcc5014d7e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 >> 172.21.15.43:0/2888899171 conn(0x7fcc500b2810 msgr2=0x7fcc500cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 shutdown_connections 2024-09-17T05:29:39.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.749+0000 7fcc57c00640 1 -- 172.21.15.43:0/2888899171 wait complete. 2024-09-17T05:29:39.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- 172.21.15.43:0/233770878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec06f940 msgr2=0x7fabec071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 --2- 172.21.15.43:0/233770878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec06f940 0x7fabec071150 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009840 tx=0x7fabdc01c790 comp rx=0 tx=0).stop 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- 172.21.15.43:0/233770878 shutdown_connections 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 --2- 172.21.15.43:0/233770878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec0b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 --2- 172.21.15.43:0/233770878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec06f940 0x7fabec071150 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- 172.21.15.43:0/233770878 >> 172.21.15.43:0/233770878 conn(0x7fabec0b2810 msgr2=0x7fabec0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- 172.21.15.43:0/233770878 shutdown_connections 2024-09-17T05:29:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- 172.21.15.43:0/233770878 wait complete. 2024-09-17T05:29:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 Processor -- start 2024-09-17T05:29:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.929+0000 7fabf2000640 1 -- start start 2024-09-17T05:29:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabec06f940 0x7fabec154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fabec16c020 con 0x7fabec160210 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabec16bea0 con 0x7fabec06f940 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fabec16c1a0 con 0x7fabec0b4570 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46040/0 (socket says 172.21.15.43:46040) 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 -- 172.21.15.43:0/3797044485 learned_addr learned my addr 172.21.15.43:0/3797044485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabe3e00640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabeb400640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabec06f940 0x7fabec154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 -- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 msgr2=0x7fabec149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 -- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabec06f940 msgr2=0x7fabec154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabec06f940 0x7fabec154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 -- 172.21.15.43:0/3797044485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec14ae90 con 0x7fabec160210 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabe3e00640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabebe00640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c730 tx=0x7fabe400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4003b80 con 0x7fabec160210 2024-09-17T05:29:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe4003d40 con 0x7fabec160210 2024-09-17T05:29:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40052d0 con 0x7fabec160210 2024-09-17T05:29:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabec14abf0 con 0x7fabec160210 2024-09-17T05:29:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabec14ff70 con 0x7fabec160210 2024-09-17T05:29:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.933+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabec14b380 con 0x7fabec160210 2024-09-17T05:29:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.937+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fabe4005680 con 0x7fabec160210 2024-09-17T05:29:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.937+0000 7fabe3400640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 0x7fabc8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.937+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4857+0+0 (secure 0 0 0) 0x7fabe4093e30 con 0x7fabec160210 2024-09-17T05:29:39.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.937+0000 7fabeb400640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 0x7fabc8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:39.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.937+0000 7fabeb400640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 0x7fabc8075740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0ae000 tx=0x7fabdc0b3040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:39.941+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fabec0b4ec0 con 0x7fabec160210 2024-09-17T05:29:40.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.121+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fabec071150 con 0x7fabec160210 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.121+0000 7fabe3400640 1 -- 172.21.15.43:0/3797044485 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7fabe405eb40 con 0x7fabec160210 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:29:40.126 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:29:40.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 msgr2=0x7fabc8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 0x7fabc8075740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0ae000 tx=0x7fabdc0b3040 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 msgr2=0x7fabec14a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c730 tx=0x7fabe400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 shutdown_connections 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fabc8073230 0x7fabc8075740 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabec160210 0x7fabec14a4d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fabec0b4570 0x7fabec149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 --2- 172.21.15.43:0/3797044485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabec06f940 0x7fabec154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 >> 172.21.15.43:0/3797044485 conn(0x7fabec0b2810 msgr2=0x7fabec161af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 shutdown_connections 2024-09-17T05:29:40.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.125+0000 7fabf2000640 1 -- 172.21.15.43:0/3797044485 wait complete. 2024-09-17T05:29:40.148 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:29:40.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:29:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 -- 172.21.15.43:0/93726938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc15b7e0 msgr2=0x7fd1cc0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 --2- 172.21.15.43:0/93726938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc0cf270 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009900 tx=0x7fd1b401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 -- 172.21.15.43:0/93726938 shutdown_connections 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 --2- 172.21.15.43:0/93726938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc0cf270 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 --2- 172.21.15.43:0/93726938 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc074c60 0x7fd1cc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 -- 172.21.15.43:0/93726938 >> 172.21.15.43:0/93726938 conn(0x7fd1cc06ce60 msgr2=0x7fd1cc06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 -- 172.21.15.43:0/93726938 shutdown_connections 2024-09-17T05:29:40.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.309+0000 7fd1d3000640 1 -- 172.21.15.43:0/93726938 wait complete. 2024-09-17T05:29:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 Processor -- start 2024-09-17T05:29:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 -- start start 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf7b0 0x7fd1cc14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc1fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc168d60 con 0x7fd1cc074c60 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc168be0 con 0x7fd1cc15b7e0 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d3000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc168ee0 con 0x7fd1cc0cf7b0 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:40.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46064/0 (socket says 172.21.15.43:46064) 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 -- 172.21.15.43:0/3721760334 learned_addr learned my addr 172.21.15.43:0/3721760334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1c3e00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf7b0 0x7fd1cc14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1d1000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc1fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 -- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf7b0 msgr2=0x7fd1cc14e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf7b0 0x7fd1cc14e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 -- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc15b7e0 msgr2=0x7fd1cc1fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc1fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 -- 172.21.15.43:0/3721760334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc0b06c0 con 0x7fd1cc074c60 2024-09-17T05:29:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.313+0000 7fd1cbe00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e7c0 tx=0x7fd1bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00ccc0 con 0x7fd1cc074c60 2024-09-17T05:29:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1bc00ce80 con 0x7fd1cc074c60 2024-09-17T05:29:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc005280 con 0x7fd1cc074c60 2024-09-17T05:29:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc14fde0 con 0x7fd1cc074c60 2024-09-17T05:29:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc0b0960 con 0x7fd1cc074c60 2024-09-17T05:29:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc0b3b80 con 0x7fd1cc074c60 2024-09-17T05:29:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.317+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1bc005420 con 0x7fd1cc074c60 2024-09-17T05:29:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.321+0000 7fd1c8c00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.321+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4857+0+0 (secure 0 0 0) 0x7fd1bc0948a0 con 0x7fd1cc074c60 2024-09-17T05:29:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.321+0000 7fd1c3e00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.321+0000 7fd1c3e00640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 0x7fd1a8075710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40af000 tx=0x7fd1b401b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.325+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1bc05f520 con 0x7fd1cc074c60 2024-09-17T05:29:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:40.505+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd1cc14eeb0 con 0x7fd1cc074c60 2024-09-17T05:29:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.329+0000 7fd1c8c00640 1 -- 172.21.15.43:0/3721760334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7fd1bc063f50 con 0x7fd1cc074c60 2024-09-17T05:29:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:29:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 msgr2=0x7fd1a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 0x7fd1a8075710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40af000 tx=0x7fd1b401b120 comp rx=0 tx=0).stop 2024-09-17T05:29:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 msgr2=0x7fd1cc14f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e7c0 tx=0x7fd1bc00ec80 comp rx=0 tx=0).stop 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 shutdown_connections 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd1a8073200 0x7fd1a8075710 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1cc15b7e0 0x7fd1cc1fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1cc0cf7b0 0x7fd1cc14e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 --2- 172.21.15.43:0/3721760334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1cc074c60 0x7fd1cc14f8a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 >> 172.21.15.43:0/3721760334 conn(0x7fd1cc06ce60 msgr2=0x7fd1cc06f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 shutdown_connections 2024-09-17T05:29:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.333+0000 7fd1d3000640 1 -- 172.21.15.43:0/3721760334 wait complete. 2024-09-17T05:29:41.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:29:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 -- 172.21.15.43:0/2149920099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b5070 msgr2=0x7f21d80b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 --2- 172.21.15.43:0/2149920099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b5070 0x7f21d80b5480 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f21d80d02c0 tx=0x7f21c001c6c0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 -- 172.21.15.43:0/2149920099 shutdown_connections 2024-09-17T05:29:41.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 --2- 172.21.15.43:0/2149920099 >> v2:172.21.15.62:3300/0 conn(0x7f21d80d08b0 0x7f21d80d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 --2- 172.21.15.43:0/2149920099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b5070 0x7f21d80b5480 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 -- 172.21.15.43:0/2149920099 >> 172.21.15.43:0/2149920099 conn(0x7f21d80691b0 msgr2=0x7f21d80695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 -- 172.21.15.43:0/2149920099 shutdown_connections 2024-09-17T05:29:41.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.525+0000 7f21d7e00640 1 -- 172.21.15.43:0/2149920099 wait complete. 2024-09-17T05:29:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 Processor -- start 2024-09-17T05:29:41.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 -- start start 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80b5070 0x7f21d815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21d80d08b0 0x7f21d81f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8169cf0 con 0x7f21d815c4e0 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8169b70 con 0x7f21d80b5070 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.529+0000 7f21d7e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f21d8169e70 con 0x7f21d80d08b0 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46072/0 (socket says 172.21.15.43:46072) 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 -- 172.21.15.43:0/1164837032 learned_addr learned my addr 172.21.15.43:0/1164837032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:41.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21cfe00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21d80d08b0 0x7f21d81f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d6a00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80b5070 0x7f21d815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 -- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21d80d08b0 msgr2=0x7f21d81f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21d80d08b0 0x7f21d81f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 -- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80b5070 msgr2=0x7f21d815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80b5070 0x7f21d815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 -- 172.21.15.43:0/1164837032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d81fc890 con 0x7f21d815c4e0 2024-09-17T05:29:41.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7400640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f21d000da40 tx=0x7f21d000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21d815c4e0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0004b30 con 0x7f21d815c4e0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0005020 con 0x7f21d815c4e0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d81fcb70 con 0x7f21d815c4e0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d81fd130 con 0x7f21d815c4e0 2024-09-17T05:29:41.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d81fc600 con 0x7f21d815c4e0 2024-09-17T05:29:41.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.533+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21d0003980 con 0x7f21d815c4e0 2024-09-17T05:29:41.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.537+0000 7f21cea00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 0x7f21b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.537+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5227+0+0 (secure 0 0 0) 0x7f21d0094a90 con 0x7f21d815c4e0 2024-09-17T05:29:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.537+0000 7f21d6a00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 0x7f21b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:41.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.537+0000 7f21d6a00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 0x7f21b4075710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f21c8007fd0 tx=0x7f21c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:41.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.541+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21d0020070 con 0x7f21d815c4e0 2024-09-17T05:29:41.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:41.717+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f21d815f890 con 0x7f21d815c4e0 2024-09-17T05:29:42.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.337+0000 7f21cea00640 1 -- 172.21.15.43:0/1164837032 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7f21d005f720 con 0x7f21d815c4e0 2024-09-17T05:29:42.340 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:29:42.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 msgr2=0x7f21b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 0x7f21b4075710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f21c8007fd0 tx=0x7f21c8009040 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 msgr2=0x7f21d81563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f21d000da40 tx=0x7f21d000df00 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 shutdown_connections 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f21b4073200 0x7f21b4075710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815c4e0 0x7f21d81563b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f21d80d08b0 0x7f21d81f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 --2- 172.21.15.43:0/1164837032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80b5070 0x7f21d815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 >> 172.21.15.43:0/1164837032 conn(0x7f21d80691b0 msgr2=0x7f21d80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:42.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.341+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 shutdown_connections 2024-09-17T05:29:42.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.345+0000 7f21d7e00640 1 -- 172.21.15.43:0/1164837032 wait complete. 2024-09-17T05:29:42.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:29:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 -- 172.21.15.43:0/4090367464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b40cc6d0 msgr2=0x7f00b406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 --2- 172.21.15.43:0/4090367464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b40cc6d0 0x7f00b406f6e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f00b40cac70 tx=0x7f00ac01cd80 comp rx=0 tx=0).stop 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 -- 172.21.15.43:0/4090367464 shutdown_connections 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 --2- 172.21.15.43:0/4090367464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b40cc6d0 0x7f00b406f6e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 --2- 172.21.15.43:0/4090367464 >> v2:172.21.15.62:3300/0 conn(0x7f00b406d570 0x7f00b406f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 -- 172.21.15.43:0/4090367464 >> 172.21.15.43:0/4090367464 conn(0x7f00b40b23d0 msgr2=0x7f00b40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 -- 172.21.15.43:0/4090367464 shutdown_connections 2024-09-17T05:29:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.549+0000 7f00bba00640 1 -- 172.21.15.43:0/4090367464 wait complete. 2024-09-17T05:29:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 Processor -- start 2024-09-17T05:29:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- start start 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00b40cc6d0 0x7f00b409cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00b409d560 0x7f00b40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41665f0 con 0x7f00b415adc0 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4166470 con 0x7f00b409d560 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f00b4166770 con 0x7f00b40cc6d0 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46096/0 (socket says 172.21.15.43:46096) 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 -- 172.21.15.43:0/1045735205 learned_addr learned my addr 172.21.15.43:0/1045735205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b9a00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00b409d560 0x7f00b40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b9000640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00b40cc6d0 0x7f00b409cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 -- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00b40cc6d0 msgr2=0x7f00b409cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00b40cc6d0 0x7f00b409cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 -- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00b409d560 msgr2=0x7f00b40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00b409d560 0x7f00b40b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 -- 172.21.15.43:0/1045735205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b409dde0 con 0x7f00b415adc0 2024-09-17T05:29:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b3e00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e7f0 tx=0x7f00a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000ccc0 con 0x7f00b415adc0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00a000ce80 con 0x7f00b415adc0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0005280 con 0x7f00b415adc0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b41fc660 con 0x7f00b415adc0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.553+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41fcc20 con 0x7f00b415adc0 2024-09-17T05:29:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.557+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b4166c40 con 0x7f00b415adc0 2024-09-17T05:29:42.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.557+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00a0005420 con 0x7f00b415adc0 2024-09-17T05:29:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.557+0000 7f00b0c00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 0x7f0094075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.557+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5609+0+0 (secure 0 0 0) 0x7f00a0094e60 con 0x7f00b415adc0 2024-09-17T05:29:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.557+0000 7f00b9000640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 0x7f0094075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:42.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.561+0000 7f00b9000640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 0x7f0094075740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00a8007fd0 tx=0x7f00a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:42.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.561+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00b409dde0 con 0x7f00b415adc0 2024-09-17T05:29:42.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:42.773+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f00b40cb820 con 0x7f00b415adc0 2024-09-17T05:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.337+0000 7f00b0c00640 1 -- 172.21.15.43:0/1045735205 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v82) ==== 174+0+0 (secure 0 0 0) 0x7f00a005f900 con 0x7f00b415adc0 2024-09-17T05:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 12 and data pool 13 2024-09-17T05:29:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 msgr2=0x7f0094075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 0x7f0094075740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00a8007fd0 tx=0x7f00a8009040 comp rx=0 tx=0).stop 2024-09-17T05:29:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 msgr2=0x7f00b40b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e7f0 tx=0x7f00a000ecb0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 shutdown_connections 2024-09-17T05:29:43.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0094073230 0x7f0094075740 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00b409d560 0x7f00b40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b415adc0 0x7f00b40b0d90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 --2- 172.21.15.43:0/1045735205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00b40cc6d0 0x7f00b409cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.345+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 >> 172.21.15.43:0/1045735205 conn(0x7f00b40b23d0 msgr2=0x7f00b40cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.349+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 shutdown_connections 2024-09-17T05:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.349+0000 7f00bba00640 1 -- 172.21.15.43:0/1045735205 wait complete. 2024-09-17T05:29:43.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 -- 172.21.15.43:0/3348499693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc06ce60 msgr2=0x7f6fbc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 --2- 172.21.15.43:0/3348499693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc06ce60 0x7f6fbc06d270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009840 tx=0x7f6fa801c790 comp rx=0 tx=0).stop 2024-09-17T05:29:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 -- 172.21.15.43:0/3348499693 shutdown_connections 2024-09-17T05:29:43.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 --2- 172.21.15.43:0/3348499693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc06ce60 0x7f6fbc06d270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 -- 172.21.15.43:0/3348499693 >> 172.21.15.43:0/3348499693 conn(0x7f6fbc0b23d0 msgr2=0x7f6fbc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 -- 172.21.15.43:0/3348499693 shutdown_connections 2024-09-17T05:29:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.557+0000 7f6fc3600640 1 -- 172.21.15.43:0/3348499693 wait complete. 2024-09-17T05:29:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 Processor -- start 2024-09-17T05:29:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- start start 2024-09-17T05:29:43.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fbc06ce60 0x7f6fbc0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc070620 0x7f6fbc09c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0ce790 con 0x7f6fbc0cb860 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0ce610 con 0x7f6fbc070620 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0ce910 con 0x7f6fbc06ce60 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46112/0 (socket says 172.21.15.43:46112) 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 -- 172.21.15.43:0/2812236551 learned_addr learned my addr 172.21.15.43:0/2812236551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fbbe00640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc070620 0x7f6fbc09c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc0c00640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fbc06ce60 0x7f6fbc0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 -- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fbc06ce60 msgr2=0x7f6fbc0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fbc06ce60 0x7f6fbc0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 -- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc070620 msgr2=0x7f6fbc09c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc070620 0x7f6fbc09c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 -- 172.21.15.43:0/2812236551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc09ce70 con 0x7f6fbc0cb860 2024-09-17T05:29:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc1600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400cb10 tx=0x7f6fb400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4016020 con 0x7f6fbc0cb860 2024-09-17T05:29:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb400ce50 con 0x7f6fbc0cb860 2024-09-17T05:29:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400ade0 con 0x7f6fbc0cb860 2024-09-17T05:29:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc0c6de0 con 0x7f6fbc0cb860 2024-09-17T05:29:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.561+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc158120 con 0x7f6fbc0cb860 2024-09-17T05:29:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc1fb6c0 con 0x7f6fbc0cb860 2024-09-17T05:29:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6fb4019020 con 0x7f6fbc0cb860 2024-09-17T05:29:43.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fb8c00640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 0x7f6f9c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb4094370 con 0x7f6fbc0cb860 2024-09-17T05:29:43.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fc0c00640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 0x7f6f9c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.565+0000 7f6fc0c00640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 0x7f6f9c075740 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000c10 tx=0x7f6fb0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:43.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.569+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6fb4020020 con 0x7f6fbc0cb860 2024-09-17T05:29:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fbc158380 con 0x7f6fbc0cb860 2024-09-17T05:29:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fb8c00640 1 -- 172.21.15.43:0/2812236551 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16685 (secure 0 0 0) 0x7f6fbc06d7b0 con 0x7f6fbc0cb860 2024-09-17T05:29:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:43.753 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":88,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:42.776874+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:29:40.518875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:29:41.731305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 msgr2=0x7f6f9c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 0x7f6f9c075740 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0000c10 tx=0x7f6fb0009040 comp rx=0 tx=0).stop 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 msgr2=0x7f6fbc157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400cb10 tx=0x7f6fb400cc00 comp rx=0 tx=0).stop 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 shutdown_connections 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6f9c073230 0x7f6f9c075740 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fbc070620 0x7f6fbc09c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cb860 0x7f6fbc157710 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 --2- 172.21.15.43:0/2812236551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6fbc06ce60 0x7f6fbc0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 >> 172.21.15.43:0/2812236551 conn(0x7f6fbc0b23d0 msgr2=0x7f6fbc09d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 shutdown_connections 2024-09-17T05:29:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.749+0000 7f6fc3600640 1 -- 172.21.15.43:0/2812236551 wait complete. 2024-09-17T05:29:43.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:43.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 -- 172.21.15.43:0/4134116661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c06ce60 msgr2=0x7f3a5c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 --2- 172.21.15.43:0/4134116661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c06d270 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009840 tx=0x7f3a4401c790 comp rx=0 tx=0).stop 2024-09-17T05:29:43.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 -- 172.21.15.43:0/4134116661 shutdown_connections 2024-09-17T05:29:43.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 --2- 172.21.15.43:0/4134116661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c06d270 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 -- 172.21.15.43:0/4134116661 >> 172.21.15.43:0/4134116661 conn(0x7f3a5c0d1160 msgr2=0x7f3a5c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:43.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.929+0000 7f3a61c00640 1 -- 172.21.15.43:0/4134116661 shutdown_connections 2024-09-17T05:29:43.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 -- 172.21.15.43:0/4134116661 wait complete. 2024-09-17T05:29:43.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 Processor -- start 2024-09-17T05:29:43.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 -- start start 2024-09-17T05:29:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0cd990 0x7f3a5c13f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c16b080 con 0x7f3a5c0748b0 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c16af00 con 0x7f3a5c06ce60 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.933+0000 7f3a61c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c16b200 con 0x7f3a5c0cd990 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59664/0 (socket says 172.21.15.43:59664) 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 -- 172.21.15.43:0/2039989184 learned_addr learned my addr 172.21.15.43:0/2039989184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a5be00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a61200640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0cd990 0x7f3a5c13f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 -- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0cd990 msgr2=0x7f3a5c13f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0cd990 0x7f3a5c13f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 -- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 msgr2=0x7f3a5c140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:43.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 -- 172.21.15.43:0/2039989184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c13e6c0 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a5be00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a53400640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009790 tx=0x7f3a440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a440b2040 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a44005030 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44005460 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c13e960 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c13ef20 con 0x7f3a5c0748b0 2024-09-17T05:29:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c06d270 con 0x7f3a5c0748b0 2024-09-17T05:29:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.937+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a440bd060 con 0x7f3a5c0748b0 2024-09-17T05:29:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.941+0000 7f3a58c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.941+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5709+0+0 (secure 0 0 0) 0x7f3a4412d690 con 0x7f3a5c0748b0 2024-09-17T05:29:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.941+0000 7f3a5be00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.941+0000 7f3a5be00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 0x7f3a38075710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0016b0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:43.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:43.945+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a44016410 con 0x7f3a5c0748b0 2024-09-17T05:29:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.121+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a5c06da90 con 0x7f3a5c0748b0 2024-09-17T05:29:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.121+0000 7f3a58c00640 1 -- 172.21.15.43:0/2039989184 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16685 (secure 0 0 0) 0x7f3a440f87d0 con 0x7f3a5c0748b0 2024-09-17T05:29:44.126 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:44.126 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":88,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:42.776874+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:29:40.518875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:29:41.731305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 msgr2=0x7f3a38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 0x7f3a38075710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0016b0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-09-17T05:29:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 msgr2=0x7f3a5c1388c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009790 tx=0x7f3a440040c0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 shutdown_connections 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a5c0cd990 0x7f3a5c13f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a5c0748b0 0x7f3a5c1388c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 --2- 172.21.15.43:0/2039989184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a5c06ce60 0x7f3a5c140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 >> 172.21.15.43:0/2039989184 conn(0x7f3a5c0d1160 msgr2=0x7f3a5c15e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 shutdown_connections 2024-09-17T05:29:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.125+0000 7f3a61c00640 1 -- 172.21.15.43:0/2039989184 wait complete. 2024-09-17T05:29:44.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 -- 172.21.15.43:0/1448844913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c0cb860 msgr2=0x7fe82c0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 --2- 172.21.15.43:0/1448844913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c0cb860 0x7fe82c0cbc70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0d12a0 tx=0x7fe82401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 -- 172.21.15.43:0/1448844913 shutdown_connections 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 --2- 172.21.15.43:0/1448844913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c0cb860 0x7fe82c0cbc70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 --2- 172.21.15.43:0/1448844913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 0x7fe82c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 --2- 172.21.15.43:0/1448844913 >> v2:172.21.15.62:3300/0 conn(0x7fe82c070620 0x7fe82c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 -- 172.21.15.43:0/1448844913 >> 172.21.15.43:0/1448844913 conn(0x7fe82c0b23d0 msgr2=0x7fe82c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 -- 172.21.15.43:0/1448844913 shutdown_connections 2024-09-17T05:29:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.305+0000 7fe832c00640 1 -- 172.21.15.43:0/1448844913 wait complete. 2024-09-17T05:29:44.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 Processor -- start 2024-09-17T05:29:44.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 -- start start 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 0x7fe82c159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe82c0cb860 0x7fe82c1fca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c0ce490 con 0x7fe82c070620 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c0ce310 con 0x7fe82c0cb860 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe832c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe82c0ce610 con 0x7fe82c06ce60 2024-09-17T05:29:44.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe823400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe823400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59678/0 (socket says 172.21.15.43:59678) 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.309+0000 7fe823400640 1 -- 172.21.15.43:0/3740801980 learned_addr learned my addr 172.21.15.43:0/3740801980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe82be00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 0x7fe82c159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe830c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe82c0cb860 0x7fe82c1fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 -- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 msgr2=0x7fe82c159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 0x7fe82c159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 -- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe82c0cb860 msgr2=0x7fe82c1fca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe82c0cb860 0x7fe82c1fca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 -- 172.21.15.43:0/3740801980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c1fd280 con 0x7fe82c070620 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe823400640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe81400e7c0 tx=0x7fe81400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81400ccc0 con 0x7fe82c070620 2024-09-17T05:29:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe81400ce80 con 0x7fe82c070620 2024-09-17T05:29:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814005280 con 0x7fe82c070620 2024-09-17T05:29:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c0c6e20 con 0x7fe82c070620 2024-09-17T05:29:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c0ca080 con 0x7fe82c070620 2024-09-17T05:29:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82c0cebc0 con 0x7fe82c070620 2024-09-17T05:29:44.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.313+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe814005420 con 0x7fe82c070620 2024-09-17T05:29:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.317+0000 7fe828c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 0x7fe808075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.317+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5709+0+0 (secure 0 0 0) 0x7fe814094a90 con 0x7fe82c070620 2024-09-17T05:29:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.317+0000 7fe82be00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 0x7fe808075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.317+0000 7fe82be00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 0x7fe808075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00ab30 tx=0x7fe81c010040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.321+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe81405f450 con 0x7fe82c070620 2024-09-17T05:29:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:44.533+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe82c0cb370 con 0x7fe82c070620 2024-09-17T05:29:45.325 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:29:45.325+0000 7f6cf1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:45.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.361+0000 7fe828c00640 1 -- 172.21.15.43:0/3740801980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7fe814063e80 con 0x7fe82c070620 2024-09-17T05:29:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 msgr2=0x7fe808075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 0x7fe808075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00ab30 tx=0x7fe81c010040 comp rx=0 tx=0).stop 2024-09-17T05:29:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 msgr2=0x7fe82c1fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe81400e7c0 tx=0x7fe81400ec80 comp rx=0 tx=0).stop 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 shutdown_connections 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe808073200 0x7fe808075710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe82c0cb860 0x7fe82c1fca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe82c070620 0x7fe82c1fc500 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 --2- 172.21.15.43:0/3740801980 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe82c06ce60 0x7fe82c159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 >> 172.21.15.43:0/3740801980 conn(0x7fe82c0b23d0 msgr2=0x7fe82c166390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 shutdown_connections 2024-09-17T05:29:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.365+0000 7fe832c00640 1 -- 172.21.15.43:0/3740801980 wait complete. 2024-09-17T05:29:45.392 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:29:45.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:29:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 -- 172.21.15.43:0/1900062510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 msgr2=0x7f7bf40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 --2- 172.21.15.43:0/1900062510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf40700a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40ccb60 tx=0x7f7bdc01d200 comp rx=0 tx=0).stop 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 -- 172.21.15.43:0/1900062510 shutdown_connections 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 --2- 172.21.15.43:0/1900062510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf40700a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 --2- 172.21.15.43:0/1900062510 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf40730b0 0x7f7bf40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 --2- 172.21.15.43:0/1900062510 >> v2:172.21.15.62:3300/0 conn(0x7f7bf415fe10 0x7f7bf40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 -- 172.21.15.43:0/1900062510 >> 172.21.15.43:0/1900062510 conn(0x7f7bf40b23d0 msgr2=0x7f7bf40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 -- 172.21.15.43:0/1900062510 shutdown_connections 2024-09-17T05:29:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.553+0000 7f7bf9800640 1 -- 172.21.15.43:0/1900062510 wait complete. 2024-09-17T05:29:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 Processor -- start 2024-09-17T05:29:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 -- start start 2024-09-17T05:29:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40730b0 0x7f7bf41f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf415fe10 0x7f7bf41f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf40ceb00 con 0x7f7bf406fc90 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf40ce980 con 0x7f7bf40730b0 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf9800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf40cec80 con 0x7f7bf415fe10 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59698/0 (socket says 172.21.15.43:59698) 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 -- 172.21.15.43:0/3276247466 learned_addr learned my addr 172.21.15.43:0/3276247466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf3400640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf415fe10 0x7f7bf41f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bebe00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40730b0 0x7f7bf41f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 -- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf415fe10 msgr2=0x7f7bf41f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf415fe10 0x7f7bf41f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 -- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40730b0 msgr2=0x7f7bf41f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40730b0 0x7f7bf41f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.557+0000 7f7bf2a00640 1 -- 172.21.15.43:0/3276247466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf415d4c0 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7bf2a00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7be400d950 tx=0x7f7be400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4017070 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7be4009d20 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf415d7a0 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be40106c0 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf415dd60 con 0x7f7bf406fc90 2024-09-17T05:29:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf40734c0 con 0x7f7bf406fc90 2024-09-17T05:29:45.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7be401a070 con 0x7f7bf406fc90 2024-09-17T05:29:45.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:45.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.561+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f7be40944e0 con 0x7f7bf406fc90 2024-09-17T05:29:45.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.565+0000 7f7bebe00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:45.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.565+0000 7f7bebe00640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0af000 tx=0x7f7bdc01afe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.569+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7be405f3e0 con 0x7f7bf406fc90 2024-09-17T05:29:45.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:45.781+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7bf40ad6a0 con 0x7f7bf406fc90 2024-09-17T05:29:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7beaa00640 1 -- 172.21.15.43:0/3276247466 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f7be405ece0 con 0x7f7bf406fc90 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 msgr2=0x7f7bd0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0af000 tx=0x7f7bdc01afe0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 msgr2=0x7f7bf4148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7be400d950 tx=0x7f7be400de10 comp rx=0 tx=0).stop 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 shutdown_connections 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf40730b0 0x7f7bf41f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf415fe10 0x7f7bf41f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 --2- 172.21.15.43:0/3276247466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf406fc90 0x7f7bf4148aa0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 >> 172.21.15.43:0/3276247466 conn(0x7f7bf40b23d0 msgr2=0x7f7bf4075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:46.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 shutdown_connections 2024-09-17T05:29:46.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.369+0000 7f7bf9800640 1 -- 172.21.15.43:0/3276247466 wait complete. 2024-09-17T05:29:46.399 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:29:46.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 -- 172.21.15.43:0/1878816326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c806fc00 msgr2=0x7f35c8070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 --2- 172.21.15.43:0/1878816326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c806fc00 0x7f35c8070010 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009bc0 tx=0x7f35c001cba0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 -- 172.21.15.43:0/1878816326 shutdown_connections 2024-09-17T05:29:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 --2- 172.21.15.43:0/1878816326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c806fc00 0x7f35c8070010 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 --2- 172.21.15.43:0/1878816326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 0x7f35c8160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 -- 172.21.15.43:0/1878816326 >> 172.21.15.43:0/1878816326 conn(0x7f35c80b24c0 msgr2=0x7f35c80b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 -- 172.21.15.43:0/1878816326 shutdown_connections 2024-09-17T05:29:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.557+0000 7f35cda00640 1 -- 172.21.15.43:0/1878816326 wait complete. 2024-09-17T05:29:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 Processor -- start 2024-09-17T05:29:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 -- start start 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 0x7f35c80ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 0x7f35c8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35c816c0b0 con 0x7f35c815eeb0 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35c816bf30 con 0x7f35c806fc00 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35cda00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f35c816c230 con 0x7f35c8160070 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35be000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35be000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59726/0 (socket says 172.21.15.43:59726) 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35be000640 1 -- 172.21.15.43:0/3954178704 learned_addr learned my addr 172.21.15.43:0/3954178704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35c6a00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 0x7f35c80ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.561+0000 7f35c7400640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 0x7f35c8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 -- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 msgr2=0x7f35c8154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 0x7f35c8154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 -- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 msgr2=0x7f35c80ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 0x7f35c80ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 -- 172.21.15.43:0/3954178704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c80cad70 con 0x7f35c815eeb0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35c6a00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 0x7f35c80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35be000640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e7f0 tx=0x7f35b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b000ccc0 con 0x7f35c815eeb0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35b000ce80 con 0x7f35c815eeb0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c80cb280 con 0x7f35c815eeb0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0005280 con 0x7f35c815eeb0 2024-09-17T05:29:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c80b3960 con 0x7f35c815eeb0 2024-09-17T05:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c80caff0 con 0x7f35c815eeb0 2024-09-17T05:29:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f35b0005420 con 0x7f35c815eeb0 2024-09-17T05:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.565+0000 7f35bf400640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 0x7f35a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.569+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f35b0094d20 con 0x7f35c815eeb0 2024-09-17T05:29:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.569+0000 7f35c6a00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 0x7f35a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.569+0000 7f35c6a00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 0x7f35a4075740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007a60 tx=0x7f35b8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.573+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f35b005fc20 con 0x7f35c815eeb0 2024-09-17T05:29:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.781+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c80cb7a0 con 0x7f35c815eeb0 2024-09-17T05:29:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.781+0000 7f35bf400640 1 -- 172.21.15.43:0/3954178704 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5216 (secure 0 0 0) 0x7f35c80cb7a0 con 0x7f35c815eeb0 2024-09-17T05:29:46.786 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:46.786 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"btime":"2024-09-17T05:29:46:366852+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:29:45.786243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6846":{"gid":6846,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/274766899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":274766899},{"type":"v1","addr":"172.21.15.43:6835","nonce":274766899}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6846,"qdb_cluster":[6846]},"id":6}]} 2024-09-17T05:29:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 86 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 msgr2=0x7f35a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 0x7f35a4075740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007a60 tx=0x7f35b8009040 comp rx=0 tx=0).stop 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 msgr2=0x7f35c8154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f35b000e7f0 tx=0x7f35b000ecb0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 shutdown_connections 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f35a4073230 0x7f35a4075740 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f35c8160070 0x7f35c8154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815eeb0 0x7f35c8154720 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 --2- 172.21.15.43:0/3954178704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35c806fc00 0x7f35c80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 >> 172.21.15.43:0/3954178704 conn(0x7f35c80b24c0 msgr2=0x7f35c80cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 shutdown_connections 2024-09-17T05:29:46.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.785+0000 7f35cda00640 1 -- 172.21.15.43:0/3954178704 wait complete. 2024-09-17T05:29:46.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 -- 172.21.15.43:0/1664267761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b806f110 msgr2=0x7fb4b80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/1664267761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b806f110 0x7fb4b80ad0b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80af0a0 tx=0x7fb4b001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 -- 172.21.15.43:0/1664267761 shutdown_connections 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/1664267761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b806f110 0x7fb4b80ad0b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/1664267761 >> v2:172.21.15.62:3300/0 conn(0x7fb4b80b3af0 0x7fb4b815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 -- 172.21.15.43:0/1664267761 >> 172.21.15.43:0/1664267761 conn(0x7fb4b80b23d0 msgr2=0x7fb4b80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 -- 172.21.15.43:0/1664267761 shutdown_connections 2024-09-17T05:29:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.961+0000 7fb4bfa00640 1 -- 172.21.15.43:0/1664267761 wait complete. 2024-09-17T05:29:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 Processor -- start 2024-09-17T05:29:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 -- start start 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4b806f110 0x7fb4b80ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 0x7fb4b8164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8168b90 con 0x7fb4b8155610 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8168a10 con 0x7fb4b8155d70 2024-09-17T05:29:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bfa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b8168d10 con 0x7fb4b806f110 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.965+0000 7fb4bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59746/0 (socket says 172.21.15.43:59746) 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 -- 172.21.15.43:0/2557088339 learned_addr learned my addr 172.21.15.43:0/2557088339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4afe00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 0x7fb4b8164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bd000640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4b806f110 0x7fb4b80ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 -- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4b806f110 msgr2=0x7fb4b80ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4b806f110 0x7fb4b80ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 -- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 msgr2=0x7fb4b8164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 0x7fb4b8164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 -- 172.21.15.43:0/2557088339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b80ca5b0 con 0x7fb4b8155610 2024-09-17T05:29:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4afe00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 0x7fb4b8164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bda00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb4b001b370 tx=0x7fb4b0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00bc070 con 0x7fb4b8155610 2024-09-17T05:29:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4b0004330 con 0x7fb4b8155610 2024-09-17T05:29:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81fc6c0 con 0x7fb4b8155610 2024-09-17T05:29:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00b2040 con 0x7fb4b8155610 2024-09-17T05:29:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b81fcc80 con 0x7fb4b8155610 2024-09-17T05:29:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b80ad0b0 con 0x7fb4b8155610 2024-09-17T05:29:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.969+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb4b00bf070 con 0x7fb4b8155610 2024-09-17T05:29:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.973+0000 7fb4b5600640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 0x7fb494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.973+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7fb4b012dfe0 con 0x7fb4b8155610 2024-09-17T05:29:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.973+0000 7fb4bd000640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 0x7fb494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.973+0000 7fb4bd000640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 0x7fb494075710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80715c0 tx=0x7fb4a8008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:46.977+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb4b0016970 con 0x7fb4b8155610 2024-09-17T05:29:47.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.149+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4b815ed80 con 0x7fb4b8155610 2024-09-17T05:29:47.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.149+0000 7fb4b5600640 1 -- 172.21.15.43:0/2557088339 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16672 (secure 0 0 0) 0x7fb4b00f9030 con 0x7fb4b8155610 2024-09-17T05:29:47.155 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:47.155 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":89,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:29:44.343047+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:29:40.518875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:29:41.731305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 msgr2=0x7fb494075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 0x7fb494075710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80715c0 tx=0x7fb4a8008040 comp rx=0 tx=0).stop 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 msgr2=0x7fb4b8164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb4b001b370 tx=0x7fb4b0005b10 comp rx=0 tx=0).stop 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 shutdown_connections 2024-09-17T05:29:47.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb494073200 0x7fb494075710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b8155d70 0x7fb4b8164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4b8155610 0x7fb4b8164cb0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 --2- 172.21.15.43:0/2557088339 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4b806f110 0x7fb4b80ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 >> 172.21.15.43:0/2557088339 conn(0x7fb4b80b23d0 msgr2=0x7fb4b80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 shutdown_connections 2024-09-17T05:29:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.153+0000 7fb4bfa00640 1 -- 172.21.15.43:0/2557088339 wait complete. 2024-09-17T05:29:47.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:29:47.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:29:47.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 -- 172.21.15.43:0/1737938699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 msgr2=0x7f90e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 --2- 172.21.15.43:0/1737938699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90e400b9d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009ee0 tx=0x7f90f401d610 comp rx=0 tx=0).stop 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9101800640 1 -- 172.21.15.43:0/1737938699 reap_dead start 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 -- 172.21.15.43:0/1737938699 shutdown_connections 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 -- 172.21.15.43:0/1737938699 >> 172.21.15.43:0/1737938699 conn(0x7f90fc0b23d0 msgr2=0x7f90fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 -- 172.21.15.43:0/1737938699 shutdown_connections 2024-09-17T05:29:47.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.333+0000 7f9103800640 1 -- 172.21.15.43:0/1737938699 wait complete. 2024-09-17T05:29:47.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 Processor -- start 2024-09-17T05:29:47.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- start start 2024-09-17T05:29:47.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 0x7f90fc0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 0x7f90fc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0ad0b0 con 0x7f90e4005160 2024-09-17T05:29:47.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0b3d00 con 0x7f90fc074620 2024-09-17T05:29:47.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0ad230 con 0x7f90fc1f5f40 2024-09-17T05:29:47.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59762/0 (socket says 172.21.15.43:59762) 2024-09-17T05:29:47.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 -- 172.21.15.43:0/3149115001 learned_addr learned my addr 172.21.15.43:0/3149115001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9101800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 0x7f90fc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f90fbe00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 0x7f90fc0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 -- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 msgr2=0x7f90fc0740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 0x7f90fc0740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 -- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 msgr2=0x7f90fc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 0x7f90fc0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 -- 172.21.15.43:0/3149115001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc074a30 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9101800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 0x7f90fc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f90fbe00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 0x7f90fc0740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9100e00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f90f0011b30 tx=0x7f90f000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000fe00 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f0004020 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004470 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc0c9e40 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.337+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc0ca400 con 0x7f90e4005160 2024-09-17T05:29:47.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.341+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc0ca600 con 0x7f90e4005160 2024-09-17T05:29:47.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.341+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f90f0020020 con 0x7f90e4005160 2024-09-17T05:29:47.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.341+0000 7f90f8c00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 0x7f90dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.341+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f90f0094d90 con 0x7f90e4005160 2024-09-17T05:29:47.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.341+0000 7f90fbe00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 0x7f90dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.345+0000 7f90fbe00640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 0x7f90dc075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06a9d0 tx=0x7f90e80067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:47.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.345+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90fc074a30 con 0x7f90e4005160 2024-09-17T05:29:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.517+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90fc0c9a10 con 0x7f90dc073200 2024-09-17T05:29:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.525+0000 7f90f8c00640 1 -- 172.21.15.43:0/3149115001 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251326 (secure 0 0 0) 0x7f90e8001680 con 0x7f90dc073200 2024-09-17T05:29:47.532 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 msgr2=0x7f90dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 0x7f90dc075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06a9d0 tx=0x7f90e80067b0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 msgr2=0x7f90fc1f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f90f0011b30 tx=0x7f90f000f3c0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 shutdown_connections 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f90dc073200 0x7f90dc075710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90fc074620 0x7f90fc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f90fc1f5f40 0x7f90fc0740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 --2- 172.21.15.43:0/3149115001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e4005160 0x7f90fc1f2a90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 >> 172.21.15.43:0/3149115001 conn(0x7f90fc0b23d0 msgr2=0x7f90fc0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 shutdown_connections 2024-09-17T05:29:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.529+0000 7f9103800640 1 -- 172.21.15.43:0/3149115001 wait complete. 2024-09-17T05:29:47.554 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-17T05:29:45.944969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":130,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259064,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715656,"statfs":{"total":773094113280,"available":772828831744,"internally_reserved":0,"allocated":9773056,"data_stored":4851543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.606284"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387862+0000","last_change":"2024-09-17T05:29:43.383927+0000","last_active":"2024-09-17T05:29:43.387862+0000","last_peered":"2024-09-17T05:29:43.387862+0000","last_clean":"2024-09-17T05:29:43.387862+0000","last_became_active":"2024-09-17T05:29:43.383511+0000","last_became_peered":"2024-09-17T05:29:43.383511+0000","last_unstale":"2024-09-17T05:29:43.387862+0000","last_undegraded":"2024-09-17T05:29:43.387862+0000","last_fullsized":"2024-09-17T05:29:43.387862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:41.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385872+0000","last_change":"2024-09-17T05:29:43.379276+0000","last_active":"2024-09-17T05:29:43.385872+0000","last_peered":"2024-09-17T05:29:43.385872+0000","last_clean":"2024-09-17T05:29:43.385872+0000","last_became_active":"2024-09-17T05:29:43.378853+0000","last_became_peered":"2024-09-17T05:29:43.378853+0000","last_unstale":"2024-09-17T05:29:43.385872+0000","last_undegraded":"2024-09-17T05:29:43.385872+0000","last_fullsized":"2024-09-17T05:29:43.385872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:19.202636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394364+0000","last_change":"2024-09-17T05:29:43.392064+0000","last_active":"2024-09-17T05:29:43.394364+0000","last_peered":"2024-09-17T05:29:43.394364+0000","last_clean":"2024-09-17T05:29:43.394364+0000","last_became_active":"2024-09-17T05:29:43.390606+0000","last_became_peered":"2024-09-17T05:29:43.390606+0000","last_unstale":"2024-09-17T05:29:43.394364+0000","last_undegraded":"2024-09-17T05:29:43.394364+0000","last_fullsized":"2024-09-17T05:29:43.394364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:46.931916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385255+0000","last_change":"2024-09-17T05:29:43.382510+0000","last_active":"2024-09-17T05:29:43.385255+0000","last_peered":"2024-09-17T05:29:43.385255+0000","last_clean":"2024-09-17T05:29:43.385255+0000","last_became_active":"2024-09-17T05:29:43.381739+0000","last_became_peered":"2024-09-17T05:29:43.381739+0000","last_unstale":"2024-09-17T05:29:43.385255+0000","last_undegraded":"2024-09-17T05:29:43.385255+0000","last_fullsized":"2024-09-17T05:29:43.385255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:46.669427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.375278+0000","last_change":"2024-09-17T05:29:43.355953+0000","last_active":"2024-09-17T05:29:43.375278+0000","last_peered":"2024-09-17T05:29:43.375278+0000","last_clean":"2024-09-17T05:29:43.375278+0000","last_became_active":"2024-09-17T05:29:43.355604+0000","last_became_peered":"2024-09-17T05:29:43.355604+0000","last_unstale":"2024-09-17T05:29:43.375278+0000","last_undegraded":"2024-09-17T05:29:43.375278+0000","last_fullsized":"2024-09-17T05:29:43.375278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.528747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394510+0000","last_change":"2024-09-17T05:29:43.392245+0000","last_active":"2024-09-17T05:29:43.394510+0000","last_peered":"2024-09-17T05:29:43.394510+0000","last_clean":"2024-09-17T05:29:43.394510+0000","last_became_active":"2024-09-17T05:29:43.391284+0000","last_became_peered":"2024-09-17T05:29:43.391284+0000","last_unstale":"2024-09-17T05:29:43.394510+0000","last_undegraded":"2024-09-17T05:29:43.394510+0000","last_fullsized":"2024-09-17T05:29:43.394510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:06.112953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387159+0000","last_change":"2024-09-17T05:29:43.384304+0000","last_active":"2024-09-17T05:29:43.387159+0000","last_peered":"2024-09-17T05:29:43.387159+0000","last_clean":"2024-09-17T05:29:43.387159+0000","last_became_active":"2024-09-17T05:29:43.383960+0000","last_became_peered":"2024-09-17T05:29:43.383960+0000","last_unstale":"2024-09-17T05:29:43.387159+0000","last_undegraded":"2024-09-17T05:29:43.387159+0000","last_fullsized":"2024-09-17T05:29:43.387159+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:16.431772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378294+0000","last_change":"2024-09-17T05:29:43.369429+0000","last_active":"2024-09-17T05:29:43.378294+0000","last_peered":"2024-09-17T05:29:43.378294+0000","last_clean":"2024-09-17T05:29:43.378294+0000","last_became_active":"2024-09-17T05:29:43.369047+0000","last_became_peered":"2024-09-17T05:29:43.369047+0000","last_unstale":"2024-09-17T05:29:43.378294+0000","last_undegraded":"2024-09-17T05:29:43.378294+0000","last_fullsized":"2024-09-17T05:29:43.378294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:41.813501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387838+0000","last_change":"2024-09-17T05:29:43.382992+0000","last_active":"2024-09-17T05:29:43.387838+0000","last_peered":"2024-09-17T05:29:43.387838+0000","last_clean":"2024-09-17T05:29:43.387838+0000","last_became_active":"2024-09-17T05:29:43.379486+0000","last_became_peered":"2024-09-17T05:29:43.379486+0000","last_unstale":"2024-09-17T05:29:43.387838+0000","last_undegraded":"2024-09-17T05:29:43.387838+0000","last_fullsized":"2024-09-17T05:29:43.387838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:45.318539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388085+0000","last_change":"2024-09-17T05:29:43.379049+0000","last_active":"2024-09-17T05:29:43.388085+0000","last_peered":"2024-09-17T05:29:43.388085+0000","last_clean":"2024-09-17T05:29:43.388085+0000","last_became_active":"2024-09-17T05:29:43.378522+0000","last_became_peered":"2024-09-17T05:29:43.378522+0000","last_unstale":"2024-09-17T05:29:43.388085+0000","last_undegraded":"2024-09-17T05:29:43.388085+0000","last_fullsized":"2024-09-17T05:29:43.388085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:55.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.346597+0000","last_change":"2024-09-17T05:29:42.346597+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.346597+0000","last_undegraded":"2024-09-17T05:29:42.346597+0000","last_fullsized":"2024-09-17T05:29:42.346597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.355915+0000","last_change":"2024-09-17T05:29:43.353863+0000","last_active":"2024-09-17T05:29:43.355915+0000","last_peered":"2024-09-17T05:29:43.355915+0000","last_clean":"2024-09-17T05:29:43.355915+0000","last_became_active":"2024-09-17T05:29:43.353478+0000","last_became_peered":"2024-09-17T05:29:43.353478+0000","last_unstale":"2024-09-17T05:29:43.355915+0000","last_undegraded":"2024-09-17T05:29:43.355915+0000","last_fullsized":"2024-09-17T05:29:43.355915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.086569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387807+0000","last_change":"2024-09-17T05:29:43.376248+0000","last_active":"2024-09-17T05:29:43.387807+0000","last_peered":"2024-09-17T05:29:43.387807+0000","last_clean":"2024-09-17T05:29:43.387807+0000","last_became_active":"2024-09-17T05:29:43.375816+0000","last_became_peered":"2024-09-17T05:29:43.375816+0000","last_unstale":"2024-09-17T05:29:43.387807+0000","last_undegraded":"2024-09-17T05:29:43.387807+0000","last_fullsized":"2024-09-17T05:29:43.387807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:46.489949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383174+0000","last_change":"2024-09-17T05:29:43.373770+0000","last_active":"2024-09-17T05:29:43.383174+0000","last_peered":"2024-09-17T05:29:43.383174+0000","last_clean":"2024-09-17T05:29:43.383174+0000","last_became_active":"2024-09-17T05:29:43.368199+0000","last_became_peered":"2024-09-17T05:29:43.368199+0000","last_unstale":"2024-09-17T05:29:43.383174+0000","last_undegraded":"2024-09-17T05:29:43.383174+0000","last_fullsized":"2024-09-17T05:29:43.383174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:41.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385833+0000","last_change":"2024-09-17T05:29:43.379355+0000","last_active":"2024-09-17T05:29:43.385833+0000","last_peered":"2024-09-17T05:29:43.385833+0000","last_clean":"2024-09-17T05:29:43.385833+0000","last_became_active":"2024-09-17T05:29:43.378938+0000","last_became_peered":"2024-09-17T05:29:43.378938+0000","last_unstale":"2024-09-17T05:29:43.385833+0000","last_undegraded":"2024-09-17T05:29:43.385833+0000","last_fullsized":"2024-09-17T05:29:43.385833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:06.632809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386653+0000","last_change":"2024-09-17T05:29:43.381429+0000","last_active":"2024-09-17T05:29:43.386653+0000","last_peered":"2024-09-17T05:29:43.386653+0000","last_clean":"2024-09-17T05:29:43.386653+0000","last_became_active":"2024-09-17T05:29:43.381134+0000","last_became_peered":"2024-09-17T05:29:43.381134+0000","last_unstale":"2024-09-17T05:29:43.386653+0000","last_undegraded":"2024-09-17T05:29:43.386653+0000","last_fullsized":"2024-09-17T05:29:43.386653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:41.235036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.360050+0000","last_change":"2024-09-17T05:29:43.354396+0000","last_active":"2024-09-17T05:29:43.360050+0000","last_peered":"2024-09-17T05:29:43.360050+0000","last_clean":"2024-09-17T05:29:43.360050+0000","last_became_active":"2024-09-17T05:29:43.354069+0000","last_became_peered":"2024-09-17T05:29:43.354069+0000","last_unstale":"2024-09-17T05:29:43.360050+0000","last_undegraded":"2024-09-17T05:29:43.360050+0000","last_fullsized":"2024-09-17T05:29:43.360050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:45.249185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.347053+0000","last_change":"2024-09-17T05:29:42.347053+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.347053+0000","last_undegraded":"2024-09-17T05:29:42.347053+0000","last_fullsized":"2024-09-17T05:29:42.347053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382160+0000","last_change":"2024-09-17T05:29:43.367817+0000","last_active":"2024-09-17T05:29:43.382160+0000","last_peered":"2024-09-17T05:29:43.382160+0000","last_clean":"2024-09-17T05:29:43.382160+0000","last_became_active":"2024-09-17T05:29:43.367477+0000","last_became_peered":"2024-09-17T05:29:43.367477+0000","last_unstale":"2024-09-17T05:29:43.382160+0000","last_undegraded":"2024-09-17T05:29:43.382160+0000","last_fullsized":"2024-09-17T05:29:43.382160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:42.445011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382836+0000","last_change":"2024-09-17T05:29:43.369104+0000","last_active":"2024-09-17T05:29:43.382836+0000","last_peered":"2024-09-17T05:29:43.382836+0000","last_clean":"2024-09-17T05:29:43.382836+0000","last_became_active":"2024-09-17T05:29:43.368433+0000","last_became_peered":"2024-09-17T05:29:43.368433+0000","last_unstale":"2024-09-17T05:29:43.382836+0000","last_undegraded":"2024-09-17T05:29:43.382836+0000","last_fullsized":"2024-09-17T05:29:43.382836+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:51.251795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384055+0000","last_change":"2024-09-17T05:29:43.372164+0000","last_active":"2024-09-17T05:29:43.384055+0000","last_peered":"2024-09-17T05:29:43.384055+0000","last_clean":"2024-09-17T05:29:43.384055+0000","last_became_active":"2024-09-17T05:29:43.371861+0000","last_became_peered":"2024-09-17T05:29:43.371861+0000","last_unstale":"2024-09-17T05:29:43.384055+0000","last_undegraded":"2024-09-17T05:29:43.384055+0000","last_fullsized":"2024-09-17T05:29:43.384055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:40.322900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383272+0000","last_change":"2024-09-17T05:29:43.373755+0000","last_active":"2024-09-17T05:29:43.383272+0000","last_peered":"2024-09-17T05:29:43.383272+0000","last_clean":"2024-09-17T05:29:43.383272+0000","last_became_active":"2024-09-17T05:29:43.369195+0000","last_became_peered":"2024-09-17T05:29:43.369195+0000","last_unstale":"2024-09-17T05:29:43.383272+0000","last_undegraded":"2024-09-17T05:29:43.383272+0000","last_fullsized":"2024-09-17T05:29:43.383272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.543278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.361825+0000","last_change":"2024-09-17T05:29:43.355656+0000","last_active":"2024-09-17T05:29:43.361825+0000","last_peered":"2024-09-17T05:29:43.361825+0000","last_clean":"2024-09-17T05:29:43.361825+0000","last_became_active":"2024-09-17T05:29:43.353985+0000","last_became_peered":"2024-09-17T05:29:43.353985+0000","last_unstale":"2024-09-17T05:29:43.361825+0000","last_undegraded":"2024-09-17T05:29:43.361825+0000","last_fullsized":"2024-09-17T05:29:43.361825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:06.497620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.347281+0000","last_change":"2024-09-17T05:29:42.347281+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.347281+0000","last_undegraded":"2024-09-17T05:29:42.347281+0000","last_fullsized":"2024-09-17T05:29:42.347281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.347061+0000","last_change":"2024-09-17T05:29:42.347061+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.347061+0000","last_undegraded":"2024-09-17T05:29:42.347061+0000","last_fullsized":"2024-09-17T05:29:42.347061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.381768+0000","last_change":"2024-09-17T05:29:43.370799+0000","last_active":"2024-09-17T05:29:43.381768+0000","last_peered":"2024-09-17T05:29:43.381768+0000","last_clean":"2024-09-17T05:29:43.381768+0000","last_became_active":"2024-09-17T05:29:43.370466+0000","last_became_peered":"2024-09-17T05:29:43.370466+0000","last_unstale":"2024-09-17T05:29:43.381768+0000","last_undegraded":"2024-09-17T05:29:43.381768+0000","last_fullsized":"2024-09-17T05:29:43.381768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:30.649792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386760+0000","last_change":"2024-09-17T05:29:43.371526+0000","last_active":"2024-09-17T05:29:43.386760+0000","last_peered":"2024-09-17T05:29:43.386760+0000","last_clean":"2024-09-17T05:29:43.386760+0000","last_became_active":"2024-09-17T05:29:43.371182+0000","last_became_peered":"2024-09-17T05:29:43.371182+0000","last_unstale":"2024-09-17T05:29:43.386760+0000","last_undegraded":"2024-09-17T05:29:43.386760+0000","last_fullsized":"2024-09-17T05:29:43.386760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:58.739001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387429+0000","last_change":"2024-09-17T05:29:43.382315+0000","last_active":"2024-09-17T05:29:43.387429+0000","last_peered":"2024-09-17T05:29:43.387429+0000","last_clean":"2024-09-17T05:29:43.387429+0000","last_became_active":"2024-09-17T05:29:43.381954+0000","last_became_peered":"2024-09-17T05:29:43.381954+0000","last_unstale":"2024-09-17T05:29:43.387429+0000","last_undegraded":"2024-09-17T05:29:43.387429+0000","last_fullsized":"2024-09-17T05:29:43.387429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:09.124150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.371828+0000","last_change":"2024-09-17T05:29:43.365278+0000","last_active":"2024-09-17T05:29:43.371828+0000","last_peered":"2024-09-17T05:29:43.371828+0000","last_clean":"2024-09-17T05:29:43.371828+0000","last_became_active":"2024-09-17T05:29:43.364900+0000","last_became_peered":"2024-09-17T05:29:43.364900+0000","last_unstale":"2024-09-17T05:29:43.371828+0000","last_undegraded":"2024-09-17T05:29:43.371828+0000","last_fullsized":"2024-09-17T05:29:43.371828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:42.879521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382921+0000","last_change":"2024-09-17T05:29:43.372074+0000","last_active":"2024-09-17T05:29:43.382921+0000","last_peered":"2024-09-17T05:29:43.382921+0000","last_clean":"2024-09-17T05:29:43.382921+0000","last_became_active":"2024-09-17T05:29:43.368715+0000","last_became_peered":"2024-09-17T05:29:43.368715+0000","last_unstale":"2024-09-17T05:29:43.382921+0000","last_undegraded":"2024-09-17T05:29:43.382921+0000","last_fullsized":"2024-09-17T05:29:43.382921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:27.875334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394488+0000","last_change":"2024-09-17T05:29:43.392195+0000","last_active":"2024-09-17T05:29:43.394488+0000","last_peered":"2024-09-17T05:29:43.394488+0000","last_clean":"2024-09-17T05:29:43.394488+0000","last_became_active":"2024-09-17T05:29:43.391041+0000","last_became_peered":"2024-09-17T05:29:43.391041+0000","last_unstale":"2024-09-17T05:29:43.394488+0000","last_undegraded":"2024-09-17T05:29:43.394488+0000","last_fullsized":"2024-09-17T05:29:43.394488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:28:35.724566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394303+0000","last_change":"2024-09-17T05:29:43.391354+0000","last_active":"2024-09-17T05:29:43.394303+0000","last_peered":"2024-09-17T05:29:43.394303+0000","last_clean":"2024-09-17T05:29:43.394303+0000","last_became_active":"2024-09-17T05:29:43.391031+0000","last_became_peered":"2024-09-17T05:29:43.391031+0000","last_unstale":"2024-09-17T05:29:43.394303+0000","last_undegraded":"2024-09-17T05:29:43.394303+0000","last_fullsized":"2024-09-17T05:29:43.394303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:23.257421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377383+0000","last_change":"2024-09-17T05:29:43.366081+0000","last_active":"2024-09-17T05:29:43.377383+0000","last_peered":"2024-09-17T05:29:43.377383+0000","last_clean":"2024-09-17T05:29:43.377383+0000","last_became_active":"2024-09-17T05:29:43.365730+0000","last_became_peered":"2024-09-17T05:29:43.365730+0000","last_unstale":"2024-09-17T05:29:43.377383+0000","last_undegraded":"2024-09-17T05:29:43.377383+0000","last_fullsized":"2024-09-17T05:29:43.377383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:58.855247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388345+0000","last_change":"2024-09-17T05:29:42.368042+0000","last_active":"2024-09-17T05:29:43.388345+0000","last_peered":"2024-09-17T05:29:43.388345+0000","last_clean":"2024-09-17T05:29:43.388345+0000","last_became_active":"2024-09-17T05:29:42.367686+0000","last_became_peered":"2024-09-17T05:29:42.367686+0000","last_unstale":"2024-09-17T05:29:43.388345+0000","last_undegraded":"2024-09-17T05:29:43.388345+0000","last_fullsized":"2024-09-17T05:29:43.388345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:17.341630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387478+0000","last_change":"2024-09-17T05:29:43.382320+0000","last_active":"2024-09-17T05:29:43.387478+0000","last_peered":"2024-09-17T05:29:43.387478+0000","last_clean":"2024-09-17T05:29:43.387478+0000","last_became_active":"2024-09-17T05:29:43.380275+0000","last_became_peered":"2024-09-17T05:29:43.380275+0000","last_unstale":"2024-09-17T05:29:43.387478+0000","last_undegraded":"2024-09-17T05:29:43.387478+0000","last_fullsized":"2024-09-17T05:29:43.387478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:31.630199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387946+0000","last_change":"2024-09-17T05:29:42.366557+0000","last_active":"2024-09-17T05:29:43.387946+0000","last_peered":"2024-09-17T05:29:43.387946+0000","last_clean":"2024-09-17T05:29:43.387946+0000","last_became_active":"2024-09-17T05:29:42.364903+0000","last_became_peered":"2024-09-17T05:29:42.364903+0000","last_unstale":"2024-09-17T05:29:43.387946+0000","last_undegraded":"2024-09-17T05:29:43.387946+0000","last_fullsized":"2024-09-17T05:29:43.387946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:08.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378239+0000","last_change":"2024-09-17T05:29:43.372852+0000","last_active":"2024-09-17T05:29:43.378239+0000","last_peered":"2024-09-17T05:29:43.378239+0000","last_clean":"2024-09-17T05:29:43.378239+0000","last_became_active":"2024-09-17T05:29:43.372522+0000","last_became_peered":"2024-09-17T05:29:43.372522+0000","last_unstale":"2024-09-17T05:29:43.378239+0000","last_undegraded":"2024-09-17T05:29:43.378239+0000","last_fullsized":"2024-09-17T05:29:43.378239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:48.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:29:42.374379+0000","last_change":"2024-09-17T05:29:42.370576+0000","last_active":"2024-09-17T05:29:42.374379+0000","last_peered":"2024-09-17T05:29:42.374379+0000","last_clean":"2024-09-17T05:29:42.374379+0000","last_became_active":"2024-09-17T05:29:42.369890+0000","last_became_peered":"2024-09-17T05:29:42.369890+0000","last_unstale":"2024-09-17T05:29:42.374379+0000","last_undegraded":"2024-09-17T05:29:42.374379+0000","last_fullsized":"2024-09-17T05:29:42.374379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:00.491152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391762+0000","last_change":"2024-09-17T05:29:43.388104+0000","last_active":"2024-09-17T05:29:43.391762+0000","last_peered":"2024-09-17T05:29:43.391762+0000","last_clean":"2024-09-17T05:29:43.391762+0000","last_became_active":"2024-09-17T05:29:43.387740+0000","last_became_peered":"2024-09-17T05:29:43.387740+0000","last_unstale":"2024-09-17T05:29:43.391762+0000","last_undegraded":"2024-09-17T05:29:43.391762+0000","last_fullsized":"2024-09-17T05:29:43.391762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:24.008001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394399+0000","last_change":"2024-09-17T05:29:42.368993+0000","last_active":"2024-09-17T05:29:43.394399+0000","last_peered":"2024-09-17T05:29:43.394399+0000","last_clean":"2024-09-17T05:29:43.394399+0000","last_became_active":"2024-09-17T05:29:42.368510+0000","last_became_peered":"2024-09-17T05:29:42.368510+0000","last_unstale":"2024-09-17T05:29:43.394399+0000","last_undegraded":"2024-09-17T05:29:43.394399+0000","last_fullsized":"2024-09-17T05:29:43.394399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:53.140678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384095+0000","last_change":"2024-09-17T05:29:43.375002+0000","last_active":"2024-09-17T05:29:43.384095+0000","last_peered":"2024-09-17T05:29:43.384095+0000","last_clean":"2024-09-17T05:29:43.384095+0000","last_became_active":"2024-09-17T05:29:43.374572+0000","last_became_peered":"2024-09-17T05:29:43.374572+0000","last_unstale":"2024-09-17T05:29:43.384095+0000","last_undegraded":"2024-09-17T05:29:43.384095+0000","last_fullsized":"2024-09-17T05:29:43.384095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:34:27.178663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388853+0000","last_change":"2024-09-17T05:29:42.374052+0000","last_active":"2024-09-17T05:29:43.388853+0000","last_peered":"2024-09-17T05:29:43.388853+0000","last_clean":"2024-09-17T05:29:43.388853+0000","last_became_active":"2024-09-17T05:29:42.373481+0000","last_became_peered":"2024-09-17T05:29:42.373481+0000","last_unstale":"2024-09-17T05:29:43.388853+0000","last_undegraded":"2024-09-17T05:29:43.388853+0000","last_fullsized":"2024-09-17T05:29:43.388853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:52.246748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386756+0000","last_change":"2024-09-17T05:29:43.383822+0000","last_active":"2024-09-17T05:29:43.386756+0000","last_peered":"2024-09-17T05:29:43.386756+0000","last_clean":"2024-09-17T05:29:43.386756+0000","last_became_active":"2024-09-17T05:29:43.383455+0000","last_became_peered":"2024-09-17T05:29:43.383455+0000","last_unstale":"2024-09-17T05:29:43.386756+0000","last_undegraded":"2024-09-17T05:29:43.386756+0000","last_fullsized":"2024-09-17T05:29:43.386756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.761213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.390966+0000","last_change":"2024-09-17T05:29:42.358478+0000","last_active":"2024-09-17T05:29:43.390966+0000","last_peered":"2024-09-17T05:29:43.390966+0000","last_clean":"2024-09-17T05:29:43.390966+0000","last_became_active":"2024-09-17T05:29:42.358158+0000","last_became_peered":"2024-09-17T05:29:42.358158+0000","last_unstale":"2024-09-17T05:29:43.390966+0000","last_undegraded":"2024-09-17T05:29:43.390966+0000","last_fullsized":"2024-09-17T05:29:43.390966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:06.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383394+0000","last_change":"2024-09-17T05:29:43.378747+0000","last_active":"2024-09-17T05:29:43.383394+0000","last_peered":"2024-09-17T05:29:43.383394+0000","last_clean":"2024-09-17T05:29:43.383394+0000","last_became_active":"2024-09-17T05:29:43.376528+0000","last_became_peered":"2024-09-17T05:29:43.376528+0000","last_unstale":"2024-09-17T05:29:43.383394+0000","last_undegraded":"2024-09-17T05:29:43.383394+0000","last_fullsized":"2024-09-17T05:29:43.383394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:58.007315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348599+0000","last_change":"2024-09-17T05:29:42.371911+0000","last_active":"2024-09-17T05:29:43.348599+0000","last_peered":"2024-09-17T05:29:43.348599+0000","last_clean":"2024-09-17T05:29:43.348599+0000","last_became_active":"2024-09-17T05:29:42.371309+0000","last_became_peered":"2024-09-17T05:29:42.371309+0000","last_unstale":"2024-09-17T05:29:43.348599+0000","last_undegraded":"2024-09-17T05:29:43.348599+0000","last_fullsized":"2024-09-17T05:29:43.348599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:33.482665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382080+0000","last_change":"2024-09-17T05:29:43.365063+0000","last_active":"2024-09-17T05:29:43.382080+0000","last_peered":"2024-09-17T05:29:43.382080+0000","last_clean":"2024-09-17T05:29:43.382080+0000","last_became_active":"2024-09-17T05:29:43.364700+0000","last_became_peered":"2024-09-17T05:29:43.364700+0000","last_unstale":"2024-09-17T05:29:43.382080+0000","last_undegraded":"2024-09-17T05:29:43.382080+0000","last_fullsized":"2024-09-17T05:29:43.382080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:02.220275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348726+0000","last_change":"2024-09-17T05:29:42.369258+0000","last_active":"2024-09-17T05:29:43.348726+0000","last_peered":"2024-09-17T05:29:43.348726+0000","last_clean":"2024-09-17T05:29:43.348726+0000","last_became_active":"2024-09-17T05:29:42.368915+0000","last_became_peered":"2024-09-17T05:29:42.368915+0000","last_unstale":"2024-09-17T05:29:43.348726+0000","last_undegraded":"2024-09-17T05:29:43.348726+0000","last_fullsized":"2024-09-17T05:29:43.348726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:26.652183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384642+0000","last_change":"2024-09-17T05:29:43.382401+0000","last_active":"2024-09-17T05:29:43.384642+0000","last_peered":"2024-09-17T05:29:43.384642+0000","last_clean":"2024-09-17T05:29:43.384642+0000","last_became_active":"2024-09-17T05:29:43.381801+0000","last_became_peered":"2024-09-17T05:29:43.381801+0000","last_unstale":"2024-09-17T05:29:43.384642+0000","last_undegraded":"2024-09-17T05:29:43.384642+0000","last_fullsized":"2024-09-17T05:29:43.384642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:31.855952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.364901+0000","last_change":"2024-09-17T05:29:42.371740+0000","last_active":"2024-09-17T05:29:43.364901+0000","last_peered":"2024-09-17T05:29:43.364901+0000","last_clean":"2024-09-17T05:29:43.364901+0000","last_became_active":"2024-09-17T05:29:42.371404+0000","last_became_peered":"2024-09-17T05:29:42.371404+0000","last_unstale":"2024-09-17T05:29:43.364901+0000","last_undegraded":"2024-09-17T05:29:43.364901+0000","last_fullsized":"2024-09-17T05:29:43.364901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:19.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391940+0000","last_change":"2024-09-17T05:29:43.387612+0000","last_active":"2024-09-17T05:29:43.391940+0000","last_peered":"2024-09-17T05:29:43.391940+0000","last_clean":"2024-09-17T05:29:43.391940+0000","last_became_active":"2024-09-17T05:29:43.387272+0000","last_became_peered":"2024-09-17T05:29:43.387272+0000","last_unstale":"2024-09-17T05:29:43.391940+0000","last_undegraded":"2024-09-17T05:29:43.391940+0000","last_fullsized":"2024-09-17T05:29:43.391940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:13.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.373898+0000","last_change":"2024-09-17T05:29:42.357065+0000","last_active":"2024-09-17T05:29:43.373898+0000","last_peered":"2024-09-17T05:29:43.373898+0000","last_clean":"2024-09-17T05:29:43.373898+0000","last_became_active":"2024-09-17T05:29:42.356652+0000","last_became_peered":"2024-09-17T05:29:42.356652+0000","last_unstale":"2024-09-17T05:29:43.373898+0000","last_undegraded":"2024-09-17T05:29:43.373898+0000","last_fullsized":"2024-09-17T05:29:43.373898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:36.558975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.349731+0000","last_change":"2024-09-17T05:29:42.349731+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.349731+0000","last_undegraded":"2024-09-17T05:29:42.349731+0000","last_fullsized":"2024-09-17T05:29:42.349731+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387623+0000","last_change":"2024-09-17T05:29:42.369472+0000","last_active":"2024-09-17T05:29:43.387623+0000","last_peered":"2024-09-17T05:29:43.387623+0000","last_clean":"2024-09-17T05:29:43.387623+0000","last_became_active":"2024-09-17T05:29:42.369153+0000","last_became_peered":"2024-09-17T05:29:42.369153+0000","last_unstale":"2024-09-17T05:29:43.387623+0000","last_undegraded":"2024-09-17T05:29:43.387623+0000","last_fullsized":"2024-09-17T05:29:43.387623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:43.126915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377485+0000","last_change":"2024-09-17T05:29:43.368486+0000","last_active":"2024-09-17T05:29:43.377485+0000","last_peered":"2024-09-17T05:29:43.377485+0000","last_clean":"2024-09-17T05:29:43.377485+0000","last_became_active":"2024-09-17T05:29:43.368116+0000","last_became_peered":"2024-09-17T05:29:43.368116+0000","last_unstale":"2024-09-17T05:29:43.377485+0000","last_undegraded":"2024-09-17T05:29:43.377485+0000","last_fullsized":"2024-09-17T05:29:43.377485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:02.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.392999+0000","last_change":"2024-09-17T05:29:42.359617+0000","last_active":"2024-09-17T05:29:43.392999+0000","last_peered":"2024-09-17T05:29:43.392999+0000","last_clean":"2024-09-17T05:29:43.392999+0000","last_became_active":"2024-09-17T05:29:42.359297+0000","last_became_peered":"2024-09-17T05:29:42.359297+0000","last_unstale":"2024-09-17T05:29:43.392999+0000","last_undegraded":"2024-09-17T05:29:43.392999+0000","last_fullsized":"2024-09-17T05:29:43.392999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:15.177568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.346462+0000","last_change":"2024-09-17T05:29:42.346462+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.346462+0000","last_undegraded":"2024-09-17T05:29:42.346462+0000","last_fullsized":"2024-09-17T05:29:42.346462+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.389979+0000","last_change":"2024-09-17T05:29:42.362233+0000","last_active":"2024-09-17T05:29:43.389979+0000","last_peered":"2024-09-17T05:29:43.389979+0000","last_clean":"2024-09-17T05:29:43.389979+0000","last_became_active":"2024-09-17T05:29:42.361884+0000","last_became_peered":"2024-09-17T05:29:42.361884+0000","last_unstale":"2024-09-17T05:29:43.389979+0000","last_undegraded":"2024-09-17T05:29:43.389979+0000","last_fullsized":"2024-09-17T05:29:43.389979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:36.467901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391802+0000","last_change":"2024-09-17T05:29:43.386438+0000","last_active":"2024-09-17T05:29:43.391802+0000","last_peered":"2024-09-17T05:29:43.391802+0000","last_clean":"2024-09-17T05:29:43.391802+0000","last_became_active":"2024-09-17T05:29:43.380150+0000","last_became_peered":"2024-09-17T05:29:43.380150+0000","last_unstale":"2024-09-17T05:29:43.391802+0000","last_undegraded":"2024-09-17T05:29:43.391802+0000","last_fullsized":"2024-09-17T05:29:43.391802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:30.318924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.363671+0000","last_change":"2024-09-17T05:29:42.377313+0000","last_active":"2024-09-17T05:29:43.363671+0000","last_peered":"2024-09-17T05:29:43.363671+0000","last_clean":"2024-09-17T05:29:43.363671+0000","last_became_active":"2024-09-17T05:29:42.376905+0000","last_became_peered":"2024-09-17T05:29:42.376905+0000","last_unstale":"2024-09-17T05:29:43.363671+0000","last_undegraded":"2024-09-17T05:29:43.363671+0000","last_fullsized":"2024-09-17T05:29:43.363671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:30.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383048+0000","last_change":"2024-09-17T05:29:43.371130+0000","last_active":"2024-09-17T05:29:43.383048+0000","last_peered":"2024-09-17T05:29:43.383048+0000","last_clean":"2024-09-17T05:29:43.383048+0000","last_became_active":"2024-09-17T05:29:43.370337+0000","last_became_peered":"2024-09-17T05:29:43.370337+0000","last_unstale":"2024-09-17T05:29:43.383048+0000","last_undegraded":"2024-09-17T05:29:43.383048+0000","last_fullsized":"2024-09-17T05:29:43.383048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:09.124150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.366403+0000","last_change":"2024-09-17T05:29:42.361941+0000","last_active":"2024-09-17T05:29:43.366403+0000","last_peered":"2024-09-17T05:29:43.366403+0000","last_clean":"2024-09-17T05:29:43.366403+0000","last_became_active":"2024-09-17T05:29:42.361629+0000","last_became_peered":"2024-09-17T05:29:42.361629+0000","last_unstale":"2024-09-17T05:29:43.366403+0000","last_undegraded":"2024-09-17T05:29:43.366403+0000","last_fullsized":"2024-09-17T05:29:43.366403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:20.753316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.346482+0000","last_change":"2024-09-17T05:29:42.346482+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.346482+0000","last_undegraded":"2024-09-17T05:29:42.346482+0000","last_fullsized":"2024-09-17T05:29:42.346482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:29:42.366680+0000","last_change":"2024-09-17T05:29:42.362907+0000","last_active":"2024-09-17T05:29:42.366680+0000","last_peered":"2024-09-17T05:29:42.366680+0000","last_clean":"2024-09-17T05:29:42.366680+0000","last_became_active":"2024-09-17T05:29:42.362473+0000","last_became_peered":"2024-09-17T05:29:42.362473+0000","last_unstale":"2024-09-17T05:29:42.366680+0000","last_undegraded":"2024-09-17T05:29:42.366680+0000","last_fullsized":"2024-09-17T05:29:42.366680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:16.901148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385356+0000","last_change":"2024-09-17T05:29:43.381561+0000","last_active":"2024-09-17T05:29:43.385356+0000","last_peered":"2024-09-17T05:29:43.385356+0000","last_clean":"2024-09-17T05:29:43.385356+0000","last_became_active":"2024-09-17T05:29:43.381180+0000","last_became_peered":"2024-09-17T05:29:43.381180+0000","last_unstale":"2024-09-17T05:29:43.385356+0000","last_undegraded":"2024-09-17T05:29:43.385356+0000","last_fullsized":"2024-09-17T05:29:43.385356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:45.535653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386259+0000","last_change":"2024-09-17T05:29:42.368851+0000","last_active":"2024-09-17T05:29:43.386259+0000","last_peered":"2024-09-17T05:29:43.386259+0000","last_clean":"2024-09-17T05:29:43.386259+0000","last_became_active":"2024-09-17T05:29:42.368562+0000","last_became_peered":"2024-09-17T05:29:42.368562+0000","last_unstale":"2024-09-17T05:29:43.386259+0000","last_undegraded":"2024-09-17T05:29:43.386259+0000","last_fullsized":"2024-09-17T05:29:43.386259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:02.347558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394051+0000","last_change":"2024-09-17T05:29:43.390055+0000","last_active":"2024-09-17T05:29:43.394051+0000","last_peered":"2024-09-17T05:29:43.394051+0000","last_clean":"2024-09-17T05:29:43.394051+0000","last_became_active":"2024-09-17T05:29:43.389143+0000","last_became_peered":"2024-09-17T05:29:43.389143+0000","last_unstale":"2024-09-17T05:29:43.394051+0000","last_undegraded":"2024-09-17T05:29:43.394051+0000","last_fullsized":"2024-09-17T05:29:43.394051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:45:34.584862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391159+0000","last_change":"2024-09-17T05:29:42.376726+0000","last_active":"2024-09-17T05:29:43.391159+0000","last_peered":"2024-09-17T05:29:43.391159+0000","last_clean":"2024-09-17T05:29:43.391159+0000","last_became_active":"2024-09-17T05:29:42.376389+0000","last_became_peered":"2024-09-17T05:29:42.376389+0000","last_unstale":"2024-09-17T05:29:43.391159+0000","last_undegraded":"2024-09-17T05:29:43.391159+0000","last_fullsized":"2024-09-17T05:29:43.391159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:57.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.381647+0000","last_change":"2024-09-17T05:29:43.379855+0000","last_active":"2024-09-17T05:29:43.381647+0000","last_peered":"2024-09-17T05:29:43.381647+0000","last_clean":"2024-09-17T05:29:43.381647+0000","last_became_active":"2024-09-17T05:29:43.379502+0000","last_became_peered":"2024-09-17T05:29:43.379502+0000","last_unstale":"2024-09-17T05:29:43.381647+0000","last_undegraded":"2024-09-17T05:29:43.381647+0000","last_fullsized":"2024-09-17T05:29:43.381647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:28.267802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.349414+0000","last_change":"2024-09-17T05:29:42.367174+0000","last_active":"2024-09-17T05:29:43.349414+0000","last_peered":"2024-09-17T05:29:43.349414+0000","last_clean":"2024-09-17T05:29:43.349414+0000","last_became_active":"2024-09-17T05:29:42.366821+0000","last_became_peered":"2024-09-17T05:29:42.366821+0000","last_unstale":"2024-09-17T05:29:43.349414+0000","last_undegraded":"2024-09-17T05:29:43.349414+0000","last_fullsized":"2024-09-17T05:29:43.349414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:08.639903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378130+0000","last_change":"2024-09-17T05:29:43.374767+0000","last_active":"2024-09-17T05:29:43.378130+0000","last_peered":"2024-09-17T05:29:43.378130+0000","last_clean":"2024-09-17T05:29:43.378130+0000","last_became_active":"2024-09-17T05:29:43.374406+0000","last_became_peered":"2024-09-17T05:29:43.374406+0000","last_unstale":"2024-09-17T05:29:43.378130+0000","last_undegraded":"2024-09-17T05:29:43.378130+0000","last_fullsized":"2024-09-17T05:29:43.378130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:06.073333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386309+0000","last_change":"2024-09-17T05:29:42.366434+0000","last_active":"2024-09-17T05:29:43.386309+0000","last_peered":"2024-09-17T05:29:43.386309+0000","last_clean":"2024-09-17T05:29:43.386309+0000","last_became_active":"2024-09-17T05:29:42.365972+0000","last_became_peered":"2024-09-17T05:29:42.365972+0000","last_unstale":"2024-09-17T05:29:43.386309+0000","last_undegraded":"2024-09-17T05:29:43.386309+0000","last_fullsized":"2024-09-17T05:29:43.386309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:53.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387775+0000","last_change":"2024-09-17T05:29:43.372992+0000","last_active":"2024-09-17T05:29:43.387775+0000","last_peered":"2024-09-17T05:29:43.387775+0000","last_clean":"2024-09-17T05:29:43.387775+0000","last_became_active":"2024-09-17T05:29:43.372271+0000","last_became_peered":"2024-09-17T05:29:43.372271+0000","last_unstale":"2024-09-17T05:29:43.387775+0000","last_undegraded":"2024-09-17T05:29:43.387775+0000","last_fullsized":"2024-09-17T05:29:43.387775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:15.893767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.365654+0000","last_change":"2024-09-17T05:29:42.368503+0000","last_active":"2024-09-17T05:29:43.365654+0000","last_peered":"2024-09-17T05:29:43.365654+0000","last_clean":"2024-09-17T05:29:43.365654+0000","last_became_active":"2024-09-17T05:29:42.368082+0000","last_became_peered":"2024-09-17T05:29:42.368082+0000","last_unstale":"2024-09-17T05:29:43.365654+0000","last_undegraded":"2024-09-17T05:29:43.365654+0000","last_fullsized":"2024-09-17T05:29:43.365654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:58.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384034+0000","last_change":"2024-09-17T05:29:43.371376+0000","last_active":"2024-09-17T05:29:43.384034+0000","last_peered":"2024-09-17T05:29:43.384034+0000","last_clean":"2024-09-17T05:29:43.384034+0000","last_became_active":"2024-09-17T05:29:43.365092+0000","last_became_peered":"2024-09-17T05:29:43.365092+0000","last_unstale":"2024-09-17T05:29:43.384034+0000","last_undegraded":"2024-09-17T05:29:43.384034+0000","last_fullsized":"2024-09-17T05:29:43.384034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:00.730402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:29:42.359435+0000","last_change":"2024-09-17T05:29:42.353808+0000","last_active":"2024-09-17T05:29:42.359435+0000","last_peered":"2024-09-17T05:29:42.359435+0000","last_clean":"2024-09-17T05:29:42.359435+0000","last_became_active":"2024-09-17T05:29:42.353481+0000","last_became_peered":"2024-09-17T05:29:42.353481+0000","last_unstale":"2024-09-17T05:29:42.359435+0000","last_undegraded":"2024-09-17T05:29:42.359435+0000","last_fullsized":"2024-09-17T05:29:42.359435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:36.791007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.376805+0000","last_change":"2024-09-17T05:29:43.361964+0000","last_active":"2024-09-17T05:29:43.376805+0000","last_peered":"2024-09-17T05:29:43.376805+0000","last_clean":"2024-09-17T05:29:43.376805+0000","last_became_active":"2024-09-17T05:29:43.361619+0000","last_became_peered":"2024-09-17T05:29:43.361619+0000","last_unstale":"2024-09-17T05:29:43.376805+0000","last_undegraded":"2024-09-17T05:29:43.376805+0000","last_fullsized":"2024-09-17T05:29:43.376805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:23.191921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.349021+0000","last_change":"2024-09-17T05:29:42.368459+0000","last_active":"2024-09-17T05:29:43.349021+0000","last_peered":"2024-09-17T05:29:43.349021+0000","last_clean":"2024-09-17T05:29:43.349021+0000","last_became_active":"2024-09-17T05:29:42.367799+0000","last_became_peered":"2024-09-17T05:29:42.367799+0000","last_unstale":"2024-09-17T05:29:43.349021+0000","last_undegraded":"2024-09-17T05:29:43.349021+0000","last_fullsized":"2024-09-17T05:29:43.349021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:38.839620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388718+0000","last_change":"2024-09-17T05:29:43.381905+0000","last_active":"2024-09-17T05:29:43.388718+0000","last_peered":"2024-09-17T05:29:43.388718+0000","last_clean":"2024-09-17T05:29:43.388718+0000","last_became_active":"2024-09-17T05:29:43.381557+0000","last_became_peered":"2024-09-17T05:29:43.381557+0000","last_unstale":"2024-09-17T05:29:43.388718+0000","last_undegraded":"2024-09-17T05:29:43.388718+0000","last_fullsized":"2024-09-17T05:29:43.388718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:30.064954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382922+0000","last_change":"2024-09-17T05:29:42.372935+0000","last_active":"2024-09-17T05:29:43.382922+0000","last_peered":"2024-09-17T05:29:43.382922+0000","last_clean":"2024-09-17T05:29:43.382922+0000","last_became_active":"2024-09-17T05:29:42.372589+0000","last_became_peered":"2024-09-17T05:29:42.372589+0000","last_unstale":"2024-09-17T05:29:43.382922+0000","last_undegraded":"2024-09-17T05:29:43.382922+0000","last_fullsized":"2024-09-17T05:29:43.382922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:47.073986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.375016+0000","last_change":"2024-09-17T05:29:43.357595+0000","last_active":"2024-09-17T05:29:43.375016+0000","last_peered":"2024-09-17T05:29:43.375016+0000","last_clean":"2024-09-17T05:29:43.375016+0000","last_became_active":"2024-09-17T05:29:43.357221+0000","last_became_peered":"2024-09-17T05:29:43.357221+0000","last_unstale":"2024-09-17T05:29:43.375016+0000","last_undegraded":"2024-09-17T05:29:43.375016+0000","last_fullsized":"2024-09-17T05:29:43.375016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:11.628247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:29:42.368037+0000","last_change":"2024-09-17T05:29:42.365183+0000","last_active":"2024-09-17T05:29:42.368037+0000","last_peered":"2024-09-17T05:29:42.368037+0000","last_clean":"2024-09-17T05:29:42.368037+0000","last_became_active":"2024-09-17T05:29:42.364603+0000","last_became_peered":"2024-09-17T05:29:42.364603+0000","last_unstale":"2024-09-17T05:29:42.368037+0000","last_undegraded":"2024-09-17T05:29:42.368037+0000","last_fullsized":"2024-09-17T05:29:42.368037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:55.415931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377631+0000","last_change":"2024-09-17T05:29:43.365326+0000","last_active":"2024-09-17T05:29:43.377631+0000","last_peered":"2024-09-17T05:29:43.377631+0000","last_clean":"2024-09-17T05:29:43.377631+0000","last_became_active":"2024-09-17T05:29:43.364972+0000","last_became_peered":"2024-09-17T05:29:43.364972+0000","last_unstale":"2024-09-17T05:29:43.377631+0000","last_undegraded":"2024-09-17T05:29:43.377631+0000","last_fullsized":"2024-09-17T05:29:43.377631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:51.037863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348168+0000","last_change":"2024-09-17T05:29:42.363604+0000","last_active":"2024-09-17T05:29:43.348168+0000","last_peered":"2024-09-17T05:29:43.348168+0000","last_clean":"2024-09-17T05:29:43.348168+0000","last_became_active":"2024-09-17T05:29:42.363231+0000","last_became_peered":"2024-09-17T05:29:42.363231+0000","last_unstale":"2024-09-17T05:29:43.348168+0000","last_undegraded":"2024-09-17T05:29:43.348168+0000","last_fullsized":"2024-09-17T05:29:43.348168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:55.703847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.389078+0000","last_change":"2024-09-17T05:29:43.382440+0000","last_active":"2024-09-17T05:29:43.389078+0000","last_peered":"2024-09-17T05:29:43.389078+0000","last_clean":"2024-09-17T05:29:43.389078+0000","last_became_active":"2024-09-17T05:29:43.382117+0000","last_became_peered":"2024-09-17T05:29:43.382117+0000","last_unstale":"2024-09-17T05:29:43.389078+0000","last_undegraded":"2024-09-17T05:29:43.389078+0000","last_fullsized":"2024-09-17T05:29:43.389078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:47.246823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.369682+0000","last_change":"2024-09-17T05:29:42.364199+0000","last_active":"2024-09-17T05:29:43.369682+0000","last_peered":"2024-09-17T05:29:43.369682+0000","last_clean":"2024-09-17T05:29:43.369682+0000","last_became_active":"2024-09-17T05:29:42.363852+0000","last_became_peered":"2024-09-17T05:29:42.363852+0000","last_unstale":"2024-09-17T05:29:43.369682+0000","last_undegraded":"2024-09-17T05:29:43.369682+0000","last_fullsized":"2024-09-17T05:29:43.369682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:26.272759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386018+0000","last_change":"2024-09-17T05:29:43.379751+0000","last_active":"2024-09-17T05:29:43.386018+0000","last_peered":"2024-09-17T05:29:43.386018+0000","last_clean":"2024-09-17T05:29:43.386018+0000","last_became_active":"2024-09-17T05:29:43.379324+0000","last_became_peered":"2024-09-17T05:29:43.379324+0000","last_unstale":"2024-09-17T05:29:43.386018+0000","last_undegraded":"2024-09-17T05:29:43.386018+0000","last_fullsized":"2024-09-17T05:29:43.386018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:57.456587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.360013+0000","last_change":"2024-09-17T05:29:42.371061+0000","last_active":"2024-09-17T05:29:43.360013+0000","last_peered":"2024-09-17T05:29:43.360013+0000","last_clean":"2024-09-17T05:29:43.360013+0000","last_became_active":"2024-09-17T05:29:42.369612+0000","last_became_peered":"2024-09-17T05:29:42.369612+0000","last_unstale":"2024-09-17T05:29:43.360013+0000","last_undegraded":"2024-09-17T05:29:43.360013+0000","last_fullsized":"2024-09-17T05:29:43.360013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.005279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.347393+0000","last_change":"2024-09-17T05:29:42.347393+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.347393+0000","last_undegraded":"2024-09-17T05:29:42.347393+0000","last_fullsized":"2024-09-17T05:29:42.347393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.370595+0000","last_change":"2024-09-17T05:29:42.357640+0000","last_active":"2024-09-17T05:29:43.370595+0000","last_peered":"2024-09-17T05:29:43.370595+0000","last_clean":"2024-09-17T05:29:43.370595+0000","last_became_active":"2024-09-17T05:29:42.357340+0000","last_became_peered":"2024-09-17T05:29:42.357340+0000","last_unstale":"2024-09-17T05:29:43.370595+0000","last_undegraded":"2024-09-17T05:29:43.370595+0000","last_fullsized":"2024-09-17T05:29:43.370595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:10.161870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387657+0000","last_change":"2024-09-17T05:29:43.380997+0000","last_active":"2024-09-17T05:29:43.387657+0000","last_peered":"2024-09-17T05:29:43.387657+0000","last_clean":"2024-09-17T05:29:43.387657+0000","last_became_active":"2024-09-17T05:29:43.380703+0000","last_became_peered":"2024-09-17T05:29:43.380703+0000","last_unstale":"2024-09-17T05:29:43.387657+0000","last_undegraded":"2024-09-17T05:29:43.387657+0000","last_fullsized":"2024-09-17T05:29:43.387657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:55.355491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.362795+0000","last_change":"2024-09-17T05:29:42.368428+0000","last_active":"2024-09-17T05:29:43.362795+0000","last_peered":"2024-09-17T05:29:43.362795+0000","last_clean":"2024-09-17T05:29:43.362795+0000","last_became_active":"2024-09-17T05:29:42.368070+0000","last_became_peered":"2024-09-17T05:29:42.368070+0000","last_unstale":"2024-09-17T05:29:43.362795+0000","last_undegraded":"2024-09-17T05:29:43.362795+0000","last_fullsized":"2024-09-17T05:29:43.362795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:16.265304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385849+0000","last_change":"2024-09-17T05:29:43.379397+0000","last_active":"2024-09-17T05:29:43.385849+0000","last_peered":"2024-09-17T05:29:43.385849+0000","last_clean":"2024-09-17T05:29:43.385849+0000","last_became_active":"2024-09-17T05:29:43.376109+0000","last_became_peered":"2024-09-17T05:29:43.376109+0000","last_unstale":"2024-09-17T05:29:43.385849+0000","last_undegraded":"2024-09-17T05:29:43.385849+0000","last_fullsized":"2024-09-17T05:29:43.385849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:53.177047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.347496+0000","last_change":"2024-09-17T05:29:42.368873+0000","last_active":"2024-09-17T05:29:43.347496+0000","last_peered":"2024-09-17T05:29:43.347496+0000","last_clean":"2024-09-17T05:29:43.347496+0000","last_became_active":"2024-09-17T05:29:42.368515+0000","last_became_peered":"2024-09-17T05:29:42.368515+0000","last_unstale":"2024-09-17T05:29:43.347496+0000","last_undegraded":"2024-09-17T05:29:43.347496+0000","last_fullsized":"2024-09-17T05:29:43.347496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:33.573774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:29:42.346658+0000","last_change":"2024-09-17T05:29:42.346658+0000","last_active":"2024-09-17T05:29:42.333338+0000","last_peered":"2024-09-17T05:29:42.333338+0000","last_clean":"2024-09-17T05:29:42.333338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:29:42.346658+0000","last_undegraded":"2024-09-17T05:29:42.346658+0000","last_fullsized":"2024-09-17T05:29:42.346658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386838+0000","last_change":"2024-09-17T05:29:42.366151+0000","last_active":"2024-09-17T05:29:43.386838+0000","last_peered":"2024-09-17T05:29:43.386838+0000","last_clean":"2024-09-17T05:29:43.386838+0000","last_became_active":"2024-09-17T05:29:42.365500+0000","last_became_peered":"2024-09-17T05:29:42.365500+0000","last_unstale":"2024-09-17T05:29:43.386838+0000","last_undegraded":"2024-09-17T05:29:43.386838+0000","last_fullsized":"2024-09-17T05:29:43.386838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:01.020554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":238,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.363207+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:29:43.363207+0000","last_peered":"2024-09-17T05:29:43.363207+0000","last_clean":"2024-09-17T05:29:43.363207+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:29:43.363207+0000","last_undegraded":"2024-09-17T05:29:43.363207+0000","last_fullsized":"2024-09-17T05:29:43.363207+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":454311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1683456,"data_stored":1053100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":461120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":461760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":461086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1671168,"data_stored":1051544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509521,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":454311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":454311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:29:47.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:29:47.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 -- 172.21.15.43:0/2033819011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 msgr2=0x7f50140ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 --2- 172.21.15.43:0/2033819011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f50140ad0b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f500c009bc0 tx=0x7f500c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 -- 172.21.15.43:0/2033819011 shutdown_connections 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 --2- 172.21.15.43:0/2033819011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f50140ad0b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 --2- 172.21.15.43:0/2033819011 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f50140b3af0 0x7f501415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 -- 172.21.15.43:0/2033819011 >> 172.21.15.43:0/2033819011 conn(0x7f50140b23d0 msgr2=0x7f50140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 -- 172.21.15.43:0/2033819011 shutdown_connections 2024-09-17T05:29:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.753+0000 7f5019600640 1 -- 172.21.15.43:0/2033819011 wait complete. 2024-09-17T05:29:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 Processor -- start 2024-09-17T05:29:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- start start 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 0x7f50140cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 0x7f50140cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5014168be0 con 0x7f501406f110 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5014168a60 con 0x7f50140cf3a0 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5014168d60 con 0x7f5014155d70 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59782/0 (socket says 172.21.15.43:59782) 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 -- 172.21.15.43:0/3921783744 learned_addr learned my addr 172.21.15.43:0/3921783744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f500be00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 0x7f50140cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5013400640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 0x7f50140cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 -- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 msgr2=0x7f50140cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 0x7f50140cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 -- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 msgr2=0x7f50140cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 0x7f50140cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 -- 172.21.15.43:0/3921783744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140cfcf0 con 0x7f501406f110 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f500be00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 0x7f50140cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5013400640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 0x7f50140cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5012a00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f500400c400 tx=0x7f500400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004011790 con 0x7f501406f110 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5004017070 con 0x7f501406f110 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004011cc0 con 0x7f501406f110 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5014160d40 con 0x7f501406f110 2024-09-17T05:29:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.757+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5014161250 con 0x7f501406f110 2024-09-17T05:29:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50140ad0b0 con 0x7f501406f110 2024-09-17T05:29:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f500401c460 con 0x7f501406f110 2024-09-17T05:29:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f500aa00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 0x7f4ff0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f5004094110 con 0x7f501406f110 2024-09-17T05:29:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f500be00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 0x7f4ff0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.761+0000 7f500be00640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 0x7f4ff0075710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005d10 tx=0x7f4ffc005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.765+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50140cfcf0 con 0x7f501406f110 2024-09-17T05:29:47.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.941+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f501415e230 con 0x7f4ff0073200 2024-09-17T05:29:47.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.945+0000 7f500aa00640 1 -- 172.21.15.43:0/3921783744 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252068 (secure 0 0 0) 0x7f501415e230 con 0x7f4ff0073200 2024-09-17T05:29:47.951 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:47.953 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 msgr2=0x7f4ff0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 0x7f4ff0075710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005d10 tx=0x7f4ffc005de0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 msgr2=0x7f5014160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f500400c400 tx=0x7f500400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 shutdown_connections 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ff0073200 0x7f4ff0075710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50140cf3a0 0x7f50140cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5014155d70 0x7f50140cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 --2- 172.21.15.43:0/3921783744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f501406f110 0x7f5014160800 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 >> 172.21.15.43:0/3921783744 conn(0x7f50140b23d0 msgr2=0x7f501415e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 shutdown_connections 2024-09-17T05:29:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:47.949+0000 7f5019600640 1 -- 172.21.15.43:0/3921783744 wait complete. 2024-09-17T05:29:47.973 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-17T05:29:47.945533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259096,"kb_used_data":9576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715624,"statfs":{"total":773094113280,"available":772828798976,"internally_reserved":0,"allocated":9805824,"data_stored":4865290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001480"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387862+0000","last_change":"2024-09-17T05:29:43.383927+0000","last_active":"2024-09-17T05:29:43.387862+0000","last_peered":"2024-09-17T05:29:43.387862+0000","last_clean":"2024-09-17T05:29:43.387862+0000","last_became_active":"2024-09-17T05:29:43.383511+0000","last_became_peered":"2024-09-17T05:29:43.383511+0000","last_unstale":"2024-09-17T05:29:43.387862+0000","last_undegraded":"2024-09-17T05:29:43.387862+0000","last_fullsized":"2024-09-17T05:29:43.387862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:41.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385872+0000","last_change":"2024-09-17T05:29:43.379276+0000","last_active":"2024-09-17T05:29:43.385872+0000","last_peered":"2024-09-17T05:29:43.385872+0000","last_clean":"2024-09-17T05:29:43.385872+0000","last_became_active":"2024-09-17T05:29:43.378853+0000","last_became_peered":"2024-09-17T05:29:43.378853+0000","last_unstale":"2024-09-17T05:29:43.385872+0000","last_undegraded":"2024-09-17T05:29:43.385872+0000","last_fullsized":"2024-09-17T05:29:43.385872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:19.202636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394364+0000","last_change":"2024-09-17T05:29:43.392064+0000","last_active":"2024-09-17T05:29:43.394364+0000","last_peered":"2024-09-17T05:29:43.394364+0000","last_clean":"2024-09-17T05:29:43.394364+0000","last_became_active":"2024-09-17T05:29:43.390606+0000","last_became_peered":"2024-09-17T05:29:43.390606+0000","last_unstale":"2024-09-17T05:29:43.394364+0000","last_undegraded":"2024-09-17T05:29:43.394364+0000","last_fullsized":"2024-09-17T05:29:43.394364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:46.931916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385255+0000","last_change":"2024-09-17T05:29:43.382510+0000","last_active":"2024-09-17T05:29:43.385255+0000","last_peered":"2024-09-17T05:29:43.385255+0000","last_clean":"2024-09-17T05:29:43.385255+0000","last_became_active":"2024-09-17T05:29:43.381739+0000","last_became_peered":"2024-09-17T05:29:43.381739+0000","last_unstale":"2024-09-17T05:29:43.385255+0000","last_undegraded":"2024-09-17T05:29:43.385255+0000","last_fullsized":"2024-09-17T05:29:43.385255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:46.669427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.375278+0000","last_change":"2024-09-17T05:29:43.355953+0000","last_active":"2024-09-17T05:29:43.375278+0000","last_peered":"2024-09-17T05:29:43.375278+0000","last_clean":"2024-09-17T05:29:43.375278+0000","last_became_active":"2024-09-17T05:29:43.355604+0000","last_became_peered":"2024-09-17T05:29:43.355604+0000","last_unstale":"2024-09-17T05:29:43.375278+0000","last_undegraded":"2024-09-17T05:29:43.375278+0000","last_fullsized":"2024-09-17T05:29:43.375278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.528747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394510+0000","last_change":"2024-09-17T05:29:43.392245+0000","last_active":"2024-09-17T05:29:43.394510+0000","last_peered":"2024-09-17T05:29:43.394510+0000","last_clean":"2024-09-17T05:29:43.394510+0000","last_became_active":"2024-09-17T05:29:43.391284+0000","last_became_peered":"2024-09-17T05:29:43.391284+0000","last_unstale":"2024-09-17T05:29:43.394510+0000","last_undegraded":"2024-09-17T05:29:43.394510+0000","last_fullsized":"2024-09-17T05:29:43.394510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:06.112953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387159+0000","last_change":"2024-09-17T05:29:43.384304+0000","last_active":"2024-09-17T05:29:43.387159+0000","last_peered":"2024-09-17T05:29:43.387159+0000","last_clean":"2024-09-17T05:29:43.387159+0000","last_became_active":"2024-09-17T05:29:43.383960+0000","last_became_peered":"2024-09-17T05:29:43.383960+0000","last_unstale":"2024-09-17T05:29:43.387159+0000","last_undegraded":"2024-09-17T05:29:43.387159+0000","last_fullsized":"2024-09-17T05:29:43.387159+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:16.431772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378294+0000","last_change":"2024-09-17T05:29:43.369429+0000","last_active":"2024-09-17T05:29:43.378294+0000","last_peered":"2024-09-17T05:29:43.378294+0000","last_clean":"2024-09-17T05:29:43.378294+0000","last_became_active":"2024-09-17T05:29:43.369047+0000","last_became_peered":"2024-09-17T05:29:43.369047+0000","last_unstale":"2024-09-17T05:29:43.378294+0000","last_undegraded":"2024-09-17T05:29:43.378294+0000","last_fullsized":"2024-09-17T05:29:43.378294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:41.813501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387838+0000","last_change":"2024-09-17T05:29:43.382992+0000","last_active":"2024-09-17T05:29:43.387838+0000","last_peered":"2024-09-17T05:29:43.387838+0000","last_clean":"2024-09-17T05:29:43.387838+0000","last_became_active":"2024-09-17T05:29:43.379486+0000","last_became_peered":"2024-09-17T05:29:43.379486+0000","last_unstale":"2024-09-17T05:29:43.387838+0000","last_undegraded":"2024-09-17T05:29:43.387838+0000","last_fullsized":"2024-09-17T05:29:43.387838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:45.318539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388085+0000","last_change":"2024-09-17T05:29:43.379049+0000","last_active":"2024-09-17T05:29:43.388085+0000","last_peered":"2024-09-17T05:29:43.388085+0000","last_clean":"2024-09-17T05:29:43.388085+0000","last_became_active":"2024-09-17T05:29:43.378522+0000","last_became_peered":"2024-09-17T05:29:43.378522+0000","last_unstale":"2024-09-17T05:29:43.388085+0000","last_undegraded":"2024-09-17T05:29:43.388085+0000","last_fullsized":"2024-09-17T05:29:43.388085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:55.563975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219700+0000","last_change":"2024-09-17T05:29:43.377185+0000","last_active":"2024-09-17T05:29:45.219700+0000","last_peered":"2024-09-17T05:29:45.219700+0000","last_clean":"2024-09-17T05:29:45.219700+0000","last_became_active":"2024-09-17T05:29:43.376807+0000","last_became_peered":"2024-09-17T05:29:43.376807+0000","last_unstale":"2024-09-17T05:29:45.219700+0000","last_undegraded":"2024-09-17T05:29:45.219700+0000","last_fullsized":"2024-09-17T05:29:45.219700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:20.297837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.355915+0000","last_change":"2024-09-17T05:29:43.353863+0000","last_active":"2024-09-17T05:29:43.355915+0000","last_peered":"2024-09-17T05:29:43.355915+0000","last_clean":"2024-09-17T05:29:43.355915+0000","last_became_active":"2024-09-17T05:29:43.353478+0000","last_became_peered":"2024-09-17T05:29:43.353478+0000","last_unstale":"2024-09-17T05:29:43.355915+0000","last_undegraded":"2024-09-17T05:29:43.355915+0000","last_fullsized":"2024-09-17T05:29:43.355915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.086569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387807+0000","last_change":"2024-09-17T05:29:43.376248+0000","last_active":"2024-09-17T05:29:43.387807+0000","last_peered":"2024-09-17T05:29:43.387807+0000","last_clean":"2024-09-17T05:29:43.387807+0000","last_became_active":"2024-09-17T05:29:43.375816+0000","last_became_peered":"2024-09-17T05:29:43.375816+0000","last_unstale":"2024-09-17T05:29:43.387807+0000","last_undegraded":"2024-09-17T05:29:43.387807+0000","last_fullsized":"2024-09-17T05:29:43.387807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:46.489949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383174+0000","last_change":"2024-09-17T05:29:43.373770+0000","last_active":"2024-09-17T05:29:43.383174+0000","last_peered":"2024-09-17T05:29:43.383174+0000","last_clean":"2024-09-17T05:29:43.383174+0000","last_became_active":"2024-09-17T05:29:43.368199+0000","last_became_peered":"2024-09-17T05:29:43.368199+0000","last_unstale":"2024-09-17T05:29:43.383174+0000","last_undegraded":"2024-09-17T05:29:43.383174+0000","last_fullsized":"2024-09-17T05:29:43.383174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:41.190744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385833+0000","last_change":"2024-09-17T05:29:43.379355+0000","last_active":"2024-09-17T05:29:43.385833+0000","last_peered":"2024-09-17T05:29:43.385833+0000","last_clean":"2024-09-17T05:29:43.385833+0000","last_became_active":"2024-09-17T05:29:43.378938+0000","last_became_peered":"2024-09-17T05:29:43.378938+0000","last_unstale":"2024-09-17T05:29:43.385833+0000","last_undegraded":"2024-09-17T05:29:43.385833+0000","last_fullsized":"2024-09-17T05:29:43.385833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:06.632809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386653+0000","last_change":"2024-09-17T05:29:43.381429+0000","last_active":"2024-09-17T05:29:43.386653+0000","last_peered":"2024-09-17T05:29:43.386653+0000","last_clean":"2024-09-17T05:29:43.386653+0000","last_became_active":"2024-09-17T05:29:43.381134+0000","last_became_peered":"2024-09-17T05:29:43.381134+0000","last_unstale":"2024-09-17T05:29:43.386653+0000","last_undegraded":"2024-09-17T05:29:43.386653+0000","last_fullsized":"2024-09-17T05:29:43.386653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:41.235036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.360050+0000","last_change":"2024-09-17T05:29:43.354396+0000","last_active":"2024-09-17T05:29:43.360050+0000","last_peered":"2024-09-17T05:29:43.360050+0000","last_clean":"2024-09-17T05:29:43.360050+0000","last_became_active":"2024-09-17T05:29:43.354069+0000","last_became_peered":"2024-09-17T05:29:43.354069+0000","last_unstale":"2024-09-17T05:29:43.360050+0000","last_undegraded":"2024-09-17T05:29:43.360050+0000","last_fullsized":"2024-09-17T05:29:43.360050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:45.249185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.218741+0000","last_change":"2024-09-17T05:29:43.370736+0000","last_active":"2024-09-17T05:29:45.218741+0000","last_peered":"2024-09-17T05:29:45.218741+0000","last_clean":"2024-09-17T05:29:45.218741+0000","last_became_active":"2024-09-17T05:29:43.370358+0000","last_became_peered":"2024-09-17T05:29:43.370358+0000","last_unstale":"2024-09-17T05:29:45.218741+0000","last_undegraded":"2024-09-17T05:29:45.218741+0000","last_fullsized":"2024-09-17T05:29:45.218741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:46.670874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382160+0000","last_change":"2024-09-17T05:29:43.367817+0000","last_active":"2024-09-17T05:29:43.382160+0000","last_peered":"2024-09-17T05:29:43.382160+0000","last_clean":"2024-09-17T05:29:43.382160+0000","last_became_active":"2024-09-17T05:29:43.367477+0000","last_became_peered":"2024-09-17T05:29:43.367477+0000","last_unstale":"2024-09-17T05:29:43.382160+0000","last_undegraded":"2024-09-17T05:29:43.382160+0000","last_fullsized":"2024-09-17T05:29:43.382160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:42.445011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382836+0000","last_change":"2024-09-17T05:29:43.369104+0000","last_active":"2024-09-17T05:29:43.382836+0000","last_peered":"2024-09-17T05:29:43.382836+0000","last_clean":"2024-09-17T05:29:43.382836+0000","last_became_active":"2024-09-17T05:29:43.368433+0000","last_became_peered":"2024-09-17T05:29:43.368433+0000","last_unstale":"2024-09-17T05:29:43.382836+0000","last_undegraded":"2024-09-17T05:29:43.382836+0000","last_fullsized":"2024-09-17T05:29:43.382836+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:51.251795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384055+0000","last_change":"2024-09-17T05:29:43.372164+0000","last_active":"2024-09-17T05:29:43.384055+0000","last_peered":"2024-09-17T05:29:43.384055+0000","last_clean":"2024-09-17T05:29:43.384055+0000","last_became_active":"2024-09-17T05:29:43.371861+0000","last_became_peered":"2024-09-17T05:29:43.371861+0000","last_unstale":"2024-09-17T05:29:43.384055+0000","last_undegraded":"2024-09-17T05:29:43.384055+0000","last_fullsized":"2024-09-17T05:29:43.384055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:40.322900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383272+0000","last_change":"2024-09-17T05:29:43.373755+0000","last_active":"2024-09-17T05:29:43.383272+0000","last_peered":"2024-09-17T05:29:43.383272+0000","last_clean":"2024-09-17T05:29:43.383272+0000","last_became_active":"2024-09-17T05:29:43.369195+0000","last_became_peered":"2024-09-17T05:29:43.369195+0000","last_unstale":"2024-09-17T05:29:43.383272+0000","last_undegraded":"2024-09-17T05:29:43.383272+0000","last_fullsized":"2024-09-17T05:29:43.383272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.543278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.361825+0000","last_change":"2024-09-17T05:29:43.355656+0000","last_active":"2024-09-17T05:29:43.361825+0000","last_peered":"2024-09-17T05:29:43.361825+0000","last_clean":"2024-09-17T05:29:43.361825+0000","last_became_active":"2024-09-17T05:29:43.353985+0000","last_became_peered":"2024-09-17T05:29:43.353985+0000","last_unstale":"2024-09-17T05:29:43.361825+0000","last_undegraded":"2024-09-17T05:29:43.361825+0000","last_fullsized":"2024-09-17T05:29:43.361825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:06.497620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.218706+0000","last_change":"2024-09-17T05:29:43.375566+0000","last_active":"2024-09-17T05:29:45.218706+0000","last_peered":"2024-09-17T05:29:45.218706+0000","last_clean":"2024-09-17T05:29:45.218706+0000","last_became_active":"2024-09-17T05:29:43.375105+0000","last_became_peered":"2024-09-17T05:29:43.375105+0000","last_unstale":"2024-09-17T05:29:45.218706+0000","last_undegraded":"2024-09-17T05:29:45.218706+0000","last_fullsized":"2024-09-17T05:29:45.218706+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:31.051875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219292+0000","last_change":"2024-09-17T05:29:43.385036+0000","last_active":"2024-09-17T05:29:45.219292+0000","last_peered":"2024-09-17T05:29:45.219292+0000","last_clean":"2024-09-17T05:29:45.219292+0000","last_became_active":"2024-09-17T05:29:43.384547+0000","last_became_peered":"2024-09-17T05:29:43.384547+0000","last_unstale":"2024-09-17T05:29:45.219292+0000","last_undegraded":"2024-09-17T05:29:45.219292+0000","last_fullsized":"2024-09-17T05:29:45.219292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:42.256511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.381768+0000","last_change":"2024-09-17T05:29:43.370799+0000","last_active":"2024-09-17T05:29:43.381768+0000","last_peered":"2024-09-17T05:29:43.381768+0000","last_clean":"2024-09-17T05:29:43.381768+0000","last_became_active":"2024-09-17T05:29:43.370466+0000","last_became_peered":"2024-09-17T05:29:43.370466+0000","last_unstale":"2024-09-17T05:29:43.381768+0000","last_undegraded":"2024-09-17T05:29:43.381768+0000","last_fullsized":"2024-09-17T05:29:43.381768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:30.649792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386760+0000","last_change":"2024-09-17T05:29:43.371526+0000","last_active":"2024-09-17T05:29:43.386760+0000","last_peered":"2024-09-17T05:29:43.386760+0000","last_clean":"2024-09-17T05:29:43.386760+0000","last_became_active":"2024-09-17T05:29:43.371182+0000","last_became_peered":"2024-09-17T05:29:43.371182+0000","last_unstale":"2024-09-17T05:29:43.386760+0000","last_undegraded":"2024-09-17T05:29:43.386760+0000","last_fullsized":"2024-09-17T05:29:43.386760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:58.739001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387429+0000","last_change":"2024-09-17T05:29:43.382315+0000","last_active":"2024-09-17T05:29:43.387429+0000","last_peered":"2024-09-17T05:29:43.387429+0000","last_clean":"2024-09-17T05:29:43.387429+0000","last_became_active":"2024-09-17T05:29:43.381954+0000","last_became_peered":"2024-09-17T05:29:43.381954+0000","last_unstale":"2024-09-17T05:29:43.387429+0000","last_undegraded":"2024-09-17T05:29:43.387429+0000","last_fullsized":"2024-09-17T05:29:43.387429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:09.124150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.371828+0000","last_change":"2024-09-17T05:29:43.365278+0000","last_active":"2024-09-17T05:29:43.371828+0000","last_peered":"2024-09-17T05:29:43.371828+0000","last_clean":"2024-09-17T05:29:43.371828+0000","last_became_active":"2024-09-17T05:29:43.364900+0000","last_became_peered":"2024-09-17T05:29:43.364900+0000","last_unstale":"2024-09-17T05:29:43.371828+0000","last_undegraded":"2024-09-17T05:29:43.371828+0000","last_fullsized":"2024-09-17T05:29:43.371828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:42.879521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382921+0000","last_change":"2024-09-17T05:29:43.372074+0000","last_active":"2024-09-17T05:29:43.382921+0000","last_peered":"2024-09-17T05:29:43.382921+0000","last_clean":"2024-09-17T05:29:43.382921+0000","last_became_active":"2024-09-17T05:29:43.368715+0000","last_became_peered":"2024-09-17T05:29:43.368715+0000","last_unstale":"2024-09-17T05:29:43.382921+0000","last_undegraded":"2024-09-17T05:29:43.382921+0000","last_fullsized":"2024-09-17T05:29:43.382921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:27.875334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394488+0000","last_change":"2024-09-17T05:29:43.392195+0000","last_active":"2024-09-17T05:29:43.394488+0000","last_peered":"2024-09-17T05:29:43.394488+0000","last_clean":"2024-09-17T05:29:43.394488+0000","last_became_active":"2024-09-17T05:29:43.391041+0000","last_became_peered":"2024-09-17T05:29:43.391041+0000","last_unstale":"2024-09-17T05:29:43.394488+0000","last_undegraded":"2024-09-17T05:29:43.394488+0000","last_fullsized":"2024-09-17T05:29:43.394488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:28:35.724566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394303+0000","last_change":"2024-09-17T05:29:43.391354+0000","last_active":"2024-09-17T05:29:43.394303+0000","last_peered":"2024-09-17T05:29:43.394303+0000","last_clean":"2024-09-17T05:29:43.394303+0000","last_became_active":"2024-09-17T05:29:43.391031+0000","last_became_peered":"2024-09-17T05:29:43.391031+0000","last_unstale":"2024-09-17T05:29:43.394303+0000","last_undegraded":"2024-09-17T05:29:43.394303+0000","last_fullsized":"2024-09-17T05:29:43.394303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:23.257421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377383+0000","last_change":"2024-09-17T05:29:43.366081+0000","last_active":"2024-09-17T05:29:43.377383+0000","last_peered":"2024-09-17T05:29:43.377383+0000","last_clean":"2024-09-17T05:29:43.377383+0000","last_became_active":"2024-09-17T05:29:43.365730+0000","last_became_peered":"2024-09-17T05:29:43.365730+0000","last_unstale":"2024-09-17T05:29:43.377383+0000","last_undegraded":"2024-09-17T05:29:43.377383+0000","last_fullsized":"2024-09-17T05:29:43.377383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:58.855247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388345+0000","last_change":"2024-09-17T05:29:42.368042+0000","last_active":"2024-09-17T05:29:43.388345+0000","last_peered":"2024-09-17T05:29:43.388345+0000","last_clean":"2024-09-17T05:29:43.388345+0000","last_became_active":"2024-09-17T05:29:42.367686+0000","last_became_peered":"2024-09-17T05:29:42.367686+0000","last_unstale":"2024-09-17T05:29:43.388345+0000","last_undegraded":"2024-09-17T05:29:43.388345+0000","last_fullsized":"2024-09-17T05:29:43.388345+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:17.341630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387478+0000","last_change":"2024-09-17T05:29:43.382320+0000","last_active":"2024-09-17T05:29:43.387478+0000","last_peered":"2024-09-17T05:29:43.387478+0000","last_clean":"2024-09-17T05:29:43.387478+0000","last_became_active":"2024-09-17T05:29:43.380275+0000","last_became_peered":"2024-09-17T05:29:43.380275+0000","last_unstale":"2024-09-17T05:29:43.387478+0000","last_undegraded":"2024-09-17T05:29:43.387478+0000","last_fullsized":"2024-09-17T05:29:43.387478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:31.630199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387946+0000","last_change":"2024-09-17T05:29:42.366557+0000","last_active":"2024-09-17T05:29:43.387946+0000","last_peered":"2024-09-17T05:29:43.387946+0000","last_clean":"2024-09-17T05:29:43.387946+0000","last_became_active":"2024-09-17T05:29:42.364903+0000","last_became_peered":"2024-09-17T05:29:42.364903+0000","last_unstale":"2024-09-17T05:29:43.387946+0000","last_undegraded":"2024-09-17T05:29:43.387946+0000","last_fullsized":"2024-09-17T05:29:43.387946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:08.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378239+0000","last_change":"2024-09-17T05:29:43.372852+0000","last_active":"2024-09-17T05:29:43.378239+0000","last_peered":"2024-09-17T05:29:43.378239+0000","last_clean":"2024-09-17T05:29:43.378239+0000","last_became_active":"2024-09-17T05:29:43.372522+0000","last_became_peered":"2024-09-17T05:29:43.372522+0000","last_unstale":"2024-09-17T05:29:43.378239+0000","last_undegraded":"2024-09-17T05:29:43.378239+0000","last_fullsized":"2024-09-17T05:29:43.378239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:48.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219930+0000","last_change":"2024-09-17T05:29:42.370576+0000","last_active":"2024-09-17T05:29:45.219930+0000","last_peered":"2024-09-17T05:29:45.219930+0000","last_clean":"2024-09-17T05:29:45.219930+0000","last_became_active":"2024-09-17T05:29:42.369890+0000","last_became_peered":"2024-09-17T05:29:42.369890+0000","last_unstale":"2024-09-17T05:29:45.219930+0000","last_undegraded":"2024-09-17T05:29:45.219930+0000","last_fullsized":"2024-09-17T05:29:45.219930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:50.988598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391762+0000","last_change":"2024-09-17T05:29:43.388104+0000","last_active":"2024-09-17T05:29:43.391762+0000","last_peered":"2024-09-17T05:29:43.391762+0000","last_clean":"2024-09-17T05:29:43.391762+0000","last_became_active":"2024-09-17T05:29:43.387740+0000","last_became_peered":"2024-09-17T05:29:43.387740+0000","last_unstale":"2024-09-17T05:29:43.391762+0000","last_undegraded":"2024-09-17T05:29:43.391762+0000","last_fullsized":"2024-09-17T05:29:43.391762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:24.008001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394399+0000","last_change":"2024-09-17T05:29:42.368993+0000","last_active":"2024-09-17T05:29:43.394399+0000","last_peered":"2024-09-17T05:29:43.394399+0000","last_clean":"2024-09-17T05:29:43.394399+0000","last_became_active":"2024-09-17T05:29:42.368510+0000","last_became_peered":"2024-09-17T05:29:42.368510+0000","last_unstale":"2024-09-17T05:29:43.394399+0000","last_undegraded":"2024-09-17T05:29:43.394399+0000","last_fullsized":"2024-09-17T05:29:43.394399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:53.140678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384095+0000","last_change":"2024-09-17T05:29:43.375002+0000","last_active":"2024-09-17T05:29:43.384095+0000","last_peered":"2024-09-17T05:29:43.384095+0000","last_clean":"2024-09-17T05:29:43.384095+0000","last_became_active":"2024-09-17T05:29:43.374572+0000","last_became_peered":"2024-09-17T05:29:43.374572+0000","last_unstale":"2024-09-17T05:29:43.384095+0000","last_undegraded":"2024-09-17T05:29:43.384095+0000","last_fullsized":"2024-09-17T05:29:43.384095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:34:27.178663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388853+0000","last_change":"2024-09-17T05:29:42.374052+0000","last_active":"2024-09-17T05:29:43.388853+0000","last_peered":"2024-09-17T05:29:43.388853+0000","last_clean":"2024-09-17T05:29:43.388853+0000","last_became_active":"2024-09-17T05:29:42.373481+0000","last_became_peered":"2024-09-17T05:29:42.373481+0000","last_unstale":"2024-09-17T05:29:43.388853+0000","last_undegraded":"2024-09-17T05:29:43.388853+0000","last_fullsized":"2024-09-17T05:29:43.388853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:52.246748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386756+0000","last_change":"2024-09-17T05:29:43.383822+0000","last_active":"2024-09-17T05:29:43.386756+0000","last_peered":"2024-09-17T05:29:43.386756+0000","last_clean":"2024-09-17T05:29:43.386756+0000","last_became_active":"2024-09-17T05:29:43.383455+0000","last_became_peered":"2024-09-17T05:29:43.383455+0000","last_unstale":"2024-09-17T05:29:43.386756+0000","last_undegraded":"2024-09-17T05:29:43.386756+0000","last_fullsized":"2024-09-17T05:29:43.386756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:26.761213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.390966+0000","last_change":"2024-09-17T05:29:42.358478+0000","last_active":"2024-09-17T05:29:43.390966+0000","last_peered":"2024-09-17T05:29:43.390966+0000","last_clean":"2024-09-17T05:29:43.390966+0000","last_became_active":"2024-09-17T05:29:42.358158+0000","last_became_peered":"2024-09-17T05:29:42.358158+0000","last_unstale":"2024-09-17T05:29:43.390966+0000","last_undegraded":"2024-09-17T05:29:43.390966+0000","last_fullsized":"2024-09-17T05:29:43.390966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:06.782962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383394+0000","last_change":"2024-09-17T05:29:43.378747+0000","last_active":"2024-09-17T05:29:43.383394+0000","last_peered":"2024-09-17T05:29:43.383394+0000","last_clean":"2024-09-17T05:29:43.383394+0000","last_became_active":"2024-09-17T05:29:43.376528+0000","last_became_peered":"2024-09-17T05:29:43.376528+0000","last_unstale":"2024-09-17T05:29:43.383394+0000","last_undegraded":"2024-09-17T05:29:43.383394+0000","last_fullsized":"2024-09-17T05:29:43.383394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:58.007315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348599+0000","last_change":"2024-09-17T05:29:42.371911+0000","last_active":"2024-09-17T05:29:43.348599+0000","last_peered":"2024-09-17T05:29:43.348599+0000","last_clean":"2024-09-17T05:29:43.348599+0000","last_became_active":"2024-09-17T05:29:42.371309+0000","last_became_peered":"2024-09-17T05:29:42.371309+0000","last_unstale":"2024-09-17T05:29:43.348599+0000","last_undegraded":"2024-09-17T05:29:43.348599+0000","last_fullsized":"2024-09-17T05:29:43.348599+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:33.482665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382080+0000","last_change":"2024-09-17T05:29:43.365063+0000","last_active":"2024-09-17T05:29:43.382080+0000","last_peered":"2024-09-17T05:29:43.382080+0000","last_clean":"2024-09-17T05:29:43.382080+0000","last_became_active":"2024-09-17T05:29:43.364700+0000","last_became_peered":"2024-09-17T05:29:43.364700+0000","last_unstale":"2024-09-17T05:29:43.382080+0000","last_undegraded":"2024-09-17T05:29:43.382080+0000","last_fullsized":"2024-09-17T05:29:43.382080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:02.220275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348726+0000","last_change":"2024-09-17T05:29:42.369258+0000","last_active":"2024-09-17T05:29:43.348726+0000","last_peered":"2024-09-17T05:29:43.348726+0000","last_clean":"2024-09-17T05:29:43.348726+0000","last_became_active":"2024-09-17T05:29:42.368915+0000","last_became_peered":"2024-09-17T05:29:42.368915+0000","last_unstale":"2024-09-17T05:29:43.348726+0000","last_undegraded":"2024-09-17T05:29:43.348726+0000","last_fullsized":"2024-09-17T05:29:43.348726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:26.652183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384642+0000","last_change":"2024-09-17T05:29:43.382401+0000","last_active":"2024-09-17T05:29:43.384642+0000","last_peered":"2024-09-17T05:29:43.384642+0000","last_clean":"2024-09-17T05:29:43.384642+0000","last_became_active":"2024-09-17T05:29:43.381801+0000","last_became_peered":"2024-09-17T05:29:43.381801+0000","last_unstale":"2024-09-17T05:29:43.384642+0000","last_undegraded":"2024-09-17T05:29:43.384642+0000","last_fullsized":"2024-09-17T05:29:43.384642+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:31.855952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.364901+0000","last_change":"2024-09-17T05:29:42.371740+0000","last_active":"2024-09-17T05:29:43.364901+0000","last_peered":"2024-09-17T05:29:43.364901+0000","last_clean":"2024-09-17T05:29:43.364901+0000","last_became_active":"2024-09-17T05:29:42.371404+0000","last_became_peered":"2024-09-17T05:29:42.371404+0000","last_unstale":"2024-09-17T05:29:43.364901+0000","last_undegraded":"2024-09-17T05:29:43.364901+0000","last_fullsized":"2024-09-17T05:29:43.364901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:19.032614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391940+0000","last_change":"2024-09-17T05:29:43.387612+0000","last_active":"2024-09-17T05:29:43.391940+0000","last_peered":"2024-09-17T05:29:43.391940+0000","last_clean":"2024-09-17T05:29:43.391940+0000","last_became_active":"2024-09-17T05:29:43.387272+0000","last_became_peered":"2024-09-17T05:29:43.387272+0000","last_unstale":"2024-09-17T05:29:43.391940+0000","last_undegraded":"2024-09-17T05:29:43.391940+0000","last_fullsized":"2024-09-17T05:29:43.391940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:13.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.373898+0000","last_change":"2024-09-17T05:29:42.357065+0000","last_active":"2024-09-17T05:29:43.373898+0000","last_peered":"2024-09-17T05:29:43.373898+0000","last_clean":"2024-09-17T05:29:43.373898+0000","last_became_active":"2024-09-17T05:29:42.356652+0000","last_became_peered":"2024-09-17T05:29:42.356652+0000","last_unstale":"2024-09-17T05:29:43.373898+0000","last_undegraded":"2024-09-17T05:29:43.373898+0000","last_fullsized":"2024-09-17T05:29:43.373898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:36.558975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219037+0000","last_change":"2024-09-17T05:29:43.384263+0000","last_active":"2024-09-17T05:29:45.219037+0000","last_peered":"2024-09-17T05:29:45.219037+0000","last_clean":"2024-09-17T05:29:45.219037+0000","last_became_active":"2024-09-17T05:29:43.383790+0000","last_became_peered":"2024-09-17T05:29:43.383790+0000","last_unstale":"2024-09-17T05:29:45.219037+0000","last_undegraded":"2024-09-17T05:29:45.219037+0000","last_fullsized":"2024-09-17T05:29:45.219037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:12.714362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387623+0000","last_change":"2024-09-17T05:29:42.369472+0000","last_active":"2024-09-17T05:29:43.387623+0000","last_peered":"2024-09-17T05:29:43.387623+0000","last_clean":"2024-09-17T05:29:43.387623+0000","last_became_active":"2024-09-17T05:29:42.369153+0000","last_became_peered":"2024-09-17T05:29:42.369153+0000","last_unstale":"2024-09-17T05:29:43.387623+0000","last_undegraded":"2024-09-17T05:29:43.387623+0000","last_fullsized":"2024-09-17T05:29:43.387623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:43.126915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377485+0000","last_change":"2024-09-17T05:29:43.368486+0000","last_active":"2024-09-17T05:29:43.377485+0000","last_peered":"2024-09-17T05:29:43.377485+0000","last_clean":"2024-09-17T05:29:43.377485+0000","last_became_active":"2024-09-17T05:29:43.368116+0000","last_became_peered":"2024-09-17T05:29:43.368116+0000","last_unstale":"2024-09-17T05:29:43.377485+0000","last_undegraded":"2024-09-17T05:29:43.377485+0000","last_fullsized":"2024-09-17T05:29:43.377485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:02.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.392999+0000","last_change":"2024-09-17T05:29:42.359617+0000","last_active":"2024-09-17T05:29:43.392999+0000","last_peered":"2024-09-17T05:29:43.392999+0000","last_clean":"2024-09-17T05:29:43.392999+0000","last_became_active":"2024-09-17T05:29:42.359297+0000","last_became_peered":"2024-09-17T05:29:42.359297+0000","last_unstale":"2024-09-17T05:29:43.392999+0000","last_undegraded":"2024-09-17T05:29:43.392999+0000","last_fullsized":"2024-09-17T05:29:43.392999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:15.177568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.218748+0000","last_change":"2024-09-17T05:29:43.379789+0000","last_active":"2024-09-17T05:29:45.218748+0000","last_peered":"2024-09-17T05:29:45.218748+0000","last_clean":"2024-09-17T05:29:45.218748+0000","last_became_active":"2024-09-17T05:29:43.379460+0000","last_became_peered":"2024-09-17T05:29:43.379460+0000","last_unstale":"2024-09-17T05:29:45.218748+0000","last_undegraded":"2024-09-17T05:29:45.218748+0000","last_fullsized":"2024-09-17T05:29:45.218748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:57.901013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.389979+0000","last_change":"2024-09-17T05:29:42.362233+0000","last_active":"2024-09-17T05:29:43.389979+0000","last_peered":"2024-09-17T05:29:43.389979+0000","last_clean":"2024-09-17T05:29:43.389979+0000","last_became_active":"2024-09-17T05:29:42.361884+0000","last_became_peered":"2024-09-17T05:29:42.361884+0000","last_unstale":"2024-09-17T05:29:43.389979+0000","last_undegraded":"2024-09-17T05:29:43.389979+0000","last_fullsized":"2024-09-17T05:29:43.389979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:36.467901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391802+0000","last_change":"2024-09-17T05:29:43.386438+0000","last_active":"2024-09-17T05:29:43.391802+0000","last_peered":"2024-09-17T05:29:43.391802+0000","last_clean":"2024-09-17T05:29:43.391802+0000","last_became_active":"2024-09-17T05:29:43.380150+0000","last_became_peered":"2024-09-17T05:29:43.380150+0000","last_unstale":"2024-09-17T05:29:43.391802+0000","last_undegraded":"2024-09-17T05:29:43.391802+0000","last_fullsized":"2024-09-17T05:29:43.391802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:30.318924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.363671+0000","last_change":"2024-09-17T05:29:42.377313+0000","last_active":"2024-09-17T05:29:43.363671+0000","last_peered":"2024-09-17T05:29:43.363671+0000","last_clean":"2024-09-17T05:29:43.363671+0000","last_became_active":"2024-09-17T05:29:42.376905+0000","last_became_peered":"2024-09-17T05:29:42.376905+0000","last_unstale":"2024-09-17T05:29:43.363671+0000","last_undegraded":"2024-09-17T05:29:43.363671+0000","last_fullsized":"2024-09-17T05:29:43.363671+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:30.666312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.383048+0000","last_change":"2024-09-17T05:29:43.371130+0000","last_active":"2024-09-17T05:29:43.383048+0000","last_peered":"2024-09-17T05:29:43.383048+0000","last_clean":"2024-09-17T05:29:43.383048+0000","last_became_active":"2024-09-17T05:29:43.370337+0000","last_became_peered":"2024-09-17T05:29:43.370337+0000","last_unstale":"2024-09-17T05:29:43.383048+0000","last_undegraded":"2024-09-17T05:29:43.383048+0000","last_fullsized":"2024-09-17T05:29:43.383048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:09.124150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.366403+0000","last_change":"2024-09-17T05:29:42.361941+0000","last_active":"2024-09-17T05:29:43.366403+0000","last_peered":"2024-09-17T05:29:43.366403+0000","last_clean":"2024-09-17T05:29:43.366403+0000","last_became_active":"2024-09-17T05:29:42.361629+0000","last_became_peered":"2024-09-17T05:29:42.361629+0000","last_unstale":"2024-09-17T05:29:43.366403+0000","last_undegraded":"2024-09-17T05:29:43.366403+0000","last_fullsized":"2024-09-17T05:29:43.366403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:20.753316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.218547+0000","last_change":"2024-09-17T05:29:43.371574+0000","last_active":"2024-09-17T05:29:45.218547+0000","last_peered":"2024-09-17T05:29:45.218547+0000","last_clean":"2024-09-17T05:29:45.218547+0000","last_became_active":"2024-09-17T05:29:43.371261+0000","last_became_peered":"2024-09-17T05:29:43.371261+0000","last_unstale":"2024-09-17T05:29:45.218547+0000","last_undegraded":"2024-09-17T05:29:45.218547+0000","last_fullsized":"2024-09-17T05:29:45.218547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:38.740497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219685+0000","last_change":"2024-09-17T05:29:42.362907+0000","last_active":"2024-09-17T05:29:45.219685+0000","last_peered":"2024-09-17T05:29:45.219685+0000","last_clean":"2024-09-17T05:29:45.219685+0000","last_became_active":"2024-09-17T05:29:42.362473+0000","last_became_peered":"2024-09-17T05:29:42.362473+0000","last_unstale":"2024-09-17T05:29:45.219685+0000","last_undegraded":"2024-09-17T05:29:45.219685+0000","last_fullsized":"2024-09-17T05:29:45.219685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:06.444145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385356+0000","last_change":"2024-09-17T05:29:43.381561+0000","last_active":"2024-09-17T05:29:43.385356+0000","last_peered":"2024-09-17T05:29:43.385356+0000","last_clean":"2024-09-17T05:29:43.385356+0000","last_became_active":"2024-09-17T05:29:43.381180+0000","last_became_peered":"2024-09-17T05:29:43.381180+0000","last_unstale":"2024-09-17T05:29:43.385356+0000","last_undegraded":"2024-09-17T05:29:43.385356+0000","last_fullsized":"2024-09-17T05:29:43.385356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:45.535653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386259+0000","last_change":"2024-09-17T05:29:42.368851+0000","last_active":"2024-09-17T05:29:43.386259+0000","last_peered":"2024-09-17T05:29:43.386259+0000","last_clean":"2024-09-17T05:29:43.386259+0000","last_became_active":"2024-09-17T05:29:42.368562+0000","last_became_peered":"2024-09-17T05:29:42.368562+0000","last_unstale":"2024-09-17T05:29:43.386259+0000","last_undegraded":"2024-09-17T05:29:43.386259+0000","last_fullsized":"2024-09-17T05:29:43.386259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:02.347558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.394051+0000","last_change":"2024-09-17T05:29:43.390055+0000","last_active":"2024-09-17T05:29:43.394051+0000","last_peered":"2024-09-17T05:29:43.394051+0000","last_clean":"2024-09-17T05:29:43.394051+0000","last_became_active":"2024-09-17T05:29:43.389143+0000","last_became_peered":"2024-09-17T05:29:43.389143+0000","last_unstale":"2024-09-17T05:29:43.394051+0000","last_undegraded":"2024-09-17T05:29:43.394051+0000","last_fullsized":"2024-09-17T05:29:43.394051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:45:34.584862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.391159+0000","last_change":"2024-09-17T05:29:42.376726+0000","last_active":"2024-09-17T05:29:43.391159+0000","last_peered":"2024-09-17T05:29:43.391159+0000","last_clean":"2024-09-17T05:29:43.391159+0000","last_became_active":"2024-09-17T05:29:42.376389+0000","last_became_peered":"2024-09-17T05:29:42.376389+0000","last_unstale":"2024-09-17T05:29:43.391159+0000","last_undegraded":"2024-09-17T05:29:43.391159+0000","last_fullsized":"2024-09-17T05:29:43.391159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:57.004593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.381647+0000","last_change":"2024-09-17T05:29:43.379855+0000","last_active":"2024-09-17T05:29:43.381647+0000","last_peered":"2024-09-17T05:29:43.381647+0000","last_clean":"2024-09-17T05:29:43.381647+0000","last_became_active":"2024-09-17T05:29:43.379502+0000","last_became_peered":"2024-09-17T05:29:43.379502+0000","last_unstale":"2024-09-17T05:29:43.381647+0000","last_undegraded":"2024-09-17T05:29:43.381647+0000","last_fullsized":"2024-09-17T05:29:43.381647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:28.267802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.349414+0000","last_change":"2024-09-17T05:29:42.367174+0000","last_active":"2024-09-17T05:29:43.349414+0000","last_peered":"2024-09-17T05:29:43.349414+0000","last_clean":"2024-09-17T05:29:43.349414+0000","last_became_active":"2024-09-17T05:29:42.366821+0000","last_became_peered":"2024-09-17T05:29:42.366821+0000","last_unstale":"2024-09-17T05:29:43.349414+0000","last_undegraded":"2024-09-17T05:29:43.349414+0000","last_fullsized":"2024-09-17T05:29:43.349414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:08.639903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.378130+0000","last_change":"2024-09-17T05:29:43.374767+0000","last_active":"2024-09-17T05:29:43.378130+0000","last_peered":"2024-09-17T05:29:43.378130+0000","last_clean":"2024-09-17T05:29:43.378130+0000","last_became_active":"2024-09-17T05:29:43.374406+0000","last_became_peered":"2024-09-17T05:29:43.374406+0000","last_unstale":"2024-09-17T05:29:43.378130+0000","last_undegraded":"2024-09-17T05:29:43.378130+0000","last_fullsized":"2024-09-17T05:29:43.378130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:06.073333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386309+0000","last_change":"2024-09-17T05:29:42.366434+0000","last_active":"2024-09-17T05:29:43.386309+0000","last_peered":"2024-09-17T05:29:43.386309+0000","last_clean":"2024-09-17T05:29:43.386309+0000","last_became_active":"2024-09-17T05:29:42.365972+0000","last_became_peered":"2024-09-17T05:29:42.365972+0000","last_unstale":"2024-09-17T05:29:43.386309+0000","last_undegraded":"2024-09-17T05:29:43.386309+0000","last_fullsized":"2024-09-17T05:29:43.386309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:53.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387775+0000","last_change":"2024-09-17T05:29:43.372992+0000","last_active":"2024-09-17T05:29:43.387775+0000","last_peered":"2024-09-17T05:29:43.387775+0000","last_clean":"2024-09-17T05:29:43.387775+0000","last_became_active":"2024-09-17T05:29:43.372271+0000","last_became_peered":"2024-09-17T05:29:43.372271+0000","last_unstale":"2024-09-17T05:29:43.387775+0000","last_undegraded":"2024-09-17T05:29:43.387775+0000","last_fullsized":"2024-09-17T05:29:43.387775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:15.893767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.365654+0000","last_change":"2024-09-17T05:29:42.368503+0000","last_active":"2024-09-17T05:29:43.365654+0000","last_peered":"2024-09-17T05:29:43.365654+0000","last_clean":"2024-09-17T05:29:43.365654+0000","last_became_active":"2024-09-17T05:29:42.368082+0000","last_became_peered":"2024-09-17T05:29:42.368082+0000","last_unstale":"2024-09-17T05:29:43.365654+0000","last_undegraded":"2024-09-17T05:29:43.365654+0000","last_fullsized":"2024-09-17T05:29:43.365654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:58.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.384034+0000","last_change":"2024-09-17T05:29:43.371376+0000","last_active":"2024-09-17T05:29:43.384034+0000","last_peered":"2024-09-17T05:29:43.384034+0000","last_clean":"2024-09-17T05:29:43.384034+0000","last_became_active":"2024-09-17T05:29:43.365092+0000","last_became_peered":"2024-09-17T05:29:43.365092+0000","last_unstale":"2024-09-17T05:29:43.384034+0000","last_undegraded":"2024-09-17T05:29:43.384034+0000","last_fullsized":"2024-09-17T05:29:43.384034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:00.730402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.218507+0000","last_change":"2024-09-17T05:29:42.353808+0000","last_active":"2024-09-17T05:29:45.218507+0000","last_peered":"2024-09-17T05:29:45.218507+0000","last_clean":"2024-09-17T05:29:45.218507+0000","last_became_active":"2024-09-17T05:29:42.353481+0000","last_became_peered":"2024-09-17T05:29:42.353481+0000","last_unstale":"2024-09-17T05:29:45.218507+0000","last_undegraded":"2024-09-17T05:29:45.218507+0000","last_fullsized":"2024-09-17T05:29:45.218507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:06.867137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.376805+0000","last_change":"2024-09-17T05:29:43.361964+0000","last_active":"2024-09-17T05:29:43.376805+0000","last_peered":"2024-09-17T05:29:43.376805+0000","last_clean":"2024-09-17T05:29:43.376805+0000","last_became_active":"2024-09-17T05:29:43.361619+0000","last_became_peered":"2024-09-17T05:29:43.361619+0000","last_unstale":"2024-09-17T05:29:43.376805+0000","last_undegraded":"2024-09-17T05:29:43.376805+0000","last_fullsized":"2024-09-17T05:29:43.376805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:23.191921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.349021+0000","last_change":"2024-09-17T05:29:42.368459+0000","last_active":"2024-09-17T05:29:43.349021+0000","last_peered":"2024-09-17T05:29:43.349021+0000","last_clean":"2024-09-17T05:29:43.349021+0000","last_became_active":"2024-09-17T05:29:42.367799+0000","last_became_peered":"2024-09-17T05:29:42.367799+0000","last_unstale":"2024-09-17T05:29:43.349021+0000","last_undegraded":"2024-09-17T05:29:43.349021+0000","last_fullsized":"2024-09-17T05:29:43.349021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:38.839620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.388718+0000","last_change":"2024-09-17T05:29:43.381905+0000","last_active":"2024-09-17T05:29:43.388718+0000","last_peered":"2024-09-17T05:29:43.388718+0000","last_clean":"2024-09-17T05:29:43.388718+0000","last_became_active":"2024-09-17T05:29:43.381557+0000","last_became_peered":"2024-09-17T05:29:43.381557+0000","last_unstale":"2024-09-17T05:29:43.388718+0000","last_undegraded":"2024-09-17T05:29:43.388718+0000","last_fullsized":"2024-09-17T05:29:43.388718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:30.064954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.382922+0000","last_change":"2024-09-17T05:29:42.372935+0000","last_active":"2024-09-17T05:29:43.382922+0000","last_peered":"2024-09-17T05:29:43.382922+0000","last_clean":"2024-09-17T05:29:43.382922+0000","last_became_active":"2024-09-17T05:29:42.372589+0000","last_became_peered":"2024-09-17T05:29:42.372589+0000","last_unstale":"2024-09-17T05:29:43.382922+0000","last_undegraded":"2024-09-17T05:29:43.382922+0000","last_fullsized":"2024-09-17T05:29:43.382922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:47.073986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.375016+0000","last_change":"2024-09-17T05:29:43.357595+0000","last_active":"2024-09-17T05:29:43.375016+0000","last_peered":"2024-09-17T05:29:43.375016+0000","last_clean":"2024-09-17T05:29:43.375016+0000","last_became_active":"2024-09-17T05:29:43.357221+0000","last_became_peered":"2024-09-17T05:29:43.357221+0000","last_unstale":"2024-09-17T05:29:43.375016+0000","last_undegraded":"2024-09-17T05:29:43.375016+0000","last_fullsized":"2024-09-17T05:29:43.375016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:11.628247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219019+0000","last_change":"2024-09-17T05:29:42.365183+0000","last_active":"2024-09-17T05:29:45.219019+0000","last_peered":"2024-09-17T05:29:45.219019+0000","last_clean":"2024-09-17T05:29:45.219019+0000","last_became_active":"2024-09-17T05:29:42.364603+0000","last_became_peered":"2024-09-17T05:29:42.364603+0000","last_unstale":"2024-09-17T05:29:45.219019+0000","last_undegraded":"2024-09-17T05:29:45.219019+0000","last_fullsized":"2024-09-17T05:29:45.219019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:33.217586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.377631+0000","last_change":"2024-09-17T05:29:43.365326+0000","last_active":"2024-09-17T05:29:43.377631+0000","last_peered":"2024-09-17T05:29:43.377631+0000","last_clean":"2024-09-17T05:29:43.377631+0000","last_became_active":"2024-09-17T05:29:43.364972+0000","last_became_peered":"2024-09-17T05:29:43.364972+0000","last_unstale":"2024-09-17T05:29:43.377631+0000","last_undegraded":"2024-09-17T05:29:43.377631+0000","last_fullsized":"2024-09-17T05:29:43.377631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:51.037863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.348168+0000","last_change":"2024-09-17T05:29:42.363604+0000","last_active":"2024-09-17T05:29:43.348168+0000","last_peered":"2024-09-17T05:29:43.348168+0000","last_clean":"2024-09-17T05:29:43.348168+0000","last_became_active":"2024-09-17T05:29:42.363231+0000","last_became_peered":"2024-09-17T05:29:42.363231+0000","last_unstale":"2024-09-17T05:29:43.348168+0000","last_undegraded":"2024-09-17T05:29:43.348168+0000","last_fullsized":"2024-09-17T05:29:43.348168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:55.703847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.389078+0000","last_change":"2024-09-17T05:29:43.382440+0000","last_active":"2024-09-17T05:29:43.389078+0000","last_peered":"2024-09-17T05:29:43.389078+0000","last_clean":"2024-09-17T05:29:43.389078+0000","last_became_active":"2024-09-17T05:29:43.382117+0000","last_became_peered":"2024-09-17T05:29:43.382117+0000","last_unstale":"2024-09-17T05:29:43.389078+0000","last_undegraded":"2024-09-17T05:29:43.389078+0000","last_fullsized":"2024-09-17T05:29:43.389078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:47.246823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.369682+0000","last_change":"2024-09-17T05:29:42.364199+0000","last_active":"2024-09-17T05:29:43.369682+0000","last_peered":"2024-09-17T05:29:43.369682+0000","last_clean":"2024-09-17T05:29:43.369682+0000","last_became_active":"2024-09-17T05:29:42.363852+0000","last_became_peered":"2024-09-17T05:29:42.363852+0000","last_unstale":"2024-09-17T05:29:43.369682+0000","last_undegraded":"2024-09-17T05:29:43.369682+0000","last_fullsized":"2024-09-17T05:29:43.369682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:26.272759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386018+0000","last_change":"2024-09-17T05:29:43.379751+0000","last_active":"2024-09-17T05:29:43.386018+0000","last_peered":"2024-09-17T05:29:43.386018+0000","last_clean":"2024-09-17T05:29:43.386018+0000","last_became_active":"2024-09-17T05:29:43.379324+0000","last_became_peered":"2024-09-17T05:29:43.379324+0000","last_unstale":"2024-09-17T05:29:43.386018+0000","last_undegraded":"2024-09-17T05:29:43.386018+0000","last_fullsized":"2024-09-17T05:29:43.386018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:57.456587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.360013+0000","last_change":"2024-09-17T05:29:42.371061+0000","last_active":"2024-09-17T05:29:43.360013+0000","last_peered":"2024-09-17T05:29:43.360013+0000","last_clean":"2024-09-17T05:29:43.360013+0000","last_became_active":"2024-09-17T05:29:42.369612+0000","last_became_peered":"2024-09-17T05:29:42.369612+0000","last_unstale":"2024-09-17T05:29:43.360013+0000","last_undegraded":"2024-09-17T05:29:43.360013+0000","last_fullsized":"2024-09-17T05:29:43.360013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.005279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219007+0000","last_change":"2024-09-17T05:29:43.379580+0000","last_active":"2024-09-17T05:29:45.219007+0000","last_peered":"2024-09-17T05:29:45.219007+0000","last_clean":"2024-09-17T05:29:45.219007+0000","last_became_active":"2024-09-17T05:29:43.379214+0000","last_became_peered":"2024-09-17T05:29:43.379214+0000","last_unstale":"2024-09-17T05:29:45.219007+0000","last_undegraded":"2024-09-17T05:29:45.219007+0000","last_fullsized":"2024-09-17T05:29:45.219007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:50.484190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.370595+0000","last_change":"2024-09-17T05:29:42.357640+0000","last_active":"2024-09-17T05:29:43.370595+0000","last_peered":"2024-09-17T05:29:43.370595+0000","last_clean":"2024-09-17T05:29:43.370595+0000","last_became_active":"2024-09-17T05:29:42.357340+0000","last_became_peered":"2024-09-17T05:29:42.357340+0000","last_unstale":"2024-09-17T05:29:43.370595+0000","last_undegraded":"2024-09-17T05:29:43.370595+0000","last_fullsized":"2024-09-17T05:29:43.370595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:10.161870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.387657+0000","last_change":"2024-09-17T05:29:43.380997+0000","last_active":"2024-09-17T05:29:43.387657+0000","last_peered":"2024-09-17T05:29:43.387657+0000","last_clean":"2024-09-17T05:29:43.387657+0000","last_became_active":"2024-09-17T05:29:43.380703+0000","last_became_peered":"2024-09-17T05:29:43.380703+0000","last_unstale":"2024-09-17T05:29:43.387657+0000","last_undegraded":"2024-09-17T05:29:43.387657+0000","last_fullsized":"2024-09-17T05:29:43.387657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:55.355491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.362795+0000","last_change":"2024-09-17T05:29:42.368428+0000","last_active":"2024-09-17T05:29:43.362795+0000","last_peered":"2024-09-17T05:29:43.362795+0000","last_clean":"2024-09-17T05:29:43.362795+0000","last_became_active":"2024-09-17T05:29:42.368070+0000","last_became_peered":"2024-09-17T05:29:42.368070+0000","last_unstale":"2024-09-17T05:29:43.362795+0000","last_undegraded":"2024-09-17T05:29:43.362795+0000","last_fullsized":"2024-09-17T05:29:43.362795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:16.265304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.385849+0000","last_change":"2024-09-17T05:29:43.379397+0000","last_active":"2024-09-17T05:29:43.385849+0000","last_peered":"2024-09-17T05:29:43.385849+0000","last_clean":"2024-09-17T05:29:43.385849+0000","last_became_active":"2024-09-17T05:29:43.376109+0000","last_became_peered":"2024-09-17T05:29:43.376109+0000","last_unstale":"2024-09-17T05:29:43.385849+0000","last_undegraded":"2024-09-17T05:29:43.385849+0000","last_fullsized":"2024-09-17T05:29:43.385849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:53.177047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.347496+0000","last_change":"2024-09-17T05:29:42.368873+0000","last_active":"2024-09-17T05:29:43.347496+0000","last_peered":"2024-09-17T05:29:43.347496+0000","last_clean":"2024-09-17T05:29:43.347496+0000","last_became_active":"2024-09-17T05:29:42.368515+0000","last_became_peered":"2024-09-17T05:29:42.368515+0000","last_unstale":"2024-09-17T05:29:43.347496+0000","last_undegraded":"2024-09-17T05:29:43.347496+0000","last_fullsized":"2024-09-17T05:29:43.347496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:33.573774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-17T05:29:45.219334+0000","last_change":"2024-09-17T05:29:43.377148+0000","last_active":"2024-09-17T05:29:45.219334+0000","last_peered":"2024-09-17T05:29:45.219334+0000","last_clean":"2024-09-17T05:29:45.219334+0000","last_became_active":"2024-09-17T05:29:43.376804+0000","last_became_peered":"2024-09-17T05:29:43.376804+0000","last_unstale":"2024-09-17T05:29:45.219334+0000","last_undegraded":"2024-09-17T05:29:45.219334+0000","last_fullsized":"2024-09-17T05:29:45.219334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:42.333338+0000","last_clean_scrub_stamp":"2024-09-17T05:29:42.333338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:45.237984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.386838+0000","last_change":"2024-09-17T05:29:42.366151+0000","last_active":"2024-09-17T05:29:43.386838+0000","last_peered":"2024-09-17T05:29:43.386838+0000","last_clean":"2024-09-17T05:29:43.386838+0000","last_became_active":"2024-09-17T05:29:42.365500+0000","last_became_peered":"2024-09-17T05:29:42.365500+0000","last_unstale":"2024-09-17T05:29:43.386838+0000","last_undegraded":"2024-09-17T05:29:43.386838+0000","last_fullsized":"2024-09-17T05:29:43.386838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:29:41.330615+0000","last_clean_scrub_stamp":"2024-09-17T05:29:41.330615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:01.020554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":238,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-17T05:29:43.363207+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:29:43.363207+0000","last_peered":"2024-09-17T05:29:43.363207+0000","last_clean":"2024-09-17T05:29:43.363207+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:29:43.363207+0000","last_undegraded":"2024-09-17T05:29:43.363207+0000","last_fullsized":"2024-09-17T05:29:43.363207+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":454311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509522,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1683456,"data_stored":1053100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":461120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":461760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":461086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1671168,"data_stored":1051544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509521,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":454311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":468058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:29:47.977 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:29:47.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:29:48.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- 172.21.15.43:0/1006346114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 msgr2=0x7f7fec15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 --2- 172.21.15.43:0/1006346114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec15c7e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009840 tx=0x7f7fe001c760 comp rx=0 tx=0).stop 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- 172.21.15.43:0/1006346114 shutdown_connections 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 --2- 172.21.15.43:0/1006346114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec0b3af0 0x7f7fec0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 --2- 172.21.15.43:0/1006346114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec15c7e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- 172.21.15.43:0/1006346114 >> 172.21.15.43:0/1006346114 conn(0x7f7fec0b23d0 msgr2=0x7f7fec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- 172.21.15.43:0/1006346114 shutdown_connections 2024-09-17T05:29:48.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- 172.21.15.43:0/1006346114 wait complete. 2024-09-17T05:29:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 Processor -- start 2024-09-17T05:29:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 -- start start 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.181+0000 7f7ff4400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec06f110 0x7f7fec151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fec1f7010 0x7f7fec0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec168bc0 con 0x7f7fec155d70 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec168a40 con 0x7f7fec1f7010 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec168d40 con 0x7f7fec06f110 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59798/0 (socket says 172.21.15.43:59798) 2024-09-17T05:29:48.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 -- 172.21.15.43:0/2427118519 learned_addr learned my addr 172.21.15.43:0/2427118519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1a00640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec06f110 0x7f7fec151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff2400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fec1f7010 0x7f7fec0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 -- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec06f110 msgr2=0x7f7fec151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec06f110 0x7f7fec151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 -- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fec1f7010 msgr2=0x7f7fec0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fec1f7010 0x7f7fec0ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 -- 172.21.15.43:0/2427118519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec152350 con 0x7f7fec155d70 2024-09-17T05:29:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff1000640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c430 tx=0x7f7fd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8011790 con 0x7f7fec155d70 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fd8017070 con 0x7f7fec155d70 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8011cc0 con 0x7f7fec155d70 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec0cbb00 con 0x7f7fec155d70 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec0cc0c0 con 0x7f7fec155d70 2024-09-17T05:29:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec0ad0b0 con 0x7f7fec155d70 2024-09-17T05:29:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.185+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7fd801c460 con 0x7f7fec155d70 2024-09-17T05:29:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.189+0000 7f7fe9600640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 0x7f7fcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.189+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f7fd8093e10 con 0x7f7fec155d70 2024-09-17T05:29:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.189+0000 7f7ff1a00640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 0x7f7fcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.189+0000 7f7ff1a00640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 0x7f7fcc075710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00af000 tx=0x7f7fe001afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.193+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7fd8097030 con 0x7f7fec155d70 2024-09-17T05:29:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.429+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7fec1f74d0 con 0x7f7fec155d70 2024-09-17T05:29:48.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7fe9600640 1 -- 172.21.15.43:0/2427118519 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7f7fd8021020 con 0x7f7fec155d70 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:29:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:29:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 msgr2=0x7f7fcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 0x7f7fcc075710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00af000 tx=0x7f7fe001afa0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 msgr2=0x7f7fec152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c430 tx=0x7f7fd800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 shutdown_connections 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fcc073200 0x7f7fcc075710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fec1f7010 0x7f7fec0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fec155d70 0x7f7fec152620 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 --2- 172.21.15.43:0/2427118519 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7fec06f110 0x7f7fec151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 >> 172.21.15.43:0/2427118519 conn(0x7f7fec0b23d0 msgr2=0x7f7fec06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 shutdown_connections 2024-09-17T05:29:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.433+0000 7f7ff4400640 1 -- 172.21.15.43:0/2427118519 wait complete. 2024-09-17T05:29:48.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 -- 172.21.15.43:0/4076917232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 msgr2=0x7f52cc00ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 --2- 172.21.15.43:0/4076917232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52cc00ba90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f52e815bc80 tx=0x7f52e001caf0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 -- 172.21.15.43:0/4076917232 shutdown_connections 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 --2- 172.21.15.43:0/4076917232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52cc00ba90 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 -- 172.21.15.43:0/4076917232 >> 172.21.15.43:0/4076917232 conn(0x7f52e80b23d0 msgr2=0x7f52e80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 -- 172.21.15.43:0/4076917232 shutdown_connections 2024-09-17T05:29:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.613+0000 7f52edc00640 1 -- 172.21.15.43:0/4076917232 wait complete. 2024-09-17T05:29:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 Processor -- start 2024-09-17T05:29:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- start start 2024-09-17T05:29:48.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e80c3930 0x7f52e80c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 0x7f52e8091af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52e815cb70 con 0x7f52cc0053e0 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52e815c9f0 con 0x7f52e81f5170 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80b33e0 con 0x7f52e80c3930 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59806/0 (socket says 172.21.15.43:59806) 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 -- 172.21.15.43:0/770100048 learned_addr learned my addr 172.21.15.43:0/770100048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52dfe00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e80c3930 0x7f52e80c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e7400640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 0x7f52e8091af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 -- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e80c3930 msgr2=0x7f52e80c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e80c3930 0x7f52e80c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 -- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 msgr2=0x7f52e8091af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 0x7f52e8091af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 -- 172.21.15.43:0/770100048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80c40d0 con 0x7f52cc0053e0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e7400640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 0x7f52e8091af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52e6a00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f52d8001870 tx=0x7f52d8012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80139e0 con 0x7f52cc0053e0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d8013b80 con 0x7f52cc0053e0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e80c43b0 con 0x7f52cc0053e0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8010480 con 0x7f52cc0053e0 2024-09-17T05:29:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.617+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e80c4970 con 0x7f52cc0053e0 2024-09-17T05:29:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.621+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e806f800 con 0x7f52cc0053e0 2024-09-17T05:29:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.621+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f52d8010620 con 0x7f52cc0053e0 2024-09-17T05:29:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.621+0000 7f52dea00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 0x7f52c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.621+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f52d8098590 con 0x7f52cc0053e0 2024-09-17T05:29:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.621+0000 7f52dfe00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 0x7f52c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.625+0000 7f52dfe00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 0x7f52c4075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52e815b650 tx=0x7f52d00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.625+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52d80636b0 con 0x7f52cc0053e0 2024-09-17T05:29:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.837+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52e80aee50 con 0x7f52cc0053e0 2024-09-17T05:29:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.837+0000 7f52dea00640 1 -- 172.21.15.43:0/770100048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5216 (secure 0 0 0) 0x7f52d8062fb0 con 0x7f52cc0053e0 2024-09-17T05:29:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:48.842 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"btime":"2024-09-17T05:29:46:366852+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:29:45.786243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6846":{"gid":6846,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/274766899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":274766899},{"type":"v1","addr":"172.21.15.43:6835","nonce":274766899}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6846,"qdb_cluster":[6846]},"id":6}]} 2024-09-17T05:29:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 86 2024-09-17T05:29:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 msgr2=0x7f52c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 0x7f52c4075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52e815b650 tx=0x7f52d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 msgr2=0x7f52e80946e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f52d8001870 tx=0x7f52d8012a60 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 shutdown_connections 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c4073200 0x7f52c4075710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e81f5170 0x7f52e8091af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e80c3930 0x7f52e80c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 --2- 172.21.15.43:0/770100048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52cc0053e0 0x7f52e80946e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 >> 172.21.15.43:0/770100048 conn(0x7f52e80b23d0 msgr2=0x7f52e80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 shutdown_connections 2024-09-17T05:29:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:48.841+0000 7f52edc00640 1 -- 172.21.15.43:0/770100048 wait complete. 2024-09-17T05:29:49.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:29:45.786243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6846}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6846': {'gid': 6846, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/274766899', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 274766899}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 274766899}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6846, 'qdb_cluster': [6846]} 2024-09-17T05:29:49.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:29:49.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:29:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.425+0000 7f89b0000640 1 -- 172.21.15.43:0/3714665989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a80b3af0 msgr2=0x7f89a815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.425+0000 7f89b0000640 1 --2- 172.21.15.43:0/3714665989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a80b3af0 0x7f89a815c7e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8994009870 tx=0x7f899401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.425+0000 7f89b0000640 1 -- 172.21.15.43:0/3714665989 shutdown_connections 2024-09-17T05:29:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.425+0000 7f89b0000640 1 --2- 172.21.15.43:0/3714665989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a80b3af0 0x7f89a815c7e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- 172.21.15.43:0/3714665989 >> 172.21.15.43:0/3714665989 conn(0x7f89a80b23d0 msgr2=0x7f89a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- 172.21.15.43:0/3714665989 shutdown_connections 2024-09-17T05:29:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- 172.21.15.43:0/3714665989 wait complete. 2024-09-17T05:29:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 Processor -- start 2024-09-17T05:29:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- start start 2024-09-17T05:29:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89a8155d70 0x7f89a8166c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 0x7f89a80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8169e50 con 0x7f89a80ca8e0 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8169cd0 con 0x7f89a8155d70 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.429+0000 7f89b0000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89a8169fd0 con 0x7f89a806f110 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:49752/0 (socket says 172.21.15.43:49752) 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 -- 172.21.15.43:0/2446204467 learned_addr learned my addr 172.21.15.43:0/2446204467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ae000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 0x7f89a80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89acc00640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89a8155d70 0x7f89a8166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 -- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89a8155d70 msgr2=0x7f89a8166c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89a8155d70 0x7f89a8166c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 -- 172.21.15.43:0/2446204467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 msgr2=0x7f89a80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 0x7f89a80cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 -- 172.21.15.43:0/2446204467 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a80cb230 con 0x7f89a806f110 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ae000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 0x7f89a80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89ad600640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c490 tx=0x7f899c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89a5600640 1 -- 172.21.15.43:0/2446204467 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c017070 con 0x7f89a806f110 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89a5600640 1 -- 172.21.15.43:0/2446204467 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899c0053e0 con 0x7f89a806f110 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a8167180 con 0x7f89a806f110 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89a5600640 1 -- 172.21.15.43:0/2446204467 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c003860 con 0x7f89a806f110 2024-09-17T05:29:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a80c6a60 con 0x7f89a806f110 2024-09-17T05:29:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.433+0000 7f89a5600640 1 -- 172.21.15.43:0/2446204467 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f899c01a070 con 0x7f89a806f110 2024-09-17T05:29:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89a5600640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 0x7f8988075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 Processor -- start 2024-09-17T05:29:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89a5600640 1 -- 172.21.15.43:0/2446204467 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f899c0935e0 con 0x7f89a806f110 2024-09-17T05:29:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 -- start start 2024-09-17T05:29:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89acc00640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 0x7f8988075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89acc00640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 0x7f8988075740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f89940af000 tx=0x7f899401b010 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d2362be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d21faca0 0x5627d2363120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627d2237410 con 0x5627d1e21460 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5627d2174b40 con 0x5627d21faca0 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5627d2163f70 con 0x5627d2211950 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59826/0 (socket says 172.21.15.43:59826) 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 -- 172.21.15.43:0/2765209662 learned_addr learned my addr 172.21.15.43:0/2765209662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ae000640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d21faca0 0x5627d2363120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89acc00640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d2362be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 -- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 msgr2=0x5627d2362be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d2362be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 -- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d21faca0 msgr2=0x5627d2363120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d21faca0 0x5627d2363120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 -- 172.21.15.43:0/2765209662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d212b5b0 con 0x5627d1e21460 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89acc00640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d2362be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f89ad600640 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f899c05dca0 tx=0x7f899c05dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0244c00bd78580 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f8999c00640 1 -- 172.21.15.43:0/2765209662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0629d0 con 0x5627d1e21460 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f8999c00640 1 -- 172.21.15.43:0/2765209662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899c067880 con 0x5627d1e21460 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.437+0000 7f8999c00640 1 -- 172.21.15.43:0/2765209662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c063da0 con 0x5627d1e21460 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- 172.21.15.43:0/2765209662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 msgr2=0x5627d23626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f899c05dca0 tx=0x7f899c05dab0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- 172.21.15.43:0/2765209662 shutdown_connections 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d21faca0 0x5627d2363120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d2362be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- 172.21.15.43:0/2765209662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d1e21460 0x5627d23626a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- 172.21.15.43:0/2765209662 >> 172.21.15.43:0/2765209662 conn(0x5627d22e7d90 msgr2=0x5627d2317690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- 172.21.15.43:0/2765209662 shutdown_connections 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- 172.21.15.43:0/2765209662 wait complete. 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- start start 2024-09-17T05:29:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 Processor -- start 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2369920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d236a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627d20fe5f0 con 0x5627d21faca0 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5627d2167d80 con 0x5627d1e21460 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5627d210a520 con 0x5627d2211950 2024-09-17T05:29:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89b7722000 10 client.0 _ll_register_callbacks cb 0x5627d210e830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59830/0 (socket says 172.21.15.43:59830) 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89acc00640 1 -- 172.21.15.43:0/650884422 learned_addr learned my addr 172.21.15.43:0/650884422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89ae000640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d236a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.441+0000 7f89ad600640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2369920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 -- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 msgr2=0x5627d236a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d236a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 -- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 msgr2=0x5627d2369920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2369920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 -- 172.21.15.43:0/650884422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d2102190 con 0x5627d21faca0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89acc00640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8994009600 tx=0x7f89940091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbf7c426209620 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89940c7020 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f8994005510 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627d20fc110 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f89940d1050 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 10 client.6978 fetch_fsmap learned FSMap version 86 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x5627d21e8f10 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5627d20fc110 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f89940d4020 con 0x5627d21faca0 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 10 client.6978 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 10 client.6978 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 20 client.6978 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e067d0 0x5627d2373d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 4 client.6978 mds_command: new command op to 6846 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 --> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d2176a90 con 0x5627d1e067d0 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89ad600640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e067d0 0x5627d2373d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89ad600640 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e067d0 0x5627d2373d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8958fdfc3db3b745 server_cookie=a53cb193526c442c in_seq=0 out_seq=0 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 10 client.6978 ms_handle_connect on v2:172.21.15.43:6833/274766899 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 1 -- 172.21.15.43:0/650884422 <== mds.0 v2:172.21.15.43:6833/274766899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5627d2176a90 con 0x5627d1e067d0 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 10 client.6978 handle_command_reply: tid=0 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 client.6978 shutdown 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 msgr2=0x5627d2369e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8994009600 tx=0x7f89940091e0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 shutdown_connections 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e067d0 0x5627d2373d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d2211950 0x5627d236a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2369e60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 --2- 172.21.15.43:0/650884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2369920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f898e000640 0 client.6978 ms_handle_reset on v2:172.21.15.43:6833/274766899 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 >> 172.21.15.43:0/650884422 conn(0x5627d22e7d90 msgr2=0x5627d231e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 shutdown_connections 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.445+0000 7f89b7722000 1 -- 172.21.15.43:0/650884422 wait complete. 2024-09-17T05:29:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.449+0000 7f89b7722000 20 client.6978 trim_cache size 0 max 16384 2024-09-17T05:29:49.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.465+0000 7f89b7722000 1 Processor -- start 2024-09-17T05:29:49.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.465+0000 7f89b7722000 1 -- start start 2024-09-17T05:29:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 0x5627d2316cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2317210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f89a82053b0 con 0x5627d21faca0 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8204520 con 0x5627d1e21460 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f89a81fc530 con 0x5627d22e7d90 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ae000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 0x5627d2316cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ae000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59838/0 (socket says 172.21.15.43:59838) 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89acc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2317210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ae000640 1 -- 172.21.15.43:0/2330834586 learned_addr learned my addr 172.21.15.43:0/2330834586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 -- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 msgr2=0x5627d2317210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2317210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 -- 172.21.15.43:0/2330834586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 msgr2=0x5627d2317750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 -- 172.21.15.43:0/2330834586 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d21bfdb0 con 0x5627d22e7d90 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ae000640 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89ad600640 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 0x5627d2316cd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f899c091a70 tx=0x7f899c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e1d5158b54021a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f8999200640 1 -- 172.21.15.43:0/2330834586 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c063b60 con 0x5627d22e7d90 2024-09-17T05:29:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f8999200640 1 -- 172.21.15.43:0/2330834586 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899c063d00 con 0x5627d22e7d90 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f8999200640 1 -- 172.21.15.43:0/2330834586 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c05d420 con 0x5627d22e7d90 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 msgr2=0x5627d2316cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 0x5627d2316cd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f899c091a70 tx=0x7f899c014040 comp rx=0 tx=0).stop 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- 172.21.15.43:0/2330834586 shutdown_connections 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d1e21460 0x5627d2317210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d2317750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 --2- 172.21.15.43:0/2330834586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d22e7d90 0x5627d2316cd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- 172.21.15.43:0/2330834586 >> 172.21.15.43:0/2330834586 conn(0x5627d231e740 msgr2=0x5627d2360620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- 172.21.15.43:0/2330834586 shutdown_connections 2024-09-17T05:29:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- 172.21.15.43:0/2330834586 wait complete. 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.469+0000 7f89b7722000 1 -- start start 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 Processor -- start 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d1e21460 0x5627d2319db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d22e7d90 0x5627d2394220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627d20fe5f0 con 0x5627d21faca0 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89a81fcb80 con 0x5627d22e7d90 2024-09-17T05:29:49.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5627d21e8a80 con 0x5627d1e21460 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 10 client.0 _ll_register_callbacks cb 0x7f8990003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59852/0 (socket says 172.21.15.43:59852) 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 -- 172.21.15.43:0/2203744155 learned_addr learned my addr 172.21.15.43:0/2203744155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89ad600640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d1e21460 0x5627d2319db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89ae000640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d22e7d90 0x5627d2394220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 -- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d1e21460 msgr2=0x5627d2319db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d1e21460 0x5627d2319db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 -- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d22e7d90 msgr2=0x5627d2394220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d22e7d90 0x5627d2394220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 -- 172.21.15.43:0/2203744155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d212aab0 con 0x5627d21faca0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89acc00640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f89940a5fd0 tx=0x7f8994004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96102513a3d2b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89940c7070 con 0x5627d21faca0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f89940046d0 con 0x5627d21faca0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f899c063d00 con 0x5627d21faca0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f89940e3050 con 0x5627d21faca0 2024-09-17T05:29:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 10 client.6984 fetch_fsmap learned FSMap version 86 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x5627d2109070 con 0x5627d21faca0 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899c063d00 con 0x5627d21faca0 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f89940e6020 con 0x5627d21faca0 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 10 client.6984 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 10 client.6984 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 20 client.6984 populate_metadata read hostname 'smithi043' 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e1fe80 0x5627d23d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 4 client.6984 mds_command: new command op to 6846 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.473+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 --> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] -- command(tid 0: {"prefix": "status"}) -- 0x5627d2172540 con 0x5627d1e1fe80 2024-09-17T05:29:49.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89ad600640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e1fe80 0x5627d23d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f898e000640 10 client.6984 ms_handle_connect on v2:172.21.15.43:6833/274766899 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89ad600640 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e1fe80 0x5627d23d84c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642d69bdb23e3847 server_cookie=278f9e176d6a652c in_seq=0 out_seq=0 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f898e000640 1 -- 172.21.15.43:0/2203744155 <== mds.0 v2:172.21.15.43:6833/274766899 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5627d2172540 con 0x5627d1e1fe80 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f898e000640 10 client.6984 handle_command_reply: tid=0 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 client.6984 shutdown 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 msgr2=0x5627d231a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f89940a5fd0 tx=0x7f8994004470 comp rx=0 tx=0).stop 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 shutdown_connections 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:6833/274766899,v1:172.21.15.43:6835/274766899] conn(0x5627d1e1fe80 0x5627d23d84c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627d22e7d90 0x5627d2394220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627d21faca0 0x5627d231a2f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f898e000640 0 client.6984 ms_handle_reset on v2:172.21.15.43:6833/274766899 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 --2- 172.21.15.43:0/2203744155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627d1e21460 0x5627d2319db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 >> 172.21.15.43:0/2203744155 conn(0x5627d231e740 msgr2=0x5627d23736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 shutdown_connections 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 1 -- 172.21.15.43:0/2203744155 wait complete. 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b7722000 20 client.6984 trim_cache size 0 max 16384 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:29:49.488 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6846, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.1367615000000004, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 86, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 89, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 89, 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.492879908000001 2024-09-17T05:29:49.489 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:29:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 msgr2=0x7f8988075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 0x7f8988075740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f89940af000 tx=0x7f899401b010 comp rx=0 tx=0).stop 2024-09-17T05:29:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 msgr2=0x7f89a8155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c490 tx=0x7f899c00c950 comp rx=0 tx=0).stop 2024-09-17T05:29:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 shutdown_connections 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8988073230 0x7f8988075740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89a8155d70 0x7f89a8166c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80ca8e0 0x7f89a80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 --2- 172.21.15.43:0/2446204467 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f89a806f110 0x7f89a8155340 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 >> 172.21.15.43:0/2446204467 conn(0x7f89a80b23d0 msgr2=0x7f89a80b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 shutdown_connections 2024-09-17T05:29:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.477+0000 7f89b0000640 1 -- 172.21.15.43:0/2446204467 wait complete. 2024-09-17T05:29:49.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.653+0000 7f5865800640 1 -- 172.21.15.43:0/4096706462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600b7ce0 msgr2=0x7f58600d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.653+0000 7f5865800640 1 --2- 172.21.15.43:0/4096706462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600b7ce0 0x7f58600d0840 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f58600d23a0 tx=0x7f585001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.653+0000 7f5865800640 1 -- 172.21.15.43:0/4096706462 shutdown_connections 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 --2- 172.21.15.43:0/4096706462 >> v2:172.21.15.62:3300/0 conn(0x7f58600d0d80 0x7f58600cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 --2- 172.21.15.43:0/4096706462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600b7ce0 0x7f58600d0840 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- 172.21.15.43:0/4096706462 >> 172.21.15.43:0/4096706462 conn(0x7f58600cee80 msgr2=0x7f58600cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- 172.21.15.43:0/4096706462 shutdown_connections 2024-09-17T05:29:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- 172.21.15.43:0/4096706462 wait complete. 2024-09-17T05:29:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 Processor -- start 2024-09-17T05:29:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- start start 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 0x7f586006f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 0x7f58601fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5860169e10 con 0x7f58600cd3b0 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5860169c90 con 0x7f58600b7ce0 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.657+0000 7f5865800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5860169f90 con 0x7f58600d0d80 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59878/0 (socket says 172.21.15.43:59878) 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 -- 172.21.15.43:0/3543391165 learned_addr learned my addr 172.21.15.43:0/3543391165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f585f400640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 0x7f58601fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f585ea00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 0x7f586006f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 -- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 msgr2=0x7f58601fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 0x7f58601fbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 -- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 msgr2=0x7f586006f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 0x7f586006f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 -- 172.21.15.43:0/3543391165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586014be20 con 0x7f58600cd3b0 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f585ea00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 0x7f586006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f585f400640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 0x7f58601fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5857e00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f584800d6a0 tx=0x7f584800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848017070 con 0x7f58600cd3b0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5848009b10 con 0x7f58600cd3b0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586015c130 con 0x7f58600cd3b0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848010460 con 0x7f58600cd3b0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586015c6f0 con 0x7f58600cd3b0 2024-09-17T05:29:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58600d0840 con 0x7f58600cd3b0 2024-09-17T05:29:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.661+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f584801a070 con 0x7f58600cd3b0 2024-09-17T05:29:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.665+0000 7f5856a00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 0x7f583c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.665+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f5848094460 con 0x7f58600cd3b0 2024-09-17T05:29:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.665+0000 7f585ea00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 0x7f583c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.665+0000 7f585ea00640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 0x7f583c075710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58500af000 tx=0x7f585001b0e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.669+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f584805f3e0 con 0x7f58600cd3b0 2024-09-17T05:29:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.877+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58600af260 con 0x7f58600cd3b0 2024-09-17T05:29:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.877+0000 7f5856a00640 1 -- 172.21.15.43:0/3543391165 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f584805ece0 con 0x7f58600cd3b0 2024-09-17T05:29:49.882 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:29:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 msgr2=0x7f583c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 0x7f583c075710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58500af000 tx=0x7f585001b0e0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 msgr2=0x7f586014b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f584800d6a0 tx=0x7f584800db60 comp rx=0 tx=0).stop 2024-09-17T05:29:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 shutdown_connections 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f583c073200 0x7f583c075710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58600d0d80 0x7f58601fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600cd3b0 0x7f586014b110 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 --2- 172.21.15.43:0/3543391165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58600b7ce0 0x7f586006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 >> 172.21.15.43:0/3543391165 conn(0x7f58600cee80 msgr2=0x7f58600712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 shutdown_connections 2024-09-17T05:29:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:49.881+0000 7f5865800640 1 -- 172.21.15.43:0/3543391165 wait complete. 2024-09-17T05:29:49.904 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:29:49.905 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:29:49.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:49.905 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:29:49.915 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:29:49.916 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:29:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:29:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:49.921 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:29:49.986 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:29:49.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:49.988 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:29:50.040 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:29:50.041 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:29:50.041 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:29:50.041 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:29:50.041 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:29:50.041 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:29:50.041 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:29:50.041 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:29:50.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:29:50.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:29:50.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:29:50.042 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:29:50.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:29:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:29:50.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.183 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:29:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:29:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:29:50.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:29:50.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:29:50.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:29:50.260 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:50.312 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:29:50.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:29:50.312 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:50.367 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:29:50.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:29:50.473 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:29:50.473 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:29:50.473 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:29:50.492 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:29:50.492 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:29:50.492 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:29:50.492 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:29:50.492 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:29:50.502 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:29:50.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:29:50.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:50.526 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:29:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:29:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:29:50.632 INFO:teuthology.orchestra.run.smithi043.stdout:client6673 2024-09-17T05:29:50.633 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:50.633 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:50.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.642 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-17T05:29:50.701 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:50.701 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:50.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:50.756 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-17T05:29:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:6+0 records in 2024-09-17T05:29:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:6+0 records out 2024-09-17T05:29:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0860559 s, 73.1 MB/s 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:29:50.900 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-09-17T05:29:50.901 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:29:50.975 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:29:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "st_mode": 33204, 2024-09-17T05:29:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "st_ino": 1099511627777, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_dev": 44, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_nlink": 1, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_uid": 1000, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_gid": 1271, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_size": 6291456, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_atime": 1726550990.8094525, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_mtime": 1726550990.845452, 2024-09-17T05:29:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "st_ctime": 1726550990.845452 2024-09-17T05:29:50.977 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:29:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:29:50.985 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:50.985 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:51.038 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:51.039 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:51.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:29:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- 172.21.15.43:0/3503271332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d4c004810 msgr2=0x7f3d4c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 --2- 172.21.15.43:0/3503271332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d4c004810 0x7f3d4c004c20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009d10 tx=0x7f3d5001d440 comp rx=0 tx=0).stop 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6a600640 1 -- 172.21.15.43:0/3503271332 reap_dead start 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- 172.21.15.43:0/3503271332 shutdown_connections 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- 172.21.15.43:0/3503271332 >> 172.21.15.43:0/3503271332 conn(0x7f3d640b24c0 msgr2=0x7f3d640b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- 172.21.15.43:0/3503271332 shutdown_connections 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- 172.21.15.43:0/3503271332 wait complete. 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 Processor -- start 2024-09-17T05:29:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.293+0000 7f3d6c600640 1 -- start start 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d4c004810 0x7f3d641ed510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 0x7f3d640b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d640700e0 con 0x7f3d64139c60 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6406ff60 con 0x7f3d640cef50 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3d64070260 con 0x7f3d4c004810 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59904/0 (socket says 172.21.15.43:59904) 2024-09-17T05:29:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 -- 172.21.15.43:0/2712800061 learned_addr learned my addr 172.21.15.43:0/2712800061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69c00640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d4c004810 0x7f3d641ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6a600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 0x7f3d640b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 -- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d4c004810 msgr2=0x7f3d641ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d4c004810 0x7f3d641ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 -- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 msgr2=0x7f3d640b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 0x7f3d640b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 -- 172.21.15.43:0/2712800061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d640cf360 con 0x7f3d64139c60 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6a600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 0x7f3d640b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d69200640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3d500af040 tx=0x7f3d500098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b1040 con 0x7f3d64139c60 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d500bb070 con 0x7f3d64139c60 2024-09-17T05:29:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d641fb9a0 con 0x7f3d64139c60 2024-09-17T05:29:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50005320 con 0x7f3d64139c60 2024-09-17T05:29:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d641fbf20 con 0x7f3d64139c60 2024-09-17T05:29:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.297+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d641f91f0 con 0x7f3d64139c60 2024-09-17T05:29:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.301+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d500046d0 con 0x7f3d64139c60 2024-09-17T05:29:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.301+0000 7f3d61600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 0x7f3d44075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.301+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f3d5012f4b0 con 0x7f3d64139c60 2024-09-17T05:29:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.301+0000 7f3d69c00640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 0x7f3d44075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.301+0000 7f3d69c00640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 0x7f3d44075740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001ec0 tx=0x7f3d58006730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.305+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d640cf360 con 0x7f3d64139c60 2024-09-17T05:29:51.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.493+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3d640b1530 con 0x7f3d64139c60 2024-09-17T05:29:51.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d61600640 1 -- 172.21.15.43:0/2712800061 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7f3d50017180 con 0x7f3d64139c60 2024-09-17T05:29:51.499 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:29:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 msgr2=0x7f3d44075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 0x7f3d44075740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001ec0 tx=0x7f3d58006730 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 msgr2=0x7f3d640cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3d500af040 tx=0x7f3d500098e0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 shutdown_connections 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d44073230 0x7f3d44075740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d640cef50 0x7f3d640b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d64139c60 0x7f3d640cea10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 --2- 172.21.15.43:0/2712800061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d4c004810 0x7f3d641ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 >> 172.21.15.43:0/2712800061 conn(0x7f3d640b24c0 msgr2=0x7f3d6415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 shutdown_connections 2024-09-17T05:29:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.497+0000 7f3d6c600640 1 -- 172.21.15.43:0/2712800061 wait complete. 2024-09-17T05:29:51.521 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:29:51.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:51.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:29:51.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:29:51.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:51.572 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:29:51.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:51.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:51.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:51.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.789+0000 7fd4d7e00640 1 -- 172.21.15.43:0/324813058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 msgr2=0x7fd4d00cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.789+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/324813058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00cce70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd4d009bbe0 tx=0x7fd4bc01c820 comp rx=0 tx=0).stop 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- 172.21.15.43:0/324813058 shutdown_connections 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/324813058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00cce70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- 172.21.15.43:0/324813058 >> 172.21.15.43:0/324813058 conn(0x7fd4d00cee80 msgr2=0x7fd4d00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- 172.21.15.43:0/324813058 shutdown_connections 2024-09-17T05:29:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- 172.21.15.43:0/324813058 wait complete. 2024-09-17T05:29:51.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 Processor -- start 2024-09-17T05:29:51.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- start start 2024-09-17T05:29:51.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4d00b7ce0 0x7fd4d0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4d00cd3b0 0x7fd4d00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d016b030 con 0x7fd4d00d0f50 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d016aeb0 con 0x7fd4d00cd3b0 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.793+0000 7fd4d7e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d016b1b0 con 0x7fd4d00b7ce0 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59916/0 (socket says 172.21.15.43:59916) 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 -- 172.21.15.43:0/3361278795 learned_addr learned my addr 172.21.15.43:0/3361278795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d4a00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4d00cd3b0 0x7fd4d00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5400640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4d00b7ce0 0x7fd4d0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 -- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4d00b7ce0 msgr2=0x7fd4d0158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4d00b7ce0 0x7fd4d0158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 -- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4d00cd3b0 msgr2=0x7fd4d00b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4d00cd3b0 0x7fd4d00b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 -- 172.21.15.43:0/3361278795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d015e0c0 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d5e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400ba10 tx=0x7fd4c400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4019070 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c400cc30 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d015e3a0 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40146a0 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d015e960 con 0x7fd4d00d0f50 2024-09-17T05:29:51.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d00cce70 con 0x7fd4d00d0f50 2024-09-17T05:29:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.797+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4c401c070 con 0x7fd4d00d0f50 2024-09-17T05:29:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.801+0000 7fd4cd600640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 0x7fd4b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.801+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7fd4c4093e30 con 0x7fd4d00d0f50 2024-09-17T05:29:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.801+0000 7fd4d5400640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 0x7fd4b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.801+0000 7fd4d5400640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 0x7fd4b0075740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8007fd0 tx=0x7fd4c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:51.805+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd4c400ce20 con 0x7fd4d00d0f50 2024-09-17T05:29:52.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.017+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d00b2890 con 0x7fd4d00d0f50 2024-09-17T05:29:52.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4cd600640 1 -- 172.21.15.43:0/3361278795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5216 (secure 0 0 0) 0x7fd4c405e770 con 0x7fd4d00d0f50 2024-09-17T05:29:52.023 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:52.024 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"btime":"2024-09-17T05:29:46:366852+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:29:45.786243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6846":{"gid":6846,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/274766899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":274766899},{"type":"v1","addr":"172.21.15.43:6835","nonce":274766899}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6846,"qdb_cluster":[6846]},"id":6}]} 2024-09-17T05:29:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 86 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 msgr2=0x7fd4b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 0x7fd4b0075740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8007fd0 tx=0x7fd4c8009040 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 msgr2=0x7fd4d00b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400ba10 tx=0x7fd4c400bed0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 shutdown_connections 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4b0073230 0x7fd4b0075740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4d00cd3b0 0x7fd4d00b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4d00d0f50 0x7fd4d00b67c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 --2- 172.21.15.43:0/3361278795 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4d00b7ce0 0x7fd4d0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 >> 172.21.15.43:0/3361278795 conn(0x7fd4d00cee80 msgr2=0x7fd4d0072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 shutdown_connections 2024-09-17T05:29:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.021+0000 7fd4d7e00640 1 -- 172.21.15.43:0/3361278795 wait complete. 2024-09-17T05:29:52.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T05:29:52.231 INFO:teuthology.orchestra.run.smithi043.stdout:{"message":"","return_code":0} 2024-09-17T05:29:52.232 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T05:29:52.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:29:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 -- 172.21.15.43:0/1950627752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc070620 msgr2=0x7f49fc0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 --2- 172.21.15.43:0/1950627752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc070620 0x7f49fc0b30d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009840 tx=0x7f49ec01d9c0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 -- 172.21.15.43:0/1950627752 shutdown_connections 2024-09-17T05:29:52.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 --2- 172.21.15.43:0/1950627752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc070620 0x7f49fc0b30d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 -- 172.21.15.43:0/1950627752 >> 172.21.15.43:0/1950627752 conn(0x7f49fc0b23d0 msgr2=0x7f49fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 -- 172.21.15.43:0/1950627752 shutdown_connections 2024-09-17T05:29:52.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.389+0000 7f4a02200640 1 -- 172.21.15.43:0/1950627752 wait complete. 2024-09-17T05:29:52.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 Processor -- start 2024-09-17T05:29:52.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 -- start start 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49fc06ce60 0x7f49fc152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc070620 0x7f49fc09d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0ce820 con 0x7f49fc0cb860 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0ce6a0 con 0x7f49fc070620 2024-09-17T05:29:52.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f4a02200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0ce9a0 con 0x7f49fc06ce60 2024-09-17T05:29:52.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fb400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49fc06ce60 0x7f49fc152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59926/0 (socket says 172.21.15.43:59926) 2024-09-17T05:29:52.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 -- 172.21.15.43:0/3695550552 learned_addr learned my addr 172.21.15.43:0/3695550552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49faa00640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc070620 0x7f49fc09d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 -- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49fc06ce60 msgr2=0x7f49fc152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49fc06ce60 0x7f49fc152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 -- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc070620 msgr2=0x7f49fc09d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc070620 0x7f49fc09d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 -- 172.21.15.43:0/3695550552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc153260 con 0x7f49fc0cb860 2024-09-17T05:29:52.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.393+0000 7f49fbe00640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f49f400cb10 tx=0x7f49f400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4016020 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f400ce50 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc153530 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f400adc0 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc153af0 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc0b30d0 con 0x7f49fc0cb860 2024-09-17T05:29:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f49f4019020 con 0x7f49fc0cb860 2024-09-17T05:29:52.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 0x7f49dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f49f4094080 con 0x7f49fc0cb860 2024-09-17T05:29:52.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.397+0000 7f49fb400640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 0x7f49dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.401+0000 7f49fb400640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 0x7f49dc075740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0b0000 tx=0x7f49ec01c200 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.401+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f49f4020030 con 0x7f49fc0cb860 2024-09-17T05:29:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.617+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49fc0d04e0 con 0x7f49fc0cb860 2024-09-17T05:29:52.619 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:29:52.617+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:29:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.617+0000 7f49f3400640 1 -- 172.21.15.43:0/3695550552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) ==== 161+0+0 (secure 0 0 0) 0x7f49f405e9b0 con 0x7f49fc0cb860 2024-09-17T05:29:52.622 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:29:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 msgr2=0x7f49dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 0x7f49dc075740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0b0000 tx=0x7f49ec01c200 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 msgr2=0x7f49fc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f49f400cb10 tx=0x7f49f400cc00 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 shutdown_connections 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f49dc073230 0x7f49dc075740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49fc070620 0x7f49fc09d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49fc0cb860 0x7f49fc0ad0b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 --2- 172.21.15.43:0/3695550552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49fc06ce60 0x7f49fc152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 >> 172.21.15.43:0/3695550552 conn(0x7f49fc0b23d0 msgr2=0x7f49fc0af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 shutdown_connections 2024-09-17T05:29:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.621+0000 7f4a02200640 1 -- 172.21.15.43:0/3695550552 wait complete. 2024-09-17T05:29:52.636 INFO:tasks.ceph.mds.c.smithi043.stderr: -5019> 2024-09-17T05:29:45.325+0000 7f6cf1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:29:52.647 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-09-17T05:29:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 Processor -- start 2024-09-17T05:29:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 -- start start 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271a9d70 0x55b1271f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271d8cb0 0x55b1271ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1270f7a00 con 0x55b12725fe40 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b127155f40 con 0x55b1271d8cb0 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.701+0000 7f1f40028080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b127260890 con 0x55b1271a9d70 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59934/0 (socket says 172.21.15.43:59934) 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 -- 172.21.15.43:0/346390799 learned_addr learned my addr 172.21.15.43:0/346390799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39200640 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271a9d70 0x55b1271f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f33e00640 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271d8cb0 0x55b1271ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 -- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271a9d70 msgr2=0x55b1271f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271a9d70 0x55b1271f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 -- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271d8cb0 msgr2=0x55b1271ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271d8cb0 0x55b1271ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 -- 172.21.15.43:0/346390799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1270d0c10 con 0x55b12725fe40 2024-09-17T05:29:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f39c00640 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1f3401b940 tx=0x7f1f3401aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5024dac606d34c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f32a00640 1 -- 172.21.15.43:0/346390799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004020 con 0x55b12725fe40 2024-09-17T05:29:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f32a00640 1 -- 172.21.15.43:0/346390799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f340bb470 con 0x55b12725fe40 2024-09-17T05:29:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f32a00640 1 -- 172.21.15.43:0/346390799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f340042d0 con 0x55b12725fe40 2024-09-17T05:29:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 -- 172.21.15.43:0/346390799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 msgr2=0x55b1271eeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1f3401b940 tx=0x7f1f3401aab0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 -- 172.21.15.43:0/346390799 shutdown_connections 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271d8cb0 0x55b1271ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b1271eeeb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 --2- 172.21.15.43:0/346390799 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271a9d70 0x55b1271f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 -- 172.21.15.43:0/346390799 >> 172.21.15.43:0/346390799 conn(0x55b127195d80 msgr2=0x55b1271d8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 -- 172.21.15.43:0/346390799 shutdown_connections 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 -- 172.21.15.43:0/346390799 wait complete. 2024-09-17T05:29:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.705+0000 7f1f40028080 1 Processor -- start 2024-09-17T05:29:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- start start 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271a9d70 0x55b12725ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271d8cb0 0x55b127317960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b1271f0a20 con 0x55b12725fe40 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b12727c420 con 0x55b1271a9d70 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b1271f0cc0 con 0x55b1271d8cb0 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59950/0 (socket says 172.21.15.43:59950) 2024-09-17T05:29:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 learned_addr learned my addr 172.21.15.43:0/2110309675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271a9d70 0x55b12725ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271d8cb0 0x55b127317960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271d8cb0 msgr2=0x55b127317960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b1271d8cb0 0x55b127317960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271a9d70 msgr2=0x55b12725ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b1271a9d70 0x55b12725ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1273180b0 con 0x55b12725fe40 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1f34004fd0 tx=0x7f1f340bba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f31600640 1 -- 172.21.15.43:0/2110309675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004020 con 0x55b12725fe40 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b127318350 con 0x55b12725fe40 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f31600640 1 -- 172.21.15.43:0/2110309675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f340cb070 con 0x55b12725fe40 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f31600640 1 -- 172.21.15.43:0/2110309675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f340c56a0 con 0x55b12725fe40 2024-09-17T05:29:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.709+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b127318890 con 0x55b12725fe40 2024-09-17T05:29:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f31600640 1 -- 172.21.15.43:0/2110309675 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f340c3040 con 0x55b12725fe40 2024-09-17T05:29:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f31600640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 0x7f1f180740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f31600640 1 -- 172.21.15.43:0/2110309675 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f1f34133240 con 0x55b12725fe40 2024-09-17T05:29:52.718 INFO:teuthology.orchestra.run.smithi043.stdout:���� 2024-09-17T05:29:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: �� 2024-09-17T05:29:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 0x7f1f180740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b127318fd0 0x55b127321490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.0 12:00000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x55b127318fd0 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b127318fd0 0x55b127321490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.713+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 0x7f1f180740f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d820 tx=0x7f1f2c00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b127318fd0 0x55b127321490 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2400d040 con 0x55b127318fd0 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f1f24002550 0x7f1f24010540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:08000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24002550 2024-09-17T05:29:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f1f24002550 0x7f1f24010540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.717+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f1f24002550 0x7f1f24010540 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.721+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(2 [pgnls start_epoch 90 out=177b] v89'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f1f2c00aa40 con 0x7f1f24002550 2024-09-17T05:29:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.721+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 0x7f1f2c0168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.721+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:3 12.8 12:10000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.721+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 0x7f1f2c0168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.721+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 0x7f1f2c0168f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(3 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:4 12.18 12:18000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(4 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f1f240113d0 0x7f1f240117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:5 12.4 12:20000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f1f240113d0 0x7f1f240117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.725+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f1f240113d0 0x7f1f240117e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 1 ==== osd_op_reply(5 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:6 12.14 12:28000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x55b127318fd0 2024-09-17T05:29:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(6 [pgnls start_epoch 90 out=149b] v89'18 uv18 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:7 12.c 12:30000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(7 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f1f24026040 0x7f1f24027970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:8 12.1c 12:38000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24026040 2024-09-17T05:29:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f1f24026040 0x7f1f24027970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.729+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f1f24026040 0x7f1f24027970 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(8 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:9 12.2 12:40000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 2 ==== osd_op_reply(9 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f1f34021480 0x7f1f340fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:10 12.12 12:48000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f34021480 2024-09-17T05:29:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.733+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f1f34021480 0x7f1f340fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f1f34021480 0x7f1f340fc8f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.4 v2:172.21.15.62:6818/1078882315 1 ==== osd_op_reply(10 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f34021480 2024-09-17T05:29:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:11 12.a 12:50000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f34021480 2024-09-17T05:29:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.4 v2:172.21.15.62:6818/1078882315 2 ==== osd_op_reply(11 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f34021480 2024-09-17T05:29:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:12 12.1a 12:58000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 3 ==== osd_op_reply(12 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.6 12:60000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24002550 2024-09-17T05:29:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(13 [pgnls start_epoch 90 out=77b] v89'3 uv3 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:14 12.16 12:68000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 4 ==== osd_op_reply(14 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:15 12.e 12:70000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(15 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:16 12.1e 12:78000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 5 ==== osd_op_reply(16 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f1f3401e860 0x7f1f34106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:17 12.1 12:80000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3401e860 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f1f3401e860 0x7f1f34106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.737+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f1f3401e860 0x7f1f34106af0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(17 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f3401e860 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:18 12.11 12:88000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f240113d0 2024-09-17T05:29:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.2 v2:172.21.15.43:6808/3128122281 6 ==== osd_op_reply(18 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f240113d0 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.9 12:90000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24002550 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(19 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:20 12.19 12:98000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(20 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:21 12.5 12:a0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x55b127318fd0 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(21 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:22 12.15 12:a8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3401e860 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(22 [pgnls start_epoch 90 out=97b] v88'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f3401e860 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f1f3400b350 0x7f1f340c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:23 12.d 12:b0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3400b350 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f1f3400b350 0x7f1f340c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.741+0000 7f1f39200640 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f1f3400b350 0x7f1f340c5840 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(23 [pgnls start_epoch 90 out=74b] v88'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.1d 12:b8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24002550 2024-09-17T05:29:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(24 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:25 12.3 12:c0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f34021480 2024-09-17T05:29:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.4 v2:172.21.15.62:6818/1078882315 3 ==== osd_op_reply(25 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f34021480 2024-09-17T05:29:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:26 12.13 12:c8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f24026040 2024-09-17T05:29:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(26 [pgnls start_epoch 90 out=98b] v89'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.745+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:27 12.b 12:d0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(27 [pgnls start_epoch 90 out=76b] v89'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:28 12.1b 12:d8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x55b127318fd0 2024-09-17T05:29:52.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(28 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:29 12.7 12:e0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f2c017040 2024-09-17T05:29:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(29 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:30 12.17 12:e8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3401e860 2024-09-17T05:29:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(30 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f3401e860 2024-09-17T05:29:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:31 12.f 12:f0000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3400b350 2024-09-17T05:29:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(31 [pgnls start_epoch 90 out=73b] v88'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:32 12.1f 12:f8000000:%01:::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55b1273219d0 con 0x7f1f3400b350 2024-09-17T05:29:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(32 [pgnls start_epoch 90 out=101b] v89'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127305a20 con 0x7f1f24002550 2024-09-17T05:29:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:34 12.10 12:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127305dc0 con 0x7f1f24002550 2024-09-17T05:29:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv5 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.749+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:35 12.10 12:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127306160 con 0x7f1f24002550 2024-09-17T05:29:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv5 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:36 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127306550 con 0x7f1f24002550 2024-09-17T05:29:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:37 12.10 12:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127321d70 con 0x7f1f24002550 2024-09-17T05:29:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:38 12.10 12:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127322110 con 0x7f1f24002550 2024-09-17T05:29:52.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv7 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:39 12.10 12:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:40 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273234c0 con 0x7f1f24002550 2024-09-17T05:29:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:41 12.10 12:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:42 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:43 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:44 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273234c0 con 0x7f1f24002550 2024-09-17T05:29:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=533b] v0'0 uv10 ondisk = 0) ==== 164+0+533 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.762 INFO:teuthology.orchestra.run.smithi043.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (��f8D?0(��f8D?08`��f�d28`��f�d2���� �� sixmegs_head�i�����f�d2����@@ `����������f�d2��f8D?0(8`��f�d28`��f�d2 2024-09-17T05:29:52.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.753+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:45 12.10 12:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:46 12.10 12:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:47 12.10 12:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:48 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273234c0 con 0x7f1f24002550 2024-09-17T05:29:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:49 12.10 12:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:50 12.10 12:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:51 12.10 12:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x7f1f24002550 2024-09-17T05:29:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:52 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273234c0 con 0x7f1f24002550 2024-09-17T05:29:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:53 12.14 12:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273224b0 con 0x55b127318fd0 2024-09-17T05:29:52.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:54 12.14 12:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127322850 con 0x55b127318fd0 2024-09-17T05:29:52.768 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:29:52.768 INFO:teuthology.orchestra.run.smithi043.stdout:����������f8D?0����`������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)���f�=)�C@@ ����������f�=)(��f�=)8��f�=)8��fKV���fKV�����NH��fKV������������� �� 2024-09-17T05:29:52.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.757+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:55 12.14 12:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127322bf0 con 0x55b127318fd0 2024-09-17T05:29:52.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.769 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:56 12.14 12:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323fa0 con 0x55b127318fd0 2024-09-17T05:29:52.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:57 12.14 12:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127322f90 con 0x55b127318fd0 2024-09-17T05:29:52.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv18 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:58 12.14 12:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323380 con 0x55b127318fd0 2024-09-17T05:29:52.771 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�9@�9@�A@@@ ���� �� 2024-09-17T05:29:52.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv18 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:59 12.14 12:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.772 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:60 12.14 12:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv18 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:61 12.14 12:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ���A�A�A�A��XS&YA1e@��XS&YA1h^T��f���2 ((88 2 ( 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:( 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:8 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:��ft��8 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:��ft�� ��������@0@��XS&YA1�� 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:���fP�|Q�.�+� entity_id0hostname smithi043kernel_version6.11.0-rc5-g2ca44a2f35caroot/��7 ����@��XS&YA1�����f�:Nn���������@��XS&YA1��fr�Qsetattr� C�����������f���C@@ ����������fKV�(8��f��8��fKV���fKV�����NH��fKV�����������������[@��XS&YA1�����f}�)mkdir�3 (��f�=)(��f�=)8��f�=)8��f�=)  �subdir�����������f�=)�A��������������������f�=)��f�=)(88����������f�=)������������2 ((88  C�����������f�=)�C@@ ����������f�=)(��f�=)8��f�=)8��fKV���fKV�����NH��fKV�������������������@��XS&YA1�����f�$w0opencL3 (��f�=)(��f�=)8��f8D?08��f�=)  �subdir�����������f8D?0�A��������������������f8D?0��f�=)(��f8D?08��f8D?08��f8D?0����������f�=)������������V (��f8D?0(��f8D?08��f8D?08��f8D?00 sixmegs�����������f8D?0����@@ ����������f8D?0��f8D?0@(88����������f8D?0������������!!��������h@��XS&YA1�����f�ak1 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:cap updateT3 (��f�=)(��f�=)8 ��f}^�08��f�=)  �subdir�����������f8D?0�A��������������������f8D?0��f�=)(��f8D?08 ��f}^�08 ��f}^�0����������f�=)������������^ (��f8D?0(��f8D?08 ��f}^�08 ��f}^�08 &sixmegs�����������f}^�0����@@ ����������f}^�0��f8D?0�(8 ��f}^�08 ��f}^�0����������f8D?0���� ����������������@��XS&YA1�����f*ؔ2 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:cap updateT3  (��f�=)(��f�=)8P��f���18��f�=)  �subdir�����������f8D?0�A��������������������f8D?0��f�=)(��f8D?08P��f���18P��f���1����������f�=)������������^ (��f8D?0(��f8D?08P��f���18P��f���18 &sixmegs�����������f���1����@@ P����������f���1��f8D?0�(8P��f���18P��f���1����������f8D?0����P�����������������@��XS&YA1�����f[�Z 2024-09-17T05:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:cap updateT3  (��f�=)(��f�=)8`��f�d28��f�=)  �subdir�������� 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:���f8D?0�A��������������������f8D?0��f�=)(��f8D?08`��f�d28`��f�d2 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:����������f�=)������������^  (��f8D?0(��f8D?08`��f�d28`��f�d28 &sixmegs�����������f�d2����@@ `����������f�d2��f8D?0(8`��f�d28`��f�d2����������f8D?0����`����������������}'@��XS&YA1�� 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:���f�;� |Q�.�+���3/@��XS&YA1s i _ ��f�t�!check_inode_max_size 3  (��f�=)(��f�=)8`��f�d28��f�=)  �subdir�������� ���f8D?0�A��������������������f8D?0��f�=)(��f8D?08`��f�d28`��f�d2 ����������f�=)������������>  (��f8D?0(��f8D?08`��f�d28`��f�d2 sixmegs�������� 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:���f�d2����@@ `����������f�d2��f8D?0(8`��f�d28`��f�d2 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:����������f8D?0����`�������� sixmegs�������� 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:���f�d2����@@ `����������f�d2��f8D?0(8`��f�d28`��f�d2 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:����������f8D?0����`�����������������/@��XS&YA18.$�g9@��XS&YA1h^T��f�� 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout: 2  (��f�=)(��f�=)8`��f�d28��f�=) 2 ( 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:( 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:8 2024-09-17T05:29:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:��ft��8 2024-09-17T05:29:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(61 200.00000001 [read 0~16801 out=16801b] v0'0 uv16 ondisk = 0) ==== 156+0+16801 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:62 12.14 12:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.776 INFO:teuthology.orchestra.run.smithi043.stdout:��ft�� ��������@�9@��XS&YA1^TJ��f��; scatter_writebehind2 (��f�=)(��f�=)8`��f�d28`��f�d2  C�����������f�=)�C@@ ����������f�=)(��f�=)8`��f�d28��fKV���fKV�����NH��fKV�����������������/=@���� �� 2024-09-17T05:29:52.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:63 12.14 12:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.777 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:64 12.14 12:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.761+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:65 12.14 12:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:66 12.14 12:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:67 12.14 12:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:68 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x55b127318fd0 2024-09-17T05:29:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x55b127318fd0 2024-09-17T05:29:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:69 12.1c 12:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24026040 2024-09-17T05:29:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:70 12.1c 12:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24026040 2024-09-17T05:29:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:71 12.1c 12:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24026040 2024-09-17T05:29:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:72 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24026040 2024-09-17T05:29:52.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.765+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:73 12.6 12:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24002550 2024-09-17T05:29:52.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv3 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:74 12.6 12:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24002550 2024-09-17T05:29:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.784 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-09-17T05:29:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:75 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24002550 2024-09-17T05:29:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv3 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:76 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f24002550 2024-09-17T05:29:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:77 12.e 12:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f2c017040 2024-09-17T05:29:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:78 12.e 12:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f2c017040 2024-09-17T05:29:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:79 12.e 12:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f2c017040 2024-09-17T05:29:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:80 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f2c017040 2024-09-17T05:29:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.769+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:81 12.15 12:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323720 con 0x7f1f3401e860 2024-09-17T05:29:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f3401e860 2024-09-17T05:29:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:82 12.15 12:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127323af0 con 0x7f1f3401e860 2024-09-17T05:29:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f34002050 con 0x7f1f3401e860 2024-09-17T05:29:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:83 12.15 12:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127324340 con 0x7f1f3401e860 2024-09-17T05:29:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:84 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b1273257f0 con 0x7f1f3401e860 2024-09-17T05:29:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:85 12.15 12:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127324760 con 0x7f1f3401e860 2024-09-17T05:29:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:86 12.15 12:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127324b80 con 0x7f1f3401e860 2024-09-17T05:29:52.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.791 INFO:teuthology.orchestra.run.smithi043.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-09-17T05:29:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:87 12.15 12:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325060 con 0x7f1f3401e860 2024-09-17T05:29:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.792 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:88 12.15 12:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3401e860 2024-09-17T05:29:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f34111040 con 0x7f1f3401e860 2024-09-17T05:29:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:89 12.d 12:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:90 12.d 12:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.794 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-09-17T05:29:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:91 12.d 12:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.795 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:92 12.d 12:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.773+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:93 12.1d 12:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24002550 2024-09-17T05:29:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:94 12.1d 12:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24002550 2024-09-17T05:29:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:95 12.1d 12:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24002550 2024-09-17T05:29:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:96 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24002550 2024-09-17T05:29:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24002550 2024-09-17T05:29:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.800 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-09-17T05:29:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.800 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.777+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:101 12.13 12:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:102 12.13 12:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.802 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-09-17T05:29:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:103 12.13 12:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ( 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stdout:( 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stdout:8 2024-09-17T05:29:52.803 INFO:teuthology.orchestra.run.smithi043.stdout:��ft��8 2024-09-17T05:29:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:104 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f24026040 2024-09-17T05:29:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f24026040 2024-09-17T05:29:52.804 INFO:teuthology.orchestra.run.smithi043.stdout:��ft������ ��PJ 2024-09-17T05:29:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.b 12:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.b 12:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.805 INFO:teuthology.orchestra.run.smithi043.stdout: stray0_head�i�����f�d�@A������������������f�d�(8��f�d�8��f�d���f�d������������� stray1_head�i�����f��@A������������������f��(8��f��8��f����f�������������� stray2_head�i�����f�@A������������������f�(8��f�8��f���f������������� stray3_head�i�����f+��@A������������������f+��(8��f+��8��f+����f+�������������� stray4_head�i�����f�N�@A������������������f�N�(8��f�N�8��f�N���f�N������������� stray5_head�i�����fF~�@A������������������fF~�(8��fF~�8��fF~���fF~������������� stray6_head�i�����f�@A������������������f�(8��f�8��f���f������������� stray7_head�i�����f�}�@A������������������f�}�(8��f�}�8��f�}���f�}������������� stray8_head�i�����f���@A������������������f���(8��f���8��f�����f��������������� stray9_head�i��� ��ft��@A������������������ft��(8��ft��8��ft����ft������������������ �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-09-17T05:29:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127326c20 con 0x7f1f2c017040 2024-09-17T05:29:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.781+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 17 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:111 12.7 12:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325c10 con 0x7f1f2c017040 2024-09-17T05:29:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 18 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:112 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f2c017040 2024-09-17T05:29:52.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f33e00640 1 -- 172.21.15.43:0/2110309675 <== osd.7 v2:172.21.15.62:6810/1426001971 19 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2401f040 con 0x7f1f2c017040 2024-09-17T05:29:52.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.810 INFO:teuthology.orchestra.run.smithi043.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-09-17T05:29:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.811 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.f 12:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.785+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.812 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)���f�Ϫ@A������������������f�Ϫ( 2024-09-17T05:29:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.813 INFO:teuthology.orchestra.run.smithi043.stdout:8 ��ft��8 ��ft����f�Ϫ����NH������������ �� 2024-09-17T05:29:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.814 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� 2024-09-17T05:29:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.814 INFO:teuthology.orchestra.run.smithi043.stdout:���� �� ��("+ 2024-09-17T05:29:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b127325020 con 0x7f1f3400b350 2024-09-17T05:29:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39200640 1 -- 172.21.15.43:0/2110309675 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=524b] v0'0 uv6 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x7f1f2c0097d0 con 0x7f1f3400b350 2024-09-17T05:29:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b127318fd0 msgr2=0x55b127321490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b127318fd0 0x55b127321490 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f1f3401e860 msgr2=0x7f1f34106af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f1f3401e860 0x7f1f34106af0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f1f240113d0 msgr2=0x7f1f240117e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f1f240113d0 0x7f1f240117e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f1f3400b350 msgr2=0x7f1f340c5840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f1f3400b350 0x7f1f340c5840 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f1f34021480 msgr2=0x7f1f340fc8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f1f34021480 0x7f1f340fc8f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f1f24026040 msgr2=0x7f1f24027970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f1f24026040 0x7f1f24027970 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 reap_dead start 2024-09-17T05:29:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f1f24002550 msgr2=0x7f1f24010540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f1f24002550 0x7f1f24010540 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 reap_dead start 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f39c00640 1 -- 172.21.15.43:0/2110309675 reap_dead start 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 msgr2=0x7f1f2c0168f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 0x7f1f2c0168f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 msgr2=0x7f1f180740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 0x7f1f180740f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d820 tx=0x7f1f2c00f040 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 msgr2=0x55b127262380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1f34004fd0 tx=0x7f1f340bba70 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 shutdown_connections 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f1f2c017040 0x7f1f2c0168f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f18071be0 0x7f1f180740f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 --2- 172.21.15.43:0/2110309675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12725fe40 0x55b127262380 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 >> 172.21.15.43:0/2110309675 conn(0x55b127195d80 msgr2=0x55b1271d8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 shutdown_connections 2024-09-17T05:29:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.789+0000 7f1f40028080 1 -- 172.21.15.43:0/2110309675 wait complete. 2024-09-17T05:29:52.821 INFO:teuthology.orchestra.run.smithi043.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@ _parent ���� ��  (��f�=)(��f�=)8`��f�d28��f�=)���� ��  subdir_head�i�����f8D?0�A��������������������f8D?0��f�=)(��f8D?08`��f�d28`��f�d2 ����������f�=)���������������� �� �� 2024-09-17T05:29:52.821 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-09-17T05:29:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.873+0000 7f7995fa9080 1 Processor -- start 2024-09-17T05:29:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.873+0000 7f7995fa9080 1 -- start start 2024-09-17T05:29:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 0x55bd9484fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd947b99e0 0x55bd947b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd946bda00 con 0x55bd948284d0 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bd9471bf40 con 0x55bd9476f360 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55bd9482ad10 con 0x55bd947b99e0 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59968/0 (socket says 172.21.15.43:59968) 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 -- 172.21.15.43:0/1041756368 learned_addr learned my addr 172.21.15.43:0/1041756368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798f200640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 0x55bd9484fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798e800640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd947b99e0 0x55bd947b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 -- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd947b99e0 msgr2=0x55bd947b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd947b99e0 0x55bd947b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 -- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 msgr2=0x55bd9484fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 0x55bd9484fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 -- 172.21.15.43:0/1041756368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd94696c10 con 0x55bd948284d0 2024-09-17T05:29:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798f200640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 0x55bd9484fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798fc00640 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f79880015c0 tx=0x7f79880019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a8d718b31c73b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798d400640 1 -- 172.21.15.43:0/1041756368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988017070 con 0x55bd948284d0 2024-09-17T05:29:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798d400640 1 -- 172.21.15.43:0/1041756368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79880027b0 con 0x55bd948284d0 2024-09-17T05:29:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f798d400640 1 -- 172.21.15.43:0/1041756368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988002c40 con 0x55bd948284d0 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- 172.21.15.43:0/1041756368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 msgr2=0x55bd947b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f79880015c0 tx=0x7f79880019c0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- 172.21.15.43:0/1041756368 shutdown_connections 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b7600 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd947b99e0 0x55bd947b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 --2- 172.21.15.43:0/1041756368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9476f360 0x55bd9484fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- 172.21.15.43:0/1041756368 >> 172.21.15.43:0/1041756368 conn(0x55bd9475bd80 msgr2=0x55bd948278e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- 172.21.15.43:0/1041756368 shutdown_connections 2024-09-17T05:29:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.877+0000 7f7995fa9080 1 -- 172.21.15.43:0/1041756368 wait complete. 2024-09-17T05:29:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 Processor -- start 2024-09-17T05:29:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- start start 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd9476f360 0x55bd948dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd947b99e0 0x55bd94820630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bd947b9500 con 0x55bd948284d0 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bd947b9260 con 0x55bd947b99e0 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55bd94825cf0 con 0x55bd9476f360 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798f200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd9476f360 0x55bd948dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59982/0 (socket says 172.21.15.43:59982) 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 learned_addr learned my addr 172.21.15.43:0/3345211888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd947b99e0 0x55bd94820630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd9476f360 msgr2=0x55bd948dcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bd9476f360 0x55bd948dcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd947b99e0 msgr2=0x55bd94820630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd947b99e0 0x55bd94820630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd948dd290 con 0x55bd948284d0 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd947b99e0 0x55bd94820630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7988001ec0 tx=0x7f79880126c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7987e00640 1 -- 172.21.15.43:0/3345211888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988021020 con 0x55bd948284d0 2024-09-17T05:29:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7987e00640 1 -- 172.21.15.43:0/3345211888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7988018040 con 0x55bd948284d0 2024-09-17T05:29:52.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd948dd530 con 0x55bd948284d0 2024-09-17T05:29:52.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7987e00640 1 -- 172.21.15.43:0/3345211888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988017070 con 0x55bd948284d0 2024-09-17T05:29:52.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.881+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd948dda70 con 0x55bd948284d0 2024-09-17T05:29:52.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f7987e00640 1 -- 172.21.15.43:0/3345211888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7988024020 con 0x55bd948284d0 2024-09-17T05:29:52.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f7987e00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 0x7f79700740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f7987e00640 1 -- 172.21.15.43:0/3345211888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f7988093fd0 con 0x55bd948284d0 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55bd948df230 0x55bd948de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x55bd948df230 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 0x7f79700740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55bd948df230 0x55bd948de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 0x7f79700740f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f798000c6f0 tx=0x7f798000f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.885+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55bd948df230 0x55bd948de1b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x55bd948df230 2024-09-17T05:29:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f797c01bd50 0x7f797c0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c01bd50 2024-09-17T05:29:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f797c01bd50 0x7f797c0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f797c01bd50 0x7f797c0aa3d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(2 [pgnls start_epoch 90 out=177b] v89'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f798000a730 con 0x7f797c01bd50 2024-09-17T05:29:52.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 0x7f798000b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798000b040 2024-09-17T05:29:52.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 0x7f798000b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 0x7f798000b450 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(3 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798000b040 2024-09-17T05:29:52.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(4 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f797c00b040 0x7f797c0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.889+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c00b040 2024-09-17T05:29:52.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f797c00b040 0x7f797c0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f797c00b040 0x7f797c0c4400 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 1 ==== osd_op_reply(5 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x55bd948df230 2024-09-17T05:29:52.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(6 [pgnls start_epoch 90 out=149b] v89'18 uv18 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f797c0ab980 con 0x55bd948df230 2024-09-17T05:29:52.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798000b040 2024-09-17T05:29:52.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(7 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f797c0b53b0 0x7f797c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c0b53b0 2024-09-17T05:29:52.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f797c0b53b0 0x7f797c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f797c0b53b0 0x7f797c0b57c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(8 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f798000a730 con 0x7f797c0b53b0 2024-09-17T05:29:52.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c00b040 2024-09-17T05:29:52.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 2 ==== osd_op_reply(9 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f798805ecb0 0x7f798805d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798805ecb0 2024-09-17T05:29:52.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.893+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f798805ecb0 0x7f798805d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f798805ecb0 0x7f798805d720 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.4 v2:172.21.15.62:6818/1078882315 1 ==== osd_op_reply(10 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798805ecb0 2024-09-17T05:29:52.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798805ecb0 2024-09-17T05:29:52.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.4 v2:172.21.15.62:6818/1078882315 2 ==== osd_op_reply(11 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798805ecb0 2024-09-17T05:29:52.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c00b040 2024-09-17T05:29:52.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 3 ==== osd_op_reply(12 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c01bd50 2024-09-17T05:29:52.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(13 [pgnls start_epoch 90 out=77b] v89'3 uv3 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f798000a730 con 0x7f797c01bd50 2024-09-17T05:29:52.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f797c00b040 2024-09-17T05:29:52.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 4 ==== osd_op_reply(14 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55bd948de6f0 con 0x7f798000b040 2024-09-17T05:29:52.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(15 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f797c00b040 2024-09-17T05:29:52.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 5 ==== osd_op_reply(16 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f7988067ea0 0x7f7988065620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f7988067ea0 2024-09-17T05:29:52.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f7988067ea0 0x7f7988065620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.897+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f7988067ea0 0x7f7988065620 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(17 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f7988067ea0 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f797c00b040 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.2 v2:172.21.15.43:6808/3128122281 6 ==== osd_op_reply(18 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f797c00b040 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f797c01bd50 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(19 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798000a730 con 0x7f797c01bd50 2024-09-17T05:29:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798000b040 2024-09-17T05:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(20 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x55bd948df230 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(21 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x55bd948df230 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f7988067ea0 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(22 [pgnls start_epoch 90 out=97b] v88'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f798800f070 con 0x7f7988067ea0 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f798807eba0 0x7f7988092860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798807eba0 2024-09-17T05:29:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f798807eba0 0x7f7988092860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f798807eba0 0x7f7988092860 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(23 [pgnls start_epoch 90 out=74b] v88'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f798000a730 con 0x7f798807eba0 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f797c01bd50 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(24 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f798000a730 con 0x7f797c01bd50 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.901+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798805ecb0 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.4 v2:172.21.15.62:6818/1078882315 3 ==== osd_op_reply(25 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798805ecb0 2024-09-17T05:29:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f797c0b53b0 2024-09-17T05:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(26 [pgnls start_epoch 90 out=98b] v89'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f798000a730 con 0x7f797c0b53b0 2024-09-17T05:29:52.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798000b040 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(27 [pgnls start_epoch 90 out=76b] v89'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x55bd948df230 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(28 [pgnls start_epoch 90 out=49b] v88'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f797c0ab980 con 0x55bd948df230 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798000b040 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(29 [pgnls start_epoch 90 out=73b] v88'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f797c0ab980 con 0x7f798000b040 2024-09-17T05:29:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798e800640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f7988067ea0 2024-09-17T05:29:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(30 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f798800f070 con 0x7f7988067ea0 2024-09-17T05:29:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798807eba0 2024-09-17T05:29:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(31 [pgnls start_epoch 90 out=73b] v88'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f798000a730 con 0x7f798807eba0 2024-09-17T05:29:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x7f797c0df060 con 0x7f798807eba0 2024-09-17T05:29:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798f200640 1 -- 172.21.15.43:0/3345211888 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(32 [pgnls start_epoch 90 out=101b] v89'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f798000a730 con 0x7f798807eba0 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:601.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:602.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:10000000000.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:600.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:603.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:1.00000000.inode 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:200.00000000 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:200.00000001 2024-09-17T05:29:52.911 INFO:teuthology.orchestra.run.smithi043.stdout:606.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:607.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:mds0_openfiles.0 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:608.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:604.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:500.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:mds_snaptable 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:605.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:mds0_inotable 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:100.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:mds0_sessionmap 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:609.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:400.00000000 2024-09-17T05:29:52.912 INFO:teuthology.orchestra.run.smithi043.stdout:100.00000000.inode 2024-09-17T05:29:52.913 INFO:teuthology.orchestra.run.smithi043.stdout:1.00000000 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55bd948df230 msgr2=0x55bd948de1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55bd948df230 0x55bd948de1b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f7988067ea0 msgr2=0x7f7988065620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f7988067ea0 0x7f7988065620 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f797c00b040 msgr2=0x7f797c0c4400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7f797c00b040 0x7f797c0c4400 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f798807eba0 msgr2=0x7f7988092860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f798807eba0 0x7f7988092860 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f798805ecb0 msgr2=0x7f798805d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f798805ecb0 0x7f798805d720 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f797c0b53b0 msgr2=0x7f797c0b57c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f797c0b53b0 0x7f797c0b57c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 reap_dead start 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f797c01bd50 msgr2=0x7f797c0aa3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f797c01bd50 0x7f797c0aa3d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 msgr2=0x7f798000b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 0x7f798000b450 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f798fc00640 1 -- 172.21.15.43:0/3345211888 reap_dead start 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 msgr2=0x7f79700740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 0x7f79700740f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f798000c6f0 tx=0x7f798000f040 comp rx=0 tx=0).stop 2024-09-17T05:29:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 msgr2=0x55bd947b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7988001ec0 tx=0x7f79880126c0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 shutdown_connections 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f798000b040 0x7f798000b450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7970071be0 0x7f79700740f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 --2- 172.21.15.43:0/3345211888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bd948284d0 0x55bd947b8540 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 >> 172.21.15.43:0/3345211888 conn(0x55bd9475bd80 msgr2=0x55bd948db8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 shutdown_connections 2024-09-17T05:29:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.905+0000 7f7995fa9080 1 -- 172.21.15.43:0/3345211888 wait complete. 2024-09-17T05:29:52.917 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-09-17T05:29:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 Processor -- start 2024-09-17T05:29:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- start start 2024-09-17T05:29:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd275cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561fd26a9a00 con 0x561fd27609c0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561fd2707f40 con 0x561fd28129c0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561fd27a1910 con 0x561fd28125b0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60000/0 (socket says 172.21.15.43:60000) 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 -- 172.21.15.43:0/2022544638 learned_addr learned my addr 172.21.15.43:0/2022544638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfee000640 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd275cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfef400640 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 -- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 msgr2=0x561fd275cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd275cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 -- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd28129c0 msgr2=0x561fd2811180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 -- 172.21.15.43:0/2022544638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2682c10 con 0x561fd27609c0 2024-09-17T05:29:52.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfeea00640 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b1f0 tx=0x7fbfe000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b127a9aa814cd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfecc00640 1 -- 172.21.15.43:0/2022544638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000a040 con 0x561fd27609c0 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfecc00640 1 -- 172.21.15.43:0/2022544638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe000f620 con 0x561fd27609c0 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbfecc00640 1 -- 172.21.15.43:0/2022544638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000fa70 con 0x561fd27609c0 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- 172.21.15.43:0/2022544638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 msgr2=0x561fd28371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b1f0 tx=0x7fbfe000b6b0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- 172.21.15.43:0/2022544638 shutdown_connections 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd275cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 --2- 172.21.15.43:0/2022544638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd27609c0 0x561fd28371c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- 172.21.15.43:0/2022544638 >> 172.21.15.43:0/2022544638 conn(0x561fd2747d80 msgr2=0x561fd27a3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- 172.21.15.43:0/2022544638 shutdown_connections 2024-09-17T05:29:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.973+0000 7fbff58b6080 1 -- 172.21.15.43:0/2022544638 wait complete. 2024-09-17T05:29:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 Processor -- start 2024-09-17T05:29:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 -- start start 2024-09-17T05:29:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd27609c0 0x561fd280d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd280c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561fd2832000 con 0x561fd28129c0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561fd279d440 con 0x561fd27609c0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561fd279d6e0 con 0x561fd28125b0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd27609c0 0x561fd280d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfee000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd280c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 0x561fd2811180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60012/0 (socket says 172.21.15.43:60012) 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd27609c0 0x561fd280d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:50890/0 (socket says 172.21.15.43:50890) 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 -- 172.21.15.43:0/3790848518 learned_addr learned my addr 172.21.15.43:0/3790848518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 -- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 msgr2=0x561fd280c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561fd28125b0 0x561fd280c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 -- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd27609c0 msgr2=0x561fd280d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fd27609c0 0x561fd280d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 -- 172.21.15.43:0/3790848518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd28cc0c0 con 0x561fd28129c0 2024-09-17T05:29:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfef400640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 0x561fd2811180 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800f6e0 tx=0x7fbfe800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfe7400640 1 -- 172.21.15.43:0/3790848518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8004220 con 0x561fd28129c0 2024-09-17T05:29:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfe7400640 1 -- 172.21.15.43:0/3790848518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe8014070 con 0x561fd28129c0 2024-09-17T05:29:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbfe7400640 1 -- 172.21.15.43:0/3790848518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8010b00 con 0x561fd28129c0 2024-09-17T05:29:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.977+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561fd282ea10 con 0x561fd28129c0 2024-09-17T05:29:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.981+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd282ef50 con 0x561fd28129c0 2024-09-17T05:29:52.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.981+0000 7fbfe7400640 1 -- 172.21.15.43:0/3790848518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbfe8010ca0 con 0x561fd28129c0 2024-09-17T05:29:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.981+0000 7fbfe7400640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfd0071be0 0x7fbfd00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.981+0000 7fbfe7400640 1 -- 172.21.15.43:0/3790848518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fbfe80937d0 con 0x561fd28129c0 2024-09-17T05:29:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfeea00640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfd0071be0 0x7fbfd00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561fd28116c0 0x561fd28ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x561fd28cf060 con 0x561fd28116c0 2024-09-17T05:29:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfee000640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561fd28116c0 0x561fd28ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfeea00640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfd0071be0 0x7fbfd00740f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0003040 tx=0x7fbfe0000ee0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfee000640 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561fd28116c0 0x561fd28ceb20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfee000640 1 -- 172.21.15.43:0/3790848518 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7fbfdc0acaa0 con 0x561fd28116c0 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:header (274 bytes) : 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000020 00 00 00 00 ce 13 e9 66 38 44 3f 30 01 00 00 00 |.......f8D?0....| 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-17T05:29:52.989 INFO:teuthology.orchestra.run.smithi043.stdout:00000050 00 00 ce 13 e9 66 38 44 3f 30 01 00 00 00 00 00 |.....f8D?0......| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000a0 00 00 00 00 00 00 00 00 ce 13 e9 66 06 93 64 32 |...........f..d2| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000e0 00 00 00 00 00 00 ce 13 e9 66 06 93 64 32 00 00 |.........f..d2..| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000110 00 00 |..| 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout:00000112 2024-09-17T05:29:52.990 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:52.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561fd28116c0 msgr2=0x561fd28ceb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561fd28116c0 0x561fd28ceb20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfd0071be0 msgr2=0x7fbfd00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfd0071be0 0x7fbfd00740f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0003040 tx=0x7fbfe0000ee0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 msgr2=0x561fd2811180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 --2- 172.21.15.43:0/3790848518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561fd28129c0 0x561fd2811180 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800f6e0 tx=0x7fbfe800fae0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbfef400640 1 -- 172.21.15.43:0/3790848518 reap_dead start 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 shutdown_connections 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 >> 172.21.15.43:0/3790848518 conn(0x561fd2747d80 msgr2=0x561fd27a3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 shutdown_connections 2024-09-17T05:29:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:52.985+0000 7fbff58b6080 1 -- 172.21.15.43:0/3790848518 wait complete. 2024-09-17T05:29:52.994 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-09-17T05:29:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 Processor -- start 2024-09-17T05:29:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- start start 2024-09-17T05:29:53.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa766acb0 0x55daa770ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f3010 0x55daa7716250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daa7589a00 con 0x55daa76f48b0 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55daa75e7f40 con 0x55daa766acb0 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55daa763e760 con 0x55daa76f3010 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60032/0 (socket says 172.21.15.43:60032) 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cb600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f3010 0x55daa7716250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 -- 172.21.15.43:0/65058700 learned_addr learned my addr 172.21.15.43:0/65058700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cc000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa766acb0 0x55daa770ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 -- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f3010 msgr2=0x55daa7716250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f3010 0x55daa7716250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 -- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa766acb0 msgr2=0x55daa770ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa766acb0 0x55daa770ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 -- 172.21.15.43:0/65058700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa7562c10 con 0x55daa76f48b0 2024-09-17T05:29:53.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55cca00640 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c720 tx=0x7f55c400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4689c94ee8c016 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55ca200640 1 -- 172.21.15.43:0/65058700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4004020 con 0x55daa76f48b0 2024-09-17T05:29:53.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55ca200640 1 -- 172.21.15.43:0/65058700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c40042b0 con 0x55daa76f48b0 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55ca200640 1 -- 172.21.15.43:0/65058700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400aa80 con 0x55daa76f48b0 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- 172.21.15.43:0/65058700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 msgr2=0x55daa7716790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c720 tx=0x7f55c400cbe0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- 172.21.15.43:0/65058700 shutdown_connections 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa76f48b0 0x55daa7716790 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f3010 0x55daa7716250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 --2- 172.21.15.43:0/65058700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa766acb0 0x55daa770ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- 172.21.15.43:0/65058700 >> 172.21.15.43:0/65058700 conn(0x55daa7627d80 msgr2=0x55daa770ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- 172.21.15.43:0/65058700 shutdown_connections 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.049+0000 7f55d2de8080 1 -- 172.21.15.43:0/65058700 wait complete. 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 Processor -- start 2024-09-17T05:29:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 -- start start 2024-09-17T05:29:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 0x55daa7713d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa76f3010 0x55daa770bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f48b0 0x55daa7683250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55daa770d6f0 con 0x55daa766acb0 2024-09-17T05:29:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55daa77133f0 con 0x55daa76f3010 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55daa770c760 con 0x55daa76f48b0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 0x55daa7713d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 0x55daa7713d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60048/0 (socket says 172.21.15.43:60048) 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 -- 172.21.15.43:0/2474375372 learned_addr learned my addr 172.21.15.43:0/2474375372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cb600640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa76f3010 0x55daa770bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cca00640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f48b0 0x55daa7683250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 -- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f48b0 msgr2=0x55daa7683250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55daa76f48b0 0x55daa7683250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 -- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa76f3010 msgr2=0x55daa770bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55daa76f3010 0x55daa770bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 -- 172.21.15.43:0/2474375372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa7714530 con 0x55daa766acb0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55cc000640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 0x55daa7713d40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f55bc004f60 tx=0x7f55bc0aba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55c8e00640 1 -- 172.21.15.43:0/2474375372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0abd40 con 0x55daa766acb0 2024-09-17T05:29:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55c8e00640 1 -- 172.21.15.43:0/2474375372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55bc0bb070 con 0x55daa766acb0 2024-09-17T05:29:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55c8e00640 1 -- 172.21.15.43:0/2474375372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0b5680 con 0x55daa766acb0 2024-09-17T05:29:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.053+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55daa77147d0 con 0x55daa766acb0 2024-09-17T05:29:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa7714280 con 0x55daa766acb0 2024-09-17T05:29:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55c8e00640 1 -- 172.21.15.43:0/2474375372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55bc0b3040 con 0x55daa766acb0 2024-09-17T05:29:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55c8e00640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55ac071c10 0x7f55ac074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55c8e00640 1 -- 172.21.15.43:0/2474375372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f55bc12da90 con 0x55daa766acb0 2024-09-17T05:29:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55cb600640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55ac071c10 0x7f55ac074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55d2de8080 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55daa7681b40 0x55daa7719940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.057+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55daa763bc20 con 0x55daa7681b40 2024-09-17T05:29:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55cca00640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55daa7681b40 0x55daa7719940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55cb600640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55ac071c10 0x7f55ac074120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f55b8011570 tx=0x7f55b8001740 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55cca00640 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55daa7681b40 0x55daa7719940 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55cca00640 1 -- 172.21.15.43:0/2474375372 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f55c4002030 con 0x55daa7681b40 2024-09-17T05:29:53.064 INFO:teuthology.orchestra.run.smithi043.stdout:header (274 bytes) : 2024-09-17T05:29:53.064 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.064 INFO:teuthology.orchestra.run.smithi043.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000110 00 00 |..| 2024-09-17T05:29:53.065 INFO:teuthology.orchestra.run.smithi043.stdout:00000112 2024-09-17T05:29:53.066 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55daa7681b40 msgr2=0x55daa7719940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55daa7681b40 0x55daa7719940 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55ac071c10 msgr2=0x7f55ac074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f55ac071c10 0x7f55ac074120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f55b8011570 tx=0x7f55b8001740 comp rx=0 tx=0).stop 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 msgr2=0x55daa7713d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 --2- 172.21.15.43:0/2474375372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55daa766acb0 0x55daa7713d40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f55bc004f60 tx=0x7f55bc0aba90 comp rx=0 tx=0).stop 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55cca00640 1 -- 172.21.15.43:0/2474375372 reap_dead start 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 shutdown_connections 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 >> 172.21.15.43:0/2474375372 conn(0x55daa7627d80 msgr2=0x55daa76f2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 shutdown_connections 2024-09-17T05:29:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.061+0000 7f55d2de8080 1 -- 172.21.15.43:0/2474375372 wait complete. 2024-09-17T05:29:53.069 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-09-17T05:29:53.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.121+0000 7fbab99c0080 1 Processor -- start 2024-09-17T05:29:53.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- start start 2024-09-17T05:29:53.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab3facb0 0x5648ab49cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab49d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648ab377f40 con 0x5648ab4848b0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648ab384d00 con 0x5648ab3facb0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5648ab319a00 con 0x5648ab483010 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60062/0 (socket says 172.21.15.43:60062) 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 -- 172.21.15.43:0/1130441517 learned_addr learned my addr 172.21.15.43:0/1130441517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab2200640 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab49d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab2c00640 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab3facb0 0x5648ab49cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 -- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 msgr2=0x5648ab49d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab49d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 -- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab3facb0 msgr2=0x5648ab49cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab3facb0 0x5648ab49cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 -- 172.21.15.43:0/1130441517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ab49b790 con 0x5648ab4848b0 2024-09-17T05:29:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab3600640 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0015d0 tx=0x7fbaac006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ee47ead1b07ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab0e00640 1 -- 172.21.15.43:0/1130441517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac017070 con 0x5648ab4848b0 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab0e00640 1 -- 172.21.15.43:0/1130441517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaac0027b0 con 0x5648ab4848b0 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab0e00640 1 -- 172.21.15.43:0/1130441517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac002c40 con 0x5648ab4848b0 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- 172.21.15.43:0/1130441517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 msgr2=0x5648ab49eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0015d0 tx=0x7fbaac006f90 comp rx=0 tx=0).stop 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- 172.21.15.43:0/1130441517 shutdown_connections 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab4848b0 0x5648ab49eaa0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab49d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 --2- 172.21.15.43:0/1130441517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab3facb0 0x5648ab49cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- 172.21.15.43:0/1130441517 >> 172.21.15.43:0/1130441517 conn(0x5648ab3b7d80 msgr2=0x5648ab4834b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- 172.21.15.43:0/1130441517 shutdown_connections 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.125+0000 7fbab99c0080 1 -- 172.21.15.43:0/1130441517 wait complete. 2024-09-17T05:29:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 Processor -- start 2024-09-17T05:29:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- start start 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 0x5648ab471580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab496fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab4848b0 0x5648ab4a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5648ab482c50 con 0x5648ab3facb0 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648ab484f60 con 0x5648ab4848b0 2024-09-17T05:29:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5648ab49d820 con 0x5648ab483010 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 0x5648ab471580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 0x5648ab471580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60066/0 (socket says 172.21.15.43:60066) 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 -- 172.21.15.43:0/2500135870 learned_addr learned my addr 172.21.15.43:0/2500135870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2200640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab496fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab3600640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab4848b0 0x5648ab4a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 -- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 msgr2=0x5648ab496fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab496fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 -- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab4848b0 msgr2=0x5648ab4a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab4848b0 0x5648ab4a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 -- 172.21.15.43:0/2500135870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ab3cb6e0 con 0x5648ab3facb0 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2200640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5648ab483010 0x5648ab496fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab3600640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5648ab4848b0 0x5648ab4a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab2c00640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 0x5648ab471580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4018d10 tx=0x7fbaa40ab690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbaab400640 1 -- 172.21.15.43:0/2500135870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40ba020 con 0x5648ab3facb0 2024-09-17T05:29:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbaab400640 1 -- 172.21.15.43:0/2500135870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa40abc30 con 0x5648ab3facb0 2024-09-17T05:29:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5648ab3cb980 con 0x5648ab3facb0 2024-09-17T05:29:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbaab400640 1 -- 172.21.15.43:0/2500135870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40b63f0 con 0x5648ab3facb0 2024-09-17T05:29:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.129+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ab3cbec0 con 0x5648ab3facb0 2024-09-17T05:29:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbaab400640 1 -- 172.21.15.43:0/2500135870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbaa40bd020 con 0x5648ab3facb0 2024-09-17T05:29:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbaab400640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94071be0 0x7fba940740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbaab400640 1 -- 172.21.15.43:0/2500135870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fbaa412de60 con 0x5648ab3facb0 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbab2200640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94071be0 0x7fba940740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbab99c0080 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5648ab3cc600 0x5648ab3cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5648ab3ccf50 con 0x5648ab3cc600 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.133+0000 7fbab3600640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5648ab3cc600 0x5648ab3cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab2200640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94071be0 0x7fba940740f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000f550 tx=0x7fbaa00026c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab3600640 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5648ab3cc600 0x5648ab3cca10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab3600640 1 -- 172.21.15.43:0/2500135870 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbaac00f070 con 0x5648ab3cc600 2024-09-17T05:29:53.140 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5648ab3cc600 msgr2=0x5648ab3cca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5648ab3cc600 0x5648ab3cca10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94071be0 msgr2=0x7fba940740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba94071be0 0x7fba940740f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000f550 tx=0x7fbaa00026c0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 msgr2=0x5648ab471580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 --2- 172.21.15.43:0/2500135870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5648ab3facb0 0x5648ab471580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4018d10 tx=0x7fbaa40ab690 comp rx=0 tx=0).stop 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab3600640 1 -- 172.21.15.43:0/2500135870 reap_dead start 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 shutdown_connections 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 >> 172.21.15.43:0/2500135870 conn(0x5648ab3b7d80 msgr2=0x5648ab49af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 shutdown_connections 2024-09-17T05:29:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.137+0000 7fbab99c0080 1 -- 172.21.15.43:0/2500135870 wait complete. 2024-09-17T05:29:53.145 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-09-17T05:29:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 Processor -- start 2024-09-17T05:29:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- start start 2024-09-17T05:29:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19659ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 0x561a1965a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a195c0f40 con 0x561a196cae40 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561a195cdd00 con 0x561a196174d0 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561a19562a00 con 0x561a19643cb0 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60068/0 (socket says 172.21.15.43:60068) 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 -- 172.21.15.43:0/3370406172 learned_addr learned my addr 172.21.15.43:0/3370406172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6890e00640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19659ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f688be00640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 0x561a1965a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 -- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 msgr2=0x561a1965a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 0x561a1965a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 -- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 msgr2=0x561a19659ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19659ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 -- 172.21.15.43:0/3370406172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1976fb50 con 0x561a196cae40 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f688be00640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 0x561a1965a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6891800640 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f688c0015b0 tx=0x7f688c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38150182f5db42f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f688aa00640 1 -- 172.21.15.43:0/3370406172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c016070 con 0x561a196cae40 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f688aa00640 1 -- 172.21.15.43:0/3370406172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f688c00f820 con 0x561a196cae40 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f688aa00640 1 -- 172.21.15.43:0/3370406172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c00fc70 con 0x561a196cae40 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- 172.21.15.43:0/3370406172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 msgr2=0x561a1977d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f688c0015b0 tx=0x7f688c001a70 comp rx=0 tx=0).stop 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- 172.21.15.43:0/3370406172 shutdown_connections 2024-09-17T05:29:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a196cae40 0x561a1977d3f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a19643cb0 0x561a1965a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 --2- 172.21.15.43:0/3370406172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19659ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- 172.21.15.43:0/3370406172 >> 172.21.15.43:0/3370406172 conn(0x561a19600d80 msgr2=0x561a19612890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- 172.21.15.43:0/3370406172 shutdown_connections 2024-09-17T05:29:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.201+0000 7f6897cdc080 1 -- 172.21.15.43:0/3370406172 wait complete. 2024-09-17T05:29:53.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 Processor -- start 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 -- start start 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19636db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 0x561a196e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a196cae40 0x561a19784ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561a196e9d30 con 0x561a19643cb0 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561a196ea930 con 0x561a196174d0 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6897cdc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561a1965c100 con 0x561a196cae40 2024-09-17T05:29:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 0x561a196e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 0x561a196e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60070/0 (socket says 172.21.15.43:60070) 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 -- 172.21.15.43:0/476431893 learned_addr learned my addr 172.21.15.43:0/476431893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6891800640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a196cae40 0x561a19784ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6890e00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19636db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 -- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a196cae40 msgr2=0x561a19784ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561a196cae40 0x561a19784ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 -- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 msgr2=0x561a19636db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a196174d0 0x561a19636db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 -- 172.21.15.43:0/476431893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a19782aa0 con 0x561a19643cb0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f688be00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 0x561a196e69a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f687c09a380 tx=0x7f687c01dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.205+0000 7f6889600640 1 -- 172.21.15.43:0/476431893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0c1070 con 0x561a19643cb0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a19782d40 con 0x561a19643cb0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6889600640 1 -- 172.21.15.43:0/476431893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f687c0b0880 con 0x561a19643cb0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6889600640 1 -- 172.21.15.43:0/476431893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0b0c30 con 0x561a19643cb0 2024-09-17T05:29:53.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a19783280 con 0x561a19643cb0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6889600640 1 -- 172.21.15.43:0/476431893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f687c0c4070 con 0x561a19643cb0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6889600640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6870071c10 0x7f6870074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6889600640 1 -- 172.21.15.43:0/476431893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f687c1306b0 con 0x561a19643cb0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.209+0000 7f6890e00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6870071c10 0x7f6870074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561a19785a60 0x561a19781de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x561a19782320 con 0x561a19785a60 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6891800640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561a19785a60 0x561a19781de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6890e00640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6870071c10 0x7f6870074120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6884002ef0 tx=0x7f6884002a00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6891800640 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561a19785a60 0x561a19781de0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6891800640 1 -- 172.21.15.43:0/476431893 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f688c00f070 con 0x561a19785a60 2024-09-17T05:29:53.218 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.218 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561a19785a60 msgr2=0x561a19781de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561a19785a60 0x561a19781de0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6870071c10 msgr2=0x7f6870074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6870071c10 0x7f6870074120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6884002ef0 tx=0x7f6884002a00 comp rx=0 tx=0).stop 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 msgr2=0x561a196e69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 --2- 172.21.15.43:0/476431893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561a19643cb0 0x561a196e69a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f687c09a380 tx=0x7f687c01dcc0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6891800640 1 -- 172.21.15.43:0/476431893 reap_dead start 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 shutdown_connections 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 >> 172.21.15.43:0/476431893 conn(0x561a19600d80 msgr2=0x561a19659ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 shutdown_connections 2024-09-17T05:29:53.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.213+0000 7f6897cdc080 1 -- 172.21.15.43:0/476431893 wait complete. 2024-09-17T05:29:53.222 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-09-17T05:29:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 Processor -- start 2024-09-17T05:29:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 -- start start 2024-09-17T05:29:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed6d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ed86710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da1ec61f40 con 0x55da1ed6d010 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da1ec6ed00 con 0x55da1ed6e8b0 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55da1ec03a00 con 0x55da1ece4cb0 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce9400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed6d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60078/0 (socket says 172.21.15.43:60078) 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 -- 172.21.15.43:0/3208801224 learned_addr learned my addr 172.21.15.43:0/3208801224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce9e00640 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ed86710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 -- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 msgr2=0x55da1ed6d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed6d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 -- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 msgr2=0x55da1ed86710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ed86710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 -- 172.21.15.43:0/3208801224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1ed87280 con 0x55da1ed6d010 2024-09-17T05:29:53.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b250 tx=0x7f5cd800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c7cae366936b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce3400640 1 -- 172.21.15.43:0/3208801224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800a040 con 0x55da1ed6d010 2024-09-17T05:29:53.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce3400640 1 -- 172.21.15.43:0/3208801224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd800f620 con 0x55da1ed6d010 2024-09-17T05:29:53.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.277+0000 7f5ce3400640 1 -- 172.21.15.43:0/3208801224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800fa70 con 0x55da1ed6d010 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- 172.21.15.43:0/3208801224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 msgr2=0x55da1ed861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b250 tx=0x7f5cd800b710 comp rx=0 tx=0).stop 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- 172.21.15.43:0/3208801224 shutdown_connections 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ed86710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ed861d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/3208801224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed6d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- 172.21.15.43:0/3208801224 >> 172.21.15.43:0/3208801224 conn(0x55da1eca1d80 msgr2=0x55da1ed8a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- 172.21.15.43:0/3208801224 shutdown_connections 2024-09-17T05:29:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- 172.21.15.43:0/3208801224 wait complete. 2024-09-17T05:29:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 Processor -- start 2024-09-17T05:29:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- start start 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed846e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ee26110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ecb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55da1ed877c0 con 0x55da1ed6d010 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da1ed87520 con 0x55da1ed6e8b0 2024-09-17T05:29:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5cf01f6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55da1ed87a60 con 0x55da1ece4cb0 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ee26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ee26110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60084/0 (socket says 172.21.15.43:60084) 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 -- 172.21.15.43:0/829480931 learned_addr learned my addr 172.21.15.43:0/829480931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce9400640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce9e00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ecb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 -- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 msgr2=0x55da1ed846e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed846e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 -- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 msgr2=0x55da1ecb6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da1ed6e8b0 0x55da1ecb6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce8a00640 1 -- 172.21.15.43:0/829480931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1ecb6020 con 0x55da1ed6d010 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.281+0000 7f5ce9400640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55da1ece4cb0 0x55da1ed846e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce8a00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ee26110 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8000f70 tx=0x7f5cd80012a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 -- 172.21.15.43:0/829480931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd801f070 con 0x55da1ed6d010 2024-09-17T05:29:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 -- 172.21.15.43:0/829480931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd80197f0 con 0x55da1ed6d010 2024-09-17T05:29:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da1ecb62c0 con 0x55da1ed6d010 2024-09-17T05:29:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 -- 172.21.15.43:0/829480931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8019c40 con 0x55da1ed6d010 2024-09-17T05:29:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1ed8b560 con 0x55da1ed6d010 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 -- 172.21.15.43:0/829480931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5cd8022070 con 0x55da1ed6d010 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ccc071be0 0x7f5ccc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce2000640 1 -- 172.21.15.43:0/829480931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f5cd8094340 con 0x55da1ed6d010 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55da1ed8b050 0x55da1ed8d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce9400640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ccc071be0 0x7f5ccc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55da1ee26650 con 0x55da1ed8b050 2024-09-17T05:29:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.285+0000 7f5ce9e00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55da1ed8b050 0x55da1ed8d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5ce9400640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ccc071be0 0x7f5ccc0740f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0ab000 tx=0x7f5cdc0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5ce9e00640 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55da1ed8b050 0x55da1ed8d7c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5ce9e00640 1 -- 172.21.15.43:0/829480931 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5ce4006040 con 0x55da1ed8b050 2024-09-17T05:29:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.292 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55da1ed8b050 msgr2=0x55da1ed8d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55da1ed8b050 0x55da1ed8d7c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ccc071be0 msgr2=0x7f5ccc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ccc071be0 0x7f5ccc0740f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0ab000 tx=0x7f5cdc0b2040 comp rx=0 tx=0).stop 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 msgr2=0x55da1ee26110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 --2- 172.21.15.43:0/829480931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55da1ed6d010 0x55da1ee26110 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8000f70 tx=0x7f5cd80012a0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5ce9e00640 1 -- 172.21.15.43:0/829480931 reap_dead start 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 shutdown_connections 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 >> 172.21.15.43:0/829480931 conn(0x55da1eca1d80 msgr2=0x55da1ed6d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 shutdown_connections 2024-09-17T05:29:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.289+0000 7f5cf01f6080 1 -- 172.21.15.43:0/829480931 wait complete. 2024-09-17T05:29:53.297 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-09-17T05:29:53.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 Processor -- start 2024-09-17T05:29:53.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 -- start start 2024-09-17T05:29:53.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 0x5619600f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x561960136560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561960043a00 con 0x5619600f3fb0 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5619600a1f40 con 0x56196013e320 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5619601acb80 con 0x7f3514002350 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60092/0 (socket says 172.21.15.43:60092) 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 -- 172.21.15.43:0/111434264 learned_addr learned my addr 172.21.15.43:0/111434264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352f200640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x561960136560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352e800640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 0x5619600f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 -- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 msgr2=0x5619600f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 0x5619600f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 -- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 msgr2=0x561960136560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x561960136560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 -- 172.21.15.43:0/111434264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196001cc10 con 0x5619600f3fb0 2024-09-17T05:29:53.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352e800640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 0x5619600f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:53.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352de00640 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b640 tx=0x7f351c00bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71be09211fc58ce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352ca00640 1 -- 172.21.15.43:0/111434264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00ab80 con 0x5619600f3fb0 2024-09-17T05:29:53.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352ca00640 1 -- 172.21.15.43:0/111434264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f351c00ad20 con 0x5619600f3fb0 2024-09-17T05:29:53.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f352ca00640 1 -- 172.21.15.43:0/111434264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c008690 con 0x5619600f3fb0 2024-09-17T05:29:53.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.353+0000 7f35356dc080 1 -- 172.21.15.43:0/111434264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 msgr2=0x5619602521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b640 tx=0x7f351c00bb00 comp rx=0 tx=0).stop 2024-09-17T05:29:53.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- 172.21.15.43:0/111434264 shutdown_connections 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x561960136560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5619600f3fb0 0x5619602521c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- 172.21.15.43:0/111434264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3514002350 0x5619600f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- 172.21.15.43:0/111434264 >> 172.21.15.43:0/111434264 conn(0x5619600e1d80 msgr2=0x5619601ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- 172.21.15.43:0/111434264 shutdown_connections 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- 172.21.15.43:0/111434264 wait complete. 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 Processor -- start 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- start start 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5619600f3fb0 0x561960139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x5619602638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 0x561960263df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5619601c8440 con 0x7f3514002350 2024-09-17T05:29:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5619601c81a0 con 0x56196013e320 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f35356dc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561960251a80 con 0x5619600f3fb0 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 0x561960263df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 0x561960263df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60100/0 (socket says 172.21.15.43:60100) 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 -- 172.21.15.43:0/804902387 learned_addr learned my addr 172.21.15.43:0/804902387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352e800640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5619600f3fb0 0x561960139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352de00640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x5619602638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 -- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5619600f3fb0 msgr2=0x561960139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5619600f3fb0 0x561960139ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 -- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 msgr2=0x5619602638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56196013e320 0x5619602638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 -- 172.21.15.43:0/804902387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196013abc0 con 0x7f3514002350 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.357+0000 7f352f200640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 0x561960263df0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3514003080 tx=0x7f35280114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 -- 172.21.15.43:0/804902387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528011b50 con 0x7f3514002350 2024-09-17T05:29:53.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 -- 172.21.15.43:0/804902387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f352801a070 con 0x7f3514002350 2024-09-17T05:29:53.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56196013ae60 con 0x7f3514002350 2024-09-17T05:29:53.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 -- 172.21.15.43:0/804902387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280153f0 con 0x7f3514002350 2024-09-17T05:29:53.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196013b6c0 con 0x7f3514002350 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 -- 172.21.15.43:0/804902387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3528015590 con 0x7f3514002350 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3510071be0 0x7f35100740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f3527400640 1 -- 172.21.15.43:0/804902387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f3528095a50 con 0x7f3514002350 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f352e800640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3510071be0 0x7f35100740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f35356dc080 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561960260260 0x561960260670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.361+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x561960260bb0 con 0x561960260260 2024-09-17T05:29:53.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f352de00640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561960260260 0x561960260670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f352e800640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3510071be0 0x7f35100740f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35200ac000 tx=0x7f35200a6040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f352de00640 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561960260260 0x561960260670 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f352de00640 1 -- 172.21.15.43:0/804902387 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv3 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f351c00aca0 con 0x561960260260 2024-09-17T05:29:53.368 INFO:teuthology.orchestra.run.smithi043.stdout:header (42 bytes) : 2024-09-17T05:29:53.369 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-09-17T05:29:53.369 INFO:teuthology.orchestra.run.smithi043.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 03 00 00 |me v011.........| 2024-09-17T05:29:53.369 INFO:teuthology.orchestra.run.smithi043.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-09-17T05:29:53.369 INFO:teuthology.orchestra.run.smithi043.stdout:0000002a 2024-09-17T05:29:53.369 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561960260260 msgr2=0x561960260670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561960260260 0x561960260670 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3510071be0 msgr2=0x7f35100740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3510071be0 0x7f35100740f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35200ac000 tx=0x7f35200a6040 comp rx=0 tx=0).stop 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 msgr2=0x561960263df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 --2- 172.21.15.43:0/804902387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3514002350 0x561960263df0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3514003080 tx=0x7f35280114c0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f352f200640 1 -- 172.21.15.43:0/804902387 reap_dead start 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 shutdown_connections 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 >> 172.21.15.43:0/804902387 conn(0x5619600e1d80 msgr2=0x5619601c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 shutdown_connections 2024-09-17T05:29:53.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.365+0000 7f35356dc080 1 -- 172.21.15.43:0/804902387 wait complete. 2024-09-17T05:29:53.373 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-09-17T05:29:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 Processor -- start 2024-09-17T05:29:53.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- start start 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 0x5584a3371900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 0x5584a33971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584a3209a00 con 0x5584a33729c0 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5584a3267f40 con 0x5584a32c09c0 2024-09-17T05:29:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5584a339c2d0 con 0x5584a33725b0 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60108/0 (socket says 172.21.15.43:60108) 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 -- 172.21.15.43:0/1851365015 learned_addr learned my addr 172.21.15.43:0/1851365015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea0e00640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 0x5584a3371900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7e9be00640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 0x5584a33971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 -- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 msgr2=0x5584a33971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 0x5584a33971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 -- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 msgr2=0x5584a3371900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 0x5584a3371900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 -- 172.21.15.43:0/1851365015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a31e2c10 con 0x5584a33729c0 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea0e00640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 0x5584a3371900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7e9be00640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 0x5584a33971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea1800640 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001650 tx=0x7f7e9c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ca556bfe5b10e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7e9aa00640 1 -- 172.21.15.43:0/1851365015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c016070 con 0x5584a33729c0 2024-09-17T05:29:53.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7e9aa00640 1 -- 172.21.15.43:0/1851365015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e9c00f800 con 0x5584a33729c0 2024-09-17T05:29:53.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7e9aa00640 1 -- 172.21.15.43:0/1851365015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00fc50 con 0x5584a33729c0 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- 172.21.15.43:0/1851365015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 msgr2=0x5584a32fd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001650 tx=0x7f7e9c001b10 comp rx=0 tx=0).stop 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- 172.21.15.43:0/1851365015 shutdown_connections 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a33729c0 0x5584a32fd730 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33725b0 0x5584a33971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/1851365015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a32c09c0 0x5584a3371900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- 172.21.15.43:0/1851365015 >> 172.21.15.43:0/1851365015 conn(0x5584a32a7d80 msgr2=0x5584a3392420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- 172.21.15.43:0/1851365015 shutdown_connections 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.429+0000 7f7ea7c31080 1 -- 172.21.15.43:0/1851365015 wait complete. 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 Processor -- start 2024-09-17T05:29:53.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 -- start start 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 0x5584a3368b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a33725b0 0x5584a3369250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33729c0 0x5584a3369790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5584a3303490 con 0x5584a32c09c0 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5584a33031f0 con 0x5584a33725b0 2024-09-17T05:29:53.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea7c31080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5584a3303730 con 0x5584a33729c0 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 0x5584a3368b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 0x5584a3368b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60112/0 (socket says 172.21.15.43:60112) 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 -- 172.21.15.43:0/222552970 learned_addr learned my addr 172.21.15.43:0/222552970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7e9be00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a33725b0 0x5584a3369250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea1800640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33729c0 0x5584a3369790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 -- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33729c0 msgr2=0x5584a3369790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33729c0 0x5584a3369790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 -- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a33725b0 msgr2=0x5584a3369250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a33725b0 0x5584a3369250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 -- 172.21.15.43:0/222552970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a3304b50 con 0x5584a32c09c0 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7e9be00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584a33725b0 0x5584a3369250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea1800640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5584a33729c0 0x5584a3369790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.433+0000 7f7ea0e00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 0x5584a3368b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7e9401b710 tx=0x7f7e940ab5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 -- 172.21.15.43:0/222552970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940abc50 con 0x5584a32c09c0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 -- 172.21.15.43:0/222552970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e940b63f0 con 0x5584a32c09c0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5584a34216c0 con 0x5584a32c09c0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 -- 172.21.15.43:0/222552970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940b65e0 con 0x5584a32c09c0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a3421c00 con 0x5584a32c09c0 2024-09-17T05:29:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 -- 172.21.15.43:0/222552970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7e940be020 con 0x5584a32c09c0 2024-09-17T05:29:53.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e80071be0 0x7f7e800740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e99600640 1 -- 172.21.15.43:0/222552970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f7e9412ddd0 con 0x5584a32c09c0 2024-09-17T05:29:53.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5584a3304640 0x5584a342f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5584a336a860 con 0x5584a3304640 2024-09-17T05:29:53.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7e9be00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e80071be0 0x7f7e800740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.437+0000 7f7ea1800640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5584a3304640 0x5584a342f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7e9be00640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e80071be0 0x7f7e800740f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004fd0 tx=0x7f7e8c004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea1800640 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5584a3304640 0x5584a342f3e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea1800640 1 -- 172.21.15.43:0/222552970 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7e9c019040 con 0x5584a3304640 2024-09-17T05:29:53.444 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.444 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5584a3304640 msgr2=0x5584a342f3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5584a3304640 0x5584a342f3e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e80071be0 msgr2=0x7f7e800740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e80071be0 0x7f7e800740f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004fd0 tx=0x7f7e8c004890 comp rx=0 tx=0).stop 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 msgr2=0x5584a3368b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 --2- 172.21.15.43:0/222552970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a32c09c0 0x5584a3368b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7e9401b710 tx=0x7f7e940ab5b0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea1800640 1 -- 172.21.15.43:0/222552970 reap_dead start 2024-09-17T05:29:53.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 shutdown_connections 2024-09-17T05:29:53.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 >> 172.21.15.43:0/222552970 conn(0x5584a32a7d80 msgr2=0x5584a33920b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 shutdown_connections 2024-09-17T05:29:53.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.441+0000 7f7ea7c31080 1 -- 172.21.15.43:0/222552970 wait complete. 2024-09-17T05:29:53.449 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-09-17T05:29:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.501+0000 7fd448e53080 1 Processor -- start 2024-09-17T05:29:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- start start 2024-09-17T05:29:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300c359c0 0x562300ce2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce75b0 0x562300ce29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562300b7ea00 con 0x562300ce79c0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562300bdcf40 con 0x562300c359c0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562300d112d0 con 0x562300ce75b0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60132/0 (socket says 172.21.15.43:60132) 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 -- 172.21.15.43:0/1630511131 learned_addr learned my addr 172.21.15.43:0/1630511131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300c359c0 0x562300ce2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd441600640 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce75b0 0x562300ce29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 -- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce75b0 msgr2=0x562300ce29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce75b0 0x562300ce29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 -- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300c359c0 msgr2=0x562300ce2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300c359c0 0x562300ce2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 -- 172.21.15.43:0/1630511131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562300b57c10 con 0x562300ce79c0 2024-09-17T05:29:53.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd442a00640 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00a4b0 tx=0x7fd43c00a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e35d11765b41e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd43be00640 1 -- 172.21.15.43:0/1630511131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c015070 con 0x562300ce79c0 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd43be00640 1 -- 172.21.15.43:0/1630511131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43c00e730 con 0x562300ce79c0 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd43be00640 1 -- 172.21.15.43:0/1630511131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00eb80 con 0x562300ce79c0 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- 172.21.15.43:0/1630511131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 msgr2=0x562300c78070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00a4b0 tx=0x7fd43c00a8b0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- 172.21.15.43:0/1630511131 shutdown_connections 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300ce79c0 0x562300c78070 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce75b0 0x562300ce29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 --2- 172.21.15.43:0/1630511131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300c359c0 0x562300ce2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- 172.21.15.43:0/1630511131 >> 172.21.15.43:0/1630511131 conn(0x562300c1cd80 msgr2=0x562300d07440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- 172.21.15.43:0/1630511131 shutdown_connections 2024-09-17T05:29:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.505+0000 7fd448e53080 1 -- 172.21.15.43:0/1630511131 wait complete. 2024-09-17T05:29:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 Processor -- start 2024-09-17T05:29:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 -- start start 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 0x562300d0e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300ce75b0 0x562300d03a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce79c0 0x562300d06ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562300c72e90 con 0x562300c359c0 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562300c72bf0 con 0x562300ce75b0 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562300c73130 con 0x562300ce79c0 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 0x562300d0e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 0x562300d0e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60148/0 (socket says 172.21.15.43:60148) 2024-09-17T05:29:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 -- 172.21.15.43:0/365468409 learned_addr learned my addr 172.21.15.43:0/365468409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd441600640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300ce75b0 0x562300d03a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442a00640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce79c0 0x562300d06ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 -- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce79c0 msgr2=0x562300d06ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562300ce79c0 0x562300d06ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 -- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300ce75b0 msgr2=0x562300d03a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300ce75b0 0x562300d03a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 -- 172.21.15.43:0/365468409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562300d03f80 con 0x562300c359c0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd441600640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562300ce75b0 0x562300d03a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd442000640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 0x562300d0e780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd43401b6f0 tx=0x7fd4340abb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd43aa00640 1 -- 172.21.15.43:0/365468409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340abe10 con 0x562300c359c0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd43aa00640 1 -- 172.21.15.43:0/365468409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4340b5530 con 0x562300c359c0 2024-09-17T05:29:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd43aa00640 1 -- 172.21.15.43:0/365468409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340b5760 con 0x562300c359c0 2024-09-17T05:29:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.509+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562300d076d0 con 0x562300c359c0 2024-09-17T05:29:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562300c31410 con 0x562300c359c0 2024-09-17T05:29:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd43aa00640 1 -- 172.21.15.43:0/365468409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4340bd020 con 0x562300c359c0 2024-09-17T05:29:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd43aa00640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424071be0 0x7fd4240740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd43aa00640 1 -- 172.21.15.43:0/365468409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fd43412dd00 con 0x562300c359c0 2024-09-17T05:29:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd441600640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424071be0 0x7fd4240740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd448e53080 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562300d066f0 0x562300cdfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x562300ce01c0 con 0x562300d066f0 2024-09-17T05:29:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.513+0000 7fd442a00640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562300d066f0 0x562300cdfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd441600640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424071be0 0x7fd4240740f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd430004fd0 tx=0x7fd43000d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd442a00640 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562300d066f0 0x562300cdfc80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd442a00640 1 -- 172.21.15.43:0/365468409 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd43c00e070 con 0x562300d066f0 2024-09-17T05:29:53.520 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.520 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562300d066f0 msgr2=0x562300cdfc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562300d066f0 0x562300cdfc80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424071be0 msgr2=0x7fd4240740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424071be0 0x7fd4240740f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd430004fd0 tx=0x7fd43000d040 comp rx=0 tx=0).stop 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 msgr2=0x562300d0e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 --2- 172.21.15.43:0/365468409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562300c359c0 0x562300d0e780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd43401b6f0 tx=0x7fd4340abb60 comp rx=0 tx=0).stop 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd442a00640 1 -- 172.21.15.43:0/365468409 reap_dead start 2024-09-17T05:29:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 shutdown_connections 2024-09-17T05:29:53.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 >> 172.21.15.43:0/365468409 conn(0x562300c1cd80 msgr2=0x562300c71a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 shutdown_connections 2024-09-17T05:29:53.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.517+0000 7fd448e53080 1 -- 172.21.15.43:0/365468409 wait complete. 2024-09-17T05:29:53.525 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-09-17T05:29:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 Processor -- start 2024-09-17T05:29:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- start start 2024-09-17T05:29:53.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d1694d0 0x55ff8d23a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d21ce40 0x55ff8d164890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff8d112f40 con 0x55ff8d195cb0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ff8d11fd00 con 0x55ff8d21ce40 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ff8d0b4a00 con 0x55ff8d1694d0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60158/0 (socket says 172.21.15.43:60158) 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 -- 172.21.15.43:0/573826760 learned_addr learned my addr 172.21.15.43:0/573826760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f0510800640 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d1694d0 0x55ff8d23a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f0511200640 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d21ce40 0x55ff8d164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 -- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d1694d0 msgr2=0x55ff8d23a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d1694d0 0x55ff8d23a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 -- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d21ce40 msgr2=0x55ff8d164890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d21ce40 0x55ff8d164890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 -- 172.21.15.43:0/573826760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff8d23b950 con 0x55ff8d195cb0 2024-09-17T05:29:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050fe00640 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f04fc001e50 tx=0x7f04fc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c0d6cbd7ab94a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050ea00640 1 -- 172.21.15.43:0/573826760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc012070 con 0x55ff8d195cb0 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050ea00640 1 -- 172.21.15.43:0/573826760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04fc00e930 con 0x55ff8d195cb0 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f050ea00640 1 -- 172.21.15.43:0/573826760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00edc0 con 0x55ff8d195cb0 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- 172.21.15.43:0/573826760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 msgr2=0x55ff8d1abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f04fc001e50 tx=0x7f04fc001e80 comp rx=0 tx=0).stop 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- 172.21.15.43:0/573826760 shutdown_connections 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d21ce40 0x55ff8d164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d195cb0 0x55ff8d1abb50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 --2- 172.21.15.43:0/573826760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d1694d0 0x55ff8d23a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- 172.21.15.43:0/573826760 >> 172.21.15.43:0/573826760 conn(0x55ff8d152d80 msgr2=0x55ff8d239a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- 172.21.15.43:0/573826760 shutdown_connections 2024-09-17T05:29:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.581+0000 7f05176c1080 1 -- 172.21.15.43:0/573826760 wait complete. 2024-09-17T05:29:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 Processor -- start 2024-09-17T05:29:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- start start 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d1694d0 0x55ff8d2d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d195cb0 0x55ff8d2d3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 0x55ff8d230f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ff8d23be90 con 0x55ff8d21ce40 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ff8d23bbf0 con 0x55ff8d1694d0 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ff8d23c130 con 0x55ff8d195cb0 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 0x55ff8d230f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0510800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d1694d0 0x55ff8d2d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0510800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d1694d0 0x55ff8d2d5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:51108/0 (socket says 172.21.15.43:51108) 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 0x55ff8d230f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60166/0 (socket says 172.21.15.43:60166) 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0510800640 1 -- 172.21.15.43:0/778938205 learned_addr learned my addr 172.21.15.43:0/778938205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f050fe00640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d195cb0 0x55ff8d2d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 -- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d195cb0 msgr2=0x55ff8d2d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ff8d195cb0 0x55ff8d2d3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 -- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d1694d0 msgr2=0x55ff8d2d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff8d1694d0 0x55ff8d2d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 -- 172.21.15.43:0/778938205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff8d2319c0 con 0x55ff8d21ce40 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f0511200640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 0x55ff8d230f60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f05080ad000 tx=0x7f05080afb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f050d600640 1 -- 172.21.15.43:0/778938205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508004910 con 0x55ff8d21ce40 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f050d600640 1 -- 172.21.15.43:0/778938205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05080afdf0 con 0x55ff8d21ce40 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f050d600640 1 -- 172.21.15.43:0/778938205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080a78a0 con 0x55ff8d21ce40 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff8d231c60 con 0x55ff8d21ce40 2024-09-17T05:29:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.585+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff8d2321a0 con 0x55ff8d21ce40 2024-09-17T05:29:53.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f050d600640 1 -- 172.21.15.43:0/778938205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f05080c6020 con 0x55ff8d21ce40 2024-09-17T05:29:53.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f050d600640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04f0071be0 0x7f04f00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f050d600640 1 -- 172.21.15.43:0/778938205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f05081313f0 con 0x55ff8d21ce40 2024-09-17T05:29:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f0510800640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04f0071be0 0x7f04f00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f05176c1080 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ff8d2314a0 0x55ff8d2328e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55ff8d232e20 con 0x55ff8d2314a0 2024-09-17T05:29:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f050fe00640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ff8d2314a0 0x55ff8d2328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f0510800640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04f0071be0 0x7f04f00740f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0504011630 tx=0x7f0504002bb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.589+0000 7f050fe00640 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ff8d2314a0 0x55ff8d2328e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f050fe00640 1 -- 172.21.15.43:0/778938205 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04fc014030 con 0x55ff8d2314a0 2024-09-17T05:29:53.596 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.596 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ff8d2314a0 msgr2=0x55ff8d2328e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ff8d2314a0 0x55ff8d2328e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04f0071be0 msgr2=0x7f04f00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04f0071be0 0x7f04f00740f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0504011630 tx=0x7f0504002bb0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 msgr2=0x55ff8d230f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 --2- 172.21.15.43:0/778938205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ff8d21ce40 0x55ff8d230f60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f05080ad000 tx=0x7f05080afb90 comp rx=0 tx=0).stop 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f0511200640 1 -- 172.21.15.43:0/778938205 reap_dead start 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 shutdown_connections 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 >> 172.21.15.43:0/778938205 conn(0x55ff8d152d80 msgr2=0x55ff8d21e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 shutdown_connections 2024-09-17T05:29:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.593+0000 7f05176c1080 1 -- 172.21.15.43:0/778938205 wait complete. 2024-09-17T05:29:53.601 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-09-17T05:29:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 Processor -- start 2024-09-17T05:29:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 -- start start 2024-09-17T05:29:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70605bcb0 0x55a706101e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a706192ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a705fd8f40 con 0x55a70602cd70 2024-09-17T05:29:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a705fe5d00 con 0x55a7060e2e40 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a705f7aa00 con 0x55a70605bcb0 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60170/0 (socket says 172.21.15.43:60170) 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 -- 172.21.15.43:0/3159431503 learned_addr learned my addr 172.21.15.43:0/3159431503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984e00640 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a706192ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff983a00640 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70605bcb0 0x55a706101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 -- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70605bcb0 msgr2=0x55a706101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70605bcb0 0x55a706101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 -- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 msgr2=0x55a706192ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a706192ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 -- 172.21.15.43:0/3159431503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a706075500 con 0x55a70602cd70 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff984400640 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff97401a660 tx=0x7ff97401a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0c607a1cf55ef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff982600640 1 -- 172.21.15.43:0/3159431503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740a7820 con 0x55a70602cd70 2024-09-17T05:29:53.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff982600640 1 -- 172.21.15.43:0/3159431503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9740a7af0 con 0x55a70602cd70 2024-09-17T05:29:53.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff982600640 1 -- 172.21.15.43:0/3159431503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740b0470 con 0x55a70602cd70 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3159431503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 msgr2=0x55a706074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.657+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff97401a660 tx=0x7ff97401a7f0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3159431503 shutdown_connections 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a706192ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70605bcb0 0x55a706101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3159431503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70602cd70 0x55a706074ed0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3159431503 >> 172.21.15.43:0/3159431503 conn(0x55a706018d80 msgr2=0x55a70606d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3159431503 shutdown_connections 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3159431503 wait complete. 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 Processor -- start 2024-09-17T05:29:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- start start 2024-09-17T05:29:53.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70602cd70 0x55a7060fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 0x55a7060714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a7061890b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a70618a8a0 con 0x55a70605bcb0 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a70618a640 con 0x55a7060e2e40 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff98b2a8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a70618ab40 con 0x55a70602cd70 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 0x55a7060714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 0x55a7060714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60174/0 (socket says 172.21.15.43:60174) 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 -- 172.21.15.43:0/3147769828 learned_addr learned my addr 172.21.15.43:0/3147769828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff984400640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70602cd70 0x55a7060fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 -- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70602cd70 msgr2=0x55a7060fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff984e00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a7061890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70602cd70 0x55a7060fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 -- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 msgr2=0x55a7061890b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a7061890b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 -- 172.21.15.43:0/3147769828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7060ecfa0 con 0x55a70605bcb0 2024-09-17T05:29:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff984400640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a70602cd70 0x55a7060fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff984e00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7060e2e40 0x55a7061890b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.661+0000 7ff983a00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 0x55a7060714e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff97000dea0 tx=0x7ff97000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 -- 172.21.15.43:0/3147769828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970013070 con 0x55a70605bcb0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 -- 172.21.15.43:0/3147769828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff970018770 con 0x55a70605bcb0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a7060ed240 con 0x55a70605bcb0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 -- 172.21.15.43:0/3147769828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970018bc0 con 0x55a70605bcb0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7060ed780 con 0x55a70605bcb0 2024-09-17T05:29:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 -- 172.21.15.43:0/3147769828 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff97000f410 con 0x55a70605bcb0 2024-09-17T05:29:53.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff964071c10 0x7ff964074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff981200640 1 -- 172.21.15.43:0/3147769828 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7ff9700938a0 con 0x55a70605bcb0 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a7060eac00 0x55a706074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55a706074ac0 con 0x55a7060eac00 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff984400640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff964071c10 0x7ff964074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.665+0000 7ff984e00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a7060eac00 0x55a706074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff984400640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff964071c10 0x7ff964074120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff974000c10 tx=0x7ff974004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff984e00640 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a7060eac00 0x55a706074580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff984e00640 1 -- 172.21.15.43:0/3147769828 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7ff97c0024e0 con 0x55a7060eac00 2024-09-17T05:29:53.672 INFO:teuthology.orchestra.run.smithi043.stdout:header (274 bytes) : 2024-09-17T05:29:53.672 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.672 INFO:teuthology.orchestra.run.smithi043.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-17T05:29:53.672 INFO:teuthology.orchestra.run.smithi043.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000a0 00 00 00 00 00 00 00 00 c7 13 e9 66 74 a1 cd 14 |...........ft...| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000e0 00 00 00 00 00 00 c7 13 e9 66 74 a1 cd 14 00 00 |.........ft.....| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.673 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.674 INFO:teuthology.orchestra.run.smithi043.stdout:00000110 00 00 |..| 2024-09-17T05:29:53.674 INFO:teuthology.orchestra.run.smithi043.stdout:00000112 2024-09-17T05:29:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a7060eac00 msgr2=0x55a706074580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a7060eac00 0x55a706074580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff964071c10 msgr2=0x7ff964074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff964071c10 0x7ff964074120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff974000c10 tx=0x7ff974004890 comp rx=0 tx=0).stop 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 msgr2=0x55a7060714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 --2- 172.21.15.43:0/3147769828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a70605bcb0 0x55a7060714e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff97000dea0 tx=0x7ff97000df90 comp rx=0 tx=0).stop 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff984e00640 1 -- 172.21.15.43:0/3147769828 reap_dead start 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 shutdown_connections 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 >> 172.21.15.43:0/3147769828 conn(0x55a706018d80 msgr2=0x55a70606d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 shutdown_connections 2024-09-17T05:29:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.669+0000 7ff98b2a8080 1 -- 172.21.15.43:0/3147769828 wait complete. 2024-09-17T05:29:53.677 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-09-17T05:29:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 Processor -- start 2024-09-17T05:29:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- start start 2024-09-17T05:29:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d8724d0 0x56107d945740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d925e40 0x56107d86ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56107d7bda00 con 0x56107d89ecb0 2024-09-17T05:29:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56107d81bf40 con 0x56107d8724d0 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56107d926b30 con 0x56107d925e40 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35296/0 (socket says 172.21.15.43:35296) 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6698600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d925e40 0x56107d86ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 -- 172.21.15.43:0/1568492434 learned_addr learned my addr 172.21.15.43:0/1568492434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697c00640 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d8724d0 0x56107d945740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 -- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d925e40 msgr2=0x56107d86ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d925e40 0x56107d86ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 -- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d8724d0 msgr2=0x56107d945740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d8724d0 0x56107d945740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 -- 172.21.15.43:0/1568492434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107d796c10 con 0x56107d89ecb0 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6697200640 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f668400b5f0 tx=0x7f668400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7fba40373eb05c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6695e00640 1 -- 172.21.15.43:0/1568492434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400a890 con 0x56107d89ecb0 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6695e00640 1 -- 172.21.15.43:0/1568492434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f668400aa30 con 0x56107d89ecb0 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f6695e00640 1 -- 172.21.15.43:0/1568492434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400ae40 con 0x56107d89ecb0 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- 172.21.15.43:0/1568492434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 msgr2=0x56107d86d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f668400b5f0 tx=0x7f668400bab0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- 172.21.15.43:0/1568492434 shutdown_connections 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d925e40 0x56107d86ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d89ecb0 0x56107d86d890 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 --2- 172.21.15.43:0/1568492434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d8724d0 0x56107d945740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- 172.21.15.43:0/1568492434 >> 172.21.15.43:0/1568492434 conn(0x56107d85bd80 msgr2=0x56107d89e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- 172.21.15.43:0/1568492434 shutdown_connections 2024-09-17T05:29:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.733+0000 7f669e98c080 1 -- 172.21.15.43:0/1568492434 wait complete. 2024-09-17T05:29:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 Processor -- start 2024-09-17T05:29:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- start start 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d8724d0 0x56107d8b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d925e40 0x56107d92e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 0x56107d92e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56107d8b8600 con 0x56107d93fa60 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56107d8b5670 con 0x56107d925e40 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56107d8b1550 con 0x56107d8724d0 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 0x56107d92e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 0x56107d92e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35306/0 (socket says 172.21.15.43:35306) 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 -- 172.21.15.43:0/375842454 learned_addr learned my addr 172.21.15.43:0/375842454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6697200640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d925e40 0x56107d92e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6697c00640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d8724d0 0x56107d8b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 -- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d8724d0 msgr2=0x56107d8b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d8724d0 0x56107d8b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 -- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d925e40 msgr2=0x56107d92e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56107d925e40 0x56107d92e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 -- 172.21.15.43:0/375842454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107d940010 con 0x56107d93fa60 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6697c00640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56107d8724d0 0x56107d8b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6698600640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 0x56107d92e550 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f669000a6c0 tx=0x7f66900afbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6694a00640 1 -- 172.21.15.43:0/375842454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900a7960 con 0x56107d93fa60 2024-09-17T05:29:53.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6694a00640 1 -- 172.21.15.43:0/375842454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6690004020 con 0x56107d93fa60 2024-09-17T05:29:53.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56107d9402b0 con 0x56107d93fa60 2024-09-17T05:29:53.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f6694a00640 1 -- 172.21.15.43:0/375842454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690003420 con 0x56107d93fa60 2024-09-17T05:29:53.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.737+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107d9407f0 con 0x56107d93fa60 2024-09-17T05:29:53.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6694a00640 1 -- 172.21.15.43:0/375842454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66900c2030 con 0x56107d93fa60 2024-09-17T05:29:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6694a00640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6678071be0 0x7f66780740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6694a00640 1 -- 172.21.15.43:0/375842454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6690131c00 con 0x56107d93fa60 2024-09-17T05:29:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6697c00640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6678071be0 0x7f66780740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f669e98c080 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56107d9dff50 0x56107d9e0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x56107d9dc450 con 0x56107d9dff50 2024-09-17T05:29:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6697200640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56107d9dff50 0x56107d9e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.741+0000 7f6697c00640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6678071be0 0x7f66780740f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6688010500 tx=0x7f6688001740 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f6697200640 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56107d9dff50 0x56107d9e0360 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f6697200640 1 -- 172.21.15.43:0/375842454 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f6684004d40 con 0x56107d9dff50 2024-09-17T05:29:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:header (14 bytes) : 2024-09-17T05:29:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-09-17T05:29:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:0000000e 2024-09-17T05:29:53.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56107d9dff50 msgr2=0x56107d9e0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56107d9dff50 0x56107d9e0360 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6678071be0 msgr2=0x7f66780740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6678071be0 0x7f66780740f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6688010500 tx=0x7f6688001740 comp rx=0 tx=0).stop 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 msgr2=0x56107d92e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 --2- 172.21.15.43:0/375842454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56107d93fa60 0x56107d92e550 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f669000a6c0 tx=0x7f66900afbe0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f6698600640 1 -- 172.21.15.43:0/375842454 reap_dead start 2024-09-17T05:29:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 shutdown_connections 2024-09-17T05:29:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 >> 172.21.15.43:0/375842454 conn(0x56107d85bd80 msgr2=0x56107d945070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 shutdown_connections 2024-09-17T05:29:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.745+0000 7f669e98c080 1 -- 172.21.15.43:0/375842454 wait complete. 2024-09-17T05:29:53.753 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-09-17T05:29:53.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.805+0000 7fc7cf7e8080 1 Processor -- start 2024-09-17T05:29:53.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- start start 2024-09-17T05:29:53.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39c30c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 0x558c39c31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558c39b39a00 con 0x558c39ca1e40 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c39b97f40 con 0x558c39bee4d0 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558c39ca2b30 con 0x558c39c1acb0 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35320/0 (socket says 172.21.15.43:35320) 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 -- 172.21.15.43:0/839534342 learned_addr learned my addr 172.21.15.43:0/839534342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c3e00640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 0x558c39c31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c8a00640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39c30c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 -- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 msgr2=0x558c39c31e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 0x558c39c31e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 -- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 msgr2=0x558c39c30c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39c30c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 -- 172.21.15.43:0/839534342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c39b12c10 con 0x558c39ca1e40 2024-09-17T05:29:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c3e00640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 0x558c39c31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c9400640 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc7c401a920 tx=0x7fc7c401aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bda3f1fdb644f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c2a00640 1 -- 172.21.15.43:0/839534342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4004020 con 0x558c39ca1e40 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c2a00640 1 -- 172.21.15.43:0/839534342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7c40042b0 con 0x558c39ca1e40 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7c2a00640 1 -- 172.21.15.43:0/839534342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40bb6b0 con 0x558c39ca1e40 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/839534342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 msgr2=0x558c39c323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc7c401a920 tx=0x7fc7c401aab0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/839534342 shutdown_connections 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39ca1e40 0x558c39c323d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39c1acb0 0x558c39c31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/839534342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39c30c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/839534342 >> 172.21.15.43:0/839534342 conn(0x558c39bd7d80 msgr2=0x558c39be9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/839534342 shutdown_connections 2024-09-17T05:29:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.809+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/839534342 wait complete. 2024-09-17T05:29:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 Processor -- start 2024-09-17T05:29:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- start start 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39ca44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 0x558c39cbd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39ca1e40 0x558c39d57f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558c39bea120 con 0x558c39c1acb0 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c39c33a30 con 0x558c39bee4d0 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558c39c33cd0 con 0x558c39ca1e40 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 0x558c39cbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 0x558c39cbd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35330/0 (socket says 172.21.15.43:35330) 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 -- 172.21.15.43:0/3155715570 learned_addr learned my addr 172.21.15.43:0/3155715570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c8a00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39ca44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c9400640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39ca1e40 0x558c39d57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 -- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39ca1e40 msgr2=0x558c39d57f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558c39ca1e40 0x558c39d57f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 -- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 msgr2=0x558c39ca44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c39bee4d0 0x558c39ca44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 -- 172.21.15.43:0/3155715570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c39d4bf30 con 0x558c39c1acb0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c3e00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 0x558c39cbd550 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400a6c0 tx=0x7fc7b400ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c1600640 1 -- 172.21.15.43:0/3155715570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4008940 con 0x558c39c1acb0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c1600640 1 -- 172.21.15.43:0/3155715570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b4008ae0 con 0x558c39c1acb0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558c39d4c1d0 con 0x558c39c1acb0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7c1600640 1 -- 172.21.15.43:0/3155715570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4016420 con 0x558c39c1acb0 2024-09-17T05:29:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.813+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c39d4a320 con 0x558c39c1acb0 2024-09-17T05:29:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c1600640 1 -- 172.21.15.43:0/3155715570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b401f440 con 0x558c39c1acb0 2024-09-17T05:29:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c1600640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7a8071be0 0x7fc7a80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c1600640 1 -- 172.21.15.43:0/3155715570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fc7b4092f60 con 0x558c39c1acb0 2024-09-17T05:29:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c8a00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7a8071be0 0x7fc7a80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558c39d5b4a0 0x558c39d5b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x558c39d5bdf0 con 0x558c39d5b4a0 2024-09-17T05:29:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c9400640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558c39d5b4a0 0x558c39d5b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c8a00640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7a8071be0 0x7fc7a80740f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc003f00 tx=0x7fc7bc003840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.817+0000 7fc7c9400640 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558c39d5b4a0 0x558c39d5b8b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7c9400640 1 -- 172.21.15.43:0/3155715570 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7c4002050 con 0x558c39d5b4a0 2024-09-17T05:29:53.824 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558c39d5b4a0 msgr2=0x558c39d5b8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558c39d5b4a0 0x558c39d5b8b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7a8071be0 msgr2=0x7fc7a80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7a8071be0 0x7fc7a80740f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc003f00 tx=0x7fc7bc003840 comp rx=0 tx=0).stop 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 msgr2=0x558c39cbd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 --2- 172.21.15.43:0/3155715570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c39c1acb0 0x558c39cbd550 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400a6c0 tx=0x7fc7b400ab80 comp rx=0 tx=0).stop 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7c9400640 1 -- 172.21.15.43:0/3155715570 reap_dead start 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 shutdown_connections 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 >> 172.21.15.43:0/3155715570 conn(0x558c39bd7d80 msgr2=0x558c39be9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 shutdown_connections 2024-09-17T05:29:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.821+0000 7fc7cf7e8080 1 -- 172.21.15.43:0/3155715570 wait complete. 2024-09-17T05:29:53.829 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-09-17T05:29:53.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.881+0000 7f19450c3080 1 Processor -- start 2024-09-17T05:29:53.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.881+0000 7f19450c3080 1 -- start start 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd18a915b0 0x55dd18b43ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a247a0 0x55dd18b443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dd18928a00 con 0x55dd189df9c0 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dd18986f40 con 0x55dd18a247a0 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55dd18a20e50 con 0x55dd18a915b0 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35358/0 (socket says 172.21.15.43:35358) 2024-09-17T05:29:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 -- 172.21.15.43:0/2186812698 learned_addr learned my addr 172.21.15.43:0/2186812698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193ec00640 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a247a0 0x55dd18b443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193d800640 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd18a915b0 0x55dd18b43ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 -- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd18a915b0 msgr2=0x55dd18b43ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd18a915b0 0x55dd18b43ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 -- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a247a0 msgr2=0x55dd18b443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a247a0 0x55dd18b443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 -- 172.21.15.43:0/2186812698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd18901c10 con 0x55dd189df9c0 2024-09-17T05:29:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f193e200640 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1930017e20 tx=0x7f1930017fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba109882470689c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f1937e00640 1 -- 172.21.15.43:0/2186812698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300ab980 con 0x55dd189df9c0 2024-09-17T05:29:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f1937e00640 1 -- 172.21.15.43:0/2186812698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19300b9070 con 0x55dd189df9c0 2024-09-17T05:29:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f1937e00640 1 -- 172.21.15.43:0/2186812698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300abd80 con 0x55dd189df9c0 2024-09-17T05:29:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- 172.21.15.43:0/2186812698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 msgr2=0x55dd18a24260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1930017e20 tx=0x7f1930017fb0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- 172.21.15.43:0/2186812698 shutdown_connections 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a247a0 0x55dd18b443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd18a915b0 0x55dd18b43ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 --2- 172.21.15.43:0/2186812698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd189df9c0 0x55dd18a24260 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- 172.21.15.43:0/2186812698 >> 172.21.15.43:0/2186812698 conn(0x55dd189c6d80 msgr2=0x55dd18a1cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- 172.21.15.43:0/2186812698 shutdown_connections 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.885+0000 7f19450c3080 1 -- 172.21.15.43:0/2186812698 wait complete. 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 Processor -- start 2024-09-17T05:29:53.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- start start 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd189df9c0 0x55dd18a8b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 0x55dd18a8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a915b0 0x55dd18a8e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dd18a90c60 con 0x55dd18a247a0 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dd18a1b9a0 con 0x55dd18a915b0 2024-09-17T05:29:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55dd18a1bc40 con 0x55dd189df9c0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 0x55dd18a8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 0x55dd18a8dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35368/0 (socket says 172.21.15.43:35368) 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 -- 172.21.15.43:0/4216031464 learned_addr learned my addr 172.21.15.43:0/4216031464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193e200640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd189df9c0 0x55dd18a8b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193ec00640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a915b0 0x55dd18a8e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 -- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd189df9c0 msgr2=0x55dd18a8b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dd189df9c0 0x55dd18a8b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 -- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a915b0 msgr2=0x55dd18a8e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd18a915b0 0x55dd18a8e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 -- 172.21.15.43:0/4216031464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd18ab30c0 con 0x55dd18a247a0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f193d800640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 0x55dd18a8dad0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f192c00c260 tx=0x7f192c00c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f1936a00640 1 -- 172.21.15.43:0/4216031464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c015020 con 0x55dd18a247a0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f1936a00640 1 -- 172.21.15.43:0/4216031464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f192c011750 con 0x55dd18a247a0 2024-09-17T05:29:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f1936a00640 1 -- 172.21.15.43:0/4216031464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00a040 con 0x55dd18a247a0 2024-09-17T05:29:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dd18ab3360 con 0x55dd18a247a0 2024-09-17T05:29:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.889+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd18ab38a0 con 0x55dd18a247a0 2024-09-17T05:29:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f1936a00640 1 -- 172.21.15.43:0/4216031464 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f192c018020 con 0x55dd18a247a0 2024-09-17T05:29:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f1936a00640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1920071be0 0x7f19200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f1936a00640 1 -- 172.21.15.43:0/4216031464 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f192c093100 con 0x55dd18a247a0 2024-09-17T05:29:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f193e200640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1920071be0 0x7f19200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f19450c3080 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dd18ab0160 0x55dd18ab0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55dd18ab0ab0 con 0x55dd18ab0160 2024-09-17T05:29:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f193ec00640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dd18ab0160 0x55dd18ab0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f193e200640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1920071be0 0x7f19200740f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1930004f60 tx=0x7f1930004950 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f193ec00640 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dd18ab0160 0x55dd18ab0570 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f193ec00640 1 -- 172.21.15.43:0/4216031464 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f193800b040 con 0x55dd18ab0160 2024-09-17T05:29:53.899 INFO:teuthology.orchestra.run.smithi043.stdout:header (0 bytes) : 2024-09-17T05:29:53.900 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dd18ab0160 msgr2=0x55dd18ab0570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.893+0000 7f19450c3080 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dd18ab0160 0x55dd18ab0570 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1920071be0 msgr2=0x7f19200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1920071be0 0x7f19200740f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1930004f60 tx=0x7f1930004950 comp rx=0 tx=0).stop 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 msgr2=0x55dd18a8dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 --2- 172.21.15.43:0/4216031464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd18a247a0 0x55dd18a8dad0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f192c00c260 tx=0x7f192c00c720 comp rx=0 tx=0).stop 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f193ec00640 1 -- 172.21.15.43:0/4216031464 reap_dead start 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 shutdown_connections 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 >> 172.21.15.43:0/4216031464 conn(0x55dd189c6d80 msgr2=0x55dd18a8bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 shutdown_connections 2024-09-17T05:29:53.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.897+0000 7f19450c3080 1 -- 172.21.15.43:0/4216031464 wait complete. 2024-09-17T05:29:53.904 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-09-17T05:29:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.957+0000 7f753b136080 1 Processor -- start 2024-09-17T05:29:53.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.957+0000 7f753b136080 1 -- start start 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111ba6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 0x558111ba7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558111a1da00 con 0x558111b865b0 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558111a7bf40 con 0x558111b869c0 2024-09-17T05:29:53.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558111b158c0 con 0x558111ad49c0 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35394/0 (socket says 172.21.15.43:35394) 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 -- 172.21.15.43:0/1235421270 learned_addr learned my addr 172.21.15.43:0/1235421270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7534200640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111ba6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7534c00640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 0x558111ba7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 -- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 msgr2=0x558111ba6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111ba6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 -- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 msgr2=0x558111ba7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:53.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 0x558111ba7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 -- 172.21.15.43:0/1235421270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581119f6c10 con 0x558111b865b0 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7534c00640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 0x558111ba7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7533800640 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f752000c640 tx=0x7f752000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada170594d2b05aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7532400640 1 -- 172.21.15.43:0/1235421270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752000aa30 con 0x558111b865b0 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7532400640 1 -- 172.21.15.43:0/1235421270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f752000abd0 con 0x558111b865b0 2024-09-17T05:29:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f7532400640 1 -- 172.21.15.43:0/1235421270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520015440 con 0x558111b865b0 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- 172.21.15.43:0/1235421270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 msgr2=0x558111b80e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f752000c640 tx=0x7f752000cb00 comp rx=0 tx=0).stop 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- 172.21.15.43:0/1235421270 shutdown_connections 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b869c0 0x558111ba7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b865b0 0x558111b80e10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 --2- 172.21.15.43:0/1235421270 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111ba6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- 172.21.15.43:0/1235421270 >> 172.21.15.43:0/1235421270 conn(0x558111abbd80 msgr2=0x558111b84fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- 172.21.15.43:0/1235421270 shutdown_connections 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.961+0000 7f753b136080 1 -- 172.21.15.43:0/1235421270 wait complete. 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 Processor -- start 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- start start 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111b7f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b865b0 0x558111baab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 0x558111b0feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558111b849e0 con 0x558111b869c0 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558111b84c80 con 0x558111b865b0 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558111b82000 con 0x558111ad49c0 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 0x558111b0feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111b7f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 0x558111b0feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35410/0 (socket says 172.21.15.43:35410) 2024-09-17T05:29:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 -- 172.21.15.43:0/4279074077 learned_addr learned my addr 172.21.15.43:0/4279074077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7533800640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b865b0 0x558111baab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 -- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 msgr2=0x558111b7f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111b7f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 -- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b865b0 msgr2=0x558111baab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558111b865b0 0x558111baab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 -- 172.21.15.43:0/4279074077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558111bad200 con 0x558111b869c0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534200640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558111ad49c0 0x558111b7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7534c00640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 0x558111b0feb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f752c0ad040 tx=0x7f752c0aa510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7531000640 1 -- 172.21.15.43:0/4279074077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c0ba030 con 0x558111b869c0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7531000640 1 -- 172.21.15.43:0/4279074077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f752c0aab70 con 0x558111b869c0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f7531000640 1 -- 172.21.15.43:0/4279074077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c0af040 con 0x558111b869c0 2024-09-17T05:29:53.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558111bad4a0 con 0x558111b869c0 2024-09-17T05:29:53.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.965+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558111bad9e0 con 0x558111b869c0 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7531000640 1 -- 172.21.15.43:0/4279074077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f752c0bd030 con 0x558111b869c0 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7531000640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7514071be0 0x7f75140740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7531000640 1 -- 172.21.15.43:0/4279074077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f752c12da10 con 0x558111b869c0 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f753b136080 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558111bab040 0x558111c40680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x558111c40bc0 con 0x558111bab040 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7534200640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7514071be0 0x7f75140740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7533800640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558111bab040 0x558111c40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7534200640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7514071be0 0x7f75140740f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f752400e930 tx=0x7f7524002a00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.969+0000 7f7533800640 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558111bab040 0x558111c40680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f7533800640 1 -- 172.21.15.43:0/4279074077 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f752001d040 con 0x558111bab040 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:header (274 bytes) : 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000020 00 00 00 00 ce 13 e9 66 8c 3d 18 29 00 00 00 00 |.......f.=.)....| 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-17T05:29:53.976 INFO:teuthology.orchestra.run.smithi043.stdout:00000050 00 00 ce 13 e9 66 8c 3d 18 29 00 00 00 00 00 00 |.....f.=.)......| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000a0 00 00 00 00 00 00 00 00 ce 13 e9 66 06 93 64 32 |...........f..d2| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000e0 00 00 00 00 00 00 ce 13 e9 66 8c 3d 18 29 00 00 |.........f.=.)..| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:* 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000110 00 00 |..| 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout:00000112 2024-09-17T05:29:53.977 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:53.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558111bab040 msgr2=0x558111c40680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558111bab040 0x558111c40680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7514071be0 msgr2=0x7f75140740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7514071be0 0x7f75140740f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f752400e930 tx=0x7f7524002a00 comp rx=0 tx=0).stop 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 msgr2=0x558111b0feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 --2- 172.21.15.43:0/4279074077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558111b869c0 0x558111b0feb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f752c0ad040 tx=0x7f752c0aa510 comp rx=0 tx=0).stop 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f7534c00640 1 -- 172.21.15.43:0/4279074077 reap_dead start 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 shutdown_connections 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 >> 172.21.15.43:0/4279074077 conn(0x558111abbd80 msgr2=0x558111b87330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 shutdown_connections 2024-09-17T05:29:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:53.973+0000 7f753b136080 1 -- 172.21.15.43:0/4279074077 wait complete. 2024-09-17T05:29:53.980 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-09-17T05:29:54.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.033+0000 7f0acda47080 1 Processor -- start 2024-09-17T05:29:54.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.033+0000 7f0acda47080 1 -- start start 2024-09-17T05:29:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x562909954710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x562909937920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56290982af40 con 0x562909934e40 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562909837d00 con 0x5629098814d0 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5629097cca00 con 0x5629098adcb0 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0abfe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x562909937920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x562909954710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35424/0 (socket says 172.21.15.43:35424) 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 -- 172.21.15.43:0/2404512141 learned_addr learned my addr 172.21.15.43:0/2404512141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 -- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 msgr2=0x562909937920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x562909937920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 -- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 msgr2=0x562909954710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x562909954710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 -- 172.21.15.43:0/2404512141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629099dba80 con 0x562909934e40 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00b1040 tx=0x7f0ac001aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a424c88c45445207 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac4c00640 1 -- 172.21.15.43:0/2404512141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004020 con 0x562909934e40 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac4c00640 1 -- 172.21.15.43:0/2404512141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ac00042b0 con 0x562909934e40 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0ac4c00640 1 -- 172.21.15.43:0/2404512141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00af770 con 0x562909934e40 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- 172.21.15.43:0/2404512141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 msgr2=0x56290987ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00b1040 tx=0x7f0ac001aa30 comp rx=0 tx=0).stop 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- 172.21.15.43:0/2404512141 shutdown_connections 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x56290987ca70 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x562909937920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 --2- 172.21.15.43:0/2404512141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x562909954710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- 172.21.15.43:0/2404512141 >> 172.21.15.43:0/2404512141 conn(0x56290986ad80 msgr2=0x562909880880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- 172.21.15.43:0/2404512141 shutdown_connections 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 -- 172.21.15.43:0/2404512141 wait complete. 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.037+0000 7f0acda47080 1 Processor -- start 2024-09-17T05:29:54.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- start start 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x5629099439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x5629099ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x5629099eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629099dbfc0 con 0x562909934e40 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5629099dbd20 con 0x5629098814d0 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5629099dc260 con 0x5629098adcb0 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x5629099eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x5629099eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35426/0 (socket says 172.21.15.43:35426) 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 -- 172.21.15.43:0/1573970033 learned_addr learned my addr 172.21.15.43:0/1573970033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0abfe00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x5629099ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6000640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x5629099439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 -- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 msgr2=0x5629099ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5629098adcb0 0x5629099ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 -- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 msgr2=0x5629099439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5629098814d0 0x5629099439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 -- 172.21.15.43:0/1573970033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629099eb6a0 con 0x562909934e40 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0ac6a00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x5629099eae10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000a6f0 tx=0x7f0ac00afb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0abea00640 1 -- 172.21.15.43:0/1573970033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00ba630 con 0x562909934e40 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0abea00640 1 -- 172.21.15.43:0/1573970033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ac00ba7d0 con 0x562909934e40 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0abea00640 1 -- 172.21.15.43:0/1573970033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00babc0 con 0x562909934e40 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5629099eb940 con 0x562909934e40 2024-09-17T05:29:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.041+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629099ec540 con 0x562909934e40 2024-09-17T05:29:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abea00640 1 -- 172.21.15.43:0/1573970033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ac00b8040 con 0x562909934e40 2024-09-17T05:29:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abea00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa4071c10 0x7f0aa4074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abea00640 1 -- 172.21.15.43:0/1573970033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f0ac01327c0 con 0x562909934e40 2024-09-17T05:29:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0acda47080 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562909954490 0x5629098c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0ac6000640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa4071c10 0x7f0aa4074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5629098c5e60 con 0x562909954490 2024-09-17T05:29:54.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abfe00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562909954490 0x5629098c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0ac6000640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa4071c10 0x7f0aa4074120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b6b0 tx=0x7f0ab800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abfe00640 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562909954490 0x5629098c5920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0abfe00640 1 -- 172.21.15.43:0/1573970033 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7f0ab00019a0 con 0x562909954490 2024-09-17T05:29:54.051 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs_head 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562909954490 msgr2=0x5629098c5920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.045+0000 7f0acda47080 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562909954490 0x5629098c5920 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa4071c10 msgr2=0x7f0aa4074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0aa4071c10 0x7f0aa4074120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b6b0 tx=0x7f0ab800d040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 msgr2=0x5629099eae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 --2- 172.21.15.43:0/1573970033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562909934e40 0x5629099eae10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000a6f0 tx=0x7f0ac00afb10 comp rx=0 tx=0).stop 2024-09-17T05:29:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0ac6a00640 1 -- 172.21.15.43:0/1573970033 reap_dead start 2024-09-17T05:29:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 shutdown_connections 2024-09-17T05:29:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 >> 172.21.15.43:0/1573970033 conn(0x56290986ad80 msgr2=0x562909951170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 shutdown_connections 2024-09-17T05:29:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.049+0000 7f0acda47080 1 -- 172.21.15.43:0/1573970033 wait complete. 2024-09-17T05:29:54.056 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-09-17T05:29:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 Processor -- start 2024-09-17T05:29:54.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- start start 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d2bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c10f40 0x55bbc4d2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bbc4b7bf40 con 0x55bbc4c12180 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bbc4b88d00 con 0x7f6794003c50 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55bbc4b1da00 con 0x55bbc4c10f40 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35450/0 (socket says 172.21.15.43:35450) 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 -- 172.21.15.43:0/1143346264 learned_addr learned my addr 172.21.15.43:0/1143346264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67afe00640 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d2bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b0800640 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c10f40 0x55bbc4d2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 -- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c10f40 msgr2=0x55bbc4d2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c10f40 0x55bbc4d2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 -- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 msgr2=0x55bbc4d2bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d2bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 -- 172.21.15.43:0/1143346264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc4c1a210 con 0x55bbc4c12180 2024-09-17T05:29:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67af400640 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c310 tx=0x7f679c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff087f8911c7ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67ae000640 1 -- 172.21.15.43:0/1143346264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c013020 con 0x55bbc4c12180 2024-09-17T05:29:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67ae000640 1 -- 172.21.15.43:0/1143346264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f679c00f750 con 0x55bbc4c12180 2024-09-17T05:29:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67ae000640 1 -- 172.21.15.43:0/1143346264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c00fb00 con 0x55bbc4c12180 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- 172.21.15.43:0/1143346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 msgr2=0x55bbc4c10a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c310 tx=0x7f679c00c7d0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- 172.21.15.43:0/1143346264 shutdown_connections 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c10f40 0x55bbc4d2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c12180 0x55bbc4c10a00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/1143346264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d2bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- 172.21.15.43:0/1143346264 >> 172.21.15.43:0/1143346264 conn(0x55bbc4bbbd80 msgr2=0x55bbc4ca5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- 172.21.15.43:0/1143346264 shutdown_connections 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.113+0000 7f67b6bc0080 1 -- 172.21.15.43:0/1143346264 wait complete. 2024-09-17T05:29:54.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 Processor -- start 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- start start 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 0x55bbc4ca3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c12180 0x55bbc4d3c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d3c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bbc4c85870 con 0x55bbc4c10f40 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bbc4d304f0 con 0x7f6794003c50 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55bbc4c85b10 con 0x55bbc4c12180 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 0x55bbc4ca3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 0x55bbc4ca3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35452/0 (socket says 172.21.15.43:35452) 2024-09-17T05:29:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 -- 172.21.15.43:0/3461170284 learned_addr learned my addr 172.21.15.43:0/3461170284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67af400640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c12180 0x55bbc4d3c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b0800640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d3c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 -- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c12180 msgr2=0x55bbc4d3c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55bbc4c12180 0x55bbc4d3c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 -- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 msgr2=0x55bbc4d3c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d3c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 -- 172.21.15.43:0/3461170284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc4d2f050 con 0x55bbc4c10f40 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b0800640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6794003c50 0x55bbc4d3c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67afe00640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 0x55bbc4ca3070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f67a00019d0 tx=0x7f67a000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67acc00640 1 -- 172.21.15.43:0/3461170284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000d6c0 con 0x55bbc4c10f40 2024-09-17T05:29:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67acc00640 1 -- 172.21.15.43:0/3461170284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a000d860 con 0x55bbc4c10f40 2024-09-17T05:29:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67acc00640 1 -- 172.21.15.43:0/3461170284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000dcb0 con 0x55bbc4c10f40 2024-09-17T05:29:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bbc4d2fe80 con 0x55bbc4c10f40 2024-09-17T05:29:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.117+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc4ca35b0 con 0x55bbc4c10f40 2024-09-17T05:29:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67acc00640 1 -- 172.21.15.43:0/3461170284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67a001a420 con 0x55bbc4c10f40 2024-09-17T05:29:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67acc00640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6790071be0 0x7f67900740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67acc00640 1 -- 172.21.15.43:0/3461170284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f67a0095dd0 con 0x55bbc4c10f40 2024-09-17T05:29:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55bbc4d2eb80 0x55bbc4bcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67af400640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6790071be0 0x7f67900740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55bbc4bd0130 con 0x55bbc4d2eb80 2024-09-17T05:29:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67b0800640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55bbc4d2eb80 0x55bbc4bcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67af400640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6790071be0 0x7f67900740f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f679c000c10 tx=0x7f679c01b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.121+0000 7f67b0800640 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55bbc4d2eb80 0x55bbc4bcfbf0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b0800640 1 -- 172.21.15.43:0/3461170284 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f67a8002030 con 0x55bbc4d2eb80 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55bbc4d2eb80 msgr2=0x55bbc4bcfbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55bbc4d2eb80 0x55bbc4bcfbf0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6790071be0 msgr2=0x7f67900740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6790071be0 0x7f67900740f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f679c000c10 tx=0x7f679c01b040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 msgr2=0x55bbc4ca3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 --2- 172.21.15.43:0/3461170284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bbc4c10f40 0x55bbc4ca3070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f67a00019d0 tx=0x7f67a000f840 comp rx=0 tx=0).stop 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b0800640 1 -- 172.21.15.43:0/3461170284 reap_dead start 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 shutdown_connections 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 >> 172.21.15.43:0/3461170284 conn(0x55bbc4bbbd80 msgr2=0x55bbc4ca5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 shutdown_connections 2024-09-17T05:29:54.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.125+0000 7f67b6bc0080 1 -- 172.21.15.43:0/3461170284 wait complete. 2024-09-17T05:29:54.133 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-09-17T05:29:54.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.185+0000 7ff6cab08080 1 Processor -- start 2024-09-17T05:29:54.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.185+0000 7ff6cab08080 1 -- start start 2024-09-17T05:29:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 0x564b356e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c58b0 0x564b356e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b3555aa00 con 0x564b3563bcb0 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564b355b8f40 con 0x564b356c4010 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564b3560f5b0 con 0x564b356c58b0 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35458/0 (socket says 172.21.15.43:35458) 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 -- 172.21.15.43:0/99524918 learned_addr learned my addr 172.21.15.43:0/99524918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3200640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 0x564b356e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c4600640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c58b0 0x564b356e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 -- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c58b0 msgr2=0x564b356e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c58b0 0x564b356e1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 -- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 msgr2=0x564b356e0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 0x564b356e0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 -- 172.21.15.43:0/99524918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b35533c10 con 0x564b3563bcb0 2024-09-17T05:29:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3200640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 0x564b356e0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4018e80 tx=0x7ff6b4017ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5fae8b7ffd9a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c1e00640 1 -- 172.21.15.43:0/99524918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40ba070 con 0x564b3563bcb0 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c1e00640 1 -- 172.21.15.43:0/99524918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6b40ab980 con 0x564b3563bcb0 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6c1e00640 1 -- 172.21.15.43:0/99524918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40abd90 con 0x564b3563bcb0 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- 172.21.15.43:0/99524918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 msgr2=0x564b35653280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4018e80 tx=0x7ff6b4017ff0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- 172.21.15.43:0/99524918 shutdown_connections 2024-09-17T05:29:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c4010 0x564b356e0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c58b0 0x564b356e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 --2- 172.21.15.43:0/99524918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b35653280 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- 172.21.15.43:0/99524918 >> 172.21.15.43:0/99524918 conn(0x564b355f8d80 msgr2=0x564b356e22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- 172.21.15.43:0/99524918 shutdown_connections 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 -- 172.21.15.43:0/99524918 wait complete. 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.189+0000 7ff6cab08080 1 Processor -- start 2024-09-17T05:29:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- start start 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b356e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c4010 0x564b356e21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c58b0 0x564b356dae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564b356e0650 con 0x564b3563bcb0 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564b356e08f0 con 0x564b356c58b0 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564b356e2b10 con 0x564b356c4010 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b356e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b356e1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35466/0 (socket says 172.21.15.43:35466) 2024-09-17T05:29:54.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 -- 172.21.15.43:0/2621247793 learned_addr learned my addr 172.21.15.43:0/2621247793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c4600640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c58b0 0x564b356dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3200640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c4010 0x564b356e21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 -- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c4010 msgr2=0x564b356e21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564b356c4010 0x564b356e21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 -- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c58b0 msgr2=0x564b356dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b356c58b0 0x564b356dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 -- 172.21.15.43:0/2621247793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b35652960 con 0x564b3563bcb0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c3c00640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b356e1c60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40b8040 tx=0x7ff6b40acc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c0a00640 1 -- 172.21.15.43:0/2621247793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40ba070 con 0x564b3563bcb0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c0a00640 1 -- 172.21.15.43:0/2621247793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6b40b5550 con 0x564b3563bcb0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564b356c4b40 con 0x564b3563bcb0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6c0a00640 1 -- 172.21.15.43:0/2621247793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40b59a0 con 0x564b3563bcb0 2024-09-17T05:29:54.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.193+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3564f2c0 con 0x564b3563bcb0 2024-09-17T05:29:54.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c0a00640 1 -- 172.21.15.43:0/2621247793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6b40b3070 con 0x564b3563bcb0 2024-09-17T05:29:54.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c0a00640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6a4071be0 0x7ff6a40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c0a00640 1 -- 172.21.15.43:0/2621247793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7ff6b412dfe0 con 0x564b3563bcb0 2024-09-17T05:29:54.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564b3564f760 0x564b3564fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c3200640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6a4071be0 0x7ff6a40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x564b356500b0 con 0x564b3564f760 2024-09-17T05:29:54.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c4600640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564b3564f760 0x564b3564fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c3200640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6a4071be0 0x7ff6a40740f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0004fd0 tx=0x7ff6b0002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c4600640 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564b3564f760 0x564b3564fb70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c4600640 1 -- 172.21.15.43:0/2621247793 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7ff6bc003020 con 0x564b3564f760 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564b3564f760 msgr2=0x564b3564fb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564b3564f760 0x564b3564fb70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6a4071be0 msgr2=0x7ff6a40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6a4071be0 0x7ff6a40740f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0004fd0 tx=0x7ff6b0002040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 msgr2=0x564b356e1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6cab08080 1 --2- 172.21.15.43:0/2621247793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564b3563bcb0 0x564b356e1c60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40b8040 tx=0x7ff6b40acc00 comp rx=0 tx=0).stop 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.197+0000 7ff6c4600640 1 -- 172.21.15.43:0/2621247793 reap_dead start 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.201+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 shutdown_connections 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.201+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 >> 172.21.15.43:0/2621247793 conn(0x564b355f8d80 msgr2=0x564b356c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.201+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 shutdown_connections 2024-09-17T05:29:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.201+0000 7ff6cab08080 1 -- 172.21.15.43:0/2621247793 wait complete. 2024-09-17T05:29:54.209 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-09-17T05:29:54.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.261+0000 7fc6117d9080 1 Processor -- start 2024-09-17T05:29:54.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.261+0000 7fc6117d9080 1 -- start start 2024-09-17T05:29:54.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 0x562168c63650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562168aa6f40 con 0x562168afad70 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562168ab3d00 con 0x562168bb0e40 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562168a48a00 con 0x562168b29cb0 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35470/0 (socket says 172.21.15.43:35470) 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 -- 172.21.15.43:0/4133404935 learned_addr learned my addr 172.21.15.43:0/4133404935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60b400640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60a000640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 0x562168c63650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 -- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 msgr2=0x562168c63650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 0x562168c63650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 -- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 msgr2=0x562168c63b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c63b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 -- 172.21.15.43:0/4133404935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562168b295f0 con 0x562168afad70 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60a000640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 0x562168c63650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc60aa00640 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc004f50 tx=0x7fc5fc01a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e78042b58dc5ed server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc608c00640 1 -- 172.21.15.43:0/4133404935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0a77e0 con 0x562168afad70 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc608c00640 1 -- 172.21.15.43:0/4133404935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5fc0a7ab0 con 0x562168afad70 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc608c00640 1 -- 172.21.15.43:0/4133404935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0b0470 con 0x562168afad70 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- 172.21.15.43:0/4133404935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 msgr2=0x562168b3fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc004f50 tx=0x7fc5fc01a7b0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- 172.21.15.43:0/4133404935 shutdown_connections 2024-09-17T05:29:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168b29cb0 0x562168c63650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 --2- 172.21.15.43:0/4133404935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168afad70 0x562168b3fc30 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- 172.21.15.43:0/4133404935 >> 172.21.15.43:0/4133404935 conn(0x562168ae6d80 msgr2=0x562168af8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- 172.21.15.43:0/4133404935 shutdown_connections 2024-09-17T05:29:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.265+0000 7fc6117d9080 1 -- 172.21.15.43:0/4133404935 wait complete. 2024-09-17T05:29:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 Processor -- start 2024-09-17T05:29:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- start start 2024-09-17T05:29:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168afad70 0x562168c671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 0x562168c684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c64f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562168b29890 con 0x562168b29cb0 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562168b420d0 con 0x562168bb0e40 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562168b42370 con 0x562168afad70 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 0x562168c684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 0x562168c684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35472/0 (socket says 172.21.15.43:35472) 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 -- 172.21.15.43:0/1280772619 learned_addr learned my addr 172.21.15.43:0/1280772619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60aa00640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168afad70 0x562168c671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60b400640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 -- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168afad70 msgr2=0x562168c671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562168afad70 0x562168c671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 -- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 msgr2=0x562168c64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562168bb0e40 0x562168c64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 -- 172.21.15.43:0/1280772619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562168c67720 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc60a000640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 0x562168c684e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8002eb0 tx=0x7fc5f800b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc603400640 1 -- 172.21.15.43:0/1280772619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8013070 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc603400640 1 -- 172.21.15.43:0/1280772619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f800f910 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc603400640 1 -- 172.21.15.43:0/1280772619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800fd20 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562168c68cd0 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.269+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562168c694b0 con 0x562168b29cb0 2024-09-17T05:29:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc603400640 1 -- 172.21.15.43:0/1280772619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5f8017410 con 0x562168b29cb0 2024-09-17T05:29:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc603400640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ec071be0 0x7fc5ec0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc603400640 1 -- 172.21.15.43:0/1280772619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fc5f8093290 con 0x562168b29cb0 2024-09-17T05:29:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562168b21c50 0x562168bb30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60aa00640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ec071be0 0x7fc5ec0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x562168c69bf0 con 0x562168b21c50 2024-09-17T05:29:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60b400640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562168b21c50 0x562168bb30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60aa00640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ec071be0 0x7fc5ec0740f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ba000 tx=0x7fc5fc0b3040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60b400640 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562168b21c50 0x562168bb30b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60b400640 1 -- 172.21.15.43:0/1280772619 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv18 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fc604010550 con 0x562168b21c50 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562168b21c50 msgr2=0x562168bb30b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562168b21c50 0x562168bb30b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ec071be0 msgr2=0x7fc5ec0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5ec071be0 0x7fc5ec0740f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ba000 tx=0x7fc5fc0b3040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 msgr2=0x562168c684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc6117d9080 1 --2- 172.21.15.43:0/1280772619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562168b29cb0 0x562168c684e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8002eb0 tx=0x7fc5f800b740 comp rx=0 tx=0).stop 2024-09-17T05:29:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.273+0000 7fc60b400640 1 -- 172.21.15.43:0/1280772619 reap_dead start 2024-09-17T05:29:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.277+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 shutdown_connections 2024-09-17T05:29:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.277+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 >> 172.21.15.43:0/1280772619 conn(0x562168ae6d80 msgr2=0x562168bccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.277+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 shutdown_connections 2024-09-17T05:29:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.277+0000 7fc6117d9080 1 -- 172.21.15.43:0/1280772619 wait complete. 2024-09-17T05:29:54.284 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-09-17T05:29:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.337+0000 7f6fc6a67080 1 Processor -- start 2024-09-17T05:29:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.337+0000 7f6fc6a67080 1 -- start start 2024-09-17T05:29:54.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08f63280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08f1c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa08e6aa00 con 0x55fa08fd4010 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fa08ec8f40 con 0x55fa08fd58b0 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55fa08f1f760 con 0x55fa08f637c0 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35490/0 (socket says 172.21.15.43:35490) 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 -- 172.21.15.43:0/3786606548 learned_addr learned my addr 172.21.15.43:0/3786606548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 -- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 msgr2=0x55fa08f1c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08f1c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 -- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 msgr2=0x55fa08f63280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08f63280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 -- 172.21.15.43:0/3786606548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa08e43c10 con 0x55fa08fd4010 2024-09-17T05:29:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006f40 tx=0x7f6fb000b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbf7c3f4beec7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbde00640 1 -- 172.21.15.43:0/3786606548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000e840 con 0x55fa08fd4010 2024-09-17T05:29:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbde00640 1 -- 172.21.15.43:0/3786606548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb000ead0 con 0x55fa08fd4010 2024-09-17T05:29:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fbde00640 1 -- 172.21.15.43:0/3786606548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000d390 con 0x55fa08fd4010 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- 172.21.15.43:0/3786606548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 msgr2=0x55fa08ff2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006f40 tx=0x7f6fb000b740 comp rx=0 tx=0).stop 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- 172.21.15.43:0/3786606548 shutdown_connections 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08f63280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08f1c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/3786606548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08ff2760 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- 172.21.15.43:0/3786606548 >> 172.21.15.43:0/3786606548 conn(0x55fa08f08d80 msgr2=0x55fa08ff2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- 172.21.15.43:0/3786606548 shutdown_connections 2024-09-17T05:29:54.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- 172.21.15.43:0/3786606548 wait complete. 2024-09-17T05:29:54.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 Processor -- start 2024-09-17T05:29:54.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.341+0000 7f6fc6a67080 1 -- start start 2024-09-17T05:29:54.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08ff2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08fd2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08fee540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fa08ff2ca0 con 0x55fa08fd4010 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fa08fd2a80 con 0x55fa08fd58b0 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55fa08fed620 con 0x55fa08f637c0 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08fd2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08fd2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35498/0 (socket says 172.21.15.43:35498) 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 -- 172.21.15.43:0/940866917 learned_addr learned my addr 172.21.15.43:0/940866917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08ff2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc0600640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08fee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 -- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 msgr2=0x55fa08ff2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fa08f637c0 0x55fa08ff2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 -- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 msgr2=0x55fa08fee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fa08fd58b0 0x55fa08fee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 -- 172.21.15.43:0/940866917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa08fed8c0 con 0x55fa08fd4010 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbf200640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08fd2540 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00a6f0 tx=0x7f6fac00abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbca00640 1 -- 172.21.15.43:0/940866917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac009850 con 0x55fa08fd4010 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbca00640 1 -- 172.21.15.43:0/940866917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fac0099f0 con 0x55fa08fd4010 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa08fedb60 con 0x55fa08fd4010 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fbca00640 1 -- 172.21.15.43:0/940866917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac009e40 con 0x55fa08fd4010 2024-09-17T05:29:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.345+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa08fee0a0 con 0x55fa08fd4010 2024-09-17T05:29:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fbca00640 1 -- 172.21.15.43:0/940866917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6fac008460 con 0x55fa08fd4010 2024-09-17T05:29:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fbca00640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fa0071be0 0x7f6fa00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fbca00640 1 -- 172.21.15.43:0/940866917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6fac093410 con 0x55fa08fd4010 2024-09-17T05:29:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fa0071be0 0x7f6fa00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55fa08fd2d40 0x55fa0908c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55fa0908c8d0 con 0x55fa08fd2d40 2024-09-17T05:29:54.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fc0600640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55fa08fd2d40 0x55fa0908c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fbfc00640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fa0071be0 0x7f6fa00740f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006e60 tx=0x7f6fb0019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.349+0000 7f6fc0600640 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55fa08fd2d40 0x55fa0908c390 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc0600640 1 -- 172.21.15.43:0/940866917 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f6fb8002050 con 0x55fa08fd2d40 2024-09-17T05:29:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55fa08fd2d40 msgr2=0x55fa0908c390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55fa08fd2d40 0x55fa0908c390 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fa0071be0 msgr2=0x7f6fa00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fa0071be0 0x7f6fa00740f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006e60 tx=0x7f6fb0019040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 msgr2=0x55fa08fd2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 --2- 172.21.15.43:0/940866917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fa08fd4010 0x55fa08fd2540 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00a6f0 tx=0x7f6fac00abb0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc0600640 1 -- 172.21.15.43:0/940866917 reap_dead start 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 shutdown_connections 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 >> 172.21.15.43:0/940866917 conn(0x55fa08f08d80 msgr2=0x55fa08ff2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 shutdown_connections 2024-09-17T05:29:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.353+0000 7f6fc6a67080 1 -- 172.21.15.43:0/940866917 wait complete. 2024-09-17T05:29:54.361 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-09-17T05:29:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.413+0000 7f9f39469080 1 Processor -- start 2024-09-17T05:29:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.413+0000 7f9f39469080 1 -- start start 2024-09-17T05:29:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d962d70 0x55b87d9ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87d9a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b87d90ef40 con 0x55b87d991cb0 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b87d91bd00 con 0x55b87da18e40 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b87d8b0a00 con 0x55b87d962d70 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35514/0 (socket says 172.21.15.43:35514) 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 -- 172.21.15.43:0/1775059379 learned_addr learned my addr 172.21.15.43:0/1775059379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f32600640 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d962d70 0x55b87d9ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f33000640 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87d9a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 -- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d962d70 msgr2=0x55b87d9ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d962d70 0x55b87d9ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 -- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 msgr2=0x55b87d9a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87d9a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 -- 172.21.15.43:0/1775059379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87d9916d0 con 0x55b87d991cb0 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f31c00640 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9f20001e60 tx=0x7f9f2000b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b3994e8560e3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f2be00640 1 -- 172.21.15.43:0/1775059379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20012070 con 0x55b87d991cb0 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f2be00640 1 -- 172.21.15.43:0/1775059379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f2000e840 con 0x55b87d991cb0 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f2be00640 1 -- 172.21.15.43:0/1775059379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000ec90 con 0x55b87d991cb0 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- 172.21.15.43:0/1775059379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 msgr2=0x55b87d98edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9f20001e60 tx=0x7f9f2000b740 comp rx=0 tx=0).stop 2024-09-17T05:29:54.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- 172.21.15.43:0/1775059379 shutdown_connections 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87d9a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d991cb0 0x55b87d98edc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 --2- 172.21.15.43:0/1775059379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d962d70 0x55b87d9ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- 172.21.15.43:0/1775059379 >> 172.21.15.43:0/1775059379 conn(0x55b87d94ed80 msgr2=0x55b87da37c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- 172.21.15.43:0/1775059379 shutdown_connections 2024-09-17T05:29:54.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.417+0000 7f9f39469080 1 -- 172.21.15.43:0/1775059379 wait complete. 2024-09-17T05:29:54.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 Processor -- start 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- start start 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 0x55b87d9a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d991cb0 0x55b87da25030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87dac2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b87d991970 con 0x55b87d962d70 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b87d9aab00 con 0x55b87da18e40 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b87d9aada0 con 0x55b87d991cb0 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 0x55b87d9a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 0x55b87d9a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35530/0 (socket says 172.21.15.43:35530) 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 -- 172.21.15.43:0/3278898713 learned_addr learned my addr 172.21.15.43:0/3278898713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f33000640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87dac2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f31c00640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d991cb0 0x55b87da25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 -- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d991cb0 msgr2=0x55b87da25030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b87d991cb0 0x55b87da25030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 -- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 msgr2=0x55b87dac2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87dac2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 -- 172.21.15.43:0/3278898713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87da256a0 con 0x55b87d962d70 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f33000640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b87da18e40 0x55b87dac2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f32600640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 0x55b87d9a7590 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401c9c0 tx=0x7f9f24004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f2aa00640 1 -- 172.21.15.43:0/3278898713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b0670 con 0x55b87d962d70 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f2aa00640 1 -- 172.21.15.43:0/3278898713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f240b0810 con 0x55b87d962d70 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b87da25940 con 0x55b87d962d70 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f2aa00640 1 -- 172.21.15.43:0/3278898713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b0c60 con 0x55b87d962d70 2024-09-17T05:29:54.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.421+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87da25e80 con 0x55b87d962d70 2024-09-17T05:29:54.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f2aa00640 1 -- 172.21.15.43:0/3278898713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f240bc3f0 con 0x55b87d962d70 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f2aa00640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f14071c10 0x7f9f14074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f2aa00640 1 -- 172.21.15.43:0/3278898713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f9f241309d0 con 0x55b87d962d70 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f39469080 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b87dac38f0 0x55b87dac3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f31c00640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f14071c10 0x7f9f14074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b87dac4240 con 0x55b87dac38f0 2024-09-17T05:29:54.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.425+0000 7f9f33000640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b87dac38f0 0x55b87dac3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f31c00640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f14071c10 0x7f9f14074120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9f20001c40 tx=0x7f9f20019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f33000640 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b87dac38f0 0x55b87dac3d00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f33000640 1 -- 172.21.15.43:0/3278898713 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f9f2c0015c0 con 0x55b87dac38f0 2024-09-17T05:29:54.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b87dac38f0 msgr2=0x55b87dac3d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b87dac38f0 0x55b87dac3d00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f14071c10 msgr2=0x7f9f14074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f14071c10 0x7f9f14074120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9f20001c40 tx=0x7f9f20019040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 msgr2=0x55b87d9a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 --2- 172.21.15.43:0/3278898713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b87d962d70 0x55b87d9a7590 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401c9c0 tx=0x7f9f24004f10 comp rx=0 tx=0).stop 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f33000640 1 -- 172.21.15.43:0/3278898713 reap_dead start 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 shutdown_connections 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 >> 172.21.15.43:0/3278898713 conn(0x55b87d94ed80 msgr2=0x55b87d960890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 shutdown_connections 2024-09-17T05:29:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.429+0000 7f9f39469080 1 -- 172.21.15.43:0/3278898713 wait complete. 2024-09-17T05:29:54.437 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-09-17T05:29:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.489+0000 7fcaa9c3a080 1 Processor -- start 2024-09-17T05:29:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- start start 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04eedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04e43e40 0x55df04eee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df04cdba00 con 0x55df04d904d0 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df04d39f40 con 0x55df04dbccb0 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55df04e44b30 con 0x55df04e43e40 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35552/0 (socket says 172.21.15.43:35552) 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 -- 172.21.15.43:0/1382737648 learned_addr learned my addr 172.21.15.43:0/1382737648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa3800640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04e43e40 0x55df04eee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2400640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04eedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 -- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04e43e40 msgr2=0x55df04eee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04e43e40 0x55df04eee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 -- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 msgr2=0x55df04eedf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04eedf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 -- 172.21.15.43:0/1382737648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df04cb4c10 con 0x55df04d904d0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2400640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04eedf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fca9400b230 tx=0x7fca9400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52374fa211c0530 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa1000640 1 -- 172.21.15.43:0/1382737648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94016070 con 0x55df04d904d0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa1000640 1 -- 172.21.15.43:0/1382737648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca9400d4c0 con 0x55df04d904d0 2024-09-17T05:29:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa1000640 1 -- 172.21.15.43:0/1382737648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400d710 con 0x55df04d904d0 2024-09-17T05:29:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/1382737648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 msgr2=0x55df04ee89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fca9400b230 tx=0x7fca9400b6f0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/1382737648 shutdown_connections 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04eedf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04e43e40 0x55df04eee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/1382737648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04d904d0 0x55df04ee89d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/1382737648 >> 172.21.15.43:0/1382737648 conn(0x55df04d79d80 msgr2=0x55df04dd3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/1382737648 shutdown_connections 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.493+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/1382737648 wait complete. 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 Processor -- start 2024-09-17T05:29:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- start start 2024-09-17T05:29:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04d904d0 0x55df04e5e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04d8f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 0x55df04efa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55df04eec610 con 0x55df04e43e40 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df04eec3b0 con 0x55df04dbccb0 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55df04eed530 con 0x55df04d904d0 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 0x55df04efa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 0x55df04efa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35568/0 (socket says 172.21.15.43:35568) 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 -- 172.21.15.43:0/2860489352 learned_addr learned my addr 172.21.15.43:0/2860489352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa2400640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04d8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 -- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04d904d0 msgr2=0x55df04e5e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04d904d0 0x55df04e5e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04d904d0 0x55df04e5e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 -- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 msgr2=0x55df04d8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04d8f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 -- 172.21.15.43:0/2860489352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df04d8c1d0 con 0x55df04e43e40 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa2400640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df04dbccb0 0x55df04d8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55df04d904d0 0x55df04e5e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa3800640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 0x55df04efa5a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0107c0 tx=0x7fca9c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fca9b400640 1 -- 172.21.15.43:0/2860489352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00ed30 con 0x55df04e43e40 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fca9b400640 1 -- 172.21.15.43:0/2860489352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca9c019440 con 0x55df04e43e40 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df04d8e900 con 0x55df04e43e40 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fca9b400640 1 -- 172.21.15.43:0/2860489352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0198b0 con 0x55df04e43e40 2024-09-17T05:29:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.497+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df04d8ee40 con 0x55df04e43e40 2024-09-17T05:29:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fca9b400640 1 -- 172.21.15.43:0/2860489352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fca9c0223f0 con 0x55df04e43e40 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fca9b400640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca84071be0 0x7fca840740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fca9b400640 1 -- 172.21.15.43:0/2860489352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fca9c0943b0 con 0x55df04e43e40 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55df04d8fac0 0x55df04daf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca84071be0 0x7fca840740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55df04e5f990 con 0x55df04d8fac0 2024-09-17T05:29:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.501+0000 7fcaa2400640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55df04d8fac0 0x55df04daf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa2e00640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca84071be0 0x7fca840740f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca94014040 tx=0x7fca94001090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa2400640 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55df04d8fac0 0x55df04daf6a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa2400640 1 -- 172.21.15.43:0/2860489352 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fca900a77c0 con 0x55df04d8fac0 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55df04d8fac0 msgr2=0x55df04daf6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55df04d8fac0 0x55df04daf6a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca84071be0 msgr2=0x7fca840740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fca84071be0 0x7fca840740f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fca94014040 tx=0x7fca94001090 comp rx=0 tx=0).stop 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 msgr2=0x55df04efa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 --2- 172.21.15.43:0/2860489352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55df04e43e40 0x55df04efa5a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0107c0 tx=0x7fca9c010c80 comp rx=0 tx=0).stop 2024-09-17T05:29:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa3800640 1 -- 172.21.15.43:0/2860489352 reap_dead start 2024-09-17T05:29:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 shutdown_connections 2024-09-17T05:29:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 >> 172.21.15.43:0/2860489352 conn(0x55df04d79d80 msgr2=0x55df04e46730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 shutdown_connections 2024-09-17T05:29:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.505+0000 7fcaa9c3a080 1 -- 172.21.15.43:0/2860489352 wait complete. 2024-09-17T05:29:54.513 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-09-17T05:29:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 Processor -- start 2024-09-17T05:29:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 -- start start 2024-09-17T05:29:54.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c6e5b0 0x560937d23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c8e280 0x560937d23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560937b05a00 con 0x560937bbc9c0 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560937b63f40 con 0x560937c8e280 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3029249080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560937bfdf00 con 0x560937c6e5b0 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35582/0 (socket says 172.21.15.43:35582) 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 -- 172.21.15.43:0/3947817194 learned_addr learned my addr 172.21.15.43:0/3947817194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3022000640 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c8e280 0x560937d23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3020c00640 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c6e5b0 0x560937d23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 -- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c6e5b0 msgr2=0x560937d23220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c6e5b0 0x560937d23220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 -- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c8e280 msgr2=0x560937d23760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c8e280 0x560937d23760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 -- 172.21.15.43:0/3947817194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560937adec10 con 0x560937bbc9c0 2024-09-17T05:29:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f3021600640 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3014019740 tx=0x7f3014017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e767fbfb9a0448b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f301b400640 1 -- 172.21.15.43:0/3947817194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140ac470 con 0x560937bbc9c0 2024-09-17T05:29:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f301b400640 1 -- 172.21.15.43:0/3947817194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30140ac610 con 0x560937bbc9c0 2024-09-17T05:29:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.569+0000 7f301b400640 1 -- 172.21.15.43:0/3947817194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140aca40 con 0x560937bbc9c0 2024-09-17T05:29:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- 172.21.15.43:0/3947817194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 msgr2=0x560937c8dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3014019740 tx=0x7f3014017e30 comp rx=0 tx=0).stop 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- 172.21.15.43:0/3947817194 shutdown_connections 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c8e280 0x560937d23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c6e5b0 0x560937d23220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- 172.21.15.43:0/3947817194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c8dd40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- 172.21.15.43:0/3947817194 >> 172.21.15.43:0/3947817194 conn(0x560937ba3d80 msgr2=0x560937bf9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- 172.21.15.43:0/3947817194 shutdown_connections 2024-09-17T05:29:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- 172.21.15.43:0/3947817194 wait complete. 2024-09-17T05:29:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 Processor -- start 2024-09-17T05:29:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- start start 2024-09-17T05:29:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c95860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c6e5b0 0x560937c6c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c8e280 0x560937c8a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560937c8b4a0 con 0x560937bbc9c0 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560937bfb820 con 0x560937c6e5b0 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3029249080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560937c6d3a0 con 0x560937c8e280 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c95860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c95860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35594/0 (socket says 172.21.15.43:35594) 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 -- 172.21.15.43:0/2714705288 learned_addr learned my addr 172.21.15.43:0/2714705288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3020c00640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c6e5b0 0x560937c6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3022000640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c8e280 0x560937c8a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 -- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c8e280 msgr2=0x560937c8a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560937c8e280 0x560937c8a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 -- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c6e5b0 msgr2=0x560937c6c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560937c6e5b0 0x560937c6c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 -- 172.21.15.43:0/2714705288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560937c6cbd0 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.573+0000 7f3021600640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c95860 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f301401b570 tx=0x7f30140acd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 -- 172.21.15.43:0/2714705288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140b7630 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 -- 172.21.15.43:0/2714705288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30140bf070 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560937c72530 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 -- 172.21.15.43:0/2714705288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140b7a50 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560937c95120 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 -- 172.21.15.43:0/2714705288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30140b7bf0 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3004071c10 0x7f3004074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f301a000640 1 -- 172.21.15.43:0/2714705288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f301412dfd0 con 0x560937bbc9c0 2024-09-17T05:29:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f3029249080 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x560937c946e0 0x560937c5dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x560937c5e150 con 0x560937c946e0 2024-09-17T05:29:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.577+0000 7f3020c00640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3004071c10 0x7f3004074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3022000640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x560937c946e0 0x560937c5dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3022000640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x560937c946e0 0x560937c5dc10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3020c00640 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3004071c10 0x7f3004074120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3010001870 tx=0x7f3010002f70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3022000640 1 -- 172.21.15.43:0/2714705288 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f301c00b040 con 0x560937c946e0 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x560937c946e0 msgr2=0x560937c5dc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x560937c946e0 0x560937c5dc10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3004071c10 msgr2=0x7f3004074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3004071c10 0x7f3004074120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3010001870 tx=0x7f3010002f70 comp rx=0 tx=0).stop 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 msgr2=0x560937c95860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 --2- 172.21.15.43:0/2714705288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560937bbc9c0 0x560937c95860 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f301401b570 tx=0x7f30140acd10 comp rx=0 tx=0).stop 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3022000640 1 -- 172.21.15.43:0/2714705288 reap_dead start 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 shutdown_connections 2024-09-17T05:29:54.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 >> 172.21.15.43:0/2714705288 conn(0x560937ba3d80 msgr2=0x560937c006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 shutdown_connections 2024-09-17T05:29:54.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.581+0000 7f3029249080 1 -- 172.21.15.43:0/2714705288 wait complete. 2024-09-17T05:29:54.590 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-09-17T05:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 Processor -- start 2024-09-17T05:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 -- start start 2024-09-17T05:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae705f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f6e60 0x55ae705f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ae7046ea00 con 0x55ae705f80e0 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ae704ccf40 con 0x55ae705f6e60 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ae705235b0 con 0x7f1558003100 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1572c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae705f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35614/0 (socket says 172.21.15.43:35614) 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 -- 172.21.15.43:0/973309815 learned_addr learned my addr 172.21.15.43:0/973309815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1572200640 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f6e60 0x55ae705f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 -- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 msgr2=0x55ae705f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae705f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 -- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f6e60 msgr2=0x55ae705f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f6e60 0x55ae705f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 -- 172.21.15.43:0/973309815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae70447c10 con 0x55ae705f80e0 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1572c00640 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae705f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1573600640 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55ae70550880 tx=0x7f156c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ed6bb708d57b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1570e00640 1 -- 172.21.15.43:0/973309815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c009270 con 0x55ae705f80e0 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1570e00640 1 -- 172.21.15.43:0/973309815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f156c007590 con 0x55ae705f80e0 2024-09-17T05:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1570e00640 1 -- 172.21.15.43:0/973309815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0079e0 con 0x55ae705f80e0 2024-09-17T05:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 -- 172.21.15.43:0/973309815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 msgr2=0x55ae705f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.649+0000 7f1579991080 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55ae70550880 tx=0x7f156c0098f0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- 172.21.15.43:0/973309815 shutdown_connections 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f6e60 0x55ae705f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f80e0 0x55ae705f84f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- 172.21.15.43:0/973309815 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae705f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- 172.21.15.43:0/973309815 >> 172.21.15.43:0/973309815 conn(0x55ae7050cd80 msgr2=0x55ae705d77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- 172.21.15.43:0/973309815 shutdown_connections 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- 172.21.15.43:0/973309815 wait complete. 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 Processor -- start 2024-09-17T05:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- start start 2024-09-17T05:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 0x55ae706919c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f80e0 0x55ae70563780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae70521aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ae705685c0 con 0x55ae705f6e60 2024-09-17T05:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ae705f8a30 con 0x55ae705f80e0 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1579991080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ae705f4400 con 0x7f1558003100 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 0x55ae706919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 0x55ae706919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35630/0 (socket says 172.21.15.43:35630) 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 -- 172.21.15.43:0/3919224778 learned_addr learned my addr 172.21.15.43:0/3919224778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1573600640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae70521aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572200640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f80e0 0x55ae70563780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 -- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 msgr2=0x55ae70521aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1558003100 0x55ae70521aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 -- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f80e0 msgr2=0x55ae70563780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f80e0 0x55ae70563780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 -- 172.21.15.43:0/3919224778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae70521fe0 con 0x55ae705f6e60 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572200640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae705f80e0 0x55ae70563780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.653+0000 7f1572c00640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 0x55ae706919c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f156400aa90 tx=0x7f156400af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 -- 172.21.15.43:0/3919224778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156401c070 con 0x55ae705f6e60 2024-09-17T05:29:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 -- 172.21.15.43:0/3919224778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1564017440 con 0x55ae705f6e60 2024-09-17T05:29:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 -- 172.21.15.43:0/3919224778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564017810 con 0x55ae705f6e60 2024-09-17T05:29:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ae7054d030 con 0x55ae705f6e60 2024-09-17T05:29:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae70692290 con 0x55ae705f6e60 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 -- 172.21.15.43:0/3919224778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f156401f070 con 0x55ae705f6e60 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1554071be0 0x7f15540740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f156b400640 1 -- 172.21.15.43:0/3919224778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f1564093f40 con 0x55ae705f6e60 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1579991080 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ae706938d0 0x55ae70520360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55ae705208a0 con 0x55ae706938d0 2024-09-17T05:29:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1572200640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1554071be0 0x7f15540740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.657+0000 7f1573600640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ae706938d0 0x55ae70520360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1572200640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1554071be0 0x7f15540740f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f156000b660 tx=0x7f1560018530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1573600640 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ae706938d0 0x55ae70520360 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1573600640 1 -- 172.21.15.43:0/3919224778 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f156c017020 con 0x55ae706938d0 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ae706938d0 msgr2=0x55ae70520360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ae706938d0 0x55ae70520360 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1554071be0 msgr2=0x7f15540740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1554071be0 0x7f15540740f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f156000b660 tx=0x7f1560018530 comp rx=0 tx=0).stop 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 msgr2=0x55ae706919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 --2- 172.21.15.43:0/3919224778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ae705f6e60 0x55ae706919c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f156400aa90 tx=0x7f156400af50 comp rx=0 tx=0).stop 2024-09-17T05:29:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1573600640 1 -- 172.21.15.43:0/3919224778 reap_dead start 2024-09-17T05:29:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 shutdown_connections 2024-09-17T05:29:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 >> 172.21.15.43:0/3919224778 conn(0x55ae7050cd80 msgr2=0x55ae705f65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 shutdown_connections 2024-09-17T05:29:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.661+0000 7f1579991080 1 -- 172.21.15.43:0/3919224778 wait complete. 2024-09-17T05:29:54.669 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-09-17T05:29:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 Processor -- start 2024-09-17T05:29:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 -- start start 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce61c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce5fde40 0x560cce61afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cce495a00 con 0x560cce576cb0 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560cce4f3f40 con 0x560cce5fde40 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560cce5fe890 con 0x560cce547d70 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35634/0 (socket says 172.21.15.43:35634) 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09fac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce61c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 -- 172.21.15.43:0/3648496733 learned_addr learned my addr 172.21.15.43:0/3648496733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09fb600640 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce5fde40 0x560cce61afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 -- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 msgr2=0x560cce61c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce61c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 -- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce5fde40 msgr2=0x560cce61afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce5fde40 0x560cce61afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 -- 172.21.15.43:0/3648496733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cce46ec10 con 0x560cce576cb0 2024-09-17T05:29:54.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f3e00640 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f09e400b2a0 tx=0x7f09e400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830c16ff114574ad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f9800640 1 -- 172.21.15.43:0/3648496733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e400a040 con 0x560cce576cb0 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f9800640 1 -- 172.21.15.43:0/3648496733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e400f620 con 0x560cce576cb0 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f09f9800640 1 -- 172.21.15.43:0/3648496733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e400fa70 con 0x560cce576cb0 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 -- 172.21.15.43:0/3648496733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 msgr2=0x560cce61aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.725+0000 7f0a01b59080 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f09e400b2a0 tx=0x7f09e400b760 comp rx=0 tx=0).stop 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- 172.21.15.43:0/3648496733 shutdown_connections 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce5fde40 0x560cce61afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce576cb0 0x560cce61aa80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- 172.21.15.43:0/3648496733 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce61c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- 172.21.15.43:0/3648496733 >> 172.21.15.43:0/3648496733 conn(0x560cce533d80 msgr2=0x560cce5763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- 172.21.15.43:0/3648496733 shutdown_connections 2024-09-17T05:29:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- 172.21.15.43:0/3648496733 wait complete. 2024-09-17T05:29:54.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 Processor -- start 2024-09-17T05:29:54.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- start start 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce6b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce576cb0 0x560cce619140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 0x560cce6b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560cce6a5920 con 0x560cce5fde40 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560cce61a410 con 0x560cce576cb0 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f0a01b59080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560cce6a5bc0 con 0x560cce547d70 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 0x560cce6b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 0x560cce6b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35640/0 (socket says 172.21.15.43:35640) 2024-09-17T05:29:54.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 -- 172.21.15.43:0/2575209030 learned_addr learned my addr 172.21.15.43:0/2575209030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09f3e00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce576cb0 0x560cce619140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fac00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce6b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 -- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 msgr2=0x560cce6b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce6b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 -- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce576cb0 msgr2=0x560cce619140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce576cb0 0x560cce619140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fb600640 1 -- 172.21.15.43:0/2575209030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cce6b3360 con 0x560cce5fde40 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09f3e00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560cce576cb0 0x560cce619140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.729+0000 7f09fac00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560cce547d70 0x560cce6b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09fb600640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 0x560cce6b3560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005010 tx=0x7f09f40afc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 -- 172.21.15.43:0/2575209030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40a7960 con 0x560cce5fde40 2024-09-17T05:29:54.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 -- 172.21.15.43:0/2575209030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09f4004020 con 0x560cce5fde40 2024-09-17T05:29:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560cce56a580 con 0x560cce5fde40 2024-09-17T05:29:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 -- 172.21.15.43:0/2575209030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4003420 con 0x560cce5fde40 2024-09-17T05:29:54.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cce56aac0 con 0x560cce5fde40 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 -- 172.21.15.43:0/2575209030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f09f40c2030 con 0x560cce5fde40 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f09d8071be0 0x7f09d80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09f3400640 1 -- 172.21.15.43:0/2575209030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f09f4131ba0 con 0x560cce5fde40 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f0a01b59080 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560cce545da0 0x560cce5697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f09fac00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f09d8071be0 0x7f09d80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.733+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x560cce569cf0 con 0x560cce545da0 2024-09-17T05:29:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f09f3e00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560cce545da0 0x560cce5697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f09fac00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f09d8071be0 0x7f09d80740f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d860 tx=0x7f09ec00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f09f3e00640 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560cce545da0 0x560cce5697b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f09f3e00640 1 -- 172.21.15.43:0/2575209030 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7f09e400d040 con 0x560cce545da0 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray0_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray1_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray2_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray3_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray4_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray5_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray6_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray7_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray8_head 2024-09-17T05:29:54.741 INFO:teuthology.orchestra.run.smithi043.stdout:stray9_head 2024-09-17T05:29:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560cce545da0 msgr2=0x560cce5697b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560cce545da0 0x560cce5697b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f09d8071be0 msgr2=0x7f09d80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f09d8071be0 0x7f09d80740f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00d860 tx=0x7f09ec00f040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 msgr2=0x560cce6b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 --2- 172.21.15.43:0/2575209030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560cce5fde40 0x560cce6b3560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005010 tx=0x7f09f40afc20 comp rx=0 tx=0).stop 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f09fb600640 1 -- 172.21.15.43:0/2575209030 reap_dead start 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 shutdown_connections 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 >> 172.21.15.43:0/2575209030 conn(0x560cce533d80 msgr2=0x560cce6a4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 shutdown_connections 2024-09-17T05:29:54.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.737+0000 7f0a01b59080 1 -- 172.21.15.43:0/2575209030 wait complete. 2024-09-17T05:29:54.746 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-09-17T05:29:54.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 Processor -- start 2024-09-17T05:29:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 -- start start 2024-09-17T05:29:54.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a0d5d70 0x560e9a11e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a11ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e9a023a00 con 0x560e9a104cb0 2024-09-17T05:29:54.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560e9a081f40 con 0x560e9a18be40 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560e9a18cdd0 con 0x560e9a0d5d70 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35642/0 (socket says 172.21.15.43:35642) 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 -- 172.21.15.43:0/934680876 learned_addr learned my addr 172.21.15.43:0/934680876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c2a00640 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a11ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c2000640 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a0d5d70 0x560e9a11e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 -- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a0d5d70 msgr2=0x560e9a11e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a0d5d70 0x560e9a11e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 -- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 msgr2=0x560e9a11ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a11ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 -- 172.21.15.43:0/934680876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e99ffcc10 con 0x560e9a104cb0 2024-09-17T05:29:54.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3c1600640 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b5c0 tx=0x7fe3b000ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793151b745175df server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3bbe00640 1 -- 172.21.15.43:0/934680876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000a950 con 0x560e9a104cb0 2024-09-17T05:29:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.801+0000 7fe3bbe00640 1 -- 172.21.15.43:0/934680876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3b000aaf0 con 0x560e9a104cb0 2024-09-17T05:29:54.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3bbe00640 1 -- 172.21.15.43:0/934680876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0008460 con 0x560e9a104cb0 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- 172.21.15.43:0/934680876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 msgr2=0x560e9a11cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b5c0 tx=0x7fe3b000ba80 comp rx=0 tx=0).stop 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- 172.21.15.43:0/934680876 shutdown_connections 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a11ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a104cb0 0x560e9a11cf50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/934680876 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a0d5d70 0x560e9a11e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- 172.21.15.43:0/934680876 >> 172.21.15.43:0/934680876 conn(0x560e9a0c1d80 msgr2=0x560e9a1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- 172.21.15.43:0/934680876 shutdown_connections 2024-09-17T05:29:54.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- 172.21.15.43:0/934680876 wait complete. 2024-09-17T05:29:54.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 Processor -- start 2024-09-17T05:29:54.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- start start 2024-09-17T05:29:54.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 0x560e9a1a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a104cb0 0x560e9a231a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a231f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e9a1ab5c0 con 0x560e9a0d5d70 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560e9a1abb00 con 0x560e9a18be40 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c8e33080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560e9a1ab860 con 0x560e9a104cb0 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 0x560e9a1a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 0x560e9a1a7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35658/0 (socket says 172.21.15.43:35658) 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c2000640 1 -- 172.21.15.43:0/1511636457 learned_addr learned my addr 172.21.15.43:0/1511636457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c1600640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a104cb0 0x560e9a231a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c2a00640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a231f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.805+0000 7fe3c2000640 1 -- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a104cb0 msgr2=0x560e9a231a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2000640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a104cb0 0x560e9a231a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2000640 1 -- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 msgr2=0x560e9a231f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2000640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a231f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2000640 1 -- 172.21.15.43:0/1511636457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9a1a7a90 con 0x560e9a0d5d70 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c1600640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560e9a104cb0 0x560e9a231a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2a00640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560e9a18be40 0x560e9a231f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c2000640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 0x560e9a1a7550 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe3b401b560 tx=0x7fe3b40a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 -- 172.21.15.43:0/1511636457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40ad070 con 0x560e9a0d5d70 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 -- 172.21.15.43:0/1511636457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3b40a9860 con 0x560e9a0d5d70 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560e9a2325a0 con 0x560e9a0d5d70 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 -- 172.21.15.43:0/1511636457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40a9c70 con 0x560e9a0d5d70 2024-09-17T05:29:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9a232ae0 con 0x560e9a0d5d70 2024-09-17T05:29:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 -- 172.21.15.43:0/1511636457 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3b40b0070 con 0x560e9a0d5d70 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe3a4071be0 0x7fe3a40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.809+0000 7fe3baa00640 1 -- 172.21.15.43:0/1511636457 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fe3b4130e30 con 0x560e9a0d5d70 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c1600640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe3a4071be0 0x7fe3a40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560e9a2334c0 0x560e9a2338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x560e9a233e10 con 0x560e9a2334c0 2024-09-17T05:29:54.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c2a00640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560e9a2334c0 0x560e9a2338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c1600640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe3a4071be0 0x7fe3a40740f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d040 tx=0x7fe3b0002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c2a00640 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560e9a2334c0 0x560e9a2338d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c2a00640 1 -- 172.21.15.43:0/1511636457 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7fe3bc006790 con 0x560e9a2334c0 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560e9a2334c0 msgr2=0x560e9a2338d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560e9a2334c0 0x560e9a2338d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe3a4071be0 msgr2=0x7fe3a40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe3a4071be0 0x7fe3a40740f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d040 tx=0x7fe3b0002fc0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 msgr2=0x560e9a1a7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 --2- 172.21.15.43:0/1511636457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e9a0d5d70 0x560e9a1a7550 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe3b401b560 tx=0x7fe3b40a8de0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c2a00640 1 -- 172.21.15.43:0/1511636457 reap_dead start 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 shutdown_connections 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 >> 172.21.15.43:0/1511636457 conn(0x560e9a0c1d80 msgr2=0x560e9a11e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 shutdown_connections 2024-09-17T05:29:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.813+0000 7fe3c8e33080 1 -- 172.21.15.43:0/1511636457 wait complete. 2024-09-17T05:29:54.822 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-09-17T05:29:54.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 Processor -- start 2024-09-17T05:29:54.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- start start 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483309cb0 0x55748331f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557483390e40 0x55748331feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557483228a00 con 0x5574832dad70 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557483286f40 con 0x557483390e40 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557483391890 con 0x557483309cb0 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35670/0 (socket says 172.21.15.43:35670) 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 -- 172.21.15.43:0/3827131684 learned_addr learned my addr 172.21.15.43:0/3827131684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3a00640 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557483390e40 0x55748331feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad2600640 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483309cb0 0x55748331f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 -- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483309cb0 msgr2=0x55748331f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483309cb0 0x55748331f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 -- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557483390e40 msgr2=0x55748331feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557483390e40 0x55748331feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 -- 172.21.15.43:0/3827131684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557483201c10 con 0x5574832dad70 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad3000640 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c560 tx=0x7f6ac400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105aa7d61e175849 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad1200640 1 -- 172.21.15.43:0/3827131684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400aa70 con 0x5574832dad70 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad1200640 1 -- 172.21.15.43:0/3827131684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac400ac10 con 0x5574832dad70 2024-09-17T05:29:54.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad1200640 1 -- 172.21.15.43:0/3827131684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4008430 con 0x5574832dad70 2024-09-17T05:29:54.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- 172.21.15.43:0/3827131684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 msgr2=0x557483322400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c560 tx=0x7f6ac400ca20 comp rx=0 tx=0).stop 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- 172.21.15.43:0/3827131684 shutdown_connections 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557483390e40 0x55748331feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483309cb0 0x55748331f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/3827131684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5574832dad70 0x557483322400 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- 172.21.15.43:0/3827131684 >> 172.21.15.43:0/3827131684 conn(0x5574832c6d80 msgr2=0x557483309590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.877+0000 7f6ad9eda080 1 -- 172.21.15.43:0/3827131684 shutdown_connections 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 -- 172.21.15.43:0/3827131684 wait complete. 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 Processor -- start 2024-09-17T05:29:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 -- start start 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574832dad70 0x5574833abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 0x557483439840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483390e40 0x55748331fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557483321a20 con 0x557483309cb0 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5574833ad420 con 0x5574832dad70 2024-09-17T05:29:54.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad9eda080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557483321cc0 con 0x557483390e40 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 0x557483439840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 0x557483439840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35672/0 (socket says 172.21.15.43:35672) 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 -- 172.21.15.43:0/1932140577 learned_addr learned my addr 172.21.15.43:0/1932140577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad3000640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574832dad70 0x5574833abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad3a00640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483390e40 0x55748331fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 -- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483390e40 msgr2=0x55748331fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557483390e40 0x55748331fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 -- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574832dad70 msgr2=0x5574833abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5574832dad70 0x5574833abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 -- 172.21.15.43:0/1932140577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557483393590 con 0x557483309cb0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6ad2600640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 0x557483439840 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000a6d0 tx=0x7f6ac000ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6acb400640 1 -- 172.21.15.43:0/1932140577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0019070 con 0x557483309cb0 2024-09-17T05:29:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6acb400640 1 -- 172.21.15.43:0/1932140577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac0008950 con 0x557483309cb0 2024-09-17T05:29:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.881+0000 7f6acb400640 1 -- 172.21.15.43:0/1932140577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0008da0 con 0x557483309cb0 2024-09-17T05:29:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557483393830 con 0x557483309cb0 2024-09-17T05:29:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557483393210 con 0x557483309cb0 2024-09-17T05:29:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6acb400640 1 -- 172.21.15.43:0/1932140577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6ac001c070 con 0x557483309cb0 2024-09-17T05:29:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6acb400640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ab4071be0 0x7f6ab40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6acb400640 1 -- 172.21.15.43:0/1932140577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6ac0092a60 con 0x557483309cb0 2024-09-17T05:29:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55748331ff90 0x5574834470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad3000640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ab4071be0 0x7f6ab40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x557483447610 con 0x55748331ff90 2024-09-17T05:29:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.885+0000 7f6ad3a00640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55748331ff90 0x5574834470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad3000640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ab4071be0 0x7f6ab40740f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c480 tx=0x7f6ac400e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad3a00640 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55748331ff90 0x5574834470d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad3a00640 1 -- 172.21.15.43:0/1932140577 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f6acc0a7d70 con 0x55748331ff90 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55748331ff90 msgr2=0x5574834470d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55748331ff90 0x5574834470d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ab4071be0 msgr2=0x7f6ab40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6ab4071be0 0x7f6ab40740f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c480 tx=0x7f6ac400e040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 msgr2=0x557483439840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 --2- 172.21.15.43:0/1932140577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557483309cb0 0x557483439840 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000a6d0 tx=0x7f6ac000ab90 comp rx=0 tx=0).stop 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad3a00640 1 -- 172.21.15.43:0/1932140577 reap_dead start 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 shutdown_connections 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 >> 172.21.15.43:0/1932140577 conn(0x5574832c6d80 msgr2=0x557483309550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 shutdown_connections 2024-09-17T05:29:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.889+0000 7f6ad9eda080 1 -- 172.21.15.43:0/1932140577 wait complete. 2024-09-17T05:29:54.897 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-09-17T05:29:54.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 Processor -- start 2024-09-17T05:29:54.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- start start 2024-09-17T05:29:54.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac5b0 0x55cc17dab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac9c0 0x55cc17dab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc17c43a00 con 0x55cc17cfa9c0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc17ca1f40 con 0x55cc17dac5b0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cc17d3b8c0 con 0x55cc17dac9c0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35690/0 (socket says 172.21.15.43:35690) 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 -- 172.21.15.43:0/1565321222 learned_addr learned my addr 172.21.15.43:0/1565321222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f6690000640 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac9c0 0x55cc17dab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668ec00640 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac5b0 0x55cc17dab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 -- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac9c0 msgr2=0x55cc17dab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac9c0 0x55cc17dab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 -- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac5b0 msgr2=0x55cc17dab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:54.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac5b0 0x55cc17dab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 -- 172.21.15.43:0/1565321222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc17c1cc10 con 0x55cc17cfa9c0 2024-09-17T05:29:54.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668f600640 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6680001ec0 tx=0x7f668000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ccfd4348dcd221 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668d800640 1 -- 172.21.15.43:0/1565321222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668000a040 con 0x55cc17cfa9c0 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668d800640 1 -- 172.21.15.43:0/1565321222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f668000f620 con 0x55cc17cfa9c0 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f668d800640 1 -- 172.21.15.43:0/1565321222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668000fa70 con 0x55cc17cfa9c0 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- 172.21.15.43:0/1565321222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 msgr2=0x55cc17d373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6680001ec0 tx=0x7f668000b680 comp rx=0 tx=0).stop 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- 172.21.15.43:0/1565321222 shutdown_connections 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac5b0 0x55cc17dab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac9c0 0x55cc17dab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 --2- 172.21.15.43:0/1565321222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17d373c0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- 172.21.15.43:0/1565321222 >> 172.21.15.43:0/1565321222 conn(0x55cc17ce1d80 msgr2=0x55cc17dcbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- 172.21.15.43:0/1565321222 shutdown_connections 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.953+0000 7f669640d080 1 -- 172.21.15.43:0/1565321222 wait complete. 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 Processor -- start 2024-09-17T05:29:54.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 -- start start 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17da94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac5b0 0x55cc17e654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac9c0 0x55cc17dafb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc17d400c0 con 0x55cc17cfa9c0 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc17d3fe20 con 0x55cc17dac9c0 2024-09-17T05:29:54.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cc17d3f720 con 0x55cc17dac5b0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17da94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17da94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35696/0 (socket says 172.21.15.43:35696) 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 -- 172.21.15.43:0/3671249192 learned_addr learned my addr 172.21.15.43:0/3671249192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668ec00640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac5b0 0x55cc17e654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f6690000640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac9c0 0x55cc17dafb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 -- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac5b0 msgr2=0x55cc17e654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cc17dac5b0 0x55cc17e654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 -- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac9c0 msgr2=0x55cc17dafb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc17dac9c0 0x55cc17dafb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 -- 172.21.15.43:0/3671249192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc17dcace0 con 0x55cc17cfa9c0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f668f600640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17da94b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6680001230 tx=0x7f6680001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f6687e00640 1 -- 172.21.15.43:0/3671249192 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001b640 con 0x55cc17cfa9c0 2024-09-17T05:29:54.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f6687e00640 1 -- 172.21.15.43:0/3671249192 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6680020020 con 0x55cc17cfa9c0 2024-09-17T05:29:54.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f6687e00640 1 -- 172.21.15.43:0/3671249192 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668001b930 con 0x55cc17cfa9c0 2024-09-17T05:29:54.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.957+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cc17dcaf80 con 0x55cc17cfa9c0 2024-09-17T05:29:54.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc17dcb4c0 con 0x55cc17cfa9c0 2024-09-17T05:29:54.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f6687e00640 1 -- 172.21.15.43:0/3671249192 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6680017070 con 0x55cc17cfa9c0 2024-09-17T05:29:54.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f6687e00640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6670071be0 0x7f66700740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f6687e00640 1 -- 172.21.15.43:0/3671249192 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6680094100 con 0x55cc17cfa9c0 2024-09-17T05:29:54.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f669640d080 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cc17e69630 0x55cc17dcb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55cc17dcbea0 con 0x55cc17e69630 2024-09-17T05:29:54.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f668ec00640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6670071be0 0x7f66700740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.961+0000 7f6690000640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cc17e69630 0x55cc17dcb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f668ec00640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6670071be0 0x7f66700740f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c8d0 tx=0x7f667c00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f6690000640 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cc17e69630 0x55cc17dcb960 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f6690000640 1 -- 172.21.15.43:0/3671249192 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f6688002050 con 0x55cc17e69630 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cc17e69630 msgr2=0x55cc17dcb960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cc17e69630 0x55cc17dcb960 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6670071be0 msgr2=0x7f66700740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6670071be0 0x7f66700740f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c8d0 tx=0x7f667c00f040 comp rx=0 tx=0).stop 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 msgr2=0x55cc17da94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 --2- 172.21.15.43:0/3671249192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc17cfa9c0 0x55cc17da94b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6680001230 tx=0x7f6680001260 comp rx=0 tx=0).stop 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f6690000640 1 -- 172.21.15.43:0/3671249192 reap_dead start 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 shutdown_connections 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 >> 172.21.15.43:0/3671249192 conn(0x55cc17ce1d80 msgr2=0x55cc17daa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 shutdown_connections 2024-09-17T05:29:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:54.965+0000 7f669640d080 1 -- 172.21.15.43:0/3671249192 wait complete. 2024-09-17T05:29:54.973 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-09-17T05:29:55.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.025+0000 7f8af2247080 1 Processor -- start 2024-09-17T05:29:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.025+0000 7f8af2247080 1 -- start start 2024-09-17T05:29:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407ad8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407ab9480 0x55e407adc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e407950a00 con 0x55e407abb8b0 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e4079aef40 con 0x55e407aba010 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e407a057f0 con 0x55e407ab9480 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35716/0 (socket says 172.21.15.43:35716) 2024-09-17T05:29:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 -- 172.21.15.43:0/1157054191 learned_addr learned my addr 172.21.15.43:0/1157054191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8aea800640 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407ad8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8aeb200640 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407ab9480 0x55e407adc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 -- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407ab9480 msgr2=0x55e407adc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407ab9480 0x55e407adc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 -- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 msgr2=0x55e407ad8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407ad8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 -- 172.21.15.43:0/1157054191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e407929c10 con 0x55e407abb8b0 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c560 tx=0x7f8ad800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc678a59f1def5c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae8a00640 1 -- 172.21.15.43:0/1157054191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800aa70 con 0x55e407abb8b0 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae8a00640 1 -- 172.21.15.43:0/1157054191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad800ac10 con 0x55e407abb8b0 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8ae8a00640 1 -- 172.21.15.43:0/1157054191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8008430 con 0x55e407abb8b0 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- 172.21.15.43:0/1157054191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 msgr2=0x55e407ad91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c560 tx=0x7f8ad800ca20 comp rx=0 tx=0).stop 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- 172.21.15.43:0/1157054191 shutdown_connections 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407ab9480 0x55e407adc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407abb8b0 0x55e407ad91a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 --2- 172.21.15.43:0/1157054191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407ad8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- 172.21.15.43:0/1157054191 >> 172.21.15.43:0/1157054191 conn(0x55e4079eed80 msgr2=0x55e407ad5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- 172.21.15.43:0/1157054191 shutdown_connections 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 -- 172.21.15.43:0/1157054191 wait complete. 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.029+0000 7f8af2247080 1 Processor -- start 2024-09-17T05:29:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- start start 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 0x55e407ad3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407a45110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407abb8b0 0x55e407ad7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e407ad6890 con 0x55e407ab9480 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e407ab9a60 con 0x55e407aba010 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e407ab9d00 con 0x55e407abb8b0 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 0x55e407ad3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 0x55e407ad3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35726/0 (socket says 172.21.15.43:35726) 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 -- 172.21.15.43:0/1070372981 learned_addr learned my addr 172.21.15.43:0/1070372981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407a45110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aeb200640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407abb8b0 0x55e407ad7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 -- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407abb8b0 msgr2=0x55e407ad7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e407abb8b0 0x55e407ad7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 -- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 msgr2=0x55e407a45110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e407aba010 0x55e407a45110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 -- 172.21.15.43:0/1070372981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e407a322c0 con 0x55e407ab9480 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8aea800640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 0x55e407ad3700 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00efd0 tx=0x7f8adc00c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8ae3400640 1 -- 172.21.15.43:0/1070372981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc015020 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8ae3400640 1 -- 172.21.15.43:0/1070372981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8adc011750 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8ae3400640 1 -- 172.21.15.43:0/1070372981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00a040 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e407a32560 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.033+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e407a449d0 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8ae3400640 1 -- 172.21.15.43:0/1070372981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8adc018020 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8ae3400640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8acc071be0 0x7f8acc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8ae3400640 1 -- 172.21.15.43:0/1070372981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f8adc0931a0 con 0x55e407ab9480 2024-09-17T05:29:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8acc071be0 0x7f8acc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e407ad3c40 0x55e407b703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55e407b70920 con 0x55e407ad3c40 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8aeb200640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e407ad3c40 0x55e407b703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8ae9e00640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8acc071be0 0x7f8acc0740f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800f040 tx=0x7f8ad8002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8aeb200640 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e407ad3c40 0x55e407b703e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8aeb200640 1 -- 172.21.15.43:0/1070372981 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7f8ae4002050 con 0x55e407ad3c40 2024-09-17T05:29:55.093 INFO:teuthology.orchestra.run.smithi043.stdout:subdir_head 2024-09-17T05:29:55.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e407ad3c40 msgr2=0x55e407b703e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e407ad3c40 0x55e407b703e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8acc071be0 msgr2=0x7f8acc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8acc071be0 0x7f8acc0740f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800f040 tx=0x7f8ad8002fc0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 msgr2=0x55e407ad3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 --2- 172.21.15.43:0/1070372981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e407ab9480 0x55e407ad3700 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00efd0 tx=0x7f8adc00c6f0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8aeb200640 1 -- 172.21.15.43:0/1070372981 reap_dead start 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 shutdown_connections 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 >> 172.21.15.43:0/1070372981 conn(0x55e4079eed80 msgr2=0x55e407ad5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 shutdown_connections 2024-09-17T05:29:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.037+0000 7f8af2247080 1 -- 172.21.15.43:0/1070372981 wait complete. 2024-09-17T05:29:55.096 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-09-17T05:29:55.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.149+0000 7fb0fb621080 1 Processor -- start 2024-09-17T05:29:55.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.149+0000 7fb0fb621080 1 -- start start 2024-09-17T05:29:55.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601280609c0 0x560128130d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 0x56012813aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560128007f40 con 0x5601281125b0 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560128014d00 con 0x5601280609c0 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560127fa9a00 con 0x560128131260 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35728/0 (socket says 172.21.15.43:35728) 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 -- 172.21.15.43:0/672539105 learned_addr learned my addr 172.21.15.43:0/672539105 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f4800640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601280609c0 0x560128130d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f5200640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 0x56012813aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 -- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 msgr2=0x56012813aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 0x56012813aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 -- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601280609c0 msgr2=0x560128130d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601280609c0 0x560128130d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 -- 172.21.15.43:0/672539105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012805bfa0 con 0x5601281125b0 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f5200640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 0x56012813aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b220 tx=0x7fb0e000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de9249c73210b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f2a00640 1 -- 172.21.15.43:0/672539105 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000a040 con 0x5601281125b0 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f2a00640 1 -- 172.21.15.43:0/672539105 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0e000f620 con 0x5601281125b0 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0f2a00640 1 -- 172.21.15.43:0/672539105 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000fa70 con 0x5601281125b0 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- 172.21.15.43:0/672539105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 msgr2=0x56012813a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b220 tx=0x7fb0e000b6e0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- 172.21.15.43:0/672539105 shutdown_connections 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560128131260 0x56012813aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5601281125b0 0x56012813a550 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 --2- 172.21.15.43:0/672539105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601280609c0 0x560128130d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- 172.21.15.43:0/672539105 >> 172.21.15.43:0/672539105 conn(0x560128047d80 msgr2=0x56012809da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- 172.21.15.43:0/672539105 shutdown_connections 2024-09-17T05:29:55.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.153+0000 7fb0fb621080 1 -- 172.21.15.43:0/672539105 wait complete. 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 Processor -- start 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- start start 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5601280609c0 0x5601281ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601281125b0 0x56012809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 0x56012810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560128131cc0 con 0x560128131260 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560128131f60 con 0x5601281125b0 2024-09-17T05:29:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5601281ce900 con 0x5601280609c0 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 0x56012810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 0x56012810cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35736/0 (socket says 172.21.15.43:35736) 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 -- 172.21.15.43:0/1312270655 learned_addr learned my addr 172.21.15.43:0/1312270655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f4800640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5601280609c0 0x5601281ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601281125b0 0x56012809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 -- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5601280609c0 msgr2=0x5601281ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5601280609c0 0x5601281ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 -- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601281125b0 msgr2=0x56012809da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601281125b0 0x56012809da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 -- 172.21.15.43:0/1312270655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012810c830 con 0x560128131260 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f5200640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 0x56012810cfb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006fa0 tx=0x7fb0ec00c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f1600640 1 -- 172.21.15.43:0/1312270655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec015030 con 0x560128131260 2024-09-17T05:29:55.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f1600640 1 -- 172.21.15.43:0/1312270655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0ec0113f0 con 0x560128131260 2024-09-17T05:29:55.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0f1600640 1 -- 172.21.15.43:0/1312270655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec011820 con 0x560128131260 2024-09-17T05:29:55.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5601281cb790 con 0x560128131260 2024-09-17T05:29:55.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.157+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601281cbcd0 con 0x560128131260 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f1600640 1 -- 172.21.15.43:0/1312270655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0ec018070 con 0x560128131260 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f1600640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0d4071be0 0x7fb0d40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f1600640 1 -- 172.21.15.43:0/1312270655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fb0ec092e00 con 0x560128131260 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0fb621080 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5601281cf030 0x5601281cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x56012809df70 con 0x5601281cf030 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f4800640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0d4071be0 0x7fb0d40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5601281cf030 0x5601281cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f4800640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0d4071be0 0x7fb0d40740f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ab000 tx=0x7fb0e80ad040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.161+0000 7fb0f3e00640 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5601281cf030 0x5601281cf440 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0f3e00640 1 -- 172.21.15.43:0/1312270655 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7fb0e000d040 con 0x5601281cf030 2024-09-17T05:29:55.168 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-09-17T05:29:52.000000+0000, size 0 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5601281cf030 msgr2=0x5601281cf440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5601281cf030 0x5601281cf440 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0d4071be0 msgr2=0x7fb0d40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0d4071be0 0x7fb0d40740f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ab000 tx=0x7fb0e80ad040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 msgr2=0x56012810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 --2- 172.21.15.43:0/1312270655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560128131260 0x56012810cfb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006fa0 tx=0x7fb0ec00c5e0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0f5200640 1 -- 172.21.15.43:0/1312270655 reap_dead start 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 shutdown_connections 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 >> 172.21.15.43:0/1312270655 conn(0x560128047d80 msgr2=0x56012809edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 shutdown_connections 2024-09-17T05:29:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.165+0000 7fb0fb621080 1 -- 172.21.15.43:0/1312270655 wait complete. 2024-09-17T05:29:55.173 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-09-17T05:29:55.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.225+0000 7f730b355080 1 Processor -- start 2024-09-17T05:29:55.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.225+0000 7f730b355080 1 -- start start 2024-09-17T05:29:55.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72dfd0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e0440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e72dedca00 con 0x55e72e0455b0 2024-09-17T05:29:55.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e72df3af40 con 0x55e72e0459c0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e72dfd4940 con 0x55e72df939c0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35760/0 (socket says 172.21.15.43:35760) 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 -- 172.21.15.43:0/1798896114 learned_addr learned my addr 172.21.15.43:0/1798896114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7304400640 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72dfd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7304e00640 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e0440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 -- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 msgr2=0x55e72dfd0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72dfd0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 -- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 msgr2=0x55e72e0440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e0440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 -- 172.21.15.43:0/1798896114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72deb5c10 con 0x55e72e0455b0 2024-09-17T05:29:55.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7303a00640 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b5f0 tx=0x7f72f000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e833fde7ba1bff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7302600640 1 -- 172.21.15.43:0/1798896114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000a950 con 0x55e72e0455b0 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7302600640 1 -- 172.21.15.43:0/1798896114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72f000aaf0 con 0x55e72e0455b0 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f7302600640 1 -- 172.21.15.43:0/1798896114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0015470 con 0x55e72e0455b0 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- 172.21.15.43:0/1798896114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 msgr2=0x55e72df8f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b5f0 tx=0x7f72f000bab0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- 172.21.15.43:0/1798896114 shutdown_connections 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e0440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72df8f9a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 --2- 172.21.15.43:0/1798896114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72dfd0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- 172.21.15.43:0/1798896114 >> 172.21.15.43:0/1798896114 conn(0x55e72df7ad80 msgr2=0x55e72dfd6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- 172.21.15.43:0/1798896114 shutdown_connections 2024-09-17T05:29:55.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.229+0000 7f730b355080 1 -- 172.21.15.43:0/1798896114 wait complete. 2024-09-17T05:29:55.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 Processor -- start 2024-09-17T05:29:55.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- start start 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72e065360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72e041e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e042370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e72e06cba0 con 0x55e72e0455b0 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e72e0630d0 con 0x55e72e0459c0 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e72e063370 con 0x55e72df939c0 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72e041e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72e041e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35768/0 (socket says 172.21.15.43:35768) 2024-09-17T05:29:55.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 -- 172.21.15.43:0/2750349507 learned_addr learned my addr 172.21.15.43:0/2750349507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7304400640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72e065360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7304e00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e042370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 -- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 msgr2=0x55e72e065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72e065360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 -- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 msgr2=0x55e72e042370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e72e0459c0 0x55e72e042370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 -- 172.21.15.43:0/2750349507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e72e064440 con 0x55e72e0455b0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7304400640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e72df939c0 0x55e72e065360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7303a00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72e041e30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f72f000bfb0 tx=0x7f72f00166b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7301200640 1 -- 172.21.15.43:0/2750349507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0016ae0 con 0x55e72e0455b0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7301200640 1 -- 172.21.15.43:0/2750349507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72f0016c80 con 0x55e72e0455b0 2024-09-17T05:29:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f7301200640 1 -- 172.21.15.43:0/2750349507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0020420 con 0x55e72e0455b0 2024-09-17T05:29:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e72e0646e0 con 0x55e72e0455b0 2024-09-17T05:29:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.233+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72e064c20 con 0x55e72e0455b0 2024-09-17T05:29:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7301200640 1 -- 172.21.15.43:0/2750349507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f72f0029440 con 0x55e72e0455b0 2024-09-17T05:29:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7301200640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f72e4071be0 0x7f72e40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7301200640 1 -- 172.21.15.43:0/2750349507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f72f0094a90 con 0x55e72e0455b0 2024-09-17T05:29:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7304400640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f72e4071be0 0x7f72e40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f730b355080 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e72dfd8c50 0x55e72e030bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55e72e031110 con 0x55e72dfd8c50 2024-09-17T05:29:55.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7304e00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e72dfd8c50 0x55e72e030bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.237+0000 7f7304400640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f72e4071be0 0x7f72e40740f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f72f4010500 tx=0x7f72f4001740 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f7304e00640 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e72dfd8c50 0x55e72e030bd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f7304e00640 1 -- 172.21.15.43:0/2750349507 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f72fc002050 con 0x55e72dfd8c50 2024-09-17T05:29:55.244 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/600.00000000 mtime 2024-09-17T05:29:43.000000+0000, size 0 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e72dfd8c50 msgr2=0x55e72e030bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e72dfd8c50 0x55e72e030bd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f72e4071be0 msgr2=0x7f72e40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f72e4071be0 0x7f72e40740f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f72f4010500 tx=0x7f72f4001740 comp rx=0 tx=0).stop 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 msgr2=0x55e72e041e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 --2- 172.21.15.43:0/2750349507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e72e0455b0 0x55e72e041e30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f72f000bfb0 tx=0x7f72f00166b0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f7304e00640 1 -- 172.21.15.43:0/2750349507 reap_dead start 2024-09-17T05:29:55.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 shutdown_connections 2024-09-17T05:29:55.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 >> 172.21.15.43:0/2750349507 conn(0x55e72df7ad80 msgr2=0x55e72e040270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 shutdown_connections 2024-09-17T05:29:55.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.241+0000 7f730b355080 1 -- 172.21.15.43:0/2750349507 wait complete. 2024-09-17T05:29:55.249 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-09-17T05:29:55.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 Processor -- start 2024-09-17T05:29:55.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 -- start start 2024-09-17T05:29:55.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420eaf20 0x55b7420a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 0x55b7420e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b74204ef40 con 0x7fefcc003b50 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b74205bd00 con 0x55b7420a46d0 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefeb585080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b741ff0a00 con 0x55b7420eaf20 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35800/0 (socket says 172.21.15.43:35800) 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 -- 172.21.15.43:0/1236886117 learned_addr learned my addr 172.21.15.43:0/1236886117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 -- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420eaf20 msgr2=0x55b7420a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420eaf20 0x55b7420a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 -- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 msgr2=0x55b7420e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe5200640 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 0x55b7420e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 0x55b7420e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 -- 172.21.15.43:0/1236886117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b742158af0 con 0x7fefcc003b50 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe5200640 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 0x55b7420e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe4800640 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001760 tx=0x7fefd8009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac93c0157e0344ed server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe2a00640 1 -- 172.21.15.43:0/1236886117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8013020 con 0x7fefcc003b50 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe2a00640 1 -- 172.21.15.43:0/1236886117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd800da00 con 0x7fefcc003b50 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.305+0000 7fefe2a00640 1 -- 172.21.15.43:0/1236886117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd800de50 con 0x7fefcc003b50 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- 172.21.15.43:0/1236886117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 msgr2=0x55b742176040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001760 tx=0x7fefd8009a30 comp rx=0 tx=0).stop 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- 172.21.15.43:0/1236886117 shutdown_connections 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b7420a46d0 0x55b7420e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420eaf20 0x55b7420a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- 172.21.15.43:0/1236886117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefcc003b50 0x55b742176040 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- 172.21.15.43:0/1236886117 >> 172.21.15.43:0/1236886117 conn(0x55b74208ed80 msgr2=0x55b742175020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- 172.21.15.43:0/1236886117 shutdown_connections 2024-09-17T05:29:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- 172.21.15.43:0/1236886117 wait complete. 2024-09-17T05:29:55.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 Processor -- start 2024-09-17T05:29:55.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- start start 2024-09-17T05:29:55.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420a46d0 0x55b742210330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 0x55b7420e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefcc003b50 0x55b7420c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b7420ea540 con 0x55b7420eaf20 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b7420ea2a0 con 0x7fefcc003b50 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefeb585080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b7420ea7e0 con 0x55b7420a46d0 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 0x55b7420e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 0x55b7420e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35812/0 (socket says 172.21.15.43:35812) 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 -- 172.21.15.43:0/2954667035 learned_addr learned my addr 172.21.15.43:0/2954667035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe4800640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420a46d0 0x55b742210330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe5200640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefcc003b50 0x55b7420c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 -- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420a46d0 msgr2=0x55b742210330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b7420a46d0 0x55b742210330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 -- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefcc003b50 msgr2=0x55b7420c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefcc003b50 0x55b7420c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe3e00640 1 -- 172.21.15.43:0/2954667035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b742174dc0 con 0x55b7420eaf20 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.309+0000 7fefe5200640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefcc003b50 0x55b7420c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe3e00640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 0x55b7420e7ff0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cbd0 tx=0x7fefd000a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 -- 172.21.15.43:0/2954667035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0013070 con 0x55b7420eaf20 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 -- 172.21.15.43:0/2954667035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd000f750 con 0x55b7420eaf20 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b742210e80 con 0x55b7420eaf20 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 -- 172.21.15.43:0/2954667035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ae60 con 0x55b7420eaf20 2024-09-17T05:29:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7422113c0 con 0x55b7420eaf20 2024-09-17T05:29:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 -- 172.21.15.43:0/2954667035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefd0016070 con 0x55b7420eaf20 2024-09-17T05:29:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefc4071be0 0x7fefc40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe1600640 1 -- 172.21.15.43:0/2954667035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fefd0092ce0 con 0x55b7420eaf20 2024-09-17T05:29:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefeb585080 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b7422109c0 0x55b7422133b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefe4800640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefc4071be0 0x7fefc40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.313+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55b7420e5570 con 0x55b7422109c0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefe5200640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b7422109c0 0x55b7422133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefe4800640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefc4071be0 0x7fefc40740f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001700 tx=0x55b7420e3410 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefe5200640 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b7422109c0 0x55b7422133b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefe5200640 1 -- 172.21.15.43:0/2954667035 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7fefdc0b5040 con 0x55b7422109c0 2024-09-17T05:29:55.321 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/1.00000000.inode mtime 2024-09-17T05:29:52.000000+0000, size 582 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b7422109c0 msgr2=0x55b7422133b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b7422109c0 0x55b7422133b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefc4071be0 msgr2=0x7fefc40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefc4071be0 0x7fefc40740f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001700 tx=0x55b7420e3410 comp rx=0 tx=0).stop 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 msgr2=0x55b7420e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 --2- 172.21.15.43:0/2954667035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b7420eaf20 0x55b7420e7ff0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cbd0 tx=0x7fefd000a630 comp rx=0 tx=0).stop 2024-09-17T05:29:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefe5200640 1 -- 172.21.15.43:0/2954667035 reap_dead start 2024-09-17T05:29:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 shutdown_connections 2024-09-17T05:29:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 >> 172.21.15.43:0/2954667035 conn(0x55b74208ed80 msgr2=0x55b742175fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 shutdown_connections 2024-09-17T05:29:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.317+0000 7fefeb585080 1 -- 172.21.15.43:0/2954667035 wait complete. 2024-09-17T05:29:55.326 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-09-17T05:29:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.381+0000 7fcb173b2080 1 Processor -- start 2024-09-17T05:29:55.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.381+0000 7fcb173b2080 1 -- start start 2024-09-17T05:29:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d75339090 0x562d752f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d7533c7e0 0x562d7533cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562d752a1f40 con 0x7fcaf4003100 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d752aed00 con 0x562d7533c7e0 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562d75243a00 con 0x562d75339090 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35836/0 (socket says 172.21.15.43:35836) 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 -- 172.21.15.43:0/2543425475 learned_addr learned my addr 172.21.15.43:0/2543425475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb0fc00640 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d75339090 0x562d752f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb11000640 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d7533c7e0 0x562d7533cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 -- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d75339090 msgr2=0x562d752f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d75339090 0x562d752f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 -- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d7533c7e0 msgr2=0x562d7533cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d7533c7e0 0x562d7533cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 -- 172.21.15.43:0/2543425475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7533d130 con 0x7fcaf4003100 2024-09-17T05:29:55.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb10600640 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000f460 tx=0x7fcb0000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c27b73a364c807 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb0e800640 1 -- 172.21.15.43:0/2543425475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb000108f0 con 0x7fcaf4003100 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb0e800640 1 -- 172.21.15.43:0/2543425475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb00010a90 con 0x7fcaf4003100 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb0e800640 1 -- 172.21.15.43:0/2543425475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000d3a0 con 0x7fcaf4003100 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- 172.21.15.43:0/2543425475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 msgr2=0x562d7533b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000f460 tx=0x7fcb0000f920 comp rx=0 tx=0).stop 2024-09-17T05:29:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- 172.21.15.43:0/2543425475 shutdown_connections 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d7533c7e0 0x562d7533cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d75339090 0x562d752f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2543425475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf4003100 0x562d7533b850 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- 172.21.15.43:0/2543425475 >> 172.21.15.43:0/2543425475 conn(0x562d752e1d80 msgr2=0x562d75339560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- 172.21.15.43:0/2543425475 shutdown_connections 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- 172.21.15.43:0/2543425475 wait complete. 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 Processor -- start 2024-09-17T05:29:55.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.385+0000 7fcb173b2080 1 -- start start 2024-09-17T05:29:55.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 0x562d753bb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d7533c7e0 0x562d753bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4003100 0x562d75319810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562d753cb300 con 0x562d75339090 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d753cb5a0 con 0x7fcaf4003100 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562d753cb840 con 0x562d7533c7e0 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 0x562d753bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 0x562d753bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35844/0 (socket says 172.21.15.43:35844) 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 -- 172.21.15.43:0/2158693050 learned_addr learned my addr 172.21.15.43:0/2158693050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb0fc00640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d7533c7e0 0x562d753bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb11000640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4003100 0x562d75319810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 -- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d7533c7e0 msgr2=0x562d753bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562d7533c7e0 0x562d753bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 -- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4003100 msgr2=0x562d75319810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4003100 0x562d75319810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 -- 172.21.15.43:0/2158693050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d75454ba0 con 0x562d75339090 2024-09-17T05:29:55.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb11000640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4003100 0x562d75319810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb10600640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 0x562d753bb760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcb00001d90 tx=0x7fcb0001b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb0d400640 1 -- 172.21.15.43:0/2158693050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001bc80 con 0x562d75339090 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb0d400640 1 -- 172.21.15.43:0/2158693050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb00014070 con 0x562d75339090 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d75454e40 con 0x562d75339090 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb0d400640 1 -- 172.21.15.43:0/2158693050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001be60 con 0x562d75339090 2024-09-17T05:29:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.389+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d75455380 con 0x562d75339090 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb0d400640 1 -- 172.21.15.43:0/2158693050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb000233a0 con 0x562d75339090 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb0d400640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf0071c10 0x7fcaf0074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb0d400640 1 -- 172.21.15.43:0/2158693050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fcb00097160 con 0x562d75339090 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb0fc00640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf0071c10 0x7fcaf0074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562d754543a0 0x562d75321560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x562d75321aa0 con 0x562d754543a0 2024-09-17T05:29:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb11000640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562d754543a0 0x562d75321560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb0fc00640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf0071c10 0x7fcaf0074120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0ab000 tx=0x7fcafc004040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb11000640 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562d754543a0 0x562d75321560 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb11000640 1 -- 172.21.15.43:0/2158693050 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv18 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fcb0800b040 con 0x562d754543a0 2024-09-17T05:29:55.399 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/200.00000000 mtime 2024-09-17T05:29:52.000000+0000, size 90 2024-09-17T05:29:55.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562d754543a0 msgr2=0x562d75321560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562d754543a0 0x562d75321560 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf0071c10 msgr2=0x7fcaf0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf0071c10 0x7fcaf0074120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0ab000 tx=0x7fcafc004040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 msgr2=0x562d753bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 --2- 172.21.15.43:0/2158693050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d75339090 0x562d753bb760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcb00001d90 tx=0x7fcb0001b700 comp rx=0 tx=0).stop 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb11000640 1 -- 172.21.15.43:0/2158693050 reap_dead start 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 shutdown_connections 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 >> 172.21.15.43:0/2158693050 conn(0x562d752e1d80 msgr2=0x562d75339560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 shutdown_connections 2024-09-17T05:29:55.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.393+0000 7fcb173b2080 1 -- 172.21.15.43:0/2158693050 wait complete. 2024-09-17T05:29:55.404 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-09-17T05:29:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.457+0000 7f09a4517080 1 Processor -- start 2024-09-17T05:29:55.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.457+0000 7f09a4517080 1 -- start start 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560ccc67dfb0 0x560ccc6c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 0x560ccc6c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ccc5cda00 con 0x7f0984002350 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560ccc62bf40 con 0x560ccc7551f0 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560ccc736dd0 con 0x560ccc67dfb0 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35856/0 (socket says 172.21.15.43:35856) 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 -- 172.21.15.43:0/1479314886 learned_addr learned my addr 172.21.15.43:0/1479314886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099cc00640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560ccc67dfb0 0x560ccc6c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099e000640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 0x560ccc6c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 -- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560ccc67dfb0 msgr2=0x560ccc6c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560ccc67dfb0 0x560ccc6c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 -- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 msgr2=0x560ccc6c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 0x560ccc6c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 -- 172.21.15.43:0/1479314886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ccc5a6c10 con 0x7f0984002350 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099e000640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 0x560ccc6c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f099d600640 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f099001c800 tx=0x7f0984004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28475552a5488df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f0997400640 1 -- 172.21.15.43:0/1479314886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09900014c0 con 0x7f0984002350 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f0997400640 1 -- 172.21.15.43:0/1479314886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0990001680 con 0x7f0984002350 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f0997400640 1 -- 172.21.15.43:0/1479314886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09900af790 con 0x7f0984002350 2024-09-17T05:29:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- 172.21.15.43:0/1479314886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 msgr2=0x560ccc67da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f099001c800 tx=0x7f0984004130 comp rx=0 tx=0).stop 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- 172.21.15.43:0/1479314886 shutdown_connections 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc7551f0 0x560ccc6c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560ccc67dfb0 0x560ccc6c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 --2- 172.21.15.43:0/1479314886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0984002350 0x560ccc67da70 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- 172.21.15.43:0/1479314886 >> 172.21.15.43:0/1479314886 conn(0x560ccc66bd80 msgr2=0x560ccc6ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- 172.21.15.43:0/1479314886 shutdown_connections 2024-09-17T05:29:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- 172.21.15.43:0/1479314886 wait complete. 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 Processor -- start 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.461+0000 7f09a4517080 1 -- start start 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc67dfb0 0x560ccc6c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 0x560ccc6a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0984002350 0x560ccc7ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ccc6c6130 con 0x560ccc7551f0 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560ccc6c5e90 con 0x560ccc67dfb0 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560ccc6c63d0 con 0x7f0984002350 2024-09-17T05:29:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 0x560ccc6a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 0x560ccc6a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35860/0 (socket says 172.21.15.43:35860) 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 -- 172.21.15.43:0/1615509654 learned_addr learned my addr 172.21.15.43:0/1615509654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099d600640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc67dfb0 0x560ccc6c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099e000640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0984002350 0x560ccc7ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 -- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0984002350 msgr2=0x560ccc7ecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0984002350 0x560ccc7ecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 -- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc67dfb0 msgr2=0x560ccc6c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ccc67dfb0 0x560ccc6c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 -- 172.21.15.43:0/1615509654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ccc7df2e0 con 0x560ccc7551f0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f099cc00640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 0x560ccc6a1b70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a720 tx=0x7f098c00abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f0996000640 1 -- 172.21.15.43:0/1615509654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c009b50 con 0x560ccc7551f0 2024-09-17T05:29:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f0996000640 1 -- 172.21.15.43:0/1615509654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f098c009cf0 con 0x560ccc7551f0 2024-09-17T05:29:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ccc7df580 con 0x560ccc7551f0 2024-09-17T05:29:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f0996000640 1 -- 172.21.15.43:0/1615509654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0086b0 con 0x560ccc7551f0 2024-09-17T05:29:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.465+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ccc7dfac0 con 0x560ccc7551f0 2024-09-17T05:29:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f0996000640 1 -- 172.21.15.43:0/1615509654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f098c008850 con 0x560ccc7551f0 2024-09-17T05:29:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f0996000640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0980071be0 0x7f09800740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f0996000640 1 -- 172.21.15.43:0/1615509654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f098c093c70 con 0x560ccc7551f0 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f099d600640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0980071be0 0x7f09800740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f09a4517080 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x560ccc7ed000 0x560ccc7dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x560ccc7ddcf0 con 0x560ccc7ed000 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f099e000640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x560ccc7ed000 0x560ccc7dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f099d600640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0980071be0 0x7f09800740f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f099001c800 tx=0x7f09900bd040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f099e000640 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x560ccc7ed000 0x560ccc7dd7b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f099e000640 1 -- 172.21.15.43:0/1615509654 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f0998002040 con 0x560ccc7ed000 2024-09-17T05:29:55.476 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/200.00000001 mtime 2024-09-17T05:29:52.000000+0000, size 16801 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x560ccc7ed000 msgr2=0x560ccc7dd7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.469+0000 7f09a4517080 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x560ccc7ed000 0x560ccc7dd7b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0980071be0 msgr2=0x7f09800740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0980071be0 0x7f09800740f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f099001c800 tx=0x7f09900bd040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 msgr2=0x560ccc6a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 --2- 172.21.15.43:0/1615509654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ccc7551f0 0x560ccc6a1b70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a720 tx=0x7f098c00abe0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f099e000640 1 -- 172.21.15.43:0/1615509654 reap_dead start 2024-09-17T05:29:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 shutdown_connections 2024-09-17T05:29:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 >> 172.21.15.43:0/1615509654 conn(0x560ccc66bd80 msgr2=0x560ccc6ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 shutdown_connections 2024-09-17T05:29:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.473+0000 7f09a4517080 1 -- 172.21.15.43:0/1615509654 wait complete. 2024-09-17T05:29:55.480 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-09-17T05:29:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 Processor -- start 2024-09-17T05:29:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 -- start start 2024-09-17T05:29:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 0x5650e10f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 0x5650e1042a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.533+0000 7f746948e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650e0f8ea00 con 0x5650e106fcb0 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e0fecf40 con 0x5650e10fcda0 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5650e10fbb40 con 0x5650e103e890 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35878/0 (socket says 172.21.15.43:35878) 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 -- 172.21.15.43:0/2758506233 learned_addr learned my addr 172.21.15.43:0/2758506233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7463000640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 0x5650e1042a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7462600640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 0x5650e10f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 -- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 msgr2=0x5650e10f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 0x5650e10f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 -- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 msgr2=0x5650e1042a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 0x5650e1042a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 -- 172.21.15.43:0/2758506233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0f67c10 con 0x5650e106fcb0 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7462600640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 0x5650e10f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7463000640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 0x5650e1042a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f7461c00640 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f745000c590 tx=0x7f745000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53702e0d05ff52f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f745be00640 1 -- 172.21.15.43:0/2758506233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000aa10 con 0x5650e106fcb0 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f745be00640 1 -- 172.21.15.43:0/2758506233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745000abb0 con 0x5650e106fcb0 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f745be00640 1 -- 172.21.15.43:0/2758506233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450008430 con 0x5650e106fcb0 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- 172.21.15.43:0/2758506233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 msgr2=0x5650e10f8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f745000c590 tx=0x7f745000ca50 comp rx=0 tx=0).stop 2024-09-17T05:29:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- 172.21.15.43:0/2758506233 shutdown_connections 2024-09-17T05:29:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e10fcda0 0x5650e1042a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e106fcb0 0x5650e10f8ae0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 --2- 172.21.15.43:0/2758506233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e103e890 0x5650e10f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- 172.21.15.43:0/2758506233 >> 172.21.15.43:0/2758506233 conn(0x5650e102cd80 msgr2=0x5650e1088df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- 172.21.15.43:0/2758506233 shutdown_connections 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- 172.21.15.43:0/2758506233 wait complete. 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 Processor -- start 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.537+0000 7f746948e080 1 -- start start 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e103e890 0x5650e10f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e106fcb0 0x5650e11ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 0x5650e10fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650e1083b90 con 0x5650e10fcda0 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650e11ad600 con 0x5650e103e890 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5650e10f9130 con 0x5650e106fcb0 2024-09-17T05:29:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 0x5650e10fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 0x5650e10fa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35894/0 (socket says 172.21.15.43:35894) 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 -- 172.21.15.43:0/3642468216 learned_addr learned my addr 172.21.15.43:0/3642468216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7461c00640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e106fcb0 0x5650e11ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7462600640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e103e890 0x5650e10f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 -- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e106fcb0 msgr2=0x5650e11ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5650e106fcb0 0x5650e11ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 -- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e103e890 msgr2=0x5650e10f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650e103e890 0x5650e10f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 -- 172.21.15.43:0/3642468216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e11aab90 con 0x5650e10fcda0 2024-09-17T05:29:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f7463000640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 0x5650e10fa5c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f745c00d4a0 tx=0x7f745c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f745aa00640 1 -- 172.21.15.43:0/3642468216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c018070 con 0x5650e10fcda0 2024-09-17T05:29:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f745aa00640 1 -- 172.21.15.43:0/3642468216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f745c0032e0 con 0x5650e10fcda0 2024-09-17T05:29:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f745aa00640 1 -- 172.21.15.43:0/3642468216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c0136a0 con 0x5650e10fcda0 2024-09-17T05:29:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5650e11aae30 con 0x5650e10fcda0 2024-09-17T05:29:55.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.541+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e11ab370 con 0x5650e10fcda0 2024-09-17T05:29:55.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f745aa00640 1 -- 172.21.15.43:0/3642468216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f745c01b070 con 0x5650e10fcda0 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f745aa00640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7444071be0 0x7f74440740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f745aa00640 1 -- 172.21.15.43:0/3642468216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f745c092420 con 0x5650e10fcda0 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7462600640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7444071be0 0x7f74440740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5650e10fab00 0x5650e10f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5650e10f7840 con 0x5650e10fab00 2024-09-17T05:29:55.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7461c00640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5650e10fab00 0x5650e10f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7462600640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7444071be0 0x7f74440740f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f74540ab000 tx=0x7f74540ad040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7461c00640 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5650e10fab00 0x5650e10f7300 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7461c00640 1 -- 172.21.15.43:0/3642468216 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f745000aa70 con 0x5650e10fab00 2024-09-17T05:29:55.550 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-09-17T05:29:52.000000+0000, size 0 2024-09-17T05:29:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5650e10fab00 msgr2=0x5650e10f7300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5650e10fab00 0x5650e10f7300 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7444071be0 msgr2=0x7f74440740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7444071be0 0x7f74440740f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f74540ab000 tx=0x7f74540ad040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 msgr2=0x5650e10fa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 --2- 172.21.15.43:0/3642468216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650e10fcda0 0x5650e10fa5c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f745c00d4a0 tx=0x7f745c00d960 comp rx=0 tx=0).stop 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f7463000640 1 -- 172.21.15.43:0/3642468216 reap_dead start 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 shutdown_connections 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 >> 172.21.15.43:0/3642468216 conn(0x5650e102cd80 msgr2=0x5650e1042a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 shutdown_connections 2024-09-17T05:29:55.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.545+0000 7f746948e080 1 -- 172.21.15.43:0/3642468216 wait complete. 2024-09-17T05:29:55.555 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-09-17T05:29:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 Processor -- start 2024-09-17T05:29:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 -- start start 2024-09-17T05:29:55.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 0x556eb0e7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e7acb0 0x556eb0e92070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556eb0d99a00 con 0x556eb0f01e40 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556eb0df7f40 con 0x556eb0e7acb0 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f4471f9c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556eb0f02b80 con 0x556eb0e4e4d0 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35916/0 (socket says 172.21.15.43:35916) 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 -- 172.21.15.43:0/1906423978 learned_addr learned my addr 172.21.15.43:0/1906423978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446b200640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 0x556eb0e7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446a800640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e7acb0 0x556eb0e92070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 -- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 msgr2=0x556eb0e7a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 0x556eb0e7a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 -- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e7acb0 msgr2=0x556eb0e92070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e7acb0 0x556eb0e92070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446bc00640 1 -- 172.21.15.43:0/1906423978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb0d72c10 con 0x556eb0f01e40 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.609+0000 7f446b200640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 0x556eb0e7a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f446bc00640 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f446400b6d0 tx=0x7f446400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef80aea39493ed91 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4469400640 1 -- 172.21.15.43:0/1906423978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464009a30 con 0x556eb0f01e40 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4469400640 1 -- 172.21.15.43:0/1906423978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4464009ce0 con 0x556eb0f01e40 2024-09-17T05:29:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4469400640 1 -- 172.21.15.43:0/1906423978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446400e6b0 con 0x556eb0f01e40 2024-09-17T05:29:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- 172.21.15.43:0/1906423978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 msgr2=0x556eb0e49890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f446400b6d0 tx=0x7f446400bb90 comp rx=0 tx=0).stop 2024-09-17T05:29:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- 172.21.15.43:0/1906423978 shutdown_connections 2024-09-17T05:29:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e7acb0 0x556eb0e92070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e49890 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1906423978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e4e4d0 0x556eb0e7a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- 172.21.15.43:0/1906423978 >> 172.21.15.43:0/1906423978 conn(0x556eb0e37d80 msgr2=0x556eb0f20150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- 172.21.15.43:0/1906423978 shutdown_connections 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- 172.21.15.43:0/1906423978 wait complete. 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 Processor -- start 2024-09-17T05:29:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- start start 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e4e4d0 0x556eb0e49ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e7acb0 0x556eb0f1e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e901c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556eb0f20970 con 0x556eb0f01e40 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556eb0f20c10 con 0x556eb0e4e4d0 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.613+0000 7f4471f9c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556eb0e77cc0 con 0x556eb0e7acb0 2024-09-17T05:29:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e901c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35920/0 (socket says 172.21.15.43:35920) 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 -- 172.21.15.43:0/1199482208 learned_addr learned my addr 172.21.15.43:0/1199482208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446a800640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e7acb0 0x556eb0f1e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446b200640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e4e4d0 0x556eb0e49ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 -- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e7acb0 msgr2=0x556eb0f1e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556eb0e7acb0 0x556eb0f1e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 -- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e4e4d0 msgr2=0x556eb0e49ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e4e4d0 0x556eb0e49ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 -- 172.21.15.43:0/1199482208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb0e950a0 con 0x556eb0f01e40 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446b200640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556eb0e4e4d0 0x556eb0e49ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f446bc00640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e901c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f446400ef00 tx=0x7f446400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4463e00640 1 -- 172.21.15.43:0/1199482208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464024070 con 0x556eb0f01e40 2024-09-17T05:29:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4463e00640 1 -- 172.21.15.43:0/1199482208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f446401f450 con 0x556eb0f01e40 2024-09-17T05:29:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556eb0e73140 con 0x556eb0f01e40 2024-09-17T05:29:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4463e00640 1 -- 172.21.15.43:0/1199482208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446401b070 con 0x556eb0f01e40 2024-09-17T05:29:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb0e73680 con 0x556eb0f01e40 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.617+0000 7f4463e00640 1 -- 172.21.15.43:0/1199482208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4464027070 con 0x556eb0f01e40 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4463e00640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f444c071be0 0x7f444c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4463e00640 1 -- 172.21.15.43:0/1199482208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f4464094990 con 0x556eb0f01e40 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556eb0e715a0 0x556eb0e719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446b200640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f444c071be0 0x7f444c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x556eb0e71ef0 con 0x556eb0e715a0 2024-09-17T05:29:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446a800640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556eb0e715a0 0x556eb0e719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446a800640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556eb0e715a0 0x556eb0e719b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446b200640 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f444c071be0 0x7f444c0740f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b530 tx=0x7f445c0056e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446a800640 1 -- 172.21.15.43:0/1199482208 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f44580a7ad0 con 0x556eb0e715a0 2024-09-17T05:29:55.626 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/500.00000000 mtime 2024-09-17T05:29:43.000000+0000, size 90 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556eb0e715a0 msgr2=0x556eb0e719b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556eb0e715a0 0x556eb0e719b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f444c071be0 msgr2=0x7f444c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f444c071be0 0x7f444c0740f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b530 tx=0x7f445c0056e0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 msgr2=0x556eb0e901c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 --2- 172.21.15.43:0/1199482208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556eb0f01e40 0x556eb0e901c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f446400ef00 tx=0x7f446400ef30 comp rx=0 tx=0).stop 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f446bc00640 1 -- 172.21.15.43:0/1199482208 reap_dead start 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 shutdown_connections 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 >> 172.21.15.43:0/1199482208 conn(0x556eb0e37d80 msgr2=0x556eb0e92070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 shutdown_connections 2024-09-17T05:29:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.621+0000 7f4471f9c080 1 -- 172.21.15.43:0/1199482208 wait complete. 2024-09-17T05:29:55.630 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-09-17T05:29:55.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 Processor -- start 2024-09-17T05:29:55.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 -- start start 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 0x5655172735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x565517275190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56551721ff40 con 0x5655172756d0 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56551722cd00 con 0x7f9124002350 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9146fc4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5655171c1a00 con 0x565517344210 2024-09-17T05:29:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35936/0 (socket says 172.21.15.43:35936) 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 -- 172.21.15.43:0/2562951949 learned_addr learned my addr 172.21.15.43:0/2562951949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140200640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 0x5655172735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f913f800640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x565517275190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 -- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 msgr2=0x565517275190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x565517275190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 -- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 msgr2=0x5655172735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 0x5655172735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140c00640 1 -- 172.21.15.43:0/2562951949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565517344c50 con 0x5655172756d0 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f913f800640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x565517275190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.685+0000 7f9140200640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 0x5655172735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9140c00640 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9138004fd0 tx=0x7f913800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea83ce693cd47e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f913e400640 1 -- 172.21.15.43:0/2562951949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913800fb30 con 0x5655172756d0 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f913e400640 1 -- 172.21.15.43:0/2562951949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9138014070 con 0x5655172756d0 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f913e400640 1 -- 172.21.15.43:0/2562951949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913800de70 con 0x5655172756d0 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- 172.21.15.43:0/2562951949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 msgr2=0x5655172b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9138004fd0 tx=0x7f913800db60 comp rx=0 tx=0).stop 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- 172.21.15.43:0/2562951949 shutdown_connections 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5655172756d0 0x5655172b4560 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x565517275190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 --2- 172.21.15.43:0/2562951949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9124002350 0x5655172735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- 172.21.15.43:0/2562951949 >> 172.21.15.43:0/2562951949 conn(0x56551725fd80 msgr2=0x5655172bb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- 172.21.15.43:0/2562951949 shutdown_connections 2024-09-17T05:29:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- 172.21.15.43:0/2562951949 wait complete. 2024-09-17T05:29:55.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 Processor -- start 2024-09-17T05:29:55.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.689+0000 7f9146fc4080 1 -- start start 2024-09-17T05:29:55.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655172756d0 0x5655173e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x5655173e0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 0x565517272250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565517344ef0 con 0x7f9124002350 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565517273100 con 0x5655172756d0 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x565517345190 con 0x565517344210 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 0x565517272250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f913f800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x5655173e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655172756d0 0x5655173e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 0x565517272250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35946/0 (socket says 172.21.15.43:35946) 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 -- 172.21.15.43:0/1895959256 learned_addr learned my addr 172.21.15.43:0/1895959256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 -- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 msgr2=0x5655173e0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x565517344210 0x5655173e0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 -- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655172756d0 msgr2=0x5655173e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655172756d0 0x5655173e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 -- 172.21.15.43:0/1895959256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655173e4ca0 con 0x7f9124002350 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9140c00640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 0x565517272250 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9138010fd0 tx=0x7f913800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f913d000640 1 -- 172.21.15.43:0/1895959256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138004020 con 0x7f9124002350 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f913d000640 1 -- 172.21.15.43:0/1895959256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9138014050 con 0x7f9124002350 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f913d000640 1 -- 172.21.15.43:0/1895959256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91380216e0 con 0x7f9124002350 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5655173e44c0 con 0x7f9124002350 2024-09-17T05:29:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655173e4a00 con 0x7f9124002350 2024-09-17T05:29:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.693+0000 7f913d000640 1 -- 172.21.15.43:0/1895959256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9138029020 con 0x7f9124002350 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f913d000640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9120071be0 0x7f91200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f913d000640 1 -- 172.21.15.43:0/1895959256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f91380954b0 con 0x7f9124002350 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9140200640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9120071be0 0x7f91200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5655172b8970 0x5655172b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5655172b92c0 con 0x5655172b8970 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f913f800640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5655172b8970 0x5655172b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9140200640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9120071be0 0x7f91200740f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f91300ad000 tx=0x7f913001aec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f913f800640 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5655172b8970 0x5655172b8d80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f913f800640 1 -- 172.21.15.43:0/1895959256 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f912c00b040 con 0x5655172b8970 2024-09-17T05:29:55.702 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/mds_snaptable mtime 2024-09-17T05:29:43.000000+0000, size 70 2024-09-17T05:29:55.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5655172b8970 msgr2=0x5655172b8d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5655172b8970 0x5655172b8d80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9120071be0 msgr2=0x7f91200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9120071be0 0x7f91200740f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f91300ad000 tx=0x7f913001aec0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 msgr2=0x565517272250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 --2- 172.21.15.43:0/1895959256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9124002350 0x565517272250 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9138010fd0 tx=0x7f913800ded0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9140c00640 1 -- 172.21.15.43:0/1895959256 reap_dead start 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 shutdown_connections 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 >> 172.21.15.43:0/1895959256 conn(0x56551725fd80 msgr2=0x5655173490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 shutdown_connections 2024-09-17T05:29:55.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.697+0000 7f9146fc4080 1 -- 172.21.15.43:0/1895959256 wait complete. 2024-09-17T05:29:55.706 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-09-17T05:29:55.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.761+0000 7f5ded731080 1 Processor -- start 2024-09-17T05:29:55.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.761+0000 7f5ded731080 1 -- start start 2024-09-17T05:29:55.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c7cacb0 0x56335c79c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c853010 0x56335c8534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56335c6e9a00 con 0x56335c8548b0 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56335c747f40 con 0x56335c7cacb0 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56335c79e740 con 0x56335c853010 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c853010 0x56335c8534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c7cacb0 0x56335c79c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35956/0 (socket says 172.21.15.43:35956) 2024-09-17T05:29:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 -- 172.21.15.43:0/3539319158 learned_addr learned my addr 172.21.15.43:0/3539319158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 -- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c853010 msgr2=0x56335c8534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c853010 0x56335c8534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 -- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c7cacb0 msgr2=0x56335c79c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c7cacb0 0x56335c79c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 -- 172.21.15.43:0/3539319158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335c6c2c10 con 0x56335c8548b0 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de7200640 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b7d0 tx=0x7f5de000bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a77693de197fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de4a00640 1 -- 172.21.15.43:0/3539319158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de000db30 con 0x56335c8548b0 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de4a00640 1 -- 172.21.15.43:0/3539319158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de000ddc0 con 0x56335c8548b0 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5de4a00640 1 -- 172.21.15.43:0/3539319158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de000f690 con 0x56335c8548b0 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- 172.21.15.43:0/3539319158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 msgr2=0x56335c8539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b7d0 tx=0x7f5de000bc90 comp rx=0 tx=0).stop 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- 172.21.15.43:0/3539319158 shutdown_connections 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c8539f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c853010 0x56335c8534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 --2- 172.21.15.43:0/3539319158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c7cacb0 0x56335c79c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- 172.21.15.43:0/3539319158 >> 172.21.15.43:0/3539319158 conn(0x56335c787d80 msgr2=0x56335c8527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- 172.21.15.43:0/3539319158 shutdown_connections 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- 172.21.15.43:0/3539319158 wait complete. 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 Processor -- start 2024-09-17T05:29:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.765+0000 7f5ded731080 1 -- start start 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c7cacb0 0x56335c852650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c853010 0x56335c79c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c90e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56335c86de70 con 0x56335c8548b0 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56335c86dbd0 con 0x56335c853010 2024-09-17T05:29:55.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56335c86e110 con 0x56335c7cacb0 2024-09-17T05:29:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c90e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c90e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35968/0 (socket says 172.21.15.43:35968) 2024-09-17T05:29:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 -- 172.21.15.43:0/3001245590 learned_addr learned my addr 172.21.15.43:0/3001245590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de5e00640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c853010 0x56335c79c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de6800640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c7cacb0 0x56335c852650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 -- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c7cacb0 msgr2=0x56335c852650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c7cacb0 0x56335c852650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 -- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c853010 msgr2=0x56335c79c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56335c853010 0x56335c79c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 -- 172.21.15.43:0/3001245590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335c85e380 con 0x56335c8548b0 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de6800640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56335c7cacb0 0x56335c852650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5de7200640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c90e860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5de000dc00 tx=0x7f5de000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ddf400640 1 -- 172.21.15.43:0/3001245590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0004020 con 0x56335c8548b0 2024-09-17T05:29:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ddf400640 1 -- 172.21.15.43:0/3001245590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de00041c0 con 0x56335c8548b0 2024-09-17T05:29:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ddf400640 1 -- 172.21.15.43:0/3001245590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00197a0 con 0x56335c8548b0 2024-09-17T05:29:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56335c85e620 con 0x56335c8548b0 2024-09-17T05:29:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.769+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335c85eb60 con 0x56335c8548b0 2024-09-17T05:29:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ddf400640 1 -- 172.21.15.43:0/3001245590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5de0019940 con 0x56335c8548b0 2024-09-17T05:29:55.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ddf400640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dc8071c10 0x7f5dc8074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ddf400640 1 -- 172.21.15.43:0/3001245590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f5de00947b0 con 0x56335c8548b0 2024-09-17T05:29:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56335c85c770 0x56335c85cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x56335c85f2a0 con 0x56335c85c770 2024-09-17T05:29:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de6800640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dc8071c10 0x7f5dc8074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de5e00640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56335c85c770 0x56335c85cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de6800640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dc8071c10 0x7f5dc8074120 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ab000 tx=0x7f5dd8002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de5e00640 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56335c85c770 0x56335c85cb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de5e00640 1 -- 172.21.15.43:0/3001245590 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f5dd400d040 con 0x56335c85c770 2024-09-17T05:29:55.779 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/mds0_inotable mtime 2024-09-17T05:29:52.000000+0000, size 34 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56335c85c770 msgr2=0x56335c85cb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56335c85c770 0x56335c85cb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dc8071c10 msgr2=0x7f5dc8074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5dc8071c10 0x7f5dc8074120 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ab000 tx=0x7f5dd8002040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 msgr2=0x56335c90e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 --2- 172.21.15.43:0/3001245590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56335c8548b0 0x56335c90e860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5de000dc00 tx=0x7f5de000dc30 comp rx=0 tx=0).stop 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5de7200640 1 -- 172.21.15.43:0/3001245590 reap_dead start 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 shutdown_connections 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 >> 172.21.15.43:0/3001245590 conn(0x56335c787d80 msgr2=0x56335c86cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.773+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 shutdown_connections 2024-09-17T05:29:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.777+0000 7f5ded731080 1 -- 172.21.15.43:0/3001245590 wait complete. 2024-09-17T05:29:55.784 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-09-17T05:29:55.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.837+0000 7f4403677080 1 Processor -- start 2024-09-17T05:29:55.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.837+0000 7f4403677080 1 -- start start 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e4002300 0x5570ecbcfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570ecbd3430 0x5570ecbb5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570eca4ba00 con 0x5570ecbd03c0 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5570ecaa9f40 con 0x5570ecbd3430 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5570ecb003a0 con 0x7f43e4002300 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35988/0 (socket says 172.21.15.43:35988) 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 -- 172.21.15.43:0/3894074204 learned_addr learned my addr 172.21.15.43:0/3894074204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fc800640 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e4002300 0x5570ecbcfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fd200640 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570ecbd3430 0x5570ecbb5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 -- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e4002300 msgr2=0x5570ecbcfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e4002300 0x5570ecbcfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 -- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570ecbd3430 msgr2=0x5570ecbb5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570ecbd3430 0x5570ecbb5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 -- 172.21.15.43:0/3894074204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570eca24c10 con 0x5570ecbd03c0 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43fbe00640 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b610 tx=0x7f43e800bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6612ee2cb15b4f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43faa00640 1 -- 172.21.15.43:0/3894074204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e800aad0 con 0x5570ecbd03c0 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43faa00640 1 -- 172.21.15.43:0/3894074204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e800ac70 con 0x5570ecbd03c0 2024-09-17T05:29:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f43faa00640 1 -- 172.21.15.43:0/3894074204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8008690 con 0x5570ecbd03c0 2024-09-17T05:29:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- 172.21.15.43:0/3894074204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 msgr2=0x5570ecbb54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b610 tx=0x7f43e800bad0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- 172.21.15.43:0/3894074204 shutdown_connections 2024-09-17T05:29:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570ecbd3430 0x5570ecbb5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd03c0 0x5570ecbb54e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 --2- 172.21.15.43:0/3894074204 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43e4002300 0x5570ecbcfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- 172.21.15.43:0/3894074204 >> 172.21.15.43:0/3894074204 conn(0x5570ecae9d80 msgr2=0x5570ecbb47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- 172.21.15.43:0/3894074204 shutdown_connections 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- 172.21.15.43:0/3894074204 wait complete. 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 Processor -- start 2024-09-17T05:29:55.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.841+0000 7f4403677080 1 -- start start 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5570ecbd03c0 0x5570ecbced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 0x5570ecb1f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4002300 0x5570ecb1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5570ecbd0b60 con 0x5570ecbd3430 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5570ecbd08c0 con 0x7f43e4002300 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5570ecbd43e0 con 0x5570ecbd03c0 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 0x5570ecb1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 0x5570ecb1f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35990/0 (socket says 172.21.15.43:35990) 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 -- 172.21.15.43:0/458919875 learned_addr learned my addr 172.21.15.43:0/458919875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fc800640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5570ecbd03c0 0x5570ecbced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fd200640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4002300 0x5570ecb1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 -- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5570ecbd03c0 msgr2=0x5570ecbced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5570ecbd03c0 0x5570ecbced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 -- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4002300 msgr2=0x5570ecb1faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43e4002300 0x5570ecb1faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 -- 172.21.15.43:0/458919875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ecc6e490 con 0x5570ecbd3430 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43fbe00640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 0x5570ecb1f560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f43e80176b0 tx=0x7f43e80176e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43f9600640 1 -- 172.21.15.43:0/458919875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8024070 con 0x5570ecbd3430 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43f9600640 1 -- 172.21.15.43:0/458919875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e8017b10 con 0x5570ecbd3430 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570ecc6e730 con 0x5570ecbd3430 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f43f9600640 1 -- 172.21.15.43:0/458919875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e801f410 con 0x5570ecbd3430 2024-09-17T05:29:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.845+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ecc6f070 con 0x5570ecbd3430 2024-09-17T05:29:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43f9600640 1 -- 172.21.15.43:0/458919875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43e8027070 con 0x5570ecbd3430 2024-09-17T05:29:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43f9600640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43dc071be0 0x7f43dc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43f9600640 1 -- 172.21.15.43:0/458919875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f43e8094c20 con 0x5570ecbd3430 2024-09-17T05:29:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fc800640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43dc071be0 0x7f43dc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5570ecc70e50 0x5570ecc71260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5570ecc717a0 con 0x5570ecc70e50 2024-09-17T05:29:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fd200640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5570ecc70e50 0x5570ecc71260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fc800640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43dc071be0 0x7f43dc0740f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43f0002000 tx=0x7f43f00b4040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fd200640 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5570ecc70e50 0x5570ecc71260 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fd200640 1 -- 172.21.15.43:0/458919875 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f43f4008b60 con 0x5570ecc70e50 2024-09-17T05:29:55.855 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/100.00000000 mtime 2024-09-17T05:29:43.000000+0000, size 0 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5570ecc70e50 msgr2=0x5570ecc71260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5570ecc70e50 0x5570ecc71260 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43dc071be0 msgr2=0x7f43dc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43dc071be0 0x7f43dc0740f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43f0002000 tx=0x7f43f00b4040 comp rx=0 tx=0).stop 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 msgr2=0x5570ecb1f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 --2- 172.21.15.43:0/458919875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5570ecbd3430 0x5570ecb1f560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f43e80176b0 tx=0x7f43e80176e0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f43fd200640 1 -- 172.21.15.43:0/458919875 reap_dead start 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 shutdown_connections 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 >> 172.21.15.43:0/458919875 conn(0x5570ecae9d80 msgr2=0x5570ecbd0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 shutdown_connections 2024-09-17T05:29:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.849+0000 7f4403677080 1 -- 172.21.15.43:0/458919875 wait complete. 2024-09-17T05:29:55.860 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-09-17T05:29:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 Processor -- start 2024-09-17T05:29:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 -- start start 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bcb2d70 0x56194bcdecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bd68e40 0x56194bd86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56194bc00a00 con 0x56194bce1cb0 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56194bc5ef40 con 0x56194bcb2d70 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa041df6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56194bd69890 con 0x56194bd68e40 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa033e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa03ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bd68e40 0x56194bd86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa033e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35998/0 (socket says 172.21.15.43:35998) 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa033e00640 1 -- 172.21.15.43:0/550636377 learned_addr learned my addr 172.21.15.43:0/550636377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.913+0000 7fa03b000640 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bcb2d70 0x56194bcdecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 -- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bd68e40 msgr2=0x56194bd86e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bd68e40 0x56194bd86e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 -- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bcb2d70 msgr2=0x56194bcdecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bcb2d70 0x56194bcdecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 -- 172.21.15.43:0/550636377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194bbd9c10 con 0x56194bce1cb0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa033e00640 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b220 tx=0x7fa02400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc1f3c9d9d44a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa039c00640 1 -- 172.21.15.43:0/550636377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400a040 con 0x56194bce1cb0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa039c00640 1 -- 172.21.15.43:0/550636377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa02400f620 con 0x56194bce1cb0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa039c00640 1 -- 172.21.15.43:0/550636377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400fa70 con 0x56194bce1cb0 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- 172.21.15.43:0/550636377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 msgr2=0x56194bcfa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b220 tx=0x7fa02400b6e0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- 172.21.15.43:0/550636377 shutdown_connections 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bd68e40 0x56194bd86e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bce1cb0 0x56194bcfa6d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 --2- 172.21.15.43:0/550636377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bcb2d70 0x56194bcdecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- 172.21.15.43:0/550636377 >> 172.21.15.43:0/550636377 conn(0x56194bc9ed80 msgr2=0x56194bcfb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- 172.21.15.43:0/550636377 shutdown_connections 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- 172.21.15.43:0/550636377 wait complete. 2024-09-17T05:29:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 Processor -- start 2024-09-17T05:29:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.917+0000 7fa041df6080 1 -- start start 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 0x56194bcf58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bce1cb0 0x56194bd83860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bd68e40 0x56194be22910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56194bce1580 con 0x56194bcb2d70 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56194bcf86f0 con 0x56194bd68e40 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56194bcf3560 con 0x56194bce1cb0 2024-09-17T05:29:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 0x56194bcf58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 0x56194bcf58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36008/0 (socket says 172.21.15.43:36008) 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 -- 172.21.15.43:0/1120468606 learned_addr learned my addr 172.21.15.43:0/1120468606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa033e00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bce1cb0 0x56194bd83860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03ba00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bd68e40 0x56194be22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 -- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bce1cb0 msgr2=0x56194bd83860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56194bce1cb0 0x56194bd83860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 -- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bd68e40 msgr2=0x56194be22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56194bd68e40 0x56194be22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 -- 172.21.15.43:0/1120468606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194bcde000 con 0x56194bcb2d70 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa03b000640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 0x56194bcf58f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00a550 tx=0x7fa02c0a7730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa033400640 1 -- 172.21.15.43:0/1120468606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0b0880 con 0x56194bcb2d70 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa033400640 1 -- 172.21.15.43:0/1120468606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa02c0b0a20 con 0x56194bcb2d70 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56194bcde2a0 con 0x56194bcb2d70 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa033400640 1 -- 172.21.15.43:0/1120468606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0b0df0 con 0x56194bcb2d70 2024-09-17T05:29:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.921+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194bd84530 con 0x56194bcb2d70 2024-09-17T05:29:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa033400640 1 -- 172.21.15.43:0/1120468606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa02c0bc3f0 con 0x56194bcb2d70 2024-09-17T05:29:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa033400640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa018071be0 0x7fa0180740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa033400640 1 -- 172.21.15.43:0/1120468606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fa02c130df0 con 0x56194bcb2d70 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56194bd83da0 0x56194bd84c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x56194bd851b0 con 0x56194bd83da0 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa033e00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa018071be0 0x7fa0180740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa03ba00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56194bd83da0 0x56194bd84c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa033e00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa018071be0 0x7fa0180740f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa024003040 tx=0x7fa024001090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa03ba00640 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56194bd83da0 0x56194bd84c70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa03ba00640 1 -- 172.21.15.43:0/1120468606 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7fa034002040 con 0x56194bd83da0 2024-09-17T05:29:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-09-17T05:29:52.000000+0000, size 0 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56194bd83da0 msgr2=0x56194bd84c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56194bd83da0 0x56194bd84c70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa018071be0 msgr2=0x7fa0180740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa018071be0 0x7fa0180740f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa024003040 tx=0x7fa024001090 comp rx=0 tx=0).stop 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 msgr2=0x56194bcf58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 --2- 172.21.15.43:0/1120468606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56194bcb2d70 0x56194bcf58f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00a550 tx=0x7fa02c0a7730 comp rx=0 tx=0).stop 2024-09-17T05:29:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa03ba00640 1 -- 172.21.15.43:0/1120468606 reap_dead start 2024-09-17T05:29:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 shutdown_connections 2024-09-17T05:29:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 >> 172.21.15.43:0/1120468606 conn(0x56194bc9ed80 msgr2=0x56194bcdecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 shutdown_connections 2024-09-17T05:29:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.925+0000 7fa041df6080 1 -- 172.21.15.43:0/1120468606 wait complete. 2024-09-17T05:29:55.935 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-09-17T05:29:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.989+0000 7fb91a003080 1 Processor -- start 2024-09-17T05:29:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.989+0000 7fb91a003080 1 -- start start 2024-09-17T05:29:55.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 0x557bb2b49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 0x557bb2b49fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bb2a22f40 con 0x557bb2a76d70 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557bb2a2fd00 con 0x557bb2aa5cb0 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557bb29c4a00 con 0x557bb2b2ce40 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36010/0 (socket says 172.21.15.43:36010) 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 -- 172.21.15.43:0/806063674 learned_addr learned my addr 172.21.15.43:0/806063674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913c00640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 0x557bb2b49fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb912800640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 0x557bb2b49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 -- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 msgr2=0x557bb2b49fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 0x557bb2b49fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 -- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 msgr2=0x557bb2b49a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 0x557bb2b49a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 -- 172.21.15.43:0/806063674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb2abbe40 con 0x557bb2a76d70 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913c00640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 0x557bb2b49fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb912800640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 0x557bb2b49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb913200640 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb904004f80 tx=0x7fb90401a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe0537cb9bf8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb911400640 1 -- 172.21.15.43:0/806063674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040a7820 con 0x557bb2a76d70 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb911400640 1 -- 172.21.15.43:0/806063674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9040a7af0 con 0x557bb2a76d70 2024-09-17T05:29:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb911400640 1 -- 172.21.15.43:0/806063674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040b0470 con 0x557bb2a76d70 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- 172.21.15.43:0/806063674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 msgr2=0x557bb2ab7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb904004f80 tx=0x7fb90401a7f0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- 172.21.15.43:0/806063674 shutdown_connections 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2aa5cb0 0x557bb2b49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2b2ce40 0x557bb2b49fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 --2- 172.21.15.43:0/806063674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2ab7740 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- 172.21.15.43:0/806063674 >> 172.21.15.43:0/806063674 conn(0x557bb2a62d80 msgr2=0x557bb2b4bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- 172.21.15.43:0/806063674 shutdown_connections 2024-09-17T05:29:56.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.993+0000 7fb91a003080 1 -- 172.21.15.43:0/806063674 wait complete. 2024-09-17T05:29:56.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 Processor -- start 2024-09-17T05:29:56.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- start start 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2bdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2aa5cb0 0x557bb2b2b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2b2ce40 0x557bb2be6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557bb2abc0e0 con 0x557bb2a76d70 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557bb2abeb00 con 0x557bb2b2ce40 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557bb2b49520 con 0x557bb2aa5cb0 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2bdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2bdb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36026/0 (socket says 172.21.15.43:36026) 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 -- 172.21.15.43:0/3893913930 learned_addr learned my addr 172.21.15.43:0/3893913930 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913c00640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2b2ce40 0x557bb2be6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb912800640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2aa5cb0 0x557bb2b2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 -- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2aa5cb0 msgr2=0x557bb2b2b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557bb2aa5cb0 0x557bb2b2b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 -- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2b2ce40 msgr2=0x557bb2be6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557bb2b2ce40 0x557bb2be6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 -- 172.21.15.43:0/3893913930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb2abb310 con 0x557bb2a76d70 2024-09-17T05:29:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb913200640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2bdb2a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb904004f80 tx=0x7fb9040a7980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb90be00640 1 -- 172.21.15.43:0/3893913930 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040b1ad0 con 0x557bb2a76d70 2024-09-17T05:29:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb90be00640 1 -- 172.21.15.43:0/3893913930 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9040c4070 con 0x557bb2a76d70 2024-09-17T05:29:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb90be00640 1 -- 172.21.15.43:0/3893913930 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040bc420 con 0x557bb2a76d70 2024-09-17T05:29:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557bb2abb5b0 con 0x557bb2a76d70 2024-09-17T05:29:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:55.997+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb2bd70f0 con 0x557bb2a76d70 2024-09-17T05:29:56.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb90be00640 1 -- 172.21.15.43:0/3893913930 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb9040bc5c0 con 0x557bb2a76d70 2024-09-17T05:29:56.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb90be00640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8f4071be0 0x7fb8f40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb90be00640 1 -- 172.21.15.43:0/3893913930 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fb9041316b0 con 0x557bb2a76d70 2024-09-17T05:29:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb91a003080 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557bb2bd7d40 0x557bb2bd8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb912800640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8f4071be0 0x7fb8f40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x557bb2bd8690 con 0x557bb2bd7d40 2024-09-17T05:29:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb913c00640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557bb2bd7d40 0x557bb2bd8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.001+0000 7fb912800640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8f4071be0 0x7fb8f40740f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb9000104e0 tx=0x7fb900002760 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb913c00640 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557bb2bd7d40 0x557bb2bd8150 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb913c00640 1 -- 172.21.15.43:0/3893913930 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fb90c0105d0 con 0x557bb2bd7d40 2024-09-17T05:29:56.008 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/400.00000000 mtime 2024-09-17T05:29:43.000000+0000, size 22 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557bb2bd7d40 msgr2=0x557bb2bd8150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557bb2bd7d40 0x557bb2bd8150 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8f4071be0 msgr2=0x7fb8f40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb8f4071be0 0x7fb8f40740f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb9000104e0 tx=0x7fb900002760 comp rx=0 tx=0).stop 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 msgr2=0x557bb2bdb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 --2- 172.21.15.43:0/3893913930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557bb2a76d70 0x557bb2bdb2a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb904004f80 tx=0x7fb9040a7980 comp rx=0 tx=0).stop 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb913c00640 1 -- 172.21.15.43:0/3893913930 reap_dead start 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 shutdown_connections 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 >> 172.21.15.43:0/3893913930 conn(0x557bb2a62d80 msgr2=0x557bb2aa53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 shutdown_connections 2024-09-17T05:29:56.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.005+0000 7fb91a003080 1 -- 172.21.15.43:0/3893913930 wait complete. 2024-09-17T05:29:56.013 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-09-17T05:29:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.065+0000 7fe7b7c1a080 1 Processor -- start 2024-09-17T05:29:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.065+0000 7fe7b7c1a080 1 -- start start 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 0x561494c71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494c6f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561494b75a00 con 0x561494c2c9c0 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561494bd3f40 con 0x561494cde5b0 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561494c6deb0 con 0x561494c71670 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36044/0 (socket says 172.21.15.43:36044) 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 -- 172.21.15.43:0/4169929020 learned_addr learned my addr 172.21.15.43:0/4169929020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7abe00640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 0x561494c71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b1800640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494c6f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 -- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 msgr2=0x561494c6f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494c6f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 -- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 msgr2=0x561494c71130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 0x561494c71130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 -- 172.21.15.43:0/4169929020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561494b4ec10 con 0x561494c2c9c0 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b1800640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494c6f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7abe00640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 0x561494c71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe7a401b080 tx=0x7fe7a4017ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667b3c5aaf837306 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7aaa00640 1 -- 172.21.15.43:0/4169929020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40ba070 con 0x561494c2c9c0 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7aaa00640 1 -- 172.21.15.43:0/4169929020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7a40ab980 con 0x561494c2c9c0 2024-09-17T05:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7aaa00640 1 -- 172.21.15.43:0/4169929020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40abdb0 con 0x561494c2c9c0 2024-09-17T05:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/4169929020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 msgr2=0x561494d91030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe7a401b080 tx=0x7fe7a4017ff0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/4169929020 shutdown_connections 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494cde5b0 0x561494c71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494c6f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/4169929020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494c2c9c0 0x561494d91030 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/4169929020 >> 172.21.15.43:0/4169929020 conn(0x561494c13d80 msgr2=0x561494c69ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/4169929020 shutdown_connections 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/4169929020 wait complete. 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 Processor -- start 2024-09-17T05:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.069+0000 7fe7b7c1a080 1 -- start start 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494c2c9c0 0x561494c679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494cd6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 0x561494cfd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561494cdddd0 con 0x561494cde5b0 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561494cfea00 con 0x561494c2c9c0 2024-09-17T05:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561494cfe760 con 0x561494c71670 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 0x561494cfd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 0x561494cfd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36048/0 (socket says 172.21.15.43:36048) 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 -- 172.21.15.43:0/1283838749 learned_addr learned my addr 172.21.15.43:0/1283838749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7abe00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494cd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494c2c9c0 0x561494c679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 -- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 msgr2=0x561494cd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561494c71670 0x561494cd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 -- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494c2c9c0 msgr2=0x561494c679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494c2c9c0 0x561494c679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 -- 172.21.15.43:0/1283838749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561494d8ec20 con 0x561494cde5b0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561494c2c9c0 0x561494c679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b1800640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 0x561494cfd690 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c5e0 tx=0x7fe7ac00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7a9600640 1 -- 172.21.15.43:0/1283838749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac004220 con 0x561494cde5b0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7a9600640 1 -- 172.21.15.43:0/1283838749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ac010800 con 0x561494cde5b0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561494d8eec0 con 0x561494cde5b0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7a9600640 1 -- 172.21.15.43:0/1283838749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac010c30 con 0x561494cde5b0 2024-09-17T05:29:56.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.073+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561494d8f400 con 0x561494cde5b0 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7a9600640 1 -- 172.21.15.43:0/1283838749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe7ac00f430 con 0x561494cde5b0 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7a9600640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe790071c10 0x7fe790074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7a9600640 1 -- 172.21.15.43:0/1283838749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fe7ac093d60 con 0x561494cde5b0 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe790071c10 0x7fe790074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561494c67f30 0x561494d9a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x561494d9aec0 con 0x561494c67f30 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7abe00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561494c67f30 0x561494d9a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b0e00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe790071c10 0x7fe790074120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4004f60 tx=0x7fe7a4002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7abe00640 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561494c67f30 0x561494d9a980 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7abe00640 1 -- 172.21.15.43:0/1283838749 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7fe79c00c640 con 0x561494c67f30 2024-09-17T05:29:56.083 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/100.00000000.inode mtime 2024-09-17T05:29:43.000000+0000, size 582 2024-09-17T05:29:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561494c67f30 msgr2=0x561494d9a980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561494c67f30 0x561494d9a980 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe790071c10 msgr2=0x7fe790074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe790071c10 0x7fe790074120 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4004f60 tx=0x7fe7a4002040 comp rx=0 tx=0).stop 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 msgr2=0x561494cfd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 --2- 172.21.15.43:0/1283838749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561494cde5b0 0x561494cfd690 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c5e0 tx=0x7fe7ac00caa0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b1800640 1 -- 172.21.15.43:0/1283838749 reap_dead start 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 shutdown_connections 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 >> 172.21.15.43:0/1283838749 conn(0x561494c13d80 msgr2=0x561494c69d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.077+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 shutdown_connections 2024-09-17T05:29:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.081+0000 7fe7b7c1a080 1 -- 172.21.15.43:0/1283838749 wait complete. 2024-09-17T05:29:56.088 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-09-17T05:29:56.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 Processor -- start 2024-09-17T05:29:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 -- start start 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f89836020 0x562f89846490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f8984f4e0 0x562f898c2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f89751a00 con 0x562f898bd0a0 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f897aff40 con 0x562f8984f4e0 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.141+0000 7feeeec4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562f898bdd90 con 0x562f89836020 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36068/0 (socket says 172.21.15.43:36068) 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 -- 172.21.15.43:0/3553661636 learned_addr learned my addr 172.21.15.43:0/3553661636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee7e00640 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f89836020 0x562f89846490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee7400640 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f8984f4e0 0x562f898c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 -- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f89836020 msgr2=0x562f89846490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f89836020 0x562f89846490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 -- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f8984f4e0 msgr2=0x562f898c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f8984f4e0 0x562f898c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 -- 172.21.15.43:0/3553661636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f8972ac10 con 0x562f898bd0a0 2024-09-17T05:29:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee8800640 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feee000b7a0 tx=0x7feee000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72196122780c78c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee6000640 1 -- 172.21.15.43:0/3553661636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee000db30 con 0x562f898bd0a0 2024-09-17T05:29:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee6000640 1 -- 172.21.15.43:0/3553661636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee000ddc0 con 0x562f898bd0a0 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feee6000640 1 -- 172.21.15.43:0/3553661636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee000f690 con 0x562f898bd0a0 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- 172.21.15.43:0/3553661636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 msgr2=0x562f898dc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7feee000b7a0 tx=0x7feee000bc60 comp rx=0 tx=0).stop 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- 172.21.15.43:0/3553661636 shutdown_connections 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f8984f4e0 0x562f898c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f898bd0a0 0x562f898dc150 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 --2- 172.21.15.43:0/3553661636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f89836020 0x562f89846490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- 172.21.15.43:0/3553661636 >> 172.21.15.43:0/3553661636 conn(0x562f897efd80 msgr2=0x562f898bfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- 172.21.15.43:0/3553661636 shutdown_connections 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- 172.21.15.43:0/3553661636 wait complete. 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 Processor -- start 2024-09-17T05:29:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.145+0000 7feeeec4e080 1 -- start start 2024-09-17T05:29:56.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f89836020 0x562f8984e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 0x562f8984aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f898bd0a0 0x562f8984afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f8984ee50 con 0x562f8984f4e0 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f898bcb70 con 0x562f89836020 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562f898c0590 con 0x562f898bd0a0 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 0x562f8984aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 0x562f8984aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36072/0 (socket says 172.21.15.43:36072) 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 -- 172.21.15.43:0/756394723 learned_addr learned my addr 172.21.15.43:0/756394723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7e00640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f89836020 0x562f8984e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee8800640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f898bd0a0 0x562f8984afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 -- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f898bd0a0 msgr2=0x562f8984afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f898bd0a0 0x562f8984afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 -- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f89836020 msgr2=0x562f8984e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f89836020 0x562f8984e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 -- 172.21.15.43:0/756394723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f8984d3d0 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee7400640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 0x562f8984aa60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7feed401bfe0 tx=0x7feed40a8da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee4c00640 1 -- 172.21.15.43:0/756394723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40ad040 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee4c00640 1 -- 172.21.15.43:0/756394723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed40a9680 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feee4c00640 1 -- 172.21.15.43:0/756394723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4004590 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f8984d670 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.149+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f8984dbb0 con 0x562f8984f4e0 2024-09-17T05:29:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee4c00640 1 -- 172.21.15.43:0/756394723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feed40b0070 con 0x562f8984f4e0 2024-09-17T05:29:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee4c00640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feec8071be0 0x7feec80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee4c00640 1 -- 172.21.15.43:0/756394723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7feed4130490 con 0x562f8984f4e0 2024-09-17T05:29:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee7e00640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feec8071be0 0x7feec80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f89971f00 0x562f89970340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x562f89970880 con 0x562f89971f00 2024-09-17T05:29:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee8800640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f89971f00 0x562f89970340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee7e00640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feec8071be0 0x7feec80740f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feed8001870 tx=0x7feed8002f70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee8800640 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f89971f00 0x562f89970340 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee8800640 1 -- 172.21.15.43:0/756394723 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7feee0002030 con 0x562f89971f00 2024-09-17T05:29:56.158 INFO:teuthology.orchestra.run.smithi043.stdout:cephfs_metadata/1.00000000 mtime 2024-09-17T05:29:52.000000+0000, size 0 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f89971f00 msgr2=0x562f89970340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f89971f00 0x562f89970340 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feec8071be0 msgr2=0x7feec80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feec8071be0 0x7feec80740f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feed8001870 tx=0x7feed8002f70 comp rx=0 tx=0).stop 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 msgr2=0x562f8984aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 --2- 172.21.15.43:0/756394723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f8984f4e0 0x562f8984aa60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7feed401bfe0 tx=0x7feed40a8da0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feee8800640 1 -- 172.21.15.43:0/756394723 reap_dead start 2024-09-17T05:29:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 shutdown_connections 2024-09-17T05:29:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 >> 172.21.15.43:0/756394723 conn(0x562f897efd80 msgr2=0x562f898d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 shutdown_connections 2024-09-17T05:29:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.153+0000 7feeeec4e080 1 -- 172.21.15.43:0/756394723 wait complete. 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-09-17T05:29:56.163 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-09-17T05:29:56.164 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-09-17T05:29:56.165 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:29:56.165 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:56.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:29:56.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:29:56.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:56.174 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:29:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:29:56.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:29:56.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:29:56.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 -- 172.21.15.43:0/3153631719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0b7ce0 msgr2=0x7f689c0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 --2- 172.21.15.43:0/3153631719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0b7ce0 0x7f689c0d0840 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f689c09bbe0 tx=0x7f689001c520 comp rx=0 tx=0).stop 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 -- 172.21.15.43:0/3153631719 shutdown_connections 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 --2- 172.21.15.43:0/3153631719 >> v2:172.21.15.62:3300/0 conn(0x7f689c0cd580 0x7f689c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 --2- 172.21.15.43:0/3153631719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0b7ce0 0x7f689c0d0840 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 -- 172.21.15.43:0/3153631719 >> 172.21.15.43:0/3153631719 conn(0x7f689c0cee80 msgr2=0x7f689c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 -- 172.21.15.43:0/3153631719 shutdown_connections 2024-09-17T05:29:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.421+0000 7f68a3a00640 1 -- 172.21.15.43:0/3153631719 wait complete. 2024-09-17T05:29:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 Processor -- start 2024-09-17T05:29:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- start start 2024-09-17T05:29:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 0x7f689c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f689c0cd580 0x7f689c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f689c16a020 con 0x7f689c0d0d80 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f689c169ea0 con 0x7f689c0b7ce0 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f689c16a1a0 con 0x7f689c0cd580 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36078/0 (socket says 172.21.15.43:36078) 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 -- 172.21.15.43:0/1161782174 learned_addr learned my addr 172.21.15.43:0/1161782174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1000640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 0x7f689c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f689be00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f689c0cd580 0x7f689c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 -- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f689c0cd580 msgr2=0x7f689c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f689c0cd580 0x7f689c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 -- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 msgr2=0x7f689c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 0x7f689c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 -- 172.21.15.43:0/1161782174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c15ef80 con 0x7f689c0d0d80 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1000640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 0x7f689c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a1a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f689400d9d0 tx=0x7f689400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894017070 con 0x7f689c0d0d80 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6894004b30 con 0x7f689c0d0d80 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400be60 con 0x7f689c0d0d80 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c15f260 con 0x7f689c0d0d80 2024-09-17T05:29:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.425+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c15f820 con 0x7f689c0d0d80 2024-09-17T05:29:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c0d0840 con 0x7f689c0d0d80 2024-09-17T05:29:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6894003980 con 0x7f689c0d0d80 2024-09-17T05:29:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f6898c00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 0x7f687c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6894094ad0 con 0x7f689c0d0d80 2024-09-17T05:29:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f68a1000640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 0x7f687c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.429+0000 7f68a1000640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 0x7f687c075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f68900ac000 tx=0x7f689001ad60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.433+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6894020020 con 0x7f689c0d0d80 2024-09-17T05:29:56.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.645+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f689c050fc0 con 0x7f689c0d0d80 2024-09-17T05:29:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:56.997+0000 7f6898c00640 1 -- 172.21.15.43:0/1161782174 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f6894003c70 con 0x7f689c0d0d80 2024-09-17T05:29:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:29:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 msgr2=0x7f687c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 0x7f687c075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f68900ac000 tx=0x7f689001ad60 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 msgr2=0x7f689c06f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f689400d9d0 tx=0x7f689400de90 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 shutdown_connections 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f687c073200 0x7f687c075710 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f689c0d0d80 0x7f689c06f4d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f689c0cd580 0x7f689c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 --2- 172.21.15.43:0/1161782174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689c0b7ce0 0x7f689c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 >> 172.21.15.43:0/1161782174 conn(0x7f689c0cee80 msgr2=0x7f689c15c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 shutdown_connections 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.001+0000 7f68a3a00640 1 -- 172.21.15.43:0/1161782174 wait complete. 2024-09-17T05:29:57.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:29:57.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- 172.21.15.43:0/536560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac740737c0 msgr2=0x7fac740700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:57.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 --2- 172.21.15.43:0/536560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac740737c0 0x7fac740700a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009cb0 tx=0x7fac6c01cc80 comp rx=0 tx=0).stop 2024-09-17T05:29:57.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- 172.21.15.43:0/536560676 shutdown_connections 2024-09-17T05:29:57.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 --2- 172.21.15.43:0/536560676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac740737c0 0x7fac740700a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- 172.21.15.43:0/536560676 >> 172.21.15.43:0/536560676 conn(0x7fac740b23d0 msgr2=0x7fac740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:57.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- 172.21.15.43:0/536560676 shutdown_connections 2024-09-17T05:29:57.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- 172.21.15.43:0/536560676 wait complete. 2024-09-17T05:29:57.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 Processor -- start 2024-09-17T05:29:57.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.185+0000 7fac7a000640 1 -- start start 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac7406fc90 0x7fac74099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac740b1490 0x7fac7415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac7416b670 con 0x7fac7415fe10 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac7416b4f0 con 0x7fac740b1490 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fac7416b7f0 con 0x7fac7406fc90 2024-09-17T05:29:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36090/0 (socket says 172.21.15.43:36090) 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 -- 172.21.15.43:0/4101933450 learned_addr learned my addr 172.21.15.43:0/4101933450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac73400640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac7406fc90 0x7fac74099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac73e00640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac740b1490 0x7fac7415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 -- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac7406fc90 msgr2=0x7fac74099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac7406fc90 0x7fac74099890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 -- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac740b1490 msgr2=0x7fac7415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac740b1490 0x7fac7415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 -- 172.21.15.43:0/4101933450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7415e430 con 0x7fac7415fe10 2024-09-17T05:29:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6be00640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c390 tx=0x7fac5c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c017070 con 0x7fac7415fe10 2024-09-17T05:29:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac5c003600 con 0x7fac7415fe10 2024-09-17T05:29:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c003a50 con 0x7fac7415fe10 2024-09-17T05:29:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7415e6d0 con 0x7fac7415fe10 2024-09-17T05:29:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7415ec90 con 0x7fac7415fe10 2024-09-17T05:29:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.189+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac740700a0 con 0x7fac7415fe10 2024-09-17T05:29:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.193+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac5c01a070 con 0x7fac7415fe10 2024-09-17T05:29:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.193+0000 7fac6b400640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 0x7fac50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.193+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fac5c093430 con 0x7fac7415fe10 2024-09-17T05:29:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.193+0000 7fac73400640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 0x7fac50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:57.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.193+0000 7fac73400640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 0x7fac50075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac64007fd0 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:57.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.197+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac5c05dd10 con 0x7fac7415fe10 2024-09-17T05:29:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:57.433+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fac740aeb90 con 0x7fac7415fe10 2024-09-17T05:29:58.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac6b400640 1 -- 172.21.15.43:0/4101933450 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v90) ==== 107+0+0 (secure 0 0 0) 0x7fac5c062740 con 0x7fac7415fe10 2024-09-17T05:29:58.012 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:29:58.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 msgr2=0x7fac50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 0x7fac50075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac64007fd0 tx=0x7fac64009040 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 msgr2=0x7fac7415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00c390 tx=0x7fac5c00c850 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 shutdown_connections 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac50073230 0x7fac50075740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac740b1490 0x7fac7415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac7415fe10 0x7fac7415a090 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 --2- 172.21.15.43:0/4101933450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fac7406fc90 0x7fac74099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 >> 172.21.15.43:0/4101933450 conn(0x7fac740b23d0 msgr2=0x7fac740b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 shutdown_connections 2024-09-17T05:29:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.009+0000 7fac7a000640 1 -- 172.21.15.43:0/4101933450 wait complete. 2024-09-17T05:29:58.037 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:29:58.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 Processor -- start 2024-09-17T05:29:58.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 -- start start 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af771cb0 0x5557af814ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af78a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557af690a00 con 0x5557af7fb8b0 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5557af6eef40 con 0x5557af7fa010 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff01b785080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5557af7456b0 con 0x5557af771cb0 2024-09-17T05:29:58.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff015400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff015400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36106/0 (socket says 172.21.15.43:36106) 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff015400640 1 -- 172.21.15.43:0/4263640537 learned_addr learned my addr 172.21.15.43:0/4263640537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff014a00640 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af771cb0 0x5557af814ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.093+0000 7ff00fe00640 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af78a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 -- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af771cb0 msgr2=0x5557af814ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af771cb0 0x5557af814ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 -- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 msgr2=0x5557af78a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af78a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 -- 172.21.15.43:0/4263640537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557af669c10 con 0x5557af7fb8b0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff015400640 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff0100ad040 tx=0x7ff0100181b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbff82ad97bea4ec server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff00ea00640 1 -- 172.21.15.43:0/4263640537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100af070 con 0x5557af7fb8b0 2024-09-17T05:29:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff00ea00640 1 -- 172.21.15.43:0/4263640537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0100aabc0 con 0x5557af7fb8b0 2024-09-17T05:29:58.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff00ea00640 1 -- 172.21.15.43:0/4263640537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100b63f0 con 0x5557af7fb8b0 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- 172.21.15.43:0/4263640537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 msgr2=0x5557af78a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff0100ad040 tx=0x7ff0100181b0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- 172.21.15.43:0/4263640537 shutdown_connections 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af78a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af7fb8b0 0x5557af78a7d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 --2- 172.21.15.43:0/4263640537 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af771cb0 0x5557af814ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- 172.21.15.43:0/4263640537 >> 172.21.15.43:0/4263640537 conn(0x5557af72ed80 msgr2=0x5557af815f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- 172.21.15.43:0/4263640537 shutdown_connections 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- 172.21.15.43:0/4263640537 wait complete. 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 Processor -- start 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.097+0000 7ff01b785080 1 -- start start 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af811d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af7fb8b0 0x5557af812240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5557af7f9c90 con 0x5557af771cb0 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5557af78ad10 con 0x5557af7fa010 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5557af814680 con 0x5557af7fb8b0 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36114/0 (socket says 172.21.15.43:36114) 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 learned_addr learned my addr 172.21.15.43:0/1371913298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af811d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff015400640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af7fb8b0 0x5557af812240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af7fb8b0 msgr2=0x5557af812240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5557af7fb8b0 0x5557af812240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 msgr2=0x5557af811d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557af7fa010 0x5557af811d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557af80f7d0 con 0x5557af771cb0 2024-09-17T05:29:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff008008720 tx=0x7ff008008be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff00d600640 1 -- 172.21.15.43:0/1371913298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00801a070 con 0x5557af771cb0 2024-09-17T05:29:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff00d600640 1 -- 172.21.15.43:0/1371913298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff00800abc0 con 0x5557af771cb0 2024-09-17T05:29:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff00d600640 1 -- 172.21.15.43:0/1371913298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008015420 con 0x5557af771cb0 2024-09-17T05:29:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5557af80fa70 con 0x5557af771cb0 2024-09-17T05:29:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557af80ffb0 con 0x5557af771cb0 2024-09-17T05:29:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.101+0000 7ff00d600640 1 -- 172.21.15.43:0/1371913298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff00801d070 con 0x5557af771cb0 2024-09-17T05:29:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff00d600640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 0x7feff40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff00d600640 1 -- 172.21.15.43:0/1371913298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7ff0080930f0 con 0x5557af771cb0 2024-09-17T05:29:58.109 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:29:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 0x7feff40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5557af8106f0 0x5557af810b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af811040 con 0x5557af8106f0 2024-09-17T05:29:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff015400640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5557af8106f0 0x5557af810b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 0x7feff40740f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff00000f5b0 tx=0x7ff000006790 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.105+0000 7ff015400640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5557af8106f0 0x5557af810b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v90'11 uv5 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8113e0 con 0x5557af8106f0 2024-09-17T05:29:58.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v90'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8117b0 con 0x5557af8106f0 2024-09-17T05:29:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v90'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.115 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:29:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.109+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd1d0 con 0x5557af8106f0 2024-09-17T05:29:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v90'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd570 con 0x5557af8106f0 2024-09-17T05:29:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v90'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc1c0 con 0x5557af8106f0 2024-09-17T05:29:58.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v90'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.113+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc560 con 0x5557af8106f0 2024-09-17T05:29:58.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.117+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v90'17 uv7 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.117+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.117+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v90'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.117+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v90'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.123 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:29:58.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v90'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v90'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.121+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v90'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v90'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v90'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.125+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.132 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:29:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v90'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v90'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v90'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.129+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bc900 con 0x5557af8106f0 2024-09-17T05:29:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.133+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v90'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.133+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.133+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v90'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.133+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v90'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v90'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.142 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:29:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v90'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.137+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8be920 con 0x5557af8106f0 2024-09-17T05:29:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v90'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8be920 con 0x5557af8106f0 2024-09-17T05:29:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v90'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v90'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.141+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.145+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v90'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.145+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.145+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v90'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.145+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v90'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.151 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:29:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8be920 con 0x5557af8106f0 2024-09-17T05:29:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v90'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8be920 con 0x5557af8106f0 2024-09-17T05:29:58.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v90'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.149+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8bd910 con 0x5557af8106f0 2024-09-17T05:29:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v90'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5557af8bde20 0x5557af8c0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c0780 con 0x5557af8bde20 2024-09-17T05:29:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5557af8bde20 0x5557af8c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5557af8bde20 0x5557af8c0240 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v90'19 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.153+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c0b20 con 0x5557af8bde20 2024-09-17T05:29:58.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.157+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v90'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.157+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c0ec0 con 0x5557af8bde20 2024-09-17T05:29:58.161 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:29:58.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.157+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v90'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.157+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1760 con 0x5557af8bde20 2024-09-17T05:29:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.157+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v90'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.161+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1b00 con 0x5557af8bde20 2024-09-17T05:29:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.161+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v90'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.161+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1ea0 con 0x5557af8bde20 2024-09-17T05:29:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.161+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v90'24 uv18 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.161+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c2240 con 0x5557af8bde20 2024-09-17T05:29:58.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v90'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c25e0 con 0x5557af8bde20 2024-09-17T05:29:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v90'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.169 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:29:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v90'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.165+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v90'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v90'29 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v90'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.169+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.173+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v90'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.176 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:29:58.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.173+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.173+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v90'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.173+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.177+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v90'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.177+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.177+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v90'34 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.177+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.181+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v90'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.181+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.185+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v90'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:29:58.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.185+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.185+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v90'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.185+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.189+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v90'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.189+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c1390 con 0x5557af8bde20 2024-09-17T05:29:58.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.189+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v90'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8bde20 2024-09-17T05:29:58.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.189+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5557af8c2fa0 0x5557af8c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.189+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c5900 con 0x5557af8c2fa0 2024-09-17T05:29:58.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5557af8c2fa0 0x5557af8c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5557af8c2fa0 0x5557af8c53c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v90'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c5ce0 con 0x5557af8c2fa0 2024-09-17T05:29:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v90'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.193+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c26d0 con 0x5557af8c2fa0 2024-09-17T05:29:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.200 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:29:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c7af0 con 0x5557af8c2fa0 2024-09-17T05:29:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c7e90 con 0x5557af8c2fa0 2024-09-17T05:29:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v90'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.197+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c6ae0 con 0x5557af8c2fa0 2024-09-17T05:29:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.201+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v90'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.201+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c6ed0 con 0x5557af8106f0 2024-09-17T05:29:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.201+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v90'4 uv3 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.201+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8c6ed0 con 0x5557af8106f0 2024-09-17T05:29:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.205+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v90'5 uv5 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.205+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c6ed0 con 0x5557af8106f0 2024-09-17T05:29:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.205+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v90'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.209 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:29:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.205+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8c6ed0 con 0x5557af8106f0 2024-09-17T05:29:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v90'7 uv7 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5557af8c8230 0x5557af8ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cab90 con 0x5557af8c8230 2024-09-17T05:29:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff015400640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5557af8c8230 0x5557af8ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff015400640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5557af8c8230 0x5557af8ca650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v90'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8cab90 con 0x5557af8c8230 2024-09-17T05:29:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.209+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v90'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.213+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cab90 con 0x5557af8c8230 2024-09-17T05:29:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.213+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.217 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:29:58.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.213+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cbba0 con 0x5557af8c8230 2024-09-17T05:29:58.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.217+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.217+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cbba0 con 0x5557af8c8230 2024-09-17T05:29:58.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.217+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v90'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.217+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cab90 con 0x5557af8c8230 2024-09-17T05:29:58.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.221+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v90'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.221+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5557af8cb0a0 0x5557af8cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.221+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cdbe0 con 0x5557af8cb0a0 2024-09-17T05:29:58.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.221+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5557af8cb0a0 0x5557af8cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.221+0000 7ff014a00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5557af8cb0a0 0x5557af8cd4c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v90'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8cdca0 con 0x5557af8cb0a0 2024-09-17T05:29:58.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.230 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:29:58.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cf0a0 con 0x5557af8cb0a0 2024-09-17T05:29:58.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v90'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.225+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8cf0a0 con 0x5557af8cb0a0 2024-09-17T05:29:58.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v90'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v90'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v90'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.229+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v90'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.237 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:29:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v90'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v90'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.233+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8ce090 con 0x5557af8cb0a0 2024-09-17T05:29:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff014a00640 1 -- 172.21.15.43:0/1371913298 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v90'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00801c030 con 0x5557af8cb0a0 2024-09-17T05:29:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5557af8ce710 0x5557af8d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1190 con 0x5557af8ce710 2024-09-17T05:29:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5557af8ce710 0x5557af8d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff00fe00640 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5557af8ce710 0x5557af8d0b30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v90'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.237+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1190 con 0x5557af8ce710 2024-09-17T05:29:58.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.241+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v90'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.241+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8ce710 2024-09-17T05:29:58.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.241+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v90'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.246 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:29:58.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.241+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8ce710 2024-09-17T05:29:58.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.245+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v90'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.245+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8ce710 2024-09-17T05:29:58.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.249+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v90'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.249+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8106f0 2024-09-17T05:29:58.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.249+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v90'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.249+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8106f0 2024-09-17T05:29:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.249+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v90'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.253+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8106f0 2024-09-17T05:29:58.257 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:29:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.253+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.253+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d25f0 con 0x5557af8106f0 2024-09-17T05:29:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.253+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.253+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d25f0 con 0x5557af8106f0 2024-09-17T05:29:58.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v90'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8106f0 2024-09-17T05:29:58.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v90'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8106f0 2024-09-17T05:29:58.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v90'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.257+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.261+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v90'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.261+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.266 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:29:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.261+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v90'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.261+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.265+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v90'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.265+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.269+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v90'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.269+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.269+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v90'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.269+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.273+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v90'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.273+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.278 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:29:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.273+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v90'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.273+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d15e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.277+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v90'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.277+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a90 con 0x5557af8c2fa0 2024-09-17T05:29:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.277+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v90'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8c2fa0 2024-09-17T05:29:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.277+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.281+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v90'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.281+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.281+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v90'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.281+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.287 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:29:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v90'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v90'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v90'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.285+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v90'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.293 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:29:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a50 con 0x5557af8c8230 2024-09-17T05:29:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.289+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8c8230 2024-09-17T05:29:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v90'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8c8230 2024-09-17T05:29:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v90'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff010002050 con 0x5557af8c8230 2024-09-17T05:29:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v90'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.293+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v90'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.301 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:29:58.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v90'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v90'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.297+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.301+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v90'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.301+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.301+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v90'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.301+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v90'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v90'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.309 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:29:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v90'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.305+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v90'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v90'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v90'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.309+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.315 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:29:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v90'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1a10 con 0x5557af8ce710 2024-09-17T05:29:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v90'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d19c0 con 0x5557af8ce710 2024-09-17T05:29:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v90'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d19c0 con 0x5557af8ce710 2024-09-17T05:29:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v90'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.313+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5557af8d1e10 con 0x5557af8ce710 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff00fe00640 1 -- 172.21.15.43:0/1371913298 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v90'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff00001b4e0 con 0x5557af8ce710 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5557af8bde20 msgr2=0x5557af8c0240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5557af8bde20 0x5557af8c0240 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5557af8cb0a0 msgr2=0x5557af8cd4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5557af8cb0a0 0x5557af8cd4c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5557af8ce710 msgr2=0x5557af8d0b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5557af8ce710 0x5557af8d0b30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5557af8c2fa0 msgr2=0x5557af8c53c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5557af8c2fa0 0x5557af8c53c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5557af8106f0 msgr2=0x5557af810b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5557af8106f0 0x5557af810b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5557af8c8230 msgr2=0x5557af8ca650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5557af8c8230 0x5557af8ca650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 reap_dead start 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 reap_dead start 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff015400640 1 -- 172.21.15.43:0/1371913298 reap_dead start 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 msgr2=0x7feff40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 0x7feff40740f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff00000f5b0 tx=0x7ff000006790 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 msgr2=0x5557af80bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff008008720 tx=0x7ff008008be0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 shutdown_connections 2024-09-17T05:29:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff4071be0 0x7feff40740f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 --2- 172.21.15.43:0/1371913298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5557af771cb0 0x5557af80bc30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 >> 172.21.15.43:0/1371913298 conn(0x5557af72ed80 msgr2=0x5557af818c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 shutdown_connections 2024-09-17T05:29:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.317+0000 7ff01b785080 1 -- 172.21.15.43:0/1371913298 wait complete. 2024-09-17T05:29:58.326 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-17T05:29:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 Processor -- start 2024-09-17T05:29:58.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 -- start start 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bcedd70 0x56199bceba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bda3e40 0x56199be48f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56199bc99f40 con 0x56199bd1ccb0 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56199bca6d00 con 0x56199bcedd70 2024-09-17T05:29:58.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.421+0000 7f3b0f049080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56199bc3ba00 con 0x56199bda3e40 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36130/0 (socket says 172.21.15.43:36130) 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 -- 172.21.15.43:0/3086250197 learned_addr learned my addr 172.21.15.43:0/3086250197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b07600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bcedd70 0x56199bceba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b08000640 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bda3e40 0x56199be48f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 -- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bda3e40 msgr2=0x56199be48f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bda3e40 0x56199be48f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 -- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bcedd70 msgr2=0x56199bceba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bcedd70 0x56199bceba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 -- 172.21.15.43:0/3086250197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199be4a490 con 0x56199bd1ccb0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3affe00640 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3af0001eb0 tx=0x7f3af000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40b49da12115102 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b06200640 1 -- 172.21.15.43:0/3086250197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0016070 con 0x56199bd1ccb0 2024-09-17T05:29:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b06200640 1 -- 172.21.15.43:0/3086250197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3af000be30 con 0x56199bd1ccb0 2024-09-17T05:29:58.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b06200640 1 -- 172.21.15.43:0/3086250197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af000d4c0 con 0x56199bd1ccb0 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- 172.21.15.43:0/3086250197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 msgr2=0x56199be489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3af0001eb0 tx=0x7f3af000b660 comp rx=0 tx=0).stop 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- 172.21.15.43:0/3086250197 shutdown_connections 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bda3e40 0x56199be48f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bd1ccb0 0x56199be489d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 --2- 172.21.15.43:0/3086250197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bcedd70 0x56199bceba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- 172.21.15.43:0/3086250197 >> 172.21.15.43:0/3086250197 conn(0x56199bcd9d80 msgr2=0x56199bd360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- 172.21.15.43:0/3086250197 shutdown_connections 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- 172.21.15.43:0/3086250197 wait complete. 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 Processor -- start 2024-09-17T05:29:58.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.425+0000 7f3b0f049080 1 -- start start 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bcedd70 0x56199bdb7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bd1ccb0 0x56199be58090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 0x56199bdbfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56199be4ac00 con 0x56199bda3e40 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56199be4a730 con 0x56199bd1ccb0 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56199be4aea0 con 0x56199bcedd70 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 0x56199bdbfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b07600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bcedd70 0x56199bdb7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 0x56199bdbfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36144/0 (socket says 172.21.15.43:36144) 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 -- 172.21.15.43:0/358653890 learned_addr learned my addr 172.21.15.43:0/358653890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3affe00640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bd1ccb0 0x56199be58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 -- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bcedd70 msgr2=0x56199bdb7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bcedd70 0x56199bdb7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 -- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bd1ccb0 msgr2=0x56199be58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56199bd1ccb0 0x56199be58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 -- 172.21.15.43:0/358653890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199be58940 con 0x56199bda3e40 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b07600640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56199bcedd70 0x56199bdb7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b08000640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 0x56199bdbfdd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3b00003590 tx=0x7f3b00003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b04e00640 1 -- 172.21.15.43:0/358653890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00017020 con 0x56199bda3e40 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b04e00640 1 -- 172.21.15.43:0/358653890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b00013680 con 0x56199bda3e40 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b04e00640 1 -- 172.21.15.43:0/358653890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00013ad0 con 0x56199bda3e40 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56199be58be0 con 0x56199bda3e40 2024-09-17T05:29:58.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.429+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56199be59120 con 0x56199bda3e40 2024-09-17T05:29:58.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b04e00640 1 -- 172.21.15.43:0/358653890 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3b0001a020 con 0x56199bda3e40 2024-09-17T05:29:58.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b04e00640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ae4071be0 0x7f3ae40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b04e00640 1 -- 172.21.15.43:0/358653890 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f3b00092df0 con 0x56199bda3e40 2024-09-17T05:29:58.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b0f049080 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56199be5b650 0x56199be5ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90) -- 0x56199be5bfa0 con 0x56199be5b650 2024-09-17T05:29:58.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b07600640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ae4071be0 0x7f3ae40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3affe00640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56199be5b650 0x56199be5ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3b07600640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ae4071be0 0x7f3ae40740f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3af80af000 tx=0x7f3af8002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.433+0000 7f3affe00640 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56199be5b650 0x56199be5ba60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3affe00640 1 -- 172.21.15.43:0/358653890 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v90'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3af000d640 con 0x56199be5b650 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56199be5b650 msgr2=0x56199be5ba60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56199be5b650 0x56199be5ba60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ae4071be0 msgr2=0x7f3ae40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ae4071be0 0x7f3ae40740f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3af80af000 tx=0x7f3af8002040 comp rx=0 tx=0).stop 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 msgr2=0x56199bdbfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 --2- 172.21.15.43:0/358653890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56199bda3e40 0x56199bdbfdd0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3b00003590 tx=0x7f3b00003a50 comp rx=0 tx=0).stop 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b08000640 1 -- 172.21.15.43:0/358653890 reap_dead start 2024-09-17T05:29:58.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 shutdown_connections 2024-09-17T05:29:58.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 >> 172.21.15.43:0/358653890 conn(0x56199bcd9d80 msgr2=0x56199bceba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 shutdown_connections 2024-09-17T05:29:58.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.437+0000 7f3b0f049080 1 -- 172.21.15.43:0/358653890 wait complete. 2024-09-17T05:29:58.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:29:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 -- 172.21.15.43:0/1663125242 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c06fc90 msgr2=0x7fa67c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 --2- 172.21.15.43:0/1663125242 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c06fc90 0x7fa67c0700a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa668009870 tx=0x7fa66801c660 comp rx=0 tx=0).stop 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 -- 172.21.15.43:0/1663125242 shutdown_connections 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 --2- 172.21.15.43:0/1663125242 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c06fc90 0x7fa67c0700a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 -- 172.21.15.43:0/1663125242 >> 172.21.15.43:0/1663125242 conn(0x7fa67c0b23d0 msgr2=0x7fa67c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 -- 172.21.15.43:0/1663125242 shutdown_connections 2024-09-17T05:29:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.601+0000 7fa681c00640 1 -- 172.21.15.43:0/1663125242 wait complete. 2024-09-17T05:29:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 Processor -- start 2024-09-17T05:29:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 -- start start 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c0730b0 0x7fa67c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c15fe10 0x7fa67c157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c16b610 con 0x7fa67c06fc90 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c16b490 con 0x7fa67c15fe10 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa681c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa67c16b790 con 0x7fa67c0730b0 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36156/0 (socket says 172.21.15.43:36156) 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 -- 172.21.15.43:0/3765887672 learned_addr learned my addr 172.21.15.43:0/3765887672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67a000640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c0730b0 0x7fa67c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67b400640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c15fe10 0x7fa67c157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 -- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c0730b0 msgr2=0x7fa67c1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c0730b0 0x7fa67c1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 -- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c15fe10 msgr2=0x7fa67c157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c15fe10 0x7fa67c157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.605+0000 7fa67aa00640 1 -- 172.21.15.43:0/3765887672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c15f180 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa67aa00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00c6e0 tx=0x7fa66c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c016020 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa66c00a930 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c15fa50 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00ad80 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c15b450 con 0x7fa67c06fc90 2024-09-17T05:29:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67c0700a0 con 0x7fa67c06fc90 2024-09-17T05:29:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa66c019020 con 0x7fa67c06fc90 2024-09-17T05:29:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 0x7fa65c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fa66c094470 con 0x7fa67c06fc90 2024-09-17T05:29:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.609+0000 7fa67a000640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 0x7fa65c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.613+0000 7fa67a000640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 0x7fa65c075710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6680af000 tx=0x7fa6680b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:58.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.617+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa66c020020 con 0x7fa67c06fc90 2024-09-17T05:29:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:58.829+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa67c15b730 con 0x7fa67c06fc90 2024-09-17T05:29:59.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.009+0000 7fa672a00640 1 -- 172.21.15.43:0/3765887672 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v91) ==== 162+0+0 (secure 0 0 0) 0x7fa67c15f180 con 0x7fa67c06fc90 2024-09-17T05:29:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 msgr2=0x7fa65c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 0x7fa65c075710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6680af000 tx=0x7fa6680b2040 comp rx=0 tx=0).stop 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 msgr2=0x7fa67c0994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00c6e0 tx=0x7fa66c00cba0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 shutdown_connections 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa65c073200 0x7fa65c075710 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67c15fe10 0x7fa67c157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa67c0730b0 0x7fa67c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 --2- 172.21.15.43:0/3765887672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67c06fc90 0x7fa67c0994b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 >> 172.21.15.43:0/3765887672 conn(0x7fa67c0b23d0 msgr2=0x7fa67c15bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 shutdown_connections 2024-09-17T05:29:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.013+0000 7fa681c00640 1 -- 172.21.15.43:0/3765887672 wait complete. 2024-09-17T05:29:59.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:29:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- 172.21.15.43:0/599298880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 msgr2=0x7fd65c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 --2- 172.21.15.43:0/599298880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c1609c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd65c09bbe0 tx=0x7fd65401caa0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- 172.21.15.43:0/599298880 shutdown_connections 2024-09-17T05:29:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 --2- 172.21.15.43:0/599298880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c1609c0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 --2- 172.21.15.43:0/599298880 >> v2:172.21.15.62:3300/0 conn(0x7fd65c0b7ce0 0x7fd65c0d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- 172.21.15.43:0/599298880 >> 172.21.15.43:0/599298880 conn(0x7fd65c0cee80 msgr2=0x7fd65c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- 172.21.15.43:0/599298880 shutdown_connections 2024-09-17T05:29:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- 172.21.15.43:0/599298880 wait complete. 2024-09-17T05:29:59.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 Processor -- start 2024-09-17T05:29:59.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.605+0000 7fd660c00640 1 -- start start 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd65c0b7ce0 0x7fd65c1fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd65c0d0d80 0x7fd65c0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c16a020 con 0x7fd65c0cd580 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c169ea0 con 0x7fd65c0b7ce0 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c16a1a0 con 0x7fd65c0d0d80 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36172/0 (socket says 172.21.15.43:36172) 2024-09-17T05:29:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 -- 172.21.15.43:0/1101819442 learned_addr learned my addr 172.21.15.43:0/1101819442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:29:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd65a800640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd65c0d0d80 0x7fd65c0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd659e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd65c0b7ce0 0x7fd65c1fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 -- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd65c0d0d80 msgr2=0x7fd65c0b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd65c0d0d80 0x7fd65c0b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 -- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd65c0b7ce0 msgr2=0x7fd65c1fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd65c0b7ce0 0x7fd65c1fc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 -- 172.21.15.43:0/1101819442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c1f8d90 con 0x7fd65c0cd580 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd653e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd64400b3f0 tx=0x7fd64400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644015020 con 0x7fd65c0cd580 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd644011920 con 0x7fd65c0cd580 2024-09-17T05:29:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644011d50 con 0x7fd65c0cd580 2024-09-17T05:29:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c1f9070 con 0x7fd65c0cd580 2024-09-17T05:29:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c1f9630 con 0x7fd65c0cd580 2024-09-17T05:29:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.609+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c157800 con 0x7fd65c0cd580 2024-09-17T05:29:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.613+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd644018020 con 0x7fd65c0cd580 2024-09-17T05:29:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.613+0000 7fd652000640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 0x7fd638075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.613+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fd644093dc0 con 0x7fd65c0cd580 2024-09-17T05:29:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.613+0000 7fd659e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 0x7fd638075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.613+0000 7fd659e00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 0x7fd638075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd65c071220 tx=0x7fd64c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.617+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd64401f030 con 0x7fd65c0cd580 2024-09-17T05:29:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd65c0d0840 con 0x7fd65c0cd580 2024-09-17T05:29:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd652000640 1 -- 172.21.15.43:0/1101819442 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5210 (secure 0 0 0) 0x7fd64405e690 con 0x7fd65c0cd580 2024-09-17T05:29:59.829 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:29:59.829 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"btime":"2024-09-17T05:29:59:016915+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:29:59.016895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6697":{"gid":6697,"name":"c","rank":0,"incarnation":92,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/273818866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":273818866},{"type":"v1","addr":"172.21.15.43:6835","nonce":273818866}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:29:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 92 2024-09-17T05:29:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 msgr2=0x7fd638075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 0x7fd638075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd65c071220 tx=0x7fd64c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 msgr2=0x7fd65c156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd64400b3f0 tx=0x7fd64400b8b0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 shutdown_connections 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd638073230 0x7fd638075740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd65c0d0d80 0x7fd65c0b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd65c0cd580 0x7fd65c156eb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 --2- 172.21.15.43:0/1101819442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd65c0b7ce0 0x7fd65c1fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 >> 172.21.15.43:0/1101819442 conn(0x7fd65c0cee80 msgr2=0x7fd65c0cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 shutdown_connections 2024-09-17T05:29:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:29:59.825+0000 7fd660c00640 1 -- 172.21.15.43:0/1101819442 wait complete. 2024-09-17T05:29:59.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:29:59.016895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6697': {'gid': 6697, 'name': 'c', 'rank': 0, 'incarnation': 92, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/273818866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 273818866}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 273818866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:29:59.852 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6697:up:replay 2024-09-17T05:29:59.852 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:30:03.633 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:03.629+0000 7f1a94200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:05.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 -- 172.21.15.43:0/3858063042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0410155d70 msgr2=0x7f041015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 --2- 172.21.15.43:0/3858063042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0410155d70 0x7f041015c7e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f04100d0c40 tx=0x7f040001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 -- 172.21.15.43:0/3858063042 shutdown_connections 2024-09-17T05:30:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 --2- 172.21.15.43:0/3858063042 >> v2:172.21.15.62:3300/0 conn(0x7f04100b3af0 0x7f04100ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 --2- 172.21.15.43:0/3858063042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0410155d70 0x7f041015c7e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 -- 172.21.15.43:0/3858063042 >> 172.21.15.43:0/3858063042 conn(0x7f04100b23d0 msgr2=0x7f04100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 -- 172.21.15.43:0/3858063042 shutdown_connections 2024-09-17T05:30:05.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.425+0000 7f0416800640 1 -- 172.21.15.43:0/3858063042 wait complete. 2024-09-17T05:30:05.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 Processor -- start 2024-09-17T05:30:05.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- start start 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0410155d70 0x7f04101552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 0x7f0410164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0410168c30 con 0x7f041006f110 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0410168ab0 con 0x7f0410164970 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0410168db0 con 0x7f0410155d70 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37336/0 (socket says 172.21.15.43:37336) 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 -- 172.21.15.43:0/3360984094 learned_addr learned my addr 172.21.15.43:0/3360984094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040fe00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 0x7f0410164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0406a00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0410155d70 0x7f04101552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 -- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0410155d70 msgr2=0x7f04101552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0410155d70 0x7f04101552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 -- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 msgr2=0x7f0410164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 0x7f0410164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 -- 172.21.15.43:0/3360984094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04101652c0 con 0x7f041006f110 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040fe00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 0x7f0410164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f040f400640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f040001b370 tx=0x7f0400009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000bb070 con 0x7f041006f110 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0400004d40 con 0x7f041006f110 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000b2040 con 0x7f041006f110 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04100cd0b0 con 0x7f041006f110 2024-09-17T05:30:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.429+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0410160f80 con 0x7f041006f110 2024-09-17T05:30:05.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.433+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04100ad0b0 con 0x7f041006f110 2024-09-17T05:30:05.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.433+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0400004260 con 0x7f041006f110 2024-09-17T05:30:05.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.433+0000 7f0407e00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 0x7f03ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.433+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f040012e2f0 con 0x7f041006f110 2024-09-17T05:30:05.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.433+0000 7f0406a00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 0x7f03ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.437+0000 7f0406a00640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 0x7f03ec075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f04101f9cc0 tx=0x7f03f8008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.437+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0400016970 con 0x7f041006f110 2024-09-17T05:30:05.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.649+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04100d1150 con 0x7f041006f110 2024-09-17T05:30:05.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.649+0000 7f0407e00640 1 -- 172.21.15.43:0/3360984094 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5217 (secure 0 0 0) 0x7f04100d1150 con 0x7f041006f110 2024-09-17T05:30:05.655 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"btime":"2024-09-17T05:30:02:035273+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:02.035266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6697":{"gid":6697,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":5,"addr":"172.21.15.43:6835/273818866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":273818866},{"type":"v1","addr":"172.21.15.43:6835","nonce":273818866}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6697,"qdb_cluster":[6697]},"id":6}]} 2024-09-17T05:30:05.655 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 95 2024-09-17T05:30:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 msgr2=0x7f03ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 0x7f03ec075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f04101f9cc0 tx=0x7f03f8008040 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 msgr2=0x7f04101603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f040001b370 tx=0x7f0400009360 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 shutdown_connections 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f03ec073200 0x7f03ec075710 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0410164970 0x7f0410164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0410155d70 0x7f04101552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 --2- 172.21.15.43:0/3360984094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041006f110 0x7f04101603d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 >> 172.21.15.43:0/3360984094 conn(0x7f04100b23d0 msgr2=0x7f041006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 shutdown_connections 2024-09-17T05:30:05.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.653+0000 7f0416800640 1 -- 172.21.15.43:0/3360984094 wait complete. 2024-09-17T05:30:05.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:30:02.035266+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6697': {'gid': 6697, 'name': 'c', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.43:6835/273818866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 273818866}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 273818866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6697, 'qdb_cluster': [6697]} 2024-09-17T05:30:05.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:30:05.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:30:05.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 -- 172.21.15.43:0/1739834535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c003b40 msgr2=0x7fc49c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 --2- 172.21.15.43:0/1739834535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c003b40 0x7fc49c003fb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009b40 tx=0x7fc4a801d250 comp rx=0 tx=0).stop 2024-09-17T05:30:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 -- 172.21.15.43:0/1739834535 shutdown_connections 2024-09-17T05:30:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 --2- 172.21.15.43:0/1739834535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c003b40 0x7fc49c003fb0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 -- 172.21.15.43:0/1739834535 >> 172.21.15.43:0/1739834535 conn(0x7fc4b80b23d0 msgr2=0x7fc4b80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 -- 172.21.15.43:0/1739834535 shutdown_connections 2024-09-17T05:30:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.833+0000 7fc4be000640 1 -- 172.21.15.43:0/1739834535 wait complete. 2024-09-17T05:30:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 Processor -- start 2024-09-17T05:30:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- start start 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc49c003b40 0x7fc4b8074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4b81eac10 0x7fc4b8159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b806d490 con 0x7fc4b815ae30 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b806d310 con 0x7fc49c003b40 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b806d610 con 0x7fc4b81eac10 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37356/0 (socket says 172.21.15.43:37356) 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 -- 172.21.15.43:0/2256234341 learned_addr learned my addr 172.21.15.43:0/2256234341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7400640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc49c003b40 0x7fc4b8074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4aea00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4b81eac10 0x7fc4b8159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 -- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4b81eac10 msgr2=0x7fc4b8159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4b81eac10 0x7fc4b8159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 -- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc49c003b40 msgr2=0x7fc4b8074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc49c003b40 0x7fc4b8074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 -- 172.21.15.43:0/2256234341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b81598c0 con 0x7fc4b815ae30 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0002240 tx=0x7fc4b000ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4afe00640 1 -- 172.21.15.43:0/2256234341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0010890 con 0x7fc4b815ae30 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4afe00640 1 -- 172.21.15.43:0/2256234341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b0018070 con 0x7fc4b815ae30 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4afe00640 1 -- 172.21.15.43:0/2256234341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0010da0 con 0x7fc4b815ae30 2024-09-17T05:30:05.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b815b780 con 0x7fc4b815ae30 2024-09-17T05:30:05.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.837+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b81f9220 con 0x7fc4b815ae30 2024-09-17T05:30:05.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.841+0000 7fc4afe00640 1 -- 172.21.15.43:0/2256234341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4b001d460 con 0x7fc4b815ae30 2024-09-17T05:30:05.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 Processor -- start 2024-09-17T05:30:05.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4afe00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 0x7fc494075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 -- start start 2024-09-17T05:30:05.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4afe00640 1 -- 172.21.15.43:0/2256234341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fc4b00946d0 con 0x7fc4b815ae30 2024-09-17T05:30:05.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4aea00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 0x7fc494075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4aea00640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 0x7fc494075740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc4b815bc10 tx=0x7fc4a0000bd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cf6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0bcb410 con 0x55e7d07b5460 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0b08b40 con 0x55e7d0b8eca0 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e7d0af7f70 con 0x55e7d0ba5950 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37366/0 (socket says 172.21.15.43:37366) 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 -- 172.21.15.43:0/1513475318 learned_addr learned my addr 172.21.15.43:0/1513475318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4aea00640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cf6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 -- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 msgr2=0x55e7d0cf6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cf6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 -- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 msgr2=0x55e7d0cf7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 -- 172.21.15.43:0/1513475318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d0abf5b0 con 0x55e7d07b5460 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4aea00640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cf6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc4a801d080 tx=0x7fc49c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5bcb89bfb2e1df server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4a7400640 1 -- 172.21.15.43:0/1513475318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80bb070 con 0x55e7d07b5460 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4a7400640 1 -- 172.21.15.43:0/1513475318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4a80055c0 con 0x55e7d07b5460 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.845+0000 7fc4a7400640 1 -- 172.21.15.43:0/1513475318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80048d0 con 0x55e7d07b5460 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1513475318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 msgr2=0x55e7d0cf66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc4a801d080 tx=0x7fc49c005840 comp rx=0 tx=0).stop 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1513475318 shutdown_connections 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cf6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1513475318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf66a0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1513475318 >> 172.21.15.43:0/1513475318 conn(0x55e7d0c7bd90 msgr2=0x55e7d0cab690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1513475318 shutdown_connections 2024-09-17T05:30:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1513475318 wait complete. 2024-09-17T05:30:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- start start 2024-09-17T05:30:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 Processor -- start 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d07b5460 0x55e7d0cfd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0a925f0 con 0x55e7d0b8eca0 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0afbd80 con 0x55e7d07b5460 2024-09-17T05:30:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e7d0a9e520 con 0x55e7d0ba5950 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4c55ce000 10 client.0 _ll_register_callbacks cb 0x55e7d0aa2830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37368/0 (socket says 172.21.15.43:37368) 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 -- 172.21.15.43:0/1296270494 learned_addr learned my addr 172.21.15.43:0/1296270494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d07b5460 0x55e7d0cfd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 -- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 msgr2=0x55e7d0cfe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cfe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 -- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d07b5460 msgr2=0x55e7d0cfd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d07b5460 0x55e7d0cfd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.849+0000 7fc4aea00640 1 -- 172.21.15.43:0/1296270494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d0a96190 con 0x55e7d0b8eca0 2024-09-17T05:30:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4aea00640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00129f0 tx=0x7fc4a0001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4846a6b18bba548b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0016070 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fc4a0011d60 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7d0a90110 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fc4a002f050 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 10 client.7503 fetch_fsmap learned FSMap version 95 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55e7d0b7cf10 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e7d0a90110 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fc4a0032020 con 0x55e7d0b8eca0 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 10 client.7503 fetch_fsmap finished waiting for FSMap version 95 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 10 client.7503 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 20 client.7503 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d079a7d0 0x55e7d0d07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 4 client.7503 mds_command: new command op to 6697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:30:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 --> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7d0b0aa90 con 0x55e7d079a7d0 2024-09-17T05:30:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d079a7d0 0x55e7d0d07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4b7400640 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d079a7d0 0x55e7d0d07d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29934aa963c8ff71 server_cookie=d8191ba8d06ce801 in_seq=0 out_seq=0 2024-09-17T05:30:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 10 client.7503 ms_handle_connect on v2:172.21.15.43:6833/273818866 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 1 -- 172.21.15.43:0/1296270494 <== mds.0 v2:172.21.15.43:6833/273818866 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc4a80cbf90 con 0x55e7d079a7d0 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 10 client.7503 handle_command_reply: tid=0 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 client.7503 shutdown 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 msgr2=0x55e7d0cfde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00129f0 tx=0x7fc4a0001130 comp rx=0 tx=0).stop 2024-09-17T05:30:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 shutdown_connections 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d079a7d0 0x55e7d0d07d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0cfe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cfde60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/1296270494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d07b5460 0x55e7d0cfd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc49be00640 0 client.7503 ms_handle_reset on v2:172.21.15.43:6833/273818866 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 >> 172.21.15.43:0/1296270494 conn(0x55e7d0c7bd90 msgr2=0x55e7d0cb2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 shutdown_connections 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 1 -- 172.21.15.43:0/1296270494 wait complete. 2024-09-17T05:30:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.853+0000 7fc4c55ce000 20 client.7503 trim_cache size 0 max 16384 2024-09-17T05:30:05.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.873+0000 7fc4c55ce000 1 Processor -- start 2024-09-17T05:30:05.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.873+0000 7fc4c55ce000 1 -- start start 2024-09-17T05:30:05.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0caacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8205e70 con 0x55e7d07b5460 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8204ee0 con 0x55e7d0b8eca0 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b81ee0e0 con 0x55e7d0ba5950 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37380/0 (socket says 172.21.15.43:37380) 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 -- 172.21.15.43:0/3730507565 learned_addr learned my addr 172.21.15.43:0/3730507565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4aea00640 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0caacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 -- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 msgr2=0x55e7d0caacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0caacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 -- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 msgr2=0x55e7d0cf5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 -- 172.21.15.43:0/3730507565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d0b53db0 con 0x55e7d07b5460 2024-09-17T05:30:05.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4b7400640 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009880 tx=0x7fc4a80091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c0f87e1cf1af08 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4a6a00640 1 -- 172.21.15.43:0/3730507565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80d0070 con 0x55e7d07b5460 2024-09-17T05:30:05.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4a6a00640 1 -- 172.21.15.43:0/3730507565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4a80c6040 con 0x55e7d07b5460 2024-09-17T05:30:05.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4a6a00640 1 -- 172.21.15.43:0/3730507565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80bb040 con 0x55e7d07b5460 2024-09-17T05:30:05.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- 172.21.15.43:0/3730507565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 msgr2=0x55e7d0cf5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009880 tx=0x7fc4a80091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- 172.21.15.43:0/3730507565 shutdown_connections 2024-09-17T05:30:05.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cf5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d0ba5950 0x55e7d0caacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/3730507565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d07b5460 0x55e7d0cf5630 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- 172.21.15.43:0/3730507565 >> 172.21.15.43:0/3730507565 conn(0x55e7d0c7bd90 msgr2=0x55e7d0d06f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- 172.21.15.43:0/3730507565 shutdown_connections 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- 172.21.15.43:0/3730507565 wait complete. 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 -- start start 2024-09-17T05:30:05.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.877+0000 7fc4c55ce000 1 Processor -- start 2024-09-17T05:30:05.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d07b5460 0x55e7d0cadcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0b7ca80 con 0x55e7d0ba5950 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7d0ac3ee0 con 0x55e7d0b8eca0 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e7d0afb090 con 0x55e7d07b5460 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 10 client.0 _ll_register_callbacks cb 0x7fc4a00125f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37394/0 (socket says 172.21.15.43:37394) 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d07b5460 0x55e7d0cadcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 -- 172.21.15.43:0/673443580 learned_addr learned my addr 172.21.15.43:0/673443580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4aea00640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 -- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d07b5460 msgr2=0x55e7d0cadcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d07b5460 0x55e7d0cadcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 -- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 msgr2=0x55e7d0cae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 -- 172.21.15.43:0/673443580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d0abeab0 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4aea00640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7e00640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000f90 tx=0x7fc4b005b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5b67a0ee7624e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b005eef0 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fc4b005a840 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a80c6040 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fc4b007e050 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 10 client.7509 fetch_fsmap learned FSMap version 95 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x55e7d0a9d070 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b0078020 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fc4b005aa40 con 0x55e7d0ba5950 2024-09-17T05:30:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 10 client.7509 fetch_fsmap finished waiting for FSMap version 95 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 10 client.7509 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 20 client.7509 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d0788830 0x55e7d0d6b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 4 client.7509 mds_command: new command op to 6697 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 --> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] -- command(tid 0: {"prefix": "status"}) -- 0x55e7d0b06540 con 0x55e7d0788830 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7400640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d0788830 0x55e7d0d6b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4b7400640 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d0788830 0x55e7d0d6b860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8960d65443ad8a7 server_cookie=26f8b67c0946df1e in_seq=0 out_seq=0 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 10 client.7509 ms_handle_connect on v2:172.21.15.43:6833/273818866 2024-09-17T05:30:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 1 -- 172.21.15.43:0/673443580 <== mds.0 v2:172.21.15.43:6833/273818866 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e7d0b06540 con 0x55e7d0788830 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc49be00640 10 client.7509 handle_command_reply: tid=0 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 client.7509 shutdown 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 msgr2=0x55e7d0d28220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.881+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000f90 tx=0x7fc4b005b660 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 shutdown_connections 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:6833/273818866,v1:172.21.15.43:6835/273818866] conn(0x55e7d0788830 0x55e7d0d6b860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7d0b8eca0 0x55e7d0cae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e7d0ba5950 0x55e7d0d28220 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 --2- 172.21.15.43:0/673443580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e7d07b5460 0x55e7d0cadcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc49be00640 0 client.7509 ms_handle_reset on v2:172.21.15.43:6833/273818866 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 >> 172.21.15.43:0/673443580 conn(0x55e7d0c7bd90 msgr2=0x55e7d0d078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 shutdown_connections 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 1 -- 172.21.15.43:0/673443580 wait complete. 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4c55ce000 20 client.7509 trim_cache size 0 max 16384 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6697, 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8677991370000004, 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 95, 2024-09-17T05:30:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 90, 2024-09-17T05:30:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-09-17T05:30:05.893 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 13.120198558 2024-09-17T05:30:05.893 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 msgr2=0x7fc494075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 0x7fc494075740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc4b815bc10 tx=0x7fc4a0000bd0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 msgr2=0x7fc4b815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0002240 tx=0x7fc4b000ea30 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 shutdown_connections 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc494073230 0x7fc494075740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b815ae30 0x7fc4b815b240 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4b81eac10 0x7fc4b8159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 --2- 172.21.15.43:0/2256234341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc49c003b40 0x7fc4b8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 >> 172.21.15.43:0/2256234341 conn(0x7fc4b80b23d0 msgr2=0x7fc4b815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 shutdown_connections 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:05.885+0000 7fc4be000640 1 -- 172.21.15.43:0/2256234341 wait complete. 2024-09-17T05:30:05.908 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:30:05.908 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-09-17T05:30:05.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:30:06.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- 172.21.15.43:0/3918015092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 msgr2=0x7f8cf00b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/3918015092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf00b20b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009900 tx=0x7f8cdc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- 172.21.15.43:0/3918015092 shutdown_connections 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/3918015092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf00b20b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- 172.21.15.43:0/3918015092 >> 172.21.15.43:0/3918015092 conn(0x7f8cf00cee70 msgr2=0x7f8cf00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- 172.21.15.43:0/3918015092 shutdown_connections 2024-09-17T05:30:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- 172.21.15.43:0/3918015092 wait complete. 2024-09-17T05:30:06.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 Processor -- start 2024-09-17T05:30:06.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- start start 2024-09-17T05:30:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf00ccf70 0x7f8cf00c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cf00d0950 0x7f8cf00c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00b2770 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00b25f0 con 0x7f8cf00ccf70 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.065+0000 7f8cf7e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf016b760 con 0x7f8cf00d0950 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37410/0 (socket says 172.21.15.43:37410) 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 -- 172.21.15.43:0/1955971431 learned_addr learned my addr 172.21.15.43:0/1955971431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf4a00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf00ccf70 0x7f8cf00c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cf00d0950 0x7f8cf00c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 -- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cf00d0950 msgr2=0x7f8cf00c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cf00d0950 0x7f8cf00c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 -- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf00ccf70 msgr2=0x7f8cf00c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf00ccf70 0x7f8cf00c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 -- 172.21.15.43:0/1955971431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf00c8bb0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf5400640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c430 tx=0x7f8ce400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4017070 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce40053c0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40038b0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01f6ff0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf00c8670 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf00b20b0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.069+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ce401a070 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.073+0000 7f8ced600640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 0x7f8cd0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.073+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f8ce40934e0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.073+0000 7f8cf4a00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 0x7f8cd0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.073+0000 7f8cf4a00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 0x7f8cd0075740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0af000 tx=0x7f8cdc01b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.077+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ce405dee0 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.289+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8cf00cfa90 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.289+0000 7f8ced600640 1 -- 172.21.15.43:0/1955971431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) ==== 53+0+73 (secure 0 0 0) 0x7f8ce4062910 con 0x7f8cf00b1ca0 2024-09-17T05:30:06.294 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 msgr2=0x7f8cd0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 0x7f8cd0075740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0af000 tx=0x7f8cdc01b120 comp rx=0 tx=0).stop 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 msgr2=0x7f8cf0150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c430 tx=0x7f8ce400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 shutdown_connections 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd0073230 0x7f8cd0075740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf00ccf70 0x7f8cf00c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cf00d0950 0x7f8cf00c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 --2- 172.21.15.43:0/1955971431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf00b1ca0 0x7f8cf0150bf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 >> 172.21.15.43:0/1955971431 conn(0x7f8cf00cee70 msgr2=0x7f8cf00b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 shutdown_connections 2024-09-17T05:30:06.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06.293+0000 7f8cf7e00640 1 -- 172.21.15.43:0/1955971431 wait complete. 2024-09-17T05:30:06.316 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:30:06.317 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:06.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.317 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.327 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42037sec preferred_lft 42037sec 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:06.328 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:30:06.329 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:06.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.330 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:30:06.388 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:30:06.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.391 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:30:06.448 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:30:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:30:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:30:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:30:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:30:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:30:06.449 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:30:06.449 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:30:06.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:30:06.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:30:06.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:30:06.450 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:30:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:30:06.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:06.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.587 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:30:06.596 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:30:06.596 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:30:06.597 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:06.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:30:06.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:06.659 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:06.711 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:06.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:06.711 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:06.762 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:30:06.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:06.864 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:06.865 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:06.865 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:06.883 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:06.883 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:06.883 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:06.883 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:06.883 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:06.893 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:06.893 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:30:06.894 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:30:06.898 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:30:06.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:06.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:06.918 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:30:07.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:30:07.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:07.020 INFO:teuthology.orchestra.run.smithi043.stdout:client7521 2024-09-17T05:30:07.020 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:07.020 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:07.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:07.030 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:30:07.074 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:30:07.081 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:07.077+0000 7f1a8f200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-17T05:30:07.082 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:30:07.082 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:30:07.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:07.082 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:30:07.083 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-09-17T05:30:07.084 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-09-17T05:30:07.084 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:07.084 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:07.131 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:07.131 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:07.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:30:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 -- 172.21.15.43:0/1176039007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 msgr2=0x7f758c0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 --2- 172.21.15.43:0/1176039007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0ccc40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f758c0cfdc0 tx=0x7f757c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 -- 172.21.15.43:0/1176039007 shutdown_connections 2024-09-17T05:30:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 --2- 172.21.15.43:0/1176039007 >> v2:172.21.15.62:3300/0 conn(0x7f758c15ca80 0x7f758c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 --2- 172.21.15.43:0/1176039007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0ccc40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 -- 172.21.15.43:0/1176039007 >> 172.21.15.43:0/1176039007 conn(0x7f758c068fe0 msgr2=0x7f758c069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 -- 172.21.15.43:0/1176039007 shutdown_connections 2024-09-17T05:30:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 -- 172.21.15.43:0/1176039007 wait complete. 2024-09-17T05:30:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.389+0000 7f758be00640 1 Processor -- start 2024-09-17T05:30:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- start start 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f758c15ca80 0x7f758c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 0x7f758c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f758c153680 con 0x7f758c0d1cb0 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f758c153500 con 0x7f758c1605e0 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f758c153800 con 0x7f758c15ca80 2024-09-17T05:30:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37438/0 (socket says 172.21.15.43:37438) 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 -- 172.21.15.43:0/2142574590 learned_addr learned my addr 172.21.15.43:0/2142574590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 -- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f758c15ca80 msgr2=0x7f758c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f758c15ca80 0x7f758c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 -- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 msgr2=0x7f758c0ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758b400640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 0x7f758c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 0x7f758c0ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 -- 172.21.15.43:0/2142574590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c06f740 con 0x7f758c0d1cb0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758b400640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 0x7f758c0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758aa00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f757c009790 tx=0x7f757c0a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0b2040 con 0x7f758c0d1cb0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f757c0bb070 con 0x7f758c0d1cb0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c003600 con 0x7f758c0d1cb0 2024-09-17T05:30:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1659b0 con 0x7f758c0d1cb0 2024-09-17T05:30:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.393+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c167510 con 0x7f758c0d1cb0 2024-09-17T05:30:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c0ccc40 con 0x7f758c0d1cb0 2024-09-17T05:30:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f757c0037a0 con 0x7f758c0d1cb0 2024-09-17T05:30:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f7582a00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 0x7f7568075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f757c12e930 con 0x7f758c0d1cb0 2024-09-17T05:30:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f7583e00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 0x7f7568075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.397+0000 7f7583e00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 0x7f7568075740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f758c1fbde0 tx=0x7f75740095a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.401+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f757c0163f0 con 0x7f758c0d1cb0 2024-09-17T05:30:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.585+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f758c167770 con 0x7f758c0d1cb0 2024-09-17T05:30:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.585+0000 7f7582a00640 1 -- 172.21.15.43:0/2142574590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) ==== 80+0+120 (secure 0 0 0) 0x7f757c0f9860 con 0x7f758c0d1cb0 2024-09-17T05:30:07.590 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:07.590 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:07.591 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 msgr2=0x7f7568075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 0x7f7568075740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f758c1fbde0 tx=0x7f75740095a0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 msgr2=0x7f758c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f757c009790 tx=0x7f757c0a5730 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 shutdown_connections 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7568073230 0x7f7568075740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f758c1605e0 0x7f758c0ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f758c15ca80 0x7f758c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 --2- 172.21.15.43:0/2142574590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f758c0d1cb0 0x7f758c0cb620 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 >> 172.21.15.43:0/2142574590 conn(0x7f758c068fe0 msgr2=0x7f758c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:07.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 shutdown_connections 2024-09-17T05:30:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.589+0000 7f758be00640 1 -- 172.21.15.43:0/2142574590 wait complete. 2024-09-17T05:30:07.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:30:07.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:07.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:30:07.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:07.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:07.637 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:07.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:07.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 -- 172.21.15.43:0/123140387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ac004810 msgr2=0x7f43ac004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 --2- 172.21.15.43:0/123140387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ac004810 0x7f43ac004c20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f43c80af0a0 tx=0x7f43b001d2a0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43ccc00640 1 -- 172.21.15.43:0/123140387 reap_dead start 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 -- 172.21.15.43:0/123140387 shutdown_connections 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 -- 172.21.15.43:0/123140387 >> 172.21.15.43:0/123140387 conn(0x7f43c80b23d0 msgr2=0x7f43c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 -- 172.21.15.43:0/123140387 shutdown_connections 2024-09-17T05:30:07.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.885+0000 7f43cec00640 1 -- 172.21.15.43:0/123140387 wait complete. 2024-09-17T05:30:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 Processor -- start 2024-09-17T05:30:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 -- start start 2024-09-17T05:30:07.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac004810 0x7f43c8160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c80c9fd0 0x7f43c80cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43c815cb50 con 0x7f43c80cfc80 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43c815c9d0 con 0x7f43ac004810 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43cec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43c81663b0 con 0x7f43c80c9fd0 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37466/0 (socket says 172.21.15.43:37466) 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 -- 172.21.15.43:0/83216627 learned_addr learned my addr 172.21.15.43:0/83216627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43c7e00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac004810 0x7f43c8160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43bfe00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c80c9fd0 0x7f43c80cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 -- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c80c9fd0 msgr2=0x7f43c80cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c80c9fd0 0x7f43c80cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 -- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac004810 msgr2=0x7f43c8160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac004810 0x7f43c8160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.889+0000 7f43ccc00640 1 -- 172.21.15.43:0/83216627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c81f7190 con 0x7f43c80cfc80 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43ccc00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f43c80b5290 tx=0x7f43c000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0003d90 con 0x7f43c80cfc80 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43c000fe50 con 0x7f43c80cfc80 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0004d90 con 0x7f43c80cfc80 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c81f6610 con 0x7f43c80cfc80 2024-09-17T05:30:07.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81f6bd0 con 0x7f43c80cfc80 2024-09-17T05:30:07.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c81f6380 con 0x7f43c80cfc80 2024-09-17T05:30:07.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43c0004030 con 0x7f43c80cfc80 2024-09-17T05:30:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.893+0000 7f43c4c00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 0x7f43a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.897+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f43c0095070 con 0x7f43c80cfc80 2024-09-17T05:30:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.897+0000 7f43c7e00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 0x7f43a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:07.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.897+0000 7f43c7e00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 0x7f43a4075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f43c80715c0 tx=0x7f43b80067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:07.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:07.901+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43c005f940 con 0x7f43c80cfc80 2024-09-17T05:30:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.109+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43c80ad0b0 con 0x7f43c80cfc80 2024-09-17T05:30:08.112 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:08.109+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.109+0000 7f43c4c00640 1 -- 172.21.15.43:0/83216627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) ==== 161+0+0 (secure 0 0 0) 0x7f43c0064370 con 0x7f43c80cfc80 2024-09-17T05:30:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 msgr2=0x7f43a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 0x7f43a4075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f43c80715c0 tx=0x7f43b80067b0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 msgr2=0x7f43c809d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f43c80b5290 tx=0x7f43c000f790 comp rx=0 tx=0).stop 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 shutdown_connections 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a4073200 0x7f43a4075710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c80cfc80 0x7f43c809d540 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c80c9fd0 0x7f43c80cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 --2- 172.21.15.43:0/83216627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac004810 0x7f43c8160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.113+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 >> 172.21.15.43:0/83216627 conn(0x7f43c80b23d0 msgr2=0x7f43c806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.117+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 shutdown_connections 2024-09-17T05:30:08.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.117+0000 7f43cec00640 1 -- 172.21.15.43:0/83216627 wait complete. 2024-09-17T05:30:08.124 INFO:tasks.ceph.mds.c.smithi043.stderr: -661> 2024-09-17T05:30:03.629+0000 7f1a94200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:08.125 INFO:tasks.ceph.mds.c.smithi043.stderr: -138> 2024-09-17T05:30:07.077+0000 7f1a8f200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-17T05:30:08.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:08.247 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:30:08.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- 172.21.15.43:0/2451374332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c0b3920 msgr2=0x7f237c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 --2- 172.21.15.43:0/2451374332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c0b3920 0x7f237c0ad0b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f237c072b60 tx=0x7f237401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- 172.21.15.43:0/2451374332 shutdown_connections 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 --2- 172.21.15.43:0/2451374332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c0b3920 0x7f237c0ad0b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 --2- 172.21.15.43:0/2451374332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c06f110 0x7f237c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 --2- 172.21.15.43:0/2451374332 >> v2:172.21.15.62:3300/0 conn(0x7f237c155d70 0x7f237c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- 172.21.15.43:0/2451374332 >> 172.21.15.43:0/2451374332 conn(0x7f237c0b23d0 msgr2=0x7f237c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- 172.21.15.43:0/2451374332 shutdown_connections 2024-09-17T05:30:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- 172.21.15.43:0/2451374332 wait complete. 2024-09-17T05:30:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 Processor -- start 2024-09-17T05:30:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.301+0000 7f2381600640 1 -- start start 2024-09-17T05:30:08.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f237c0b3920 0x7f237c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c155d70 0x7f237c0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f237c15f830 con 0x7f237c06f110 2024-09-17T05:30:08.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f237c15f6b0 con 0x7f237c0b3920 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f237c15f9b0 con 0x7f237c155d70 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37496/0 (socket says 172.21.15.43:37496) 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 -- 172.21.15.43:0/3473371239 learned_addr learned my addr 172.21.15.43:0/3473371239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2373e00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f237c0b3920 0x7f237c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237b400640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c155d70 0x7f237c0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 -- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c155d70 msgr2=0x7f237c0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c155d70 0x7f237c0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 -- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f237c0b3920 msgr2=0x7f237c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f237c0b3920 0x7f237c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 -- 172.21.15.43:0/3473371239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c0ce1a0 con 0x7f237c06f110 2024-09-17T05:30:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f237aa00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f236c00d750 tx=0x7f236c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c017070 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f236c004110 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0099d0 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c0cb770 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c0752d0 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.305+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c15fe60 con 0x7f237c06f110 2024-09-17T05:30:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.309+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f236c009b70 con 0x7f237c06f110 2024-09-17T05:30:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.309+0000 7f2372a00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 0x7f2358075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.309+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f236c0942d0 con 0x7f237c06f110 2024-09-17T05:30:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.309+0000 7f2373e00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 0x7f2358075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.309+0000 7f2373e00640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 0x7f2358075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f237c0c9ec0 tx=0x7f236400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.313+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f236c05eb50 con 0x7f237c06f110 2024-09-17T05:30:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:08.553+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f237c154a30 con 0x7f237c06f110 2024-09-17T05:30:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.113+0000 7f2372a00640 1 -- 172.21.15.43:0/3473371239 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v97) ==== 107+0+0 (secure 0 0 0) 0x7f236c063580 con 0x7f237c06f110 2024-09-17T05:30:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:30:09.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 msgr2=0x7f2358075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 0x7f2358075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f237c0c9ec0 tx=0x7f236400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 msgr2=0x7f237c074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f236c00d750 tx=0x7f236c00dc10 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 shutdown_connections 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2358073230 0x7f2358075740 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f237c0b3920 0x7f237c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f237c155d70 0x7f237c0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 --2- 172.21.15.43:0/3473371239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f237c06f110 0x7f237c074d90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 >> 172.21.15.43:0/3473371239 conn(0x7f237c0b23d0 msgr2=0x7f237c154f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 shutdown_connections 2024-09-17T05:30:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.117+0000 7f2381600640 1 -- 172.21.15.43:0/3473371239 wait complete. 2024-09-17T05:30:09.144 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 Processor -- start 2024-09-17T05:30:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 -- start start 2024-09-17T05:30:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 0x55a4df3235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df27ccb0 0x55a4df294f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4df19ba00 con 0x55a4df303e40 2024-09-17T05:30:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a4df1f9f40 con 0x55a4df27ccb0 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada6dc4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a4df304b80 con 0x55a4df2504d0 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 0x55a4df3235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 0x55a4df3235e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:51850/0 (socket says 172.21.15.43:51850) 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0000640 1 -- 172.21.15.43:0/3720915078 learned_addr learned my addr 172.21.15.43:0/3720915078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37518/0 (socket says 172.21.15.43:37518) 2024-09-17T05:30:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fad9f600640 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df27ccb0 0x55a4df294f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 -- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 msgr2=0x55a4df3235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 0x55a4df3235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 -- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df27ccb0 msgr2=0x55a4df294f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df27ccb0 0x55a4df294f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.201+0000 7fada0a00640 1 -- 172.21.15.43:0/3720915078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4df174c10 con 0x55a4df303e40 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada0a00640 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fad980ad000 tx=0x7fad9801aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a4409f7a8e0b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fad9e200640 1 -- 172.21.15.43:0/3720915078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980ba430 con 0x55a4df303e40 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fad9e200640 1 -- 172.21.15.43:0/3720915078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad980ba6c0 con 0x55a4df303e40 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fad9e200640 1 -- 172.21.15.43:0/3720915078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980ba8f0 con 0x55a4df303e40 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- 172.21.15.43:0/3720915078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 msgr2=0x55a4df292bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fad980ad000 tx=0x7fad9801aa30 comp rx=0 tx=0).stop 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- 172.21.15.43:0/3720915078 shutdown_connections 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df27ccb0 0x55a4df294f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df303e40 0x55a4df292bc0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- 172.21.15.43:0/3720915078 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df2504d0 0x55a4df3235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- 172.21.15.43:0/3720915078 >> 172.21.15.43:0/3720915078 conn(0x55a4df239d80 msgr2=0x55a4df24b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- 172.21.15.43:0/3720915078 shutdown_connections 2024-09-17T05:30:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- 172.21.15.43:0/3720915078 wait complete. 2024-09-17T05:30:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 Processor -- start 2024-09-17T05:30:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- start start 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df2504d0 0x55a4df31f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df303e40 0x55a4df3bb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a4df320770 con 0x55a4df27ccb0 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a4df296ef0 con 0x55a4df2504d0 2024-09-17T05:30:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.205+0000 7fada6dc4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a4df24c040 con 0x55a4df303e40 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37520/0 (socket says 172.21.15.43:37520) 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 learned_addr learned my addr 172.21.15.43:0/2326390383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df2504d0 0x55a4df31f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fada0a00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df303e40 0x55a4df3bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df303e40 msgr2=0x55a4df3bb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a4df303e40 0x55a4df3bb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df2504d0 msgr2=0x55a4df31f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a4df2504d0 0x55a4df31f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4df3ac660 con 0x55a4df27ccb0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00a770 tx=0x7fad8c00ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9ce00640 1 -- 172.21.15.43:0/2326390383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01b070 con 0x55a4df27ccb0 2024-09-17T05:30:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9ce00640 1 -- 172.21.15.43:0/2326390383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad8c009dc0 con 0x55a4df27ccb0 2024-09-17T05:30:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4df3ac900 con 0x55a4df27ccb0 2024-09-17T05:30:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9ce00640 1 -- 172.21.15.43:0/2326390383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0166a0 con 0x55a4df27ccb0 2024-09-17T05:30:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4df3ace40 con 0x55a4df27ccb0 2024-09-17T05:30:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.209+0000 7fad9ce00640 1 -- 172.21.15.43:0/2326390383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad8c01e070 con 0x55a4df27ccb0 2024-09-17T05:30:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fad9ce00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 0x7fad800740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fad9ce00640 1 -- 172.21.15.43:0/2326390383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7fad8c093a50 con 0x55a4df27ccb0 2024-09-17T05:30:09.216 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:09.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 0x7fad800740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55a4df3ada90 0x55a4df293e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3bb6b0 con 0x55a4df3ada90 2024-09-17T05:30:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada0a00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55a4df3ada90 0x55a4df293e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 0x7fad800740f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c10 tx=0x7fad90001840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.213+0000 7fada0a00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55a4df3ada90 0x55a4df293e90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v91'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df2943d0 con 0x55a4df3ada90 2024-09-17T05:30:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v91'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c6730 con 0x55a4df3ada90 2024-09-17T05:30:09.222 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:09.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v91'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.217+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7ae0 con 0x55a4df3ada90 2024-09-17T05:30:09.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v91'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7e80 con 0x55a4df3ada90 2024-09-17T05:30:09.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v91'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c6ad0 con 0x55a4df3ada90 2024-09-17T05:30:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v91'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.221+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c6ec0 con 0x55a4df3ada90 2024-09-17T05:30:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.225+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v91'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.225+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3c72b0 con 0x55a4df3ada90 2024-09-17T05:30:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.225+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v91'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.225+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c72b0 con 0x55a4df3ada90 2024-09-17T05:30:09.231 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:09.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v91'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v91'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v91'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.229+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v91'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v91'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.237 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:09.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v91'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.233+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v91'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v91'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v91'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.237+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 600.00000000 [create] v91'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 600.00000000 [delete] v91'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.247 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v91'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.241+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.245+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v91'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.245+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.245+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v91'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.245+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v91'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 603.00000000 [create] v91'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 603.00000000 [delete] v91'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.249+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.255 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:09.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v91'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v91'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v91'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.253+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3ada90 2024-09-17T05:30:09.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v91'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55a4df3c7620 0x55a4df3ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:30 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3c7260 con 0x55a4df3c7620 2024-09-17T05:30:09.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55a4df3c7620 0x55a4df3ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55a4df3c7620 0x55a4df3ca740 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(30 1.00000000.inode [create] v91'44 uv23 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.257+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:31 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3cacc0 con 0x55a4df3c7620 2024-09-17T05:30:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.261+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(31 1.00000000.inode [delete] v91'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.261+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cb580 con 0x55a4df3c7620 2024-09-17T05:30:09.266 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:09.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.261+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(32 1.00000000.inode [create] v91'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.261+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cbdd0 con 0x55a4df3c7620 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.265+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v91'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.265+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cc170 con 0x55a4df3c7620 2024-09-17T05:30:09.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.265+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v91'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.265+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cc510 con 0x55a4df3c7620 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(35 200.00000000 [create] v91'49 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3cc8b0 con 0x55a4df3c7620 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(36 200.00000000 [delete] v91'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.274 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:09.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(37 200.00000000 [create] v91'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.269+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v91'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v91'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(40 200.00000001 [create] v91'54 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.273+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.277+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(41 200.00000001 [delete] v91'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.277+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.282 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:09.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.277+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(42 200.00000001 [create] v91'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.277+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0e20 con 0x55a4df3c7620 2024-09-17T05:30:09.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(43 200.00000001 [write 0~16801] v91'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0e20 con 0x55a4df3c7620 2024-09-17T05:30:09.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v91'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(45 606.00000000 [create] v91'59 uv39 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.281+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.285+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(46 606.00000000 [delete] v91'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.285+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.290 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:09.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.285+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(47 606.00000000 [create] v91'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.285+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cdc60 con 0x55a4df3c7620 2024-09-17T05:30:09.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v91'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cdc60 con 0x55a4df3c7620 2024-09-17T05:30:09.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v91'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3ccc50 con 0x55a4df3c7620 2024-09-17T05:30:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v91'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3c7620 2024-09-17T05:30:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a4df3cd2d0 0x55a4df3cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3cfcc0 con 0x55a4df3cd2d0 2024-09-17T05:30:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a4df3cd2d0 0x55a4df3cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.289+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a4df3cd2d0 0x55a4df3cf6f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(51 607.00000000 [create] v91'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d01b0 con 0x55a4df3cd2d0 2024-09-17T05:30:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(52 607.00000000 [delete] v91'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0690 con 0x55a4df3cd2d0 2024-09-17T05:30:09.299 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(53 607.00000000 [create] v91'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.293+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1f40 con 0x55a4df3cd2d0 2024-09-17T05:30:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.297+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v91'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.297+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d22e0 con 0x55a4df3cd2d0 2024-09-17T05:30:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.297+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v91'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.297+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0f30 con 0x55a4df3cd2d0 2024-09-17T05:30:09.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v91'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3ada90 2024-09-17T05:30:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(57 mds0_openfiles.0 [create] v91'9 uv8 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3ada90 2024-09-17T05:30:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v91'10 uv10 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.301+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3ada90 2024-09-17T05:30:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v91'11 uv11 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.308 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3ada90 2024-09-17T05:30:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v91'12 uv12 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 0x55a4df3d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3d2680 2024-09-17T05:30:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.305+0000 7fada0a00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 0x55a4df3d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.309+0000 7fada0a00640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 0x55a4df3d4aa0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.309+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(61 608.00000000 [create] v91'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.309+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3d2680 2024-09-17T05:30:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.309+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(62 608.00000000 [delete] v91'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.309+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d1320 con 0x55a4df3d2680 2024-09-17T05:30:09.315 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(63 608.00000000 [create] v91'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d5ff0 con 0x55a4df3d2680 2024-09-17T05:30:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v91'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d5ff0 con 0x55a4df3d2680 2024-09-17T05:30:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v91'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.313+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d4fe0 con 0x55a4df3d2680 2024-09-17T05:30:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v91'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55a4df3d54f0 0x55a4df3d7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d7fc0 con 0x55a4df3d54f0 2024-09-17T05:30:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55a4df3d54f0 0x55a4df3d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada0000640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55a4df3d54f0 0x55a4df3d7910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(67 604.00000000 [create] v91'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.317+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d8080 con 0x55a4df3d54f0 2024-09-17T05:30:09.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(68 604.00000000 [delete] v91'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.324 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:09.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(69 604.00000000 [create] v91'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d9480 con 0x55a4df3d54f0 2024-09-17T05:30:09.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v91'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d9480 con 0x55a4df3d54f0 2024-09-17T05:30:09.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v91'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.321+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v91'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(73 500.00000000 [create] v91'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(74 500.00000000 [delete] v91'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.325+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(75 500.00000000 [create] v91'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.332 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v91'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d8470 con 0x55a4df3d54f0 2024-09-17T05:30:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada0000640 1 -- 172.21.15.43:0/2326390383 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v91'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad90012860 con 0x55a4df3d54f0 2024-09-17T05:30:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55a4df3d8af0 0x55a4df3daf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.329+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3db550 con 0x55a4df3d8af0 2024-09-17T05:30:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55a4df3d8af0 0x55a4df3daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fad9f600640 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55a4df3d8af0 0x55a4df3daf10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(78 mds_snaptable [create] v91'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3db920 con 0x55a4df3d8af0 2024-09-17T05:30:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(79 mds_snaptable [delete] v91'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.333+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3d8af0 2024-09-17T05:30:09.340 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.337+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(80 mds_snaptable [create] v91'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.337+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3d8af0 2024-09-17T05:30:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.337+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v91'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.337+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3d8af0 2024-09-17T05:30:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v91'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3ada90 2024-09-17T05:30:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(83 605.00000000 [create] v91'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3ada90 2024-09-17T05:30:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(84 605.00000000 [delete] v91'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.341+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3ada90 2024-09-17T05:30:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v91'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.348 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dcd50 con 0x55a4df3ada90 2024-09-17T05:30:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v91'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dcd50 con 0x55a4df3ada90 2024-09-17T05:30:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v91'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.345+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3ada90 2024-09-17T05:30:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.349+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [omap-set-header] v91'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3ada90 2024-09-17T05:30:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.349+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3cd2d0 2024-09-17T05:30:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.349+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v91'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.349+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3cd2d0 2024-09-17T05:30:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v91'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dbd40 con 0x55a4df3cd2d0 2024-09-17T05:30:09.357 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v91'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v91'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.353+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.357+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v91'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.357+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.357+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v91'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.357+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.361+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v91'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.361+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.367 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:09.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.361+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v91'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.361+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3d0b20 con 0x55a4df3cd2d0 2024-09-17T05:30:09.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.365+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v91'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.365+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3deb50 con 0x55a4df3cd2d0 2024-09-17T05:30:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.369+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v91'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3cd2d0 2024-09-17T05:30:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.369+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.373+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v91'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.373+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.373+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v91'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.373+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.381 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.377+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v91'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.377+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.381+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v91'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.381+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.381+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v91'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.381+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v91'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v91'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.389 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:09.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v91'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.385+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v91'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d2680 2024-09-17T05:30:09.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v91'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad980b5040 con 0x55a4df3d2680 2024-09-17T05:30:09.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v91'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.389+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v91'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v91'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.397 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:09.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v91'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.393+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v91'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v91'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v91'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.397+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v91'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.404 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:09.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v91'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v91'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.401+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v91'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v91'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.410 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:09.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v91'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.405+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v91'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v91'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc1f0 con 0x55a4df3d8af0 2024-09-17T05:30:09.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v91'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.409+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55a4df3dc5e0 con 0x55a4df3d8af0 2024-09-17T05:30:09.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fad9f600640 1 -- 172.21.15.43:0/2326390383 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v91'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad8c01d030 con 0x55a4df3d8af0 2024-09-17T05:30:09.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55a4df3c7620 msgr2=0x55a4df3ca740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55a4df3c7620 0x55a4df3ca740 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55a4df3d54f0 msgr2=0x55a4df3d7910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55a4df3d54f0 0x55a4df3d7910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55a4df3d8af0 msgr2=0x55a4df3daf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55a4df3d8af0 0x55a4df3daf10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a4df3cd2d0 msgr2=0x55a4df3cf6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55a4df3cd2d0 0x55a4df3cf6f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 reap_dead start 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55a4df3ada90 msgr2=0x55a4df293e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55a4df3ada90 0x55a4df293e90 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada0a00640 1 -- 172.21.15.43:0/2326390383 reap_dead start 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 msgr2=0x55a4df3d4aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 0x55a4df3d4aa0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 msgr2=0x7fad800740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 0x7fad800740f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c10 tx=0x7fad90001840 comp rx=0 tx=0).stop 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 msgr2=0x55a4df295670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00a770 tx=0x7fad8c00ac30 comp rx=0 tx=0).stop 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 shutdown_connections 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55a4df3d2680 0x55a4df3d4aa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad80071be0 0x7fad800740f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 --2- 172.21.15.43:0/2326390383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4df27ccb0 0x55a4df295670 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 >> 172.21.15.43:0/2326390383 conn(0x55a4df239d80 msgr2=0x55a4df24b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 shutdown_connections 2024-09-17T05:30:09.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.413+0000 7fada6dc4080 1 -- 172.21.15.43:0/2326390383 wait complete. 2024-09-17T05:30:09.422 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-09-17T05:30:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 Processor -- start 2024-09-17T05:30:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 -- start start 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de15cb0 0x55e74deb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9f8b0 0x55e74de2c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e74dd34a00 con 0x55e74de9e010 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e74dd92f40 con 0x55e74de15cb0 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29e6269080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e74dde96b0 con 0x55e74de9f8b0 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37532/0 (socket says 172.21.15.43:37532) 2024-09-17T05:30:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 -- 172.21.15.43:0/618856687 learned_addr learned my addr 172.21.15.43:0/618856687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29df400640 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de15cb0 0x55e74deb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dfe00640 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9f8b0 0x55e74de2c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 -- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9f8b0 msgr2=0x55e74de2c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9f8b0 0x55e74de2c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 -- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de15cb0 msgr2=0x55e74deb9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de15cb0 0x55e74deb9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 -- 172.21.15.43:0/618856687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74dd0dc10 con 0x55e74de9e010 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dea00640 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f29cc001e70 tx=0x7f29cc00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff598e7a23cfb6f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dd600640 1 -- 172.21.15.43:0/618856687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00e840 con 0x55e74de9e010 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dd600640 1 -- 172.21.15.43:0/618856687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29cc00eaf0 con 0x55e74de9e010 2024-09-17T05:30:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.477+0000 7f29dd600640 1 -- 172.21.15.43:0/618856687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00d390 con 0x55e74de9e010 2024-09-17T05:30:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- 172.21.15.43:0/618856687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 msgr2=0x55e74de2c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f29cc001e70 tx=0x7f29cc00b740 comp rx=0 tx=0).stop 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- 172.21.15.43:0/618856687 shutdown_connections 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9f8b0 0x55e74de2c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de9e010 0x55e74de2c210 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- 172.21.15.43:0/618856687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de15cb0 0x55e74deb9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- 172.21.15.43:0/618856687 >> 172.21.15.43:0/618856687 conn(0x55e74ddd2d80 msgr2=0x55e74debc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- 172.21.15.43:0/618856687 shutdown_connections 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- 172.21.15.43:0/618856687 wait complete. 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 Processor -- start 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- start start 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 0x55e74deb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9e010 0x55e74dec4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de9f8b0 0x55e74df599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e74de2f3f0 con 0x55e74de15cb0 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e74deb9790 con 0x55e74de9f8b0 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29e6269080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e74de2a930 con 0x55e74de9e010 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 0x55e74deb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 0x55e74deb4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37536/0 (socket says 172.21.15.43:37536) 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 -- 172.21.15.43:0/623702424 learned_addr learned my addr 172.21.15.43:0/623702424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29dea00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9e010 0x55e74dec4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29dfe00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de9f8b0 0x55e74df599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 -- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9e010 msgr2=0x55e74dec4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e74de9e010 0x55e74dec4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 -- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de9f8b0 msgr2=0x55e74df599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e74de9f8b0 0x55e74df599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 -- 172.21.15.43:0/623702424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74dec24f0 con 0x55e74de15cb0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.481+0000 7f29df400640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 0x55e74deb4fe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f29d000a750 tx=0x7f29d000ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 -- 172.21.15.43:0/623702424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0009b60 con 0x55e74de15cb0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 -- 172.21.15.43:0/623702424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29d0009d00 con 0x55e74de15cb0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 -- 172.21.15.43:0/623702424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0008690 con 0x55e74de15cb0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e74dec2790 con 0x55e74de15cb0 2024-09-17T05:30:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74dec2cd0 con 0x55e74de15cb0 2024-09-17T05:30:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 -- 172.21.15.43:0/623702424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f29d0008830 con 0x55e74de15cb0 2024-09-17T05:30:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f29c0071c10 0x7f29c0074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29d7e00640 1 -- 172.21.15.43:0/623702424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f29d0094060 con 0x55e74de15cb0 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29dea00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f29c0071c10 0x7f29c0074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29e6269080 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e74dec4cd0 0x55e74dde6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.485+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e91) -- 0x55e74dde6db0 con 0x55e74dec4cd0 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29dfe00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e74dec4cd0 0x55e74dde6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29dea00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f29c0071c10 0x7f29c0074120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c010 tx=0x7f29cc00e000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29dfe00640 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e74dec4cd0 0x55e74dde6870 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29dfe00640 1 -- 172.21.15.43:0/623702424 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 600.00000000 [delete] v91'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d8002050 con 0x55e74dec4cd0 2024-09-17T05:30:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e74dec4cd0 msgr2=0x55e74dde6870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e74dec4cd0 0x55e74dde6870 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f29c0071c10 msgr2=0x7f29c0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f29c0071c10 0x7f29c0074120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c010 tx=0x7f29cc00e000 comp rx=0 tx=0).stop 2024-09-17T05:30:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 msgr2=0x55e74deb4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 --2- 172.21.15.43:0/623702424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e74de15cb0 0x55e74deb4fe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f29d000a750 tx=0x7f29d000ac10 comp rx=0 tx=0).stop 2024-09-17T05:30:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29dfe00640 1 -- 172.21.15.43:0/623702424 reap_dead start 2024-09-17T05:30:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 shutdown_connections 2024-09-17T05:30:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 >> 172.21.15.43:0/623702424 conn(0x55e74ddd2d80 msgr2=0x55e74de2c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 shutdown_connections 2024-09-17T05:30:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.489+0000 7f29e6269080 1 -- 172.21.15.43:0/623702424 wait complete. 2024-09-17T05:30:09.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:09.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 -- 172.21.15.43:0/2348746637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc15f470 msgr2=0x7f96cc0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 --2- 172.21.15.43:0/2348746637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc15f470 0x7f96cc0d0390 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009900 tx=0x7f96bc01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 -- 172.21.15.43:0/2348746637 shutdown_connections 2024-09-17T05:30:09.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 --2- 172.21.15.43:0/2348746637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc0b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 --2- 172.21.15.43:0/2348746637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc15f470 0x7f96cc0d0390 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 -- 172.21.15.43:0/2348746637 >> 172.21.15.43:0/2348746637 conn(0x7f96cc0b2810 msgr2=0x7f96cc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 -- 172.21.15.43:0/2348746637 shutdown_connections 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.657+0000 7f96d2200640 1 -- 172.21.15.43:0/2348746637 wait complete. 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 Processor -- start 2024-09-17T05:30:09.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- start start 2024-09-17T05:30:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc1649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96cc15f470 0x7f96cc164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1694d0 con 0x7f96cc0b3b10 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc169350 con 0x7f96cc15f470 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc169650 con 0x7f96cc0b5610 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37564/0 (socket says 172.21.15.43:37564) 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 -- 172.21.15.43:0/248353257 learned_addr learned my addr 172.21.15.43:0/248353257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96c2a00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc1649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cbe00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96cc15f470 0x7f96cc164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 -- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 msgr2=0x7f96cc1649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc1649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 -- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96cc15f470 msgr2=0x7f96cc164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96cc15f470 0x7f96cc164ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 -- 172.21.15.43:0/248353257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc152fa0 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96c2a00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc1649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96cb400640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0b0040 tx=0x7f96bc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0b2040 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96bc004330 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc005280 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1534a0 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.661+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc153a60 con 0x7f96cc0b3b10 2024-09-17T05:30:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc0b5a20 con 0x7f96cc0b3b10 2024-09-17T05:30:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96bc005420 con 0x7f96cc0b3b10 2024-09-17T05:30:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96c3e00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 0x7f96a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f96bc0bd070 con 0x7f96cc0b3b10 2024-09-17T05:30:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96c2a00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 0x7f96a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.665+0000 7f96c2a00640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 0x7f96a8075740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96cc156060 tx=0x7f96b40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.669+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f96bc0164d0 con 0x7f96cc0b3b10 2024-09-17T05:30:09.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:09.881+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f96cc0746b0 con 0x7f96cc0b3b10 2024-09-17T05:30:10.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.121+0000 7f96c3e00640 1 -- 172.21.15.43:0/248353257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v98) ==== 162+0+0 (secure 0 0 0) 0x7f96bc0f9540 con 0x7f96cc0b3b10 2024-09-17T05:30:10.124 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:10.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 msgr2=0x7f96a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 0x7f96a8075740 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96cc156060 tx=0x7f96b40091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 msgr2=0x7f96cc1569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0b0040 tx=0x7f96bc005de0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 shutdown_connections 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f96a8073230 0x7f96a8075740 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96cc15f470 0x7f96cc164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f96cc0b5610 0x7f96cc1649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 --2- 172.21.15.43:0/248353257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96cc0b3b10 0x7f96cc1569a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 >> 172.21.15.43:0/248353257 conn(0x7f96cc0b2810 msgr2=0x7f96cc0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 shutdown_connections 2024-09-17T05:30:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.125+0000 7f96d2200640 1 -- 172.21.15.43:0/248353257 wait complete. 2024-09-17T05:30:10.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- 172.21.15.43:0/1762377571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 msgr2=0x7f9c900b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 --2- 172.21.15.43:0/1762377571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900b5a20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9c90071450 tx=0x7f9c7801ca60 comp rx=0 tx=0).stop 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- 172.21.15.43:0/1762377571 shutdown_connections 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 --2- 172.21.15.43:0/1762377571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900b5a20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 --2- 172.21.15.43:0/1762377571 >> v2:172.21.15.43:3300/0 conn(0x7f9c9015f470 0x7f9c900d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- 172.21.15.43:0/1762377571 >> 172.21.15.43:0/1762377571 conn(0x7f9c900b2810 msgr2=0x7f9c900b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- 172.21.15.43:0/1762377571 shutdown_connections 2024-09-17T05:30:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- 172.21.15.43:0/1762377571 wait complete. 2024-09-17T05:30:10.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 Processor -- start 2024-09-17T05:30:10.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.309+0000 7f9c95000640 1 -- start start 2024-09-17T05:30:10.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c900b3b10 0x7f9c90156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90169420 con 0x7f9c9015f470 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901692a0 con 0x7f9c900b5610 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9c901695a0 con 0x7f9c900b3b10 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37582/0 (socket says 172.21.15.43:37582) 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 -- 172.21.15.43:0/2675918991 learned_addr learned my addr 172.21.15.43:0/2675918991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c85400640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8de00640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c900b3b10 0x7f9c90156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 -- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c900b3b10 msgr2=0x7f9c90156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c900b3b10 0x7f9c90156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 -- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 msgr2=0x7f9c900afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 -- 172.21.15.43:0/2675918991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c901fcaf0 con 0x7f9c9015f470 2024-09-17T05:30:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c8e800640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c730 tx=0x7f9c8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88016020 con 0x7f9c9015f470 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c8800a640 con 0x7f9c9015f470 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800aa90 con 0x7f9c9015f470 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c90157560 con 0x7f9c9015f470 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c901fd090 con 0x7f9c9015f470 2024-09-17T05:30:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.313+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c901572d0 con 0x7f9c9015f470 2024-09-17T05:30:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.317+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c88019020 con 0x7f9c9015f470 2024-09-17T05:30:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.317+0000 7f9c86a00640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 0x7f9c6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.317+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f9c88093fa0 con 0x7f9c9015f470 2024-09-17T05:30:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.317+0000 7f9c8de00640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 0x7f9c6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.317+0000 7f9c8de00640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 0x7f9c6c075710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9c90069610 tx=0x7f9c800091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.321+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c88020020 con 0x7f9c9015f470 2024-09-17T05:30:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.533+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c900b5a20 con 0x7f9c9015f470 2024-09-17T05:30:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.533+0000 7f9c86a00640 1 -- 172.21.15.43:0/2675918991 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5213 (secure 0 0 0) 0x7f9c900b5f60 con 0x7f9c9015f470 2024-09-17T05:30:10.539 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:10.539 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":99,"btime":"2024-09-17T05:30:10:128951+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:10.128938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"c","rank":0,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/1981693955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1981693955},{"type":"v1","addr":"172.21.15.43:6835","nonce":1981693955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 99 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 msgr2=0x7f9c6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 0x7f9c6c075710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9c90069610 tx=0x7f9c800091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 msgr2=0x7f9c9015aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c730 tx=0x7f9c8800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 shutdown_connections 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c6c073200 0x7f9c6c075710 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c900b5610 0x7f9c900afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9015f470 0x7f9c9015aaf0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 --2- 172.21.15.43:0/2675918991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c900b3b10 0x7f9c90156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 >> 172.21.15.43:0/2675918991 conn(0x7f9c900b2810 msgr2=0x7f9c901521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 shutdown_connections 2024-09-17T05:30:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:10.537+0000 7f9c95000640 1 -- 172.21.15.43:0/2675918991 wait complete. 2024-09-17T05:30:10.561 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:30:13.151 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:13.149+0000 7fc349a00640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-17T05:30:13.151 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:13.149+0000 7fc349a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-17T05:30:13.152 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:13.149+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:13.152 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:13.149+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:14.125 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:14.121+0000 7fc34ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:15.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 -- 172.21.15.43:0/4138934809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc0d0910 msgr2=0x7f66bc06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 --2- 172.21.15.43:0/4138934809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc0d0910 0x7f66bc06ce60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009900 tx=0x7f66a801c850 comp rx=0 tx=0).stop 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 -- 172.21.15.43:0/4138934809 shutdown_connections 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 --2- 172.21.15.43:0/4138934809 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0d20 0x7f66bc06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 --2- 172.21.15.43:0/4138934809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc0d0910 0x7f66bc06ce60 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 -- 172.21.15.43:0/4138934809 >> 172.21.15.43:0/4138934809 conn(0x7f66bc0b23d0 msgr2=0x7f66bc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 -- 172.21.15.43:0/4138934809 shutdown_connections 2024-09-17T05:30:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.729+0000 7f66c3c00640 1 -- 172.21.15.43:0/4138934809 wait complete. 2024-09-17T05:30:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 Processor -- start 2024-09-17T05:30:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- start start 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0910 0x7f66bc091880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66bc0d0d20 0x7f66bc1651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0ce280 con 0x7f66bc070620 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0ce100 con 0x7f66bc0d0d20 2024-09-17T05:30:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0ce400 con 0x7f66bc0d0910 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44416/0 (socket says 172.21.15.43:44416) 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 -- 172.21.15.43:0/3456259088 learned_addr learned my addr 172.21.15.43:0/3456259088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66bbe00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0910 0x7f66bc091880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66bc0d0d20 0x7f66bc1651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 -- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0910 msgr2=0x7f66bc091880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0910 0x7f66bc091880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 -- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66bc0d0d20 msgr2=0x7f66bc1651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66bc0d0d20 0x7f66bc1651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 -- 172.21.15.43:0/3456259088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc153ad0 con 0x7f66bc070620 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c1200640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c3f0 tx=0x7f66b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0017070 con 0x7f66bc070620 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66b00053c0 con 0x7f66bc070620 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc153db0 con 0x7f66bc070620 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003860 con 0x7f66bc070620 2024-09-17T05:30:15.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.733+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc154370 con 0x7f66bc070620 2024-09-17T05:30:15.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.737+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0ce990 con 0x7f66bc070620 2024-09-17T05:30:15.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.737+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66b001a070 con 0x7f66bc070620 2024-09-17T05:30:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.737+0000 7f66b8c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 0x7f669c075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.737+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f66b0093590 con 0x7f66bc070620 2024-09-17T05:30:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.737+0000 7f66bbe00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 0x7f669c075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.741+0000 7f66bbe00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 0x7f669c075770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f66a80af000 tx=0x7f66a801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.741+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f66b005de50 con 0x7f66bc070620 2024-09-17T05:30:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.949+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66bc0cf020 con 0x7f66bc070620 2024-09-17T05:30:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.949+0000 7f66b8c00640 1 -- 172.21.15.43:0/3456259088 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+4381 (secure 0 0 0) 0x7f66b0062880 con 0x7f66bc070620 2024-09-17T05:30:15.954 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":103,"btime":"2024-09-17T05:30:13:152806+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:13.152801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:15.955 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 103 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 msgr2=0x7f669c075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 0x7f669c075770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f66a80af000 tx=0x7f66a801b090 comp rx=0 tx=0).stop 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 msgr2=0x7f66bc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c3f0 tx=0x7f66b000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 shutdown_connections 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f669c073260 0x7f669c075770 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66bc0d0d20 0x7f66bc1651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66bc0d0910 0x7f66bc091880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 --2- 172.21.15.43:0/3456259088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66bc070620 0x7f66bc154b70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 >> 172.21.15.43:0/3456259088 conn(0x7f66bc0b23d0 msgr2=0x7f66bc0cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 shutdown_connections 2024-09-17T05:30:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:15.953+0000 7f66c3c00640 1 -- 172.21.15.43:0/3456259088 wait complete. 2024-09-17T05:30:15.977 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:30:15.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 -- 172.21.15.43:0/3876845823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 msgr2=0x7f13c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3876845823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13c400b9d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f13e00c7260 tx=0x7f13d801d4b0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e4e00640 1 -- 172.21.15.43:0/3876845823 reap_dead start 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 -- 172.21.15.43:0/3876845823 shutdown_connections 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 -- 172.21.15.43:0/3876845823 >> 172.21.15.43:0/3876845823 conn(0x7f13e00b23d0 msgr2=0x7f13e00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 -- 172.21.15.43:0/3876845823 shutdown_connections 2024-09-17T05:30:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.133+0000 7f13e6e00640 1 -- 172.21.15.43:0/3876845823 wait complete. 2024-09-17T05:30:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 Processor -- start 2024-09-17T05:30:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 -- start start 2024-09-17T05:30:16.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00c3ef0 0x7f13e00cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01f5120 0x7f13e01f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00ad230 con 0x7f13c40051b0 2024-09-17T05:30:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00ad0b0 con 0x7f13e01f5120 2024-09-17T05:30:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e6e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00ad3b0 con 0x7f13e00c3ef0 2024-09-17T05:30:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44436/0 (socket says 172.21.15.43:44436) 2024-09-17T05:30:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 -- 172.21.15.43:0/3505403141 learned_addr learned my addr 172.21.15.43:0/3505403141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13d7e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00c3ef0 0x7f13e00cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13e4e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01f5120 0x7f13e01f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 -- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00c3ef0 msgr2=0x7f13e00cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00c3ef0 0x7f13e00cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 -- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01f5120 msgr2=0x7f13e01f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01f5120 0x7f13e01f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 -- 172.21.15.43:0/3505403141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e01f9620 con 0x7f13c40051b0 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.137+0000 7f13dfe00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011ae0 tx=0x7f13d0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d001e070 con 0x7f13c40051b0 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d000fcd0 con 0x7f13c40051b0 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d000fea0 con 0x7f13c40051b0 2024-09-17T05:30:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e01f9900 con 0x7f13c40051b0 2024-09-17T05:30:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01f5af0 con 0x7f13c40051b0 2024-09-17T05:30:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e006f7d0 con 0x7f13c40051b0 2024-09-17T05:30:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13d0021070 con 0x7f13c40051b0 2024-09-17T05:30:16.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 0x7f13bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.141+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f13d0094c90 con 0x7f13c40051b0 2024-09-17T05:30:16.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.145+0000 7f13d7e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 0x7f13bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.145+0000 7f13d7e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 0x7f13bc075740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f13e0075540 tx=0x7f13c80067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.145+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13d005f4c0 con 0x7f13c40051b0 2024-09-17T05:30:16.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.357+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e00af8d0 con 0x7f13c40051b0 2024-09-17T05:30:16.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.357+0000 7f13dcc00640 1 -- 172.21.15.43:0/3505403141 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+4381 (secure 0 0 0) 0x7f13d0063ef0 con 0x7f13c40051b0 2024-09-17T05:30:16.363 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:16.363 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":103,"btime":"2024-09-17T05:30:13:152806+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:13.152801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 103 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 msgr2=0x7f13bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 0x7f13bc075740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f13e0075540 tx=0x7f13c80067b0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 msgr2=0x7f13e01f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011ae0 tx=0x7f13d0009ee0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 shutdown_connections 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13bc073230 0x7f13bc075740 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01f5120 0x7f13e01f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13e00c3ef0 0x7f13e00cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 --2- 172.21.15.43:0/3505403141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c40051b0 0x7f13e01f8e60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 >> 172.21.15.43:0/3505403141 conn(0x7f13e00b23d0 msgr2=0x7f13e00aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 shutdown_connections 2024-09-17T05:30:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.361+0000 7f13e6e00640 1 -- 172.21.15.43:0/3505403141 wait complete. 2024-09-17T05:30:16.385 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:16.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- 172.21.15.43:0/2431837599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92200730b0 msgr2=0x7f92200734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 --2- 172.21.15.43:0/2431837599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92200730b0 0x7f92200734c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9218009bc0 tx=0x7f921801ca80 comp rx=0 tx=0).stop 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- 172.21.15.43:0/2431837599 shutdown_connections 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 --2- 172.21.15.43:0/2431837599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92200730b0 0x7f92200734c0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 --2- 172.21.15.43:0/2431837599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92200700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- 172.21.15.43:0/2431837599 >> 172.21.15.43:0/2431837599 conn(0x7f92200b23d0 msgr2=0x7f92200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- 172.21.15.43:0/2431837599 shutdown_connections 2024-09-17T05:30:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- 172.21.15.43:0/2431837599 wait complete. 2024-09-17T05:30:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 Processor -- start 2024-09-17T05:30:16.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.541+0000 7f9227400640 1 -- start start 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92201f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 0x7f922015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9220169f20 con 0x7f922015fe10 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9220169da0 con 0x7f92200730b0 2024-09-17T05:30:16.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f922016a0a0 con 0x7f922006fc90 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44448/0 (socket says 172.21.15.43:44448) 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 -- 172.21.15.43:0/2543288896 learned_addr learned my addr 172.21.15.43:0/2543288896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9224a00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92201f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f921fe00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 0x7f922015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 -- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 msgr2=0x7f92201f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92201f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 -- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 msgr2=0x7f922015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 0x7f922015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 -- 172.21.15.43:0/2543288896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92201fcb30 con 0x7f922015fe10 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9224a00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92201f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:16.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f921fe00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 0x7f922015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9225400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f92180b0040 tx=0x7f9218005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92180b2040 con 0x7f922015fe10 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9218004040 con 0x7f922015fe10 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201fce10 con 0x7f922015fe10 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218004470 con 0x7f922015fe10 2024-09-17T05:30:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201fa490 con 0x7f922015fe10 2024-09-17T05:30:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.545+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92201fa6f0 con 0x7f922015fe10 2024-09-17T05:30:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.549+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92180bd030 con 0x7f922015fe10 2024-09-17T05:30:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.549+0000 7f921cc00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 0x7f9200075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.549+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f921812ec60 con 0x7f922015fe10 2024-09-17T05:30:16.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.549+0000 7f9224a00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 0x7f9200075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.549+0000 7f9224a00640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 0x7f9200075710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9214000f70 tx=0x7f9214009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.553+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92180c4060 con 0x7f922015fe10 2024-09-17T05:30:16.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92200700a0 con 0x7f922015fe10 2024-09-17T05:30:16.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f921cc00640 1 -- 172.21.15.43:0/2543288896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+4381 (secure 0 0 0) 0x7f9218016790 con 0x7f922015fe10 2024-09-17T05:30:16.765 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:16.765 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":103,"btime":"2024-09-17T05:30:13:152806+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:13.152801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:16.766 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 103 2024-09-17T05:30:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 msgr2=0x7f9200075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 0x7f9200075710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9214000f70 tx=0x7f9214009040 comp rx=0 tx=0).stop 2024-09-17T05:30:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 msgr2=0x7f9220158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f92180b0040 tx=0x7f9218005a60 comp rx=0 tx=0).stop 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 shutdown_connections 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9200073200 0x7f9200075710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200730b0 0x7f922015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922015fe10 0x7f9220158050 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 --2- 172.21.15.43:0/2543288896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f922006fc90 0x7f92201f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.761+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 >> 172.21.15.43:0/2543288896 conn(0x7f92200b23d0 msgr2=0x7f92200b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.765+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 shutdown_connections 2024-09-17T05:30:16.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:16.765+0000 7f9227400640 1 -- 172.21.15.43:0/2543288896 wait complete. 2024-09-17T05:30:16.788 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-09-17T05:30:16.788 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-09-17T05:30:16.788 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:16.788 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:16.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:16.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:16.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:16.798 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:16.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:16.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 -- 172.21.15.43:0/3747980548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 msgr2=0x7f0a440700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/3747980548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a440700a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009900 tx=0x7f0a2c01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 -- 172.21.15.43:0/3747980548 shutdown_connections 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/3747980548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a440730b0 0x7f0a440734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/3747980548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a440700a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 -- 172.21.15.43:0/3747980548 >> 172.21.15.43:0/3747980548 conn(0x7f0a440b23d0 msgr2=0x7f0a440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 -- 172.21.15.43:0/3747980548 shutdown_connections 2024-09-17T05:30:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.049+0000 7f0a4ae00640 1 -- 172.21.15.43:0/3747980548 wait complete. 2024-09-17T05:30:17.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 Processor -- start 2024-09-17T05:30:17.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- start start 2024-09-17T05:30:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440730b0 0x7f0a440c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a4415fe10 0x7f0a44097450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44169fc0 con 0x7f0a4406fc90 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44169e40 con 0x7f0a440730b0 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4416a140 con 0x7f0a4415fe10 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44468/0 (socket says 172.21.15.43:44468) 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 -- 172.21.15.43:0/516681685 learned_addr learned my addr 172.21.15.43:0/516681685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a48e00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a4415fe10 0x7f0a44097450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a3b400640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440730b0 0x7f0a440c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 -- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a4415fe10 msgr2=0x7f0a44097450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a4415fe10 0x7f0a44097450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 -- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440730b0 msgr2=0x7f0a440c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440730b0 0x7f0a440c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 -- 172.21.15.43:0/516681685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4415e000 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a43e00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c730 tx=0x7f0a3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003b80 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a34003d40 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3400aa50 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4415e2e0 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.053+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4415e8a0 con 0x7f0a4406fc90 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.057+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a440700a0 con 0x7f0a4406fc90 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.057+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a3400abf0 con 0x7f0a4406fc90 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.057+0000 7f0a40c00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 0x7f0a20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.057+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f0a34093f60 con 0x7f0a4406fc90 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.057+0000 7f0a3b400640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 0x7f0a20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.061+0000 7f0a3b400640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 0x7f0a20075740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0af000 tx=0x7f0a2c01b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.061+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0a44073a00 con 0x7f0a4406fc90 2024-09-17T05:30:17.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:17.277+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0a44075060 con 0x7f0a4406fc90 2024-09-17T05:30:17.908 INFO:tasks.ceph.mds.c.smithi043.stderr: -53> 2024-09-17T05:30:13.149+0000 7fc349a00640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-17T05:30:17.908 INFO:tasks.ceph.mds.c.smithi043.stderr: -51> 2024-09-17T05:30:13.149+0000 7fc349a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-17T05:30:17.908 INFO:tasks.ceph.mds.c.smithi043.stderr: -34> 2024-09-17T05:30:14.121+0000 7fc34ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:18.033 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:30:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.169+0000 7f0a40c00640 1 -- 172.21.15.43:0/516681685 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) ==== 161+0+0 (secure 0 0 0) 0x7f0a3405e820 con 0x7f0a4406fc90 2024-09-17T05:30:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 msgr2=0x7f0a20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 0x7f0a20075740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0af000 tx=0x7f0a2c01b000 comp rx=0 tx=0).stop 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 msgr2=0x7f0a4415f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c730 tx=0x7f0a3400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 shutdown_connections 2024-09-17T05:30:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a20073230 0x7f0a20075740 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440730b0 0x7f0a440c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a4415fe10 0x7f0a44097450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 --2- 172.21.15.43:0/516681685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4406fc90 0x7f0a4415f0a0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 >> 172.21.15.43:0/516681685 conn(0x7f0a440b23d0 msgr2=0x7f0a440b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 shutdown_connections 2024-09-17T05:30:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.173+0000 7f0a4ae00640 1 -- 172.21.15.43:0/516681685 wait complete. 2024-09-17T05:30:18.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- 172.21.15.43:0/995818351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 msgr2=0x7f517406d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 --2- 172.21.15.43:0/995818351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f517406d3a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f516c009c80 tx=0x7f516c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- 172.21.15.43:0/995818351 shutdown_connections 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 --2- 172.21.15.43:0/995818351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f517406d3a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 --2- 172.21.15.43:0/995818351 >> v2:172.21.15.62:3300/0 conn(0x7f517415adc0 0x7f517406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- 172.21.15.43:0/995818351 >> 172.21.15.43:0/995818351 conn(0x7f51740b23d0 msgr2=0x7f51740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:18.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- 172.21.15.43:0/995818351 shutdown_connections 2024-09-17T05:30:18.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- 172.21.15.43:0/995818351 wait complete. 2024-09-17T05:30:18.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 Processor -- start 2024-09-17T05:30:18.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.357+0000 7f517b400640 1 -- start start 2024-09-17T05:30:18.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f5174156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:18.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 0x7f51741fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:18.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:18.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5174166240 con 0x7f517415adc0 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51741660c0 con 0x7f51740cc6d0 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f51741663c0 con 0x7f51740cc160 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44486/0 (socket says 172.21.15.43:44486) 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 -- 172.21.15.43:0/3940854329 learned_addr learned my addr 172.21.15.43:0/3940854329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5178a00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f5174156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5173e00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 0x7f51741fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 -- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 msgr2=0x7f5174156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f5174156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 -- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 msgr2=0x7f51741fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 0x7f51741fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 -- 172.21.15.43:0/3940854329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5174158a00 con 0x7f517415adc0 2024-09-17T05:30:18.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5173e00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 0x7f51741fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:18.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5179400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f516c01b450 tx=0x7f516c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:18.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0bb070 con 0x7f517415adc0 2024-09-17T05:30:18.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f516c004b40 con 0x7f517415adc0 2024-09-17T05:30:18.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5174158440 con 0x7f517415adc0 2024-09-17T05:30:18.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0b2040 con 0x7f517415adc0 2024-09-17T05:30:18.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5174158150 con 0x7f517415adc0 2024-09-17T05:30:18.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.361+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51741f9340 con 0x7f517415adc0 2024-09-17T05:30:18.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.365+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f516c0be070 con 0x7f517415adc0 2024-09-17T05:30:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.365+0000 7f5170c00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 0x7f5154075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.365+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f516c12de80 con 0x7f517415adc0 2024-09-17T05:30:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.365+0000 7f5178a00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 0x7f5154075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.365+0000 7f5178a00640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 0x7f5154075740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5168009600 tx=0x7f51680091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:18.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.369+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f516c016990 con 0x7f517415adc0 2024-09-17T05:30:18.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:18.609+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5174156af0 con 0x7f517415adc0 2024-09-17T05:30:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.177+0000 7f5170c00640 1 -- 172.21.15.43:0/3940854329 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v105) ==== 101+0+0 (secure 0 0 0) 0x7f516c0f86b0 con 0x7f517415adc0 2024-09-17T05:30:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 msgr2=0x7f5154075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 0x7f5154075740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5168009600 tx=0x7f51680091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 msgr2=0x7f51741f9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f516c01b450 tx=0x7f516c005b00 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 shutdown_connections 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5154073230 0x7f5154075740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51740cc6d0 0x7f51741fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f517415adc0 0x7f51741f9a10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 --2- 172.21.15.43:0/3940854329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f51740cc160 0x7f5174156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 >> 172.21.15.43:0/3940854329 conn(0x7f51740b23d0 msgr2=0x7f5174159160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 shutdown_connections 2024-09-17T05:30:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.181+0000 7f517b400640 1 -- 172.21.15.43:0/3940854329 wait complete. 2024-09-17T05:30:19.207 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.261+0000 7f2c44848080 1 Processor -- start 2024-09-17T05:30:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.261+0000 7f2c44848080 1 -- start start 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 0x55cb489139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 0x55cb48913f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb487afa00 con 0x55cb48861360 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cb4880df40 con 0x55cb4891abe0 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cb4891cd10 con 0x55cb488ab9e0 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44492/0 (socket says 172.21.15.43:44492) 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 -- 172.21.15.43:0/1693649902 learned_addr learned my addr 172.21.15.43:0/1693649902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3d800640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 0x55cb48913f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c37e00640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 0x55cb489139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 -- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 msgr2=0x55cb489139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 0x55cb489139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 -- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 msgr2=0x55cb48913f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 0x55cb48913f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 -- 172.21.15.43:0/1693649902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb48788c10 con 0x55cb48861360 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3d800640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 0x55cb48913f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c37e00640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 0x55cb489139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c30018f10 tx=0x7f2c30018080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42fd65c1088d734 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c36a00640 1 -- 172.21.15.43:0/1693649902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300ba070 con 0x55cb48861360 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c36a00640 1 -- 172.21.15.43:0/1693649902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c300ab980 con 0x55cb48861360 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c36a00640 1 -- 172.21.15.43:0/1693649902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300abd90 con 0x55cb48861360 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- 172.21.15.43:0/1693649902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 msgr2=0x55cb488627f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c30018f10 tx=0x7f2c30018080 comp rx=0 tx=0).stop 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- 172.21.15.43:0/1693649902 shutdown_connections 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb4891abe0 0x55cb48913f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb488ab9e0 0x55cb489139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 --2- 172.21.15.43:0/1693649902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb488627f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- 172.21.15.43:0/1693649902 >> 172.21.15.43:0/1693649902 conn(0x55cb4884dd80 msgr2=0x55cb488ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- 172.21.15.43:0/1693649902 shutdown_connections 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.265+0000 7f2c44848080 1 -- 172.21.15.43:0/1693649902 wait complete. 2024-09-17T05:30:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 Processor -- start 2024-09-17T05:30:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- start start 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb4890e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb488ab9e0 0x55cb48917cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb4891abe0 0x55cb48918230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb48917610 con 0x55cb48861360 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cb489153e0 con 0x55cb488ab9e0 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cb48939880 con 0x55cb4891abe0 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb4890e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb4890e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44506/0 (socket says 172.21.15.43:44506) 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 learned_addr learned my addr 172.21.15.43:0/757341297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb4891abe0 msgr2=0x55cb48918230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cb4891abe0 0x55cb48918230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb488ab9e0 msgr2=0x55cb48917cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb488ab9e0 0x55cb48917cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb48939b20 con 0x55cb48861360 2024-09-17T05:30:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb4890e7f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2c300b8040 tx=0x7f2c300acc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c35600640 1 -- 172.21.15.43:0/757341297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300ba070 con 0x55cb48861360 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c35600640 1 -- 172.21.15.43:0/757341297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c300b5550 con 0x55cb48861360 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c35600640 1 -- 172.21.15.43:0/757341297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300b59a0 con 0x55cb48861360 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb48939dc0 con 0x55cb48861360 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.269+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4893a300 con 0x55cb48861360 2024-09-17T05:30:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c35600640 1 -- 172.21.15.43:0/757341297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c300c5060 con 0x55cb48861360 2024-09-17T05:30:19.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c35600640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c1c071be0 0x7f2c1c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c35600640 1 -- 172.21.15.43:0/757341297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f2c3012e570 con 0x55cb48861360 2024-09-17T05:30:19.278 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c1c071be0 0x7f2c1c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cb48939370 0x55cb48911930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb48911e70 con 0x55cb48939370 2024-09-17T05:30:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c3d800640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cb48939370 0x55cb48911930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c1c071be0 0x7f2c1c0740f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c28002e70 tx=0x7f2c280023f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.273+0000 7f2c3d800640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cb48939370 0x55cb48911930 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.277+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v92'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.277+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb48912250 con 0x55cb48939370 2024-09-17T05:30:19.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.277+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v92'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.277+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489dd420 con 0x55cb48939370 2024-09-17T05:30:19.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.281+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v92'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.284 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:19.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.281+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489de7d0 con 0x55cb48939370 2024-09-17T05:30:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.281+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v92'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.281+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489deb70 con 0x55cb48939370 2024-09-17T05:30:19.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v92'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489dd7c0 con 0x55cb48939370 2024-09-17T05:30:19.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v92'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddbb0 con 0x55cb48939370 2024-09-17T05:30:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v92'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.285+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489ddfa0 con 0x55cb48939370 2024-09-17T05:30:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v92'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddfa0 con 0x55cb48939370 2024-09-17T05:30:19.293 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v92'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v92'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.289+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.293+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v92'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.293+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.293+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v92'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.293+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v92'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v92'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.302 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v92'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.297+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v92'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v92'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v92'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.301+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v92'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.310 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:19.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v92'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v92'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.305+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.309+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v92'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.309+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.309+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v92'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.309+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.313+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 603.00000000 [create] v92'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.313+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.313+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 603.00000000 [delete] v92'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.313+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.319 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:19.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v92'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v92'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v92'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.317+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb48939370 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v92'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cb489de350 0x55cb489e1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:30 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ddf50 con 0x55cb489de350 2024-09-17T05:30:19.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cb489de350 0x55cb489e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cb489de350 0x55cb489e1430 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(30 1.00000000.inode [create] v92'66 uv48 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.321+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:31 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e1970 con 0x55cb489de350 2024-09-17T05:30:19.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(31 1.00000000.inode [delete] v92'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e1d10 con 0x55cb489de350 2024-09-17T05:30:19.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(32 1.00000000.inode [create] v92'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.329 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:19.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2560 con 0x55cb489de350 2024-09-17T05:30:19.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v92'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.325+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2900 con 0x55cb489de350 2024-09-17T05:30:19.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v92'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2ca0 con 0x55cb489de350 2024-09-17T05:30:19.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(35 200.00000000 [create] v92'71 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e3040 con 0x55cb489de350 2024-09-17T05:30:19.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(36 200.00000000 [delete] v92'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.329+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e33e0 con 0x55cb489de350 2024-09-17T05:30:19.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.333+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(37 200.00000000 [create] v92'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.337 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:19.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.333+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.333+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v92'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.333+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v92'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(40 200.00000001 [create] v92'76 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(41 200.00000001 [delete] v92'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.337+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.343 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(42 200.00000001 [create] v92'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(43 200.00000001 [write 0~16801] v92'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v92'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.341+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(45 606.00000000 [create] v92'81 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(46 606.00000000 [delete] v92'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.350 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(47 606.00000000 [create] v92'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.345+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v92'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v92'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e2190 con 0x55cb489de350 2024-09-17T05:30:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v92'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489de350 2024-09-17T05:30:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cb489e3da0 0x55cb489e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e68f0 con 0x55cb489e3da0 2024-09-17T05:30:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.349+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cb489e3da0 0x55cb489e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.353+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cb489e3da0 0x55cb489e41b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.353+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(51 607.00000000 [create] v92'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.353+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e6c90 con 0x55cb489e3da0 2024-09-17T05:30:19.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.353+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(52 607.00000000 [delete] v92'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.353+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e7030 con 0x55cb489e3da0 2024-09-17T05:30:19.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(53 607.00000000 [create] v92'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.360 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:19.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e8910 con 0x55cb489e3da0 2024-09-17T05:30:19.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v92'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e8cb0 con 0x55cb489e3da0 2024-09-17T05:30:19.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v92'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.357+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e7900 con 0x55cb489e3da0 2024-09-17T05:30:19.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.361+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v92'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.361+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e7cf0 con 0x55cb48939370 2024-09-17T05:30:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.361+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(57 mds0_openfiles.0 [create] v92'13 uv12 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.361+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489e7cf0 con 0x55cb48939370 2024-09-17T05:30:19.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.365+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v92'14 uv14 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.365+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e7cf0 con 0x55cb48939370 2024-09-17T05:30:19.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.365+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v92'15 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.370 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:19.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.365+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489e7cf0 con 0x55cb48939370 2024-09-17T05:30:19.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v92'16 uv16 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cb489e9050 0x55cb489eb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eb9b0 con 0x55cb489e9050 2024-09-17T05:30:19.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c3d800640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cb489e9050 0x55cb489eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c3d800640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cb489e9050 0x55cb489eb470 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(61 608.00000000 [create] v92'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.369+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489eb9b0 con 0x55cb489e9050 2024-09-17T05:30:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(62 608.00000000 [delete] v92'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eb9b0 con 0x55cb489e9050 2024-09-17T05:30:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(63 608.00000000 [create] v92'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.377 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:19.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ec9c0 con 0x55cb489e9050 2024-09-17T05:30:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v92'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.373+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489ec9c0 con 0x55cb489e9050 2024-09-17T05:30:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v92'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eb9b0 con 0x55cb489e9050 2024-09-17T05:30:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v92'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cb489ec030 0x55cb489ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eea20 con 0x55cb489ec030 2024-09-17T05:30:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cb489ec030 0x55cb489ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c3ce00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cb489ec030 0x55cb489ee450 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(67 604.00000000 [create] v92'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.377+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489eeae0 con 0x55cb489ec030 2024-09-17T05:30:19.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(68 604.00000000 [delete] v92'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(69 604.00000000 [create] v92'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.386 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:19.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eff10 con 0x55cb489ec030 2024-09-17T05:30:19.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v92'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.381+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eff10 con 0x55cb489ec030 2024-09-17T05:30:19.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v92'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v92'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(73 500.00000000 [create] v92'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.385+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(74 500.00000000 [delete] v92'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(75 500.00000000 [create] v92'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.393 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:19.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v92'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.389+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489eef00 con 0x55cb489ec030 2024-09-17T05:30:19.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c3ce00640 1 -- 172.21.15.43:0/757341297 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v92'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c300acd70 con 0x55cb489ec030 2024-09-17T05:30:19.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cb489ef580 0x55cb489f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f1fb0 con 0x55cb489ef580 2024-09-17T05:30:19.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cb489ef580 0x55cb489f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c37e00640 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cb489ef580 0x55cb489f19a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(78 mds_snaptable [create] v92'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.393+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2380 con 0x55cb489ef580 2024-09-17T05:30:19.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(79 mds_snaptable [delete] v92'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489ef580 2024-09-17T05:30:19.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(80 mds_snaptable [create] v92'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.401 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489ef580 2024-09-17T05:30:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v92'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.397+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489ef580 2024-09-17T05:30:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v92'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb48939370 2024-09-17T05:30:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(83 605.00000000 [create] v92'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb48939370 2024-09-17T05:30:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(84 605.00000000 [delete] v92'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.401+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb48939370 2024-09-17T05:30:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v92'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.408 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f37b0 con 0x55cb48939370 2024-09-17T05:30:19.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v92'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f37b0 con 0x55cb48939370 2024-09-17T05:30:19.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v92'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.405+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb48939370 2024-09-17T05:30:19.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.409+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [omap-set-header] v92'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb48939370 2024-09-17T05:30:19.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.409+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.409+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v92'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.409+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.409+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v92'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.413+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.416 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.413+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v92'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.413+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.413+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v92'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.413+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.417+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v92'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.417+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.417+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v92'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.417+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.417+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v92'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.421+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.421+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v92'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.425 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.421+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f27a0 con 0x55cb489e3da0 2024-09-17T05:30:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.421+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v92'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.421+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f55b0 con 0x55cb489e3da0 2024-09-17T05:30:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.425+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v92'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489e3da0 2024-09-17T05:30:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.425+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.425+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v92'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.425+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.425+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v92'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.429+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.432 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:19.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.429+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v92'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.429+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.429+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v92'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.429+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v92'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v92'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.439 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:19.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v92'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.433+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f3c60 con 0x55cb489e9050 2024-09-17T05:30:19.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.437+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v92'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.437+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f3c60 con 0x55cb489e9050 2024-09-17T05:30:19.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.437+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v92'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.437+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489e9050 2024-09-17T05:30:19.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.441+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v92'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c38002040 con 0x55cb489e9050 2024-09-17T05:30:19.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.441+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.441+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v92'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.441+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.445+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v92'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.445+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.445+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v92'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.449 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:19.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.445+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.445+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v92'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.449+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.449+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v92'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.449+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.449+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v92'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.449+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v92'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.456 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:19.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v92'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v92'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v92'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.453+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v92'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v92'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.462 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:19.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v92'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.457+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f3c60 con 0x55cb489ef580 2024-09-17T05:30:19.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v92'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f3c60 con 0x55cb489ef580 2024-09-17T05:30:19.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v92'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2c50 con 0x55cb489ef580 2024-09-17T05:30:19.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v92'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.461+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55cb489f2ec0 con 0x55cb489ef580 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c37e00640 1 -- 172.21.15.43:0/757341297 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v92'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2c2800d040 con 0x55cb489ef580 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cb489de350 msgr2=0x55cb489e1430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cb489de350 0x55cb489e1430 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cb489ec030 msgr2=0x55cb489ee450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cb489ec030 0x55cb489ee450 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cb489ef580 msgr2=0x55cb489f19a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cb489ef580 0x55cb489f19a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cb489e3da0 msgr2=0x55cb489e41b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 reap_dead start 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cb489e3da0 0x55cb489e41b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cb48939370 msgr2=0x55cb48911930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cb48939370 0x55cb48911930 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cb489e9050 msgr2=0x55cb489eb470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cb489e9050 0x55cb489eb470 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c1c071be0 msgr2=0x7f2c1c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2c1c071be0 0x7f2c1c0740f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c28002e70 tx=0x7f2c280023f0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 msgr2=0x55cb4890e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 --2- 172.21.15.43:0/757341297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb48861360 0x55cb4890e7f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2c300b8040 tx=0x7f2c300acc00 comp rx=0 tx=0).stop 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c3d800640 1 -- 172.21.15.43:0/757341297 reap_dead start 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 shutdown_connections 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 >> 172.21.15.43:0/757341297 conn(0x55cb4884dd80 msgr2=0x55cb488ab0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 shutdown_connections 2024-09-17T05:30:19.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.465+0000 7f2c44848080 1 -- 172.21.15.43:0/757341297 wait complete. 2024-09-17T05:30:19.473 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-09-17T05:30:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 Processor -- start 2024-09-17T05:30:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- start start 2024-09-17T05:30:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052aeee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052af141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56052ad86a00 con 0x56052aeef9c0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56052ade4f40 con 0x56052aeef5b0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56052ae7e8c0 con 0x56052ae3d9c0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44508/0 (socket says 172.21.15.43:44508) 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 -- 172.21.15.43:0/271207986 learned_addr learned my addr 172.21.15.43:0/271207986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220200640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052aeee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f021f800640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052af141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 -- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 msgr2=0x56052aeee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052aeee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 -- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 msgr2=0x56052af141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052af141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 -- 172.21.15.43:0/271207986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052ad5fc10 con 0x56052aeef9c0 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220200640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052aeee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f021f800640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052af141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f0220c00640 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f021800a920 tx=0x7f021800ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44a561b47900ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f021e400640 1 -- 172.21.15.43:0/271207986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021800dbf0 con 0x56052aeef9c0 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f021e400640 1 -- 172.21.15.43:0/271207986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f021800dd90 con 0x56052aeef9c0 2024-09-17T05:30:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f021e400640 1 -- 172.21.15.43:0/271207986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021800f690 con 0x56052aeef9c0 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- 172.21.15.43:0/271207986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 msgr2=0x56052ae79f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f021800a920 tx=0x7f021800ade0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- 172.21.15.43:0/271207986 shutdown_connections 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052af141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052ae79f50 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 --2- 172.21.15.43:0/271207986 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052aeee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.529+0000 7f02270a4080 1 -- 172.21.15.43:0/271207986 >> 172.21.15.43:0/271207986 conn(0x56052ae24d80 msgr2=0x56052af0eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- 172.21.15.43:0/271207986 shutdown_connections 2024-09-17T05:30:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- 172.21.15.43:0/271207986 wait complete. 2024-09-17T05:30:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 Processor -- start 2024-09-17T05:30:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- start start 2024-09-17T05:30:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052af0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052afa8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052aeea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56052ae82c30 con 0x56052aeef9c0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56052ae82990 con 0x56052aeef5b0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f02270a4080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56052ae812e0 con 0x56052ae3d9c0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052af0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052aeea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052af0ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48944/0 (socket says 172.21.15.43:48944) 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 -- 172.21.15.43:0/187567332 learned_addr learned my addr 172.21.15.43:0/187567332 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f021f800640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052afa8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 -- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 msgr2=0x56052afa8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052aeef5b0 0x56052afa8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 -- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 msgr2=0x56052aeea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052aeea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 -- 172.21.15.43:0/187567332 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052afa9600 con 0x56052ae3d9c0 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220c00640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56052aeef9c0 0x56052aeea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f0220200640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052af0ba80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f021000b390 tx=0x7f021000df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.533+0000 7f021d000640 1 -- 172.21.15.43:0/187567332 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210013070 con 0x56052ae3d9c0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021d000640 1 -- 172.21.15.43:0/187567332 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f021000f8e0 con 0x56052ae3d9c0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021d000640 1 -- 172.21.15.43:0/187567332 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000fcf0 con 0x56052ae3d9c0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x56052afa98a0 con 0x56052ae3d9c0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56052afa9de0 con 0x56052ae3d9c0 2024-09-17T05:30:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021d000640 1 -- 172.21.15.43:0/187567332 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0210018460 con 0x56052ae3d9c0 2024-09-17T05:30:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021d000640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0200071c10 0x7f0200074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021d000640 1 -- 172.21.15.43:0/187567332 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f0210093810 con 0x56052ae3d9c0 2024-09-17T05:30:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.537+0000 7f021f800640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0200071c10 0x7f0200074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56052af168a0 0x56052afaa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e92) -- 0x56052afaaa60 con 0x56052af168a0 2024-09-17T05:30:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f0220c00640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56052af168a0 0x56052afaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f021f800640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0200071c10 0x7f0200074120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f020c004f50 tx=0x7f020c0ae040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f0220c00640 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56052af168a0 0x56052afaa520 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f0220c00640 1 -- 172.21.15.43:0/187567332 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [delete] v92'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f021801f040 con 0x56052af168a0 2024-09-17T05:30:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56052af168a0 msgr2=0x56052afaa520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56052af168a0 0x56052afaa520 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0200071c10 msgr2=0x7f0200074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0200071c10 0x7f0200074120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f020c004f50 tx=0x7f020c0ae040 comp rx=0 tx=0).stop 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 msgr2=0x56052af0ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 --2- 172.21.15.43:0/187567332 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56052ae3d9c0 0x56052af0ba80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f021000b390 tx=0x7f021000df90 comp rx=0 tx=0).stop 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f0220c00640 1 -- 172.21.15.43:0/187567332 reap_dead start 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 shutdown_connections 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 >> 172.21.15.43:0/187567332 conn(0x56052ae24d80 msgr2=0x56052ae80070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 shutdown_connections 2024-09-17T05:30:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.541+0000 7f02270a4080 1 -- 172.21.15.43:0/187567332 wait complete. 2024-09-17T05:30:19.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- 172.21.15.43:0/2789705887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 msgr2=0x7f4b180b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2789705887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b4b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009870 tx=0x7f4b0c01c740 comp rx=0 tx=0).stop 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- 172.21.15.43:0/2789705887 shutdown_connections 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2789705887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b4b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- 172.21.15.43:0/2789705887 >> 172.21.15.43:0/2789705887 conn(0x7f4b180b23d0 msgr2=0x7f4b180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- 172.21.15.43:0/2789705887 shutdown_connections 2024-09-17T05:30:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- 172.21.15.43:0/2789705887 wait complete. 2024-09-17T05:30:19.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 Processor -- start 2024-09-17T05:30:19.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.709+0000 7f4b1f400640 1 -- start start 2024-09-17T05:30:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 0x7f4b180990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1816aff0 con 0x7f4b1806fc90 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1816ae70 con 0x7f4b180b1180 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1816b170 con 0x7f4b1815fe10 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44528/0 (socket says 172.21.15.43:44528) 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 -- 172.21.15.43:0/2715186881 learned_addr learned my addr 172.21.15.43:0/2715186881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b17e00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1d400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 0x7f4b180990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 -- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 msgr2=0x7f4b180b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 -- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 msgr2=0x7f4b180990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 0x7f4b180990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 -- 172.21.15.43:0/2715186881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b180c35d0 con 0x7f4b1806fc90 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b17e00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1d400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 0x7f4b180990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1ca00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c01a370 tx=0x7f4b0c0a5f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0b2040 con 0x7f4b1806fc90 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b0c004320 con 0x7f4b1806fc90 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0052d0 con 0x7f4b1806fc90 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b181f8df0 con 0x7f4b1806fc90 2024-09-17T05:30:19.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181f9370 con 0x7f4b1806fc90 2024-09-17T05:30:19.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.713+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b180763c0 con 0x7f4b1806fc90 2024-09-17T05:30:19.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.717+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b0c005470 con 0x7f4b1806fc90 2024-09-17T05:30:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.717+0000 7f4b14c00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 0x7f4af8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.717+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5967+0+0 (secure 0 0 0) 0x7f4b0c12e0b0 con 0x7f4b1806fc90 2024-09-17T05:30:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.717+0000 7f4b17e00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 0x7f4af8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.717+0000 7f4b17e00640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 0x7f4af8075740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001230 tx=0x7f4b04009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.721+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b181f8df0 con 0x7f4b1806fc90 2024-09-17T05:30:19.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:19.933+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4b180760e0 con 0x7f4b1806fc90 2024-09-17T05:30:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b14c00640 1 -- 172.21.15.43:0/2715186881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v106) ==== 162+0+0 (secure 0 0 0) 0x7f4b0c0bd060 con 0x7f4b1806fc90 2024-09-17T05:30:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:20.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 msgr2=0x7f4af8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 0x7f4af8075740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001230 tx=0x7f4b04009040 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 msgr2=0x7f4b180c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c01a370 tx=0x7f4b0c0a5f70 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 shutdown_connections 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4af8073230 0x7f4af8075740 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b180b1180 0x7f4b180990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b1815fe10 0x7f4b180b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 --2- 172.21.15.43:0/2715186881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b1806fc90 0x7f4b180c2d60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 >> 172.21.15.43:0/2715186881 conn(0x7f4b180b23d0 msgr2=0x7f4b1815ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 shutdown_connections 2024-09-17T05:30:20.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.189+0000 7f4b1f400640 1 -- 172.21.15.43:0/2715186881 wait complete. 2024-09-17T05:30:20.205 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:20.201+0000 7f7b45000640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-17T05:30:20.205 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:20.201+0000 7f7b45000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:30:20.206 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:20.201+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:20.206 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:20.201+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:20.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:20.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 -- 172.21.15.43:0/3381403762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe4155d70 msgr2=0x7f2fe415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 --2- 172.21.15.43:0/3381403762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe4155d70 0x7f2fe415c7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009840 tx=0x7f2fd401c760 comp rx=0 tx=0).stop 2024-09-17T05:30:20.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 -- 172.21.15.43:0/3381403762 shutdown_connections 2024-09-17T05:30:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 --2- 172.21.15.43:0/3381403762 >> v2:172.21.15.62:3300/0 conn(0x7f2fe40b3af0 0x7f2fe40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 --2- 172.21.15.43:0/3381403762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe4155d70 0x7f2fe415c7e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 -- 172.21.15.43:0/3381403762 >> 172.21.15.43:0/3381403762 conn(0x7f2fe40b23d0 msgr2=0x7f2fe40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 -- 172.21.15.43:0/3381403762 shutdown_connections 2024-09-17T05:30:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.373+0000 7f2feb000640 1 -- 172.21.15.43:0/3381403762 wait complete. 2024-09-17T05:30:20.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 Processor -- start 2024-09-17T05:30:20.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 -- start start 2024-09-17T05:30:20.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe406f110 0x7f2fe4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe41fc4a0 0x7f2fe415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4168c10 con 0x7f2fe4155d70 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe4168a90 con 0x7f2fe406f110 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2feb000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe4168d90 con 0x7f2fe41fc4a0 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44534/0 (socket says 172.21.15.43:44534) 2024-09-17T05:30:20.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.377+0000 7f2fdb400640 1 -- 172.21.15.43:0/3176199067 learned_addr learned my addr 172.21.15.43:0/3176199067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:20.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe3e00640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe406f110 0x7f2fe4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe9000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe41fc4a0 0x7f2fe415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 -- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe41fc4a0 msgr2=0x7f2fe415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe41fc4a0 0x7f2fe415df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 -- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe406f110 msgr2=0x7f2fe4154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe406f110 0x7f2fe4154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 -- 172.21.15.43:0/3176199067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe4075040 con 0x7f2fe4155d70 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fdb400640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c400 tx=0x7f2fcc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc017070 con 0x7f2fe4155d70 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fcc003600 con 0x7f2fe4155d70 2024-09-17T05:30:20.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe4074280 con 0x7f2fe4155d70 2024-09-17T05:30:20.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003a50 con 0x7f2fe4155d70 2024-09-17T05:30:20.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe4074840 con 0x7f2fe4155d70 2024-09-17T05:30:20.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe40ad0b0 con 0x7f2fe4155d70 2024-09-17T05:30:20.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.381+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2fcc01a070 con 0x7f2fe4155d70 2024-09-17T05:30:20.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.385+0000 7f2fe0c00640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 0x7f2fc0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.385+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f2fcc093660 con 0x7f2fe4155d70 2024-09-17T05:30:20.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.385+0000 7f2fe3e00640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 0x7f2fc0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.385+0000 7f2fe3e00640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 0x7f2fc0075740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40af000 tx=0x7f2fd401afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.389+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2fcc05de30 con 0x7f2fe4155d70 2024-09-17T05:30:20.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.596+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fe41fca60 con 0x7f2fe4155d70 2024-09-17T05:30:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.596+0000 7f2fe0c00640 1 -- 172.21.15.43:0/3176199067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+4381 (secure 0 0 0) 0x7f2fcc062860 con 0x7f2fe4155d70 2024-09-17T05:30:20.601 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:20.601 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":108,"btime":"2024-09-17T05:30:20:207112+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:20.207045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 108 2024-09-17T05:30:20.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 msgr2=0x7f2fc0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 0x7f2fc0075740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40af000 tx=0x7f2fd401afa0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 msgr2=0x7f2fe41fbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c400 tx=0x7f2fcc00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 shutdown_connections 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fc0073230 0x7f2fc0075740 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2fe41fc4a0 0x7f2fe415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fe4155d70 0x7f2fe41fbf60 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 --2- 172.21.15.43:0/3176199067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe406f110 0x7f2fe4154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 >> 172.21.15.43:0/3176199067 conn(0x7f2fe40b23d0 msgr2=0x7f2fe415b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 shutdown_connections 2024-09-17T05:30:20.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.600+0000 7f2feb000640 1 -- 172.21.15.43:0/3176199067 wait complete. 2024-09-17T05:30:20.624 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:20.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:20.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 -- 172.21.15.43:0/855342723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5880b3af0 msgr2=0x7fb58815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 --2- 172.21.15.43:0/855342723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5880b3af0 0x7fb58815c7e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb5880d0c40 tx=0x7fb57001c800 comp rx=0 tx=0).stop 2024-09-17T05:30:20.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 -- 172.21.15.43:0/855342723 shutdown_connections 2024-09-17T05:30:20.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 --2- 172.21.15.43:0/855342723 >> v2:172.21.15.62:3300/0 conn(0x7fb58806f110 0x7fb5880ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 --2- 172.21.15.43:0/855342723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5880b3af0 0x7fb58815c7e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 -- 172.21.15.43:0/855342723 >> 172.21.15.43:0/855342723 conn(0x7fb5880b23d0 msgr2=0x7fb5880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 -- 172.21.15.43:0/855342723 shutdown_connections 2024-09-17T05:30:20.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.776+0000 7fb58f000640 1 -- 172.21.15.43:0/855342723 wait complete. 2024-09-17T05:30:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 Processor -- start 2024-09-17T05:30:20.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 -- start start 2024-09-17T05:30:20.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 0x7fb58809ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb588155d70 0x7fb58815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb588168b90 con 0x7fb58815b4e0 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb588168a10 con 0x7fb58806f110 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58f000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb588168d10 con 0x7fb588155d70 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44550/0 (socket says 172.21.15.43:44550) 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb58d000640 1 -- 172.21.15.43:0/999679896 learned_addr learned my addr 172.21.15.43:0/999679896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb587e00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 0x7fb58809ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.780+0000 7fb57fe00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb588155d70 0x7fb58815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 -- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb588155d70 msgr2=0x7fb58815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb588155d70 0x7fb58815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 -- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 msgr2=0x7fb58809ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 0x7fb58809ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 -- 172.21.15.43:0/999679896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880b1110 con 0x7fb58815b4e0 2024-09-17T05:30:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb587e00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 0x7fb58809ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58d000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb58000ea80 tx=0x7fb58000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000cea0 con 0x7fb58815b4e0 2024-09-17T05:30:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb580004b30 con 0x7fb58815b4e0 2024-09-17T05:30:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880cb5a0 con 0x7fb58815b4e0 2024-09-17T05:30:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580005020 con 0x7fb58815b4e0 2024-09-17T05:30:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5880cbb60 con 0x7fb58815b4e0 2024-09-17T05:30:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880ad0b0 con 0x7fb58815b4e0 2024-09-17T05:30:20.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.784+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5800051c0 con 0x7fb58815b4e0 2024-09-17T05:30:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.788+0000 7fb584c00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 0x7fb564075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.788+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7fb580094f40 con 0x7fb58815b4e0 2024-09-17T05:30:20.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.788+0000 7fb587e00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 0x7fb564075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.788+0000 7fb587e00640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 0x7fb564075740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb5880715c0 tx=0x7fb578009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:20.792+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb58005fde0 con 0x7fb58815b4e0 2024-09-17T05:30:21.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.000+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb588074e10 con 0x7fb58815b4e0 2024-09-17T05:30:21.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.000+0000 7fb584c00640 1 -- 172.21.15.43:0/999679896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+4381 (secure 0 0 0) 0x7fb58005f6e0 con 0x7fb58815b4e0 2024-09-17T05:30:21.007 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:21.007 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":108,"btime":"2024-09-17T05:30:20:207112+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:20.207045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:21.007 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 108 2024-09-17T05:30:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 msgr2=0x7fb564075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 0x7fb564075740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb5880715c0 tx=0x7fb578009040 comp rx=0 tx=0).stop 2024-09-17T05:30:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 msgr2=0x7fb58815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb58000ea80 tx=0x7fb58000ef40 comp rx=0 tx=0).stop 2024-09-17T05:30:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 shutdown_connections 2024-09-17T05:30:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb564073230 0x7fb564075740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815b4e0 0x7fb58815ac50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb588155d70 0x7fb58815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 --2- 172.21.15.43:0/999679896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58806f110 0x7fb58809ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 >> 172.21.15.43:0/999679896 conn(0x7fb5880b23d0 msgr2=0x7fb58806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 shutdown_connections 2024-09-17T05:30:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.004+0000 7fb58f000640 1 -- 172.21.15.43:0/999679896 wait complete. 2024-09-17T05:30:21.030 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:21.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 -- 172.21.15.43:0/932609699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 msgr2=0x7f7b1406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 --2- 172.21.15.43:0/932609699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b1406ce60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7b140c7260 tx=0x7f7b0401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 -- 172.21.15.43:0/932609699 shutdown_connections 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 --2- 172.21.15.43:0/932609699 >> v2:172.21.15.62:3300/0 conn(0x7f7b140cc160 0x7f7b1406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 --2- 172.21.15.43:0/932609699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b1406ce60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 -- 172.21.15.43:0/932609699 >> 172.21.15.43:0/932609699 conn(0x7f7b140b23d0 msgr2=0x7f7b140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 -- 172.21.15.43:0/932609699 shutdown_connections 2024-09-17T05:30:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.184+0000 7f7b18a00640 1 -- 172.21.15.43:0/932609699 wait complete. 2024-09-17T05:30:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 Processor -- start 2024-09-17T05:30:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 -- start start 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b140cc6d0 0x7f7b140c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b141681f0 con 0x7f7b1415adc0 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b14168070 con 0x7f7b140cc6d0 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b18a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7b14168370 con 0x7f7b140cc160 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b12800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48990/0 (socket says 172.21.15.43:48990) 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 -- 172.21.15.43:0/550160604 learned_addr learned my addr 172.21.15.43:0/550160604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b12800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44562/0 (socket says 172.21.15.43:44562) 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b0be00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b140cc6d0 0x7f7b140c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 -- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b140cc6d0 msgr2=0x7f7b140c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b140cc6d0 0x7f7b140c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 -- 172.21.15.43:0/550160604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 msgr2=0x7f7b140c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 -- 172.21.15.43:0/550160604 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b140adc90 con 0x7f7b140cc160 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b12800640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.188+0000 7f7b11e00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b0401cda0 tx=0x7f7b04005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.192+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040bb070 con 0x7f7b140cc160 2024-09-17T05:30:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.192+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b04004320 con 0x7f7b140cc160 2024-09-17T05:30:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.192+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b140adf70 con 0x7f7b140cc160 2024-09-17T05:30:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.192+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040b2040 con 0x7f7b140cc160 2024-09-17T05:30:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.192+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1415b460 con 0x7f7b140cc160 2024-09-17T05:30:21.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b140ce100 con 0x7f7b140cc160 2024-09-17T05:30:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b040be070 con 0x7f7b140cc160 2024-09-17T05:30:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b0a000640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 0x7f7af0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f7b0412db10 con 0x7f7b140cc160 2024-09-17T05:30:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b0be00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 0x7f7af0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.196+0000 7f7b0be00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 0x7f7af0075710 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b140c9910 tx=0x7f7afc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.200+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7b040164b0 con 0x7f7b140cc160 2024-09-17T05:30:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.416+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b1406d200 con 0x7f7b140cc160 2024-09-17T05:30:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.416+0000 7f7b0a000640 1 -- 172.21.15.43:0/550160604 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+4381 (secure 0 0 0) 0x7f7b040f82e0 con 0x7f7b140cc160 2024-09-17T05:30:21.421 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":108,"btime":"2024-09-17T05:30:20:207112+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1590794613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1590794613},{"type":"v1","addr":"172.21.15.62:6835","nonce":1590794613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:20.207045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 108 2024-09-17T05:30:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.416+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 msgr2=0x7f7af0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 0x7f7af0075710 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b140c9910 tx=0x7f7afc009040 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 msgr2=0x7f7b140ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b0401cda0 tx=0x7f7b04005e70 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 shutdown_connections 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7af0073200 0x7f7af0075710 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b140cc6d0 0x7f7b140c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b1415adc0 0x7f7b140c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 --2- 172.21.15.43:0/550160604 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7b140cc160 0x7f7b140ad390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 >> 172.21.15.43:0/550160604 conn(0x7f7b140b23d0 msgr2=0x7f7b140cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 shutdown_connections 2024-09-17T05:30:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.420+0000 7f7b18a00640 1 -- 172.21.15.43:0/550160604 wait complete. 2024-09-17T05:30:21.443 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-09-17T05:30:21.444 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-09-17T05:30:21.444 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:21.444 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:21.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:21.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:21.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:21.453 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:21.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:21.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 -- 172.21.15.43:0/1191380062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0155d70 msgr2=0x7f11d015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 --2- 172.21.15.43:0/1191380062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0155d70 0x7f11d015c7e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f11d0071050 tx=0x7f11c001c800 comp rx=0 tx=0).stop 2024-09-17T05:30:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 -- 172.21.15.43:0/1191380062 shutdown_connections 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 --2- 172.21.15.43:0/1191380062 >> v2:172.21.15.62:3300/0 conn(0x7f11d00b3920 0x7f11d00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 --2- 172.21.15.43:0/1191380062 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d006f110 0x7f11d00b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 --2- 172.21.15.43:0/1191380062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d0155d70 0x7f11d015c7e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 -- 172.21.15.43:0/1191380062 >> 172.21.15.43:0/1191380062 conn(0x7f11d00b23d0 msgr2=0x7f11d00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 -- 172.21.15.43:0/1191380062 shutdown_connections 2024-09-17T05:30:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.704+0000 7f11d6800640 1 -- 172.21.15.43:0/1191380062 wait complete. 2024-09-17T05:30:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 Processor -- start 2024-09-17T05:30:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- start start 2024-09-17T05:30:21.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 0x7f11d0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 0x7f11d00cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d015f870 con 0x7f11d00b3920 2024-09-17T05:30:21.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f11d015f6f0 con 0x7f11d006f110 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f11d015f9f0 con 0x7f11d0155d70 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44572/0 (socket says 172.21.15.43:44572) 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 -- 172.21.15.43:0/2823965500 learned_addr learned my addr 172.21.15.43:0/2823965500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11cfe00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 0x7f11d00cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11cf400640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 0x7f11d0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 -- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 msgr2=0x7f11d00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 0x7f11d00cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 -- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 msgr2=0x7f11d0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 0x7f11d0154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 -- 172.21.15.43:0/2823965500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d009af90 con 0x7f11d00b3920 2024-09-17T05:30:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11cfe00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 0x7f11d00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11cf400640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 0x7f11d0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c6a00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f11b800e770 tx=0x7f11b800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b800ccc0 con 0x7f11d00b3920 2024-09-17T05:30:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11b800ce60 con 0x7f11d00b3920 2024-09-17T05:30:21.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d00b0c90 con 0x7f11d00b3920 2024-09-17T05:30:21.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005280 con 0x7f11d00b3920 2024-09-17T05:30:21.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.708+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d0099aa0 con 0x7f11d00b3920 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.712+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0154e40 con 0x7f11d00b3920 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.712+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f11b8005420 con 0x7f11d00b3920 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.712+0000 7f11c7e00640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 0x7f11ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.712+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f11b8094d40 con 0x7f11d00b3920 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.712+0000 7f11cf400640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 0x7f11ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.716+0000 7f11cf400640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 0x7f11ac075740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f11c00af000 tx=0x7f11c001b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.716+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11d009af90 con 0x7f11d00b3920 2024-09-17T05:30:21.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:21.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:21.928+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f11d015fee0 con 0x7f11d00b3920 2024-09-17T05:30:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.196+0000 7f11c7e00640 1 -- 172.21.15.43:0/2823965500 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) ==== 161+0+0 (secure 0 0 0) 0x7f11b805fb80 con 0x7f11d00b3920 2024-09-17T05:30:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 msgr2=0x7f11ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 0x7f11ac075740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f11c00af000 tx=0x7f11c001b040 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 msgr2=0x7f11d0099fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f11b800e770 tx=0x7f11b800ec30 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 shutdown_connections 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f11ac073230 0x7f11ac075740 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11d0155d70 0x7f11d00cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d00b3920 0x7f11d0099fc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 --2- 172.21.15.43:0/2823965500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11d006f110 0x7f11d0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 >> 172.21.15.43:0/2823965500 conn(0x7f11d00b23d0 msgr2=0x7f11d009d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:22.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 shutdown_connections 2024-09-17T05:30:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.200+0000 7f11d6800640 1 -- 172.21.15.43:0/2823965500 wait complete. 2024-09-17T05:30:22.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:22.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- 172.21.15.43:0/2779235118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00500b5070 msgr2=0x7f00500b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 --2- 172.21.15.43:0/2779235118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00500b5070 0x7f00500b5480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0038009840 tx=0x7f003801c700 comp rx=0 tx=0).stop 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- 172.21.15.43:0/2779235118 shutdown_connections 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 --2- 172.21.15.43:0/2779235118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500d08b0 0x7f00500d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 --2- 172.21.15.43:0/2779235118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00500b5070 0x7f00500b5480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- 172.21.15.43:0/2779235118 >> 172.21.15.43:0/2779235118 conn(0x7f00500691b0 msgr2=0x7f00500695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- 172.21.15.43:0/2779235118 shutdown_connections 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- 172.21.15.43:0/2779235118 wait complete. 2024-09-17T05:30:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 Processor -- start 2024-09-17T05:30:22.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.384+0000 7f0054a00640 1 -- start start 2024-09-17T05:30:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500b5070 0x7f00501587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:22.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f005015c4e0 0x7f00500cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0050169d90 con 0x7f00500d08b0 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0050169c10 con 0x7f005015c4e0 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0050169f10 con 0x7f00500b5070 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f004e800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500b5070 0x7f00501587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44580/0 (socket says 172.21.15.43:44580) 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 -- 172.21.15.43:0/724652101 learned_addr learned my addr 172.21.15.43:0/724652101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f004f200640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f005015c4e0 0x7f00500cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 -- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500b5070 msgr2=0x7f00501587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500b5070 0x7f00501587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 -- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f005015c4e0 msgr2=0x7f00500cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f005015c4e0 0x7f00500cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 -- 172.21.15.43:0/724652101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00500ca6a0 con 0x7f00500d08b0 2024-09-17T05:30:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0045e00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f003801cbe0 tx=0x7f00380a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00380bd020 con 0x7f00500d08b0 2024-09-17T05:30:22.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00380a66d0 con 0x7f00500d08b0 2024-09-17T05:30:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0050164430 con 0x7f00500d08b0 2024-09-17T05:30:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038003600 con 0x7f00500d08b0 2024-09-17T05:30:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501649f0 con 0x7f00500d08b0 2024-09-17T05:30:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.388+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00500b5480 con 0x7f00500d08b0 2024-09-17T05:30:22.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.392+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00380c0020 con 0x7f00500d08b0 2024-09-17T05:30:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.392+0000 7f0047400640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 0x7f002c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.392+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f003812f380 con 0x7f00500d08b0 2024-09-17T05:30:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.392+0000 7f004e800640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 0x7f002c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:22.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.392+0000 7f004e800640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 0x7f002c075710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0040001870 tx=0x7f0040001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:22.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.396+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00380c7030 con 0x7f00500d08b0 2024-09-17T05:30:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:22.636+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f005006a1e0 con 0x7f00500d08b0 2024-09-17T05:30:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.204+0000 7f0047400640 1 -- 172.21.15.43:0/724652101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v110) ==== 101+0+0 (secure 0 0 0) 0x7f0038016410 con 0x7f00500d08b0 2024-09-17T05:30:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:30:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 msgr2=0x7f002c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 0x7f002c075710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0040001870 tx=0x7f0040001380 comp rx=0 tx=0).stop 2024-09-17T05:30:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 msgr2=0x7f00500cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f003801cbe0 tx=0x7f00380a64e0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 shutdown_connections 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f002c073200 0x7f002c075710 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f005015c4e0 0x7f00500cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00500d08b0 0x7f00500cae60 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 --2- 172.21.15.43:0/724652101 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00500b5070 0x7f00501587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 >> 172.21.15.43:0/724652101 conn(0x7f00500691b0 msgr2=0x7f005015b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 shutdown_connections 2024-09-17T05:30:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.208+0000 7f0054a00640 1 -- 172.21.15.43:0/724652101 wait complete. 2024-09-17T05:30:23.237 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:23.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.292+0000 7fdac897a080 1 Processor -- start 2024-09-17T05:30:23.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.292+0000 7fdac897a080 1 -- start start 2024-09-17T05:30:23.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e18590 0x555e64ecd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64ed3da0 0x555e64ecf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555e64d65a00 con 0x555e64e46cb0 2024-09-17T05:30:23.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555e64dc3f40 con 0x555e64e18590 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555e64efa8b0 con 0x555e64ed3da0 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44600/0 (socket says 172.21.15.43:44600) 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 -- 172.21.15.43:0/427423319 learned_addr learned my addr 172.21.15.43:0/427423319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1c00640 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e18590 0x555e64ecd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac2600640 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64ed3da0 0x555e64ecf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 -- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64ed3da0 msgr2=0x555e64ecf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64ed3da0 0x555e64ecf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 -- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e18590 msgr2=0x555e64ecd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e18590 0x555e64ecd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 -- 172.21.15.43:0/427423319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64d3ec10 con 0x555e64e46cb0 2024-09-17T05:30:23.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac1200640 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdab000b1f0 tx=0x7fdab000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bf5b17a2eea44e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdabb400640 1 -- 172.21.15.43:0/427423319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab000a040 con 0x555e64e46cb0 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdabb400640 1 -- 172.21.15.43:0/427423319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdab000f620 con 0x555e64e46cb0 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdabb400640 1 -- 172.21.15.43:0/427423319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab000fa70 con 0x555e64e46cb0 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- 172.21.15.43:0/427423319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 msgr2=0x555e64eece90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdab000b1f0 tx=0x7fdab000b6b0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- 172.21.15.43:0/427423319 shutdown_connections 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64ed3da0 0x555e64ecf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64e46cb0 0x555e64eece90 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 --2- 172.21.15.43:0/427423319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e18590 0x555e64ecd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- 172.21.15.43:0/427423319 >> 172.21.15.43:0/427423319 conn(0x555e64e03d80 msgr2=0x555e64e19a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- 172.21.15.43:0/427423319 shutdown_connections 2024-09-17T05:30:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.296+0000 7fdac897a080 1 -- 172.21.15.43:0/427423319 wait complete. 2024-09-17T05:30:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 Processor -- start 2024-09-17T05:30:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- start start 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64e18590 0x555e64eeeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e46cb0 0x555e64ee91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555e64ee97b0 con 0x555e64ed3da0 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555e64eceed0 con 0x555e64e46cb0 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555e64ecf130 con 0x555e64e18590 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44610/0 (socket says 172.21.15.43:44610) 2024-09-17T05:30:23.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 learned_addr learned my addr 172.21.15.43:0/1162858032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64e18590 0x555e64eeeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e46cb0 0x555e64ee91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64e18590 msgr2=0x555e64eeeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555e64e18590 0x555e64eeeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e46cb0 msgr2=0x555e64ee91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555e64e46cb0 0x555e64ee91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64eedeb0 con 0x555e64ed3da0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac2600640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdabc018170 tx=0x7fdabc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdaba000640 1 -- 172.21.15.43:0/1162858032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ba030 con 0x555e64ed3da0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdaba000640 1 -- 172.21.15.43:0/1162858032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdabc0b0040 con 0x555e64ed3da0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdaba000640 1 -- 172.21.15.43:0/1162858032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0af040 con 0x555e64ed3da0 2024-09-17T05:30:23.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e64eee150 con 0x555e64ed3da0 2024-09-17T05:30:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.300+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e64eee690 con 0x555e64ed3da0 2024-09-17T05:30:23.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdaba000640 1 -- 172.21.15.43:0/1162858032 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdabc0b6610 con 0x555e64ed3da0 2024-09-17T05:30:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdaba000640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 0x7fdaa40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdaba000640 1 -- 172.21.15.43:0/1162858032 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7fdabc12e7a0 con 0x555e64ed3da0 2024-09-17T05:30:23.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 0x7fdaa40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.311 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555e64f87d30 0x555e64f88140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f88680 con 0x555e64f87d30 2024-09-17T05:30:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.304+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555e64f87d30 0x555e64f88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.308+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 0x7fdaa40740f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdab40041c0 tx=0x7fdab4004150 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.308+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555e64f87d30 0x555e64f88140 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.308+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v93'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.308+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f942f0 con 0x555e64f87d30 2024-09-17T05:30:23.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.312+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v93'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.312+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94690 con 0x555e64f87d30 2024-09-17T05:30:23.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.312+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v93'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.318 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:23.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.312+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f95a40 con 0x555e64f87d30 2024-09-17T05:30:23.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v93'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f95de0 con 0x555e64f87d30 2024-09-17T05:30:23.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v93'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94a30 con 0x555e64f87d30 2024-09-17T05:30:23.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v93'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.316+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94e20 con 0x555e64f87d30 2024-09-17T05:30:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v93'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f94e20 con 0x555e64f87d30 2024-09-17T05:30:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v93'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v93'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.327 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:23.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.320+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.324+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v93'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.324+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.324+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v93'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.324+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v93'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v93'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v93'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.328+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v93'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.336 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v93'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v93'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.332+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.336+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v93'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.336+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.336+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v93'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.336+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v93'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v93'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v93'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.347 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:23.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.340+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.344+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v93'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.344+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.344+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v93'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.344+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v93'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v93'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v93'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.348+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v93'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.356 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:23.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v93'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v93'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.352+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f87d30 2024-09-17T05:30:23.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v93'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555e64f96180 0x555e64f985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f94dd0 con 0x555e64f96180 2024-09-17T05:30:23.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555e64f96180 0x555e64f985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555e64f96180 0x555e64f985a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.362 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v93'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.356+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f98ae0 con 0x555e64f96180 2024-09-17T05:30:23.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v93'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f98e80 con 0x555e64f96180 2024-09-17T05:30:23.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v93'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f99220 con 0x555e64f96180 2024-09-17T05:30:23.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 200.00000000 [create] v93'91 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.360+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f995c0 con 0x555e64f96180 2024-09-17T05:30:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.364+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 200.00000000 [delete] v93'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.364+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f99960 con 0x555e64f96180 2024-09-17T05:30:23.370 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.364+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v93'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.364+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f99d00 con 0x555e64f96180 2024-09-17T05:30:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v93'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0f0 con 0x555e64f96180 2024-09-17T05:30:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v93'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0f0 con 0x555e64f96180 2024-09-17T05:30:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000001 [create] v93'96 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.368+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000001 [delete] v93'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v93'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.377 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e270 con 0x555e64f96180 2024-09-17T05:30:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v93'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.372+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e270 con 0x555e64f96180 2024-09-17T05:30:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.376+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v93'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.376+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.376+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 606.00000000 [create] v93'101 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.376+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 606.00000000 [delete] v93'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.385 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v93'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9b0b0 con 0x555e64f96180 2024-09-17T05:30:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v93'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.380+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9b0b0 con 0x555e64f96180 2024-09-17T05:30:23.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v93'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9a0a0 con 0x555e64f96180 2024-09-17T05:30:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v93'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64f96180 2024-09-17T05:30:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555e64f9a9a0 0x555e64f9adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9d490 con 0x555e64f9a9a0 2024-09-17T05:30:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555e64f9a9a0 0x555e64f9adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.384+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555e64f9a9a0 0x555e64f9adb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(51 607.00000000 [create] v93'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f9d490 con 0x555e64f9a9a0 2024-09-17T05:30:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(52 607.00000000 [delete] v93'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9d490 con 0x555e64f9a9a0 2024-09-17T05:30:23.394 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(53 607.00000000 [create] v93'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.388+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e540 con 0x555e64f9a9a0 2024-09-17T05:30:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v93'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f9a9a0 2024-09-17T05:30:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v93'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f9a9a0 2024-09-17T05:30:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v93'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.392+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f87d30 2024-09-17T05:30:23.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.396+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v93'17 uv16 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.396+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f87d30 2024-09-17T05:30:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.396+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v93'18 uv18 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.396+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f87d30 2024-09-17T05:30:23.403 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v93'19 uv19 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64f9e500 con 0x555e64f87d30 2024-09-17T05:30:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v93'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555e64f9dac0 0x555e64f9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa0930 con 0x555e64f9dac0 2024-09-17T05:30:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac2600640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555e64f9dac0 0x555e64f9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.400+0000 7fdac2600640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555e64f9dac0 0x555e64f9ded0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.404+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(61 608.00000000 [create] v93'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.404+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa0cd0 con 0x555e64f9dac0 2024-09-17T05:30:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.404+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(62 608.00000000 [delete] v93'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.404+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa10f0 con 0x555e64f9dac0 2024-09-17T05:30:23.414 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(63 608.00000000 [create] v93'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa29a0 con 0x555e64f9dac0 2024-09-17T05:30:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v93'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa2d40 con 0x555e64f9dac0 2024-09-17T05:30:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v93'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.408+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1990 con 0x555e64f9dac0 2024-09-17T05:30:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v93'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555e64fa2240 0x555e64fa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa5630 con 0x555e64fa2240 2024-09-17T05:30:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555e64fa2240 0x555e64fa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac1c00640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555e64fa2240 0x555e64fa50f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(67 604.00000000 [create] v93'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.412+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa5630 con 0x555e64fa2240 2024-09-17T05:30:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(68 604.00000000 [delete] v93'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa5630 con 0x555e64fa2240 2024-09-17T05:30:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(69 604.00000000 [create] v93'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.422 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1590 con 0x555e64fa2240 2024-09-17T05:30:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v93'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.416+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1550 con 0x555e64fa2240 2024-09-17T05:30:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v93'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1550 con 0x555e64fa2240 2024-09-17T05:30:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v93'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1550 con 0x555e64fa2240 2024-09-17T05:30:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(73 500.00000000 [create] v93'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.420+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa1530 con 0x555e64fa2240 2024-09-17T05:30:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(74 500.00000000 [delete] v93'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1530 con 0x555e64fa2240 2024-09-17T05:30:23.429 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(75 500.00000000 [create] v93'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1530 con 0x555e64fa2240 2024-09-17T05:30:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v93'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.424+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa1530 con 0x555e64fa2240 2024-09-17T05:30:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac1c00640 1 -- 172.21.15.43:0/1162858032 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v93'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab4008920 con 0x555e64fa2240 2024-09-17T05:30:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555e64fa5ff0 0x555e64fa8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8990 con 0x555e64fa5ff0 2024-09-17T05:30:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555e64fa5ff0 0x555e64fa8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac1200640 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555e64fa5ff0 0x555e64fa8410 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(78 mds_snaptable [create] v93'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.428+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa8990 con 0x555e64fa5ff0 2024-09-17T05:30:23.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.432+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(79 mds_snaptable [delete] v93'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.432+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64fa5ff0 2024-09-17T05:30:23.439 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:23.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(80 mds_snaptable [create] v93'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64fa5ff0 2024-09-17T05:30:23.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v93'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64fa5ff0 2024-09-17T05:30:23.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v93'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.436+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f87d30 2024-09-17T05:30:23.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.440+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(83 605.00000000 [create] v93'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.440+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f87d30 2024-09-17T05:30:23.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.440+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(84 605.00000000 [delete] v93'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.440+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f87d30 2024-09-17T05:30:23.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(85 605.00000000 [create] v93'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:23.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa9dc0 con 0x555e64f87d30 2024-09-17T05:30:23.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v93'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa9dc0 con 0x555e64f87d30 2024-09-17T05:30:23.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v93'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.444+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f87d30 2024-09-17T05:30:23.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v93'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f87d30 2024-09-17T05:30:23.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v93'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v93'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.448+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.452+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v93'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.456 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:23.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.452+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.452+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v93'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.452+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v93'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v93'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v93'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.456+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.460+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v93'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.464 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:23.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.460+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa8db0 con 0x555e64f9a9a0 2024-09-17T05:30:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.460+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v93'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.460+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa5700 con 0x555e64f9a9a0 2024-09-17T05:30:23.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.464+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v93'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64f9a9a0 2024-09-17T05:30:23.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.464+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.464+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v93'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.464+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v93'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v93'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.473 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v93'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.468+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.472+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v93'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.472+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.472+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v93'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.472+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:23.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v93'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v93'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v93'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.476+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64f9dac0 2024-09-17T05:30:23.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v93'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdabc002050 con 0x555e64f9dac0 2024-09-17T05:30:23.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v93'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v93'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.480+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.487 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v93'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v93'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v93'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.484+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v93'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v93'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.494 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:23.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v93'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.488+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v93'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v93'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v93'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.492+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v93'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.501 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:23.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v93'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v93'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.496+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v93'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa56c0 con 0x555e64fa5ff0 2024-09-17T05:30:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v93'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x555e64fa9600 con 0x555e64fa5ff0 2024-09-17T05:30:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac1200640 1 -- 172.21.15.43:0/1162858032 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v93'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdab000d040 con 0x555e64fa5ff0 2024-09-17T05:30:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555e64f96180 msgr2=0x555e64f985a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555e64f96180 0x555e64f985a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555e64fa2240 msgr2=0x555e64fa50f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555e64fa2240 0x555e64fa50f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555e64fa5ff0 msgr2=0x555e64fa8410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555e64fa5ff0 0x555e64fa8410 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555e64f9a9a0 msgr2=0x555e64f9adb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555e64f9a9a0 0x555e64f9adb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555e64f87d30 msgr2=0x555e64f88140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555e64f87d30 0x555e64f88140 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 reap_dead start 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555e64f9dac0 msgr2=0x555e64f9ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555e64f9dac0 0x555e64f9ded0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac2600640 1 -- 172.21.15.43:0/1162858032 reap_dead start 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 msgr2=0x7fdaa40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 0x7fdaa40740f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdab40041c0 tx=0x7fdab4004150 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 msgr2=0x555e64edac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdabc018170 tx=0x7fdabc004a50 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 shutdown_connections 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdaa4071be0 0x7fdaa40740f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 --2- 172.21.15.43:0/1162858032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e64ed3da0 0x555e64edac80 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 >> 172.21.15.43:0/1162858032 conn(0x555e64e03d80 msgr2=0x555e64e5c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 shutdown_connections 2024-09-17T05:30:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.500+0000 7fdac897a080 1 -- 172.21.15.43:0/1162858032 wait complete. 2024-09-17T05:30:23.512 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-09-17T05:30:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 Processor -- start 2024-09-17T05:30:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 -- start start 2024-09-17T05:30:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d613d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f98b0 0x55db1d5f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55db1d48ea00 con 0x55db1d5f8010 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55db1d4ecf40 con 0x55db1d56fcb0 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f51245080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55db1d5435b0 con 0x55db1d5f98b0 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.612+0000 7f4f49a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44616/0 (socket says 172.21.15.43:44616) 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 -- 172.21.15.43:0/2225643357 learned_addr learned my addr 172.21.15.43:0/2225643357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f4ae00640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f98b0 0x55db1d5f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f4a400640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d613d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 -- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f98b0 msgr2=0x55db1d5f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f98b0 0x55db1d5f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 -- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 msgr2=0x55db1d613d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d613d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 -- 172.21.15.43:0/2225643357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1d467c10 con 0x55db1d5f8010 2024-09-17T05:30:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f4a400640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d613d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:23.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4f38004f50 tx=0x7f4f38017f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce22cc5ce4a60dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f43e00640 1 -- 172.21.15.43:0/2225643357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380ab5f0 con 0x55db1d5f8010 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f43e00640 1 -- 172.21.15.43:0/2225643357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f380b9070 con 0x55db1d5f8010 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f43e00640 1 -- 172.21.15.43:0/2225643357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380abc90 con 0x55db1d5f8010 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- 172.21.15.43:0/2225643357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 msgr2=0x55db1d6142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4f38004f50 tx=0x7f4f38017f10 comp rx=0 tx=0).stop 2024-09-17T05:30:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- 172.21.15.43:0/2225643357 shutdown_connections 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f98b0 0x55db1d5f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f8010 0x55db1d6142c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 --2- 172.21.15.43:0/2225643357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d613d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- 172.21.15.43:0/2225643357 >> 172.21.15.43:0/2225643357 conn(0x55db1d52cd80 msgr2=0x55db1d6162f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- 172.21.15.43:0/2225643357 shutdown_connections 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- 172.21.15.43:0/2225643357 wait complete. 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 Processor -- start 2024-09-17T05:30:23.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.616+0000 7f4f51245080 1 -- start start 2024-09-17T05:30:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d612db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f8010 0x55db1d613d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f98b0 0x55db1d6142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55db1d5f9500 con 0x55db1d5f98b0 2024-09-17T05:30:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55db1d5894a0 con 0x55db1d56fcb0 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55db1d589200 con 0x55db1d5f8010 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f4ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f98b0 0x55db1d6142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f4a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d612db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f8010 0x55db1d613d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f4a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d612db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:41192/0 (socket says 172.21.15.43:41192) 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f4a400640 1 -- 172.21.15.43:0/2842384507 learned_addr learned my addr 172.21.15.43:0/2842384507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 -- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 msgr2=0x55db1d612db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db1d56fcb0 0x55db1d612db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 -- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f98b0 msgr2=0x55db1d6142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f98b0 0x55db1d6142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 -- 172.21.15.43:0/2842384507 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1d5f7c10 con 0x55db1d5f8010 2024-09-17T05:30:23.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f4ae00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1d5f98b0 0x55db1d6142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:23.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f49a00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f8010 0x55db1d613d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c10 tx=0x7f4f380adb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f42a00640 1 -- 172.21.15.43:0/2842384507 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380addc0 con 0x55db1d5f8010 2024-09-17T05:30:23.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f42a00640 1 -- 172.21.15.43:0/2842384507 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f380b9040 con 0x55db1d5f8010 2024-09-17T05:30:23.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f42a00640 1 -- 172.21.15.43:0/2842384507 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380aba70 con 0x55db1d5f8010 2024-09-17T05:30:23.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db1d6b0f10 con 0x55db1d5f8010 2024-09-17T05:30:23.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.620+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55db1d6b1450 con 0x55db1d5f8010 2024-09-17T05:30:23.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f42a00640 1 -- 172.21.15.43:0/2842384507 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f380c7030 con 0x55db1d5f8010 2024-09-17T05:30:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f42a00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4f2c071be0 0x7f4f2c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f42a00640 1 -- 172.21.15.43:0/2842384507 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f4f3812e220 con 0x55db1d5f8010 2024-09-17T05:30:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f4a400640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4f2c071be0 0x7f4f2c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f51245080 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55db1d6b18f0 0x55db1d6b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x55db1d6132f0 con 0x55db1d6b18f0 2024-09-17T05:30:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f4ae00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55db1d6b18f0 0x55db1d6b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f4a400640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4f2c071be0 0x7f4f2c0740f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c004fd0 tx=0x7f4f3c002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f4ae00640 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55db1d6b18f0 0x55db1d6b1d00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.624+0000 7f4f4ae00640 1 -- 172.21.15.43:0/2842384507 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [delete] v93'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f44002bf0 con 0x55db1d6b18f0 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55db1d6b18f0 msgr2=0x55db1d6b1d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55db1d6b18f0 0x55db1d6b1d00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4f2c071be0 msgr2=0x7f4f2c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4f2c071be0 0x7f4f2c0740f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c004fd0 tx=0x7f4f3c002040 comp rx=0 tx=0).stop 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f8010 msgr2=0x55db1d613d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 --2- 172.21.15.43:0/2842384507 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55db1d5f8010 0x55db1d613d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f38000c10 tx=0x7f4f380adb10 comp rx=0 tx=0).stop 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f4ae00640 1 -- 172.21.15.43:0/2842384507 reap_dead start 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 shutdown_connections 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 >> 172.21.15.43:0/2842384507 conn(0x55db1d52cd80 msgr2=0x55db1d6162d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 shutdown_connections 2024-09-17T05:30:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.628+0000 7f4f51245080 1 -- 172.21.15.43:0/2842384507 wait complete. 2024-09-17T05:30:23.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 -- 172.21.15.43:0/3949454927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f116415d8d0 msgr2=0x7f11640d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 --2- 172.21.15.43:0/3949454927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f116415d8d0 0x7f11640d26e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f115c009bc0 tx=0x7f115c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 -- 172.21.15.43:0/3949454927 shutdown_connections 2024-09-17T05:30:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 --2- 172.21.15.43:0/3949454927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f116415d8d0 0x7f11640d26e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 --2- 172.21.15.43:0/3949454927 >> v2:172.21.15.62:3300/0 conn(0x7f116415d4c0 0x7f11640d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 -- 172.21.15.43:0/3949454927 >> 172.21.15.43:0/3949454927 conn(0x7f11640691b0 msgr2=0x7f11640695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 -- 172.21.15.43:0/3949454927 shutdown_connections 2024-09-17T05:30:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.792+0000 7f1163200640 1 -- 172.21.15.43:0/3949454927 wait complete. 2024-09-17T05:30:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 Processor -- start 2024-09-17T05:30:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 -- start start 2024-09-17T05:30:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 0x7f11640c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 0x7f116415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1164153490 con 0x7f116415d8d0 2024-09-17T05:30:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1164153310 con 0x7f116415d4c0 2024-09-17T05:30:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1163200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1164153610 con 0x7f11640ccf00 2024-09-17T05:30:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f115be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.796+0000 7f1161e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 0x7f11640c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:38604/0 (socket says 172.21.15.43:38604) 2024-09-17T05:30:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1162800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 0x7f116415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 -- 172.21.15.43:0/738262946 learned_addr learned my addr 172.21.15.43:0/738262946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 -- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 msgr2=0x7f11640c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 0x7f11640c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 -- 172.21.15.43:0/738262946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 msgr2=0x7f116415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 0x7f116415eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 -- 172.21.15.43:0/738262946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116415c7b0 con 0x7f116415d4c0 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1161e00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 0x7f11640c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1162800640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 0x7f116415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115be00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e7c0 tx=0x7f114c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00cc60 con 0x7f116415d4c0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f114c00ce20 con 0x7f116415d4c0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c005280 con 0x7f116415d4c0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1164155800 con 0x7f116415d4c0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116414f240 con 0x7f116415d4c0 2024-09-17T05:30:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1164153ac0 con 0x7f116415d4c0 2024-09-17T05:30:23.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.800+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f114c005420 con 0x7f116415d4c0 2024-09-17T05:30:23.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.804+0000 7f115a000640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 0x7f1140075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.804+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6053+0+0 (secure 0 0 0) 0x7f114c095140 con 0x7f116415d4c0 2024-09-17T05:30:23.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.804+0000 7f1161e00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 0x7f1140075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.804+0000 7f1161e00640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 0x7f1140075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f11540046c0 tx=0x7f1154007ec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:23.808+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f114c05fa20 con 0x7f116415d4c0 2024-09-17T05:30:24.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.028+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f116406a1e0 con 0x7f116415d4c0 2024-09-17T05:30:24.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f115a000640 1 -- 172.21.15.43:0/738262946 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v111) ==== 162+0+0 (secure 0 0 0) 0x7f114c064450 con 0x7f116415d4c0 2024-09-17T05:30:24.216 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 msgr2=0x7f1140075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 0x7f1140075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f11540046c0 tx=0x7f1154007ec0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 msgr2=0x7f1164153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e7c0 tx=0x7f114c00ec80 comp rx=0 tx=0).stop 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 shutdown_connections 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1140073200 0x7f1140075710 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116415d4c0 0x7f1164153cd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116415d8d0 0x7f116415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 --2- 172.21.15.43:0/738262946 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f11640ccf00 0x7f11640c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 >> 172.21.15.43:0/738262946 conn(0x7f11640691b0 msgr2=0x7f11640b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 shutdown_connections 2024-09-17T05:30:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.212+0000 7f1163200640 1 -- 172.21.15.43:0/738262946 wait complete. 2024-09-17T05:30:24.224 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:24.226+0000 7f4623200640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-17T05:30:24.225 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:24.220+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:24.225 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:24.220+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:24.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- 172.21.15.43:0/4161292626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80b6b10 msgr2=0x7f27b806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 --2- 172.21.15.43:0/4161292626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80b6b10 0x7f27b806ce60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009930 tx=0x7f27a801c810 comp rx=0 tx=0).stop 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- 172.21.15.43:0/4161292626 shutdown_connections 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 --2- 172.21.15.43:0/4161292626 >> v2:172.21.15.62:3300/0 conn(0x7f27b80cd9a0 0x7f27b80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 --2- 172.21.15.43:0/4161292626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80b6b10 0x7f27b806ce60 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- 172.21.15.43:0/4161292626 >> 172.21.15.43:0/4161292626 conn(0x7f27b80b2e80 msgr2=0x7f27b80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- 172.21.15.43:0/4161292626 shutdown_connections 2024-09-17T05:30:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- 172.21.15.43:0/4161292626 wait complete. 2024-09-17T05:30:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 Processor -- start 2024-09-17T05:30:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.396+0000 7f27bd000640 1 -- start start 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27b80b2330 0x7f27b8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27b80b6b10 0x7f27b81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27b816a100 con 0x7f27b80cd9a0 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8169f80 con 0x7f27b80b6b10 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f27b816a280 con 0x7f27b80b2330 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57146/0 (socket says 172.21.15.43:57146) 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 -- 172.21.15.43:0/2849411768 learned_addr learned my addr 172.21.15.43:0/2849411768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b5e00640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27b80b2330 0x7f27b8158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27afe00640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27b80b6b10 0x7f27b81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 -- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27b80b2330 msgr2=0x7f27b8158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27b80b2330 0x7f27b8158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 -- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27b80b6b10 msgr2=0x7f27b81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27b80b6b10 0x7f27b81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 -- 172.21.15.43:0/2849411768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81527f0 con 0x7f27b80cd9a0 2024-09-17T05:30:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27b6800640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e970 tx=0x7f27b000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004d30 con 0x7f27b80cd9a0 2024-09-17T05:30:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27b000cb20 con 0x7f27b80cd9a0 2024-09-17T05:30:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b8152c90 con 0x7f27b80cd9a0 2024-09-17T05:30:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0005020 con 0x7f27b80cd9a0 2024-09-17T05:30:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b8157760 con 0x7f27b80cd9a0 2024-09-17T05:30:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.400+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b806ce60 con 0x7f27b80cd9a0 2024-09-17T05:30:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.404+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27b00051c0 con 0x7f27b80cd9a0 2024-09-17T05:30:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.404+0000 7f27ae000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.404+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f27b0094d40 con 0x7f27b80cd9a0 2024-09-17T05:30:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.404+0000 7f27b5e00640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.404+0000 7f27b5e00640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 0x7f2794075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f27a80af000 tx=0x7f27a801b050 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.408+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27b0016030 con 0x7f27b80cd9a0 2024-09-17T05:30:24.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27b806dd50 con 0x7f27b80cd9a0 2024-09-17T05:30:24.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27ae000640 1 -- 172.21.15.43:0/2849411768 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3556 (secure 0 0 0) 0x7f27b005f4c0 con 0x7f27b80cd9a0 2024-09-17T05:30:24.625 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:24.625 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":113,"btime":"2024-09-17T05:30:24:226077+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:24.226013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:24.626 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 113 2024-09-17T05:30:24.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 msgr2=0x7f2794075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 0x7f2794075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f27a80af000 tx=0x7f27a801b050 comp rx=0 tx=0).stop 2024-09-17T05:30:24.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 msgr2=0x7f27b8165920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e970 tx=0x7f27b000ee30 comp rx=0 tx=0).stop 2024-09-17T05:30:24.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 shutdown_connections 2024-09-17T05:30:24.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27b80b6b10 0x7f27b81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b80cd9a0 0x7f27b8165920 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 --2- 172.21.15.43:0/2849411768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27b80b2330 0x7f27b8158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 >> 172.21.15.43:0/2849411768 conn(0x7f27b80b2e80 msgr2=0x7f27b80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 shutdown_connections 2024-09-17T05:30:24.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.620+0000 7f27bd000640 1 -- 172.21.15.43:0/2849411768 wait complete. 2024-09-17T05:30:24.647 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:24.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 -- 172.21.15.43:0/2023023835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0b3af0 msgr2=0x7f36fc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 --2- 172.21.15.43:0/2023023835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0b3af0 0x7f36fc15c7e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009840 tx=0x7f36e401c790 comp rx=0 tx=0).stop 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 -- 172.21.15.43:0/2023023835 shutdown_connections 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 --2- 172.21.15.43:0/2023023835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0b3af0 0x7f36fc15c7e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 -- 172.21.15.43:0/2023023835 >> 172.21.15.43:0/2023023835 conn(0x7f36fc0b23d0 msgr2=0x7f36fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 -- 172.21.15.43:0/2023023835 shutdown_connections 2024-09-17T05:30:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.800+0000 7f3701c00640 1 -- 172.21.15.43:0/2023023835 wait complete. 2024-09-17T05:30:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 Processor -- start 2024-09-17T05:30:24.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 -- start start 2024-09-17T05:30:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36fc06f110 0x7f36fc09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 0x7f36fc0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc169d70 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc169bf0 con 0x7f36fc155d70 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f3701c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc169ef0 con 0x7f36fc06f110 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57160/0 (socket says 172.21.15.43:57160) 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 -- 172.21.15.43:0/2308399381 learned_addr learned my addr 172.21.15.43:0/2308399381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36faa00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36fc06f110 0x7f36fc09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36f2000640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 0x7f36fc0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 -- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36fc06f110 msgr2=0x7f36fc09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36fc06f110 0x7f36fc09d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 -- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 msgr2=0x7f36fc0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 0x7f36fc0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 -- 172.21.15.43:0/2308399381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc0cb0e0 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36f2000640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 0x7f36fc0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36fb400640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ba40 tx=0x7f36f400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.804+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4003b80 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f4003e10 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400cbe0 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc09e020 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc0c6590 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc15c7e0 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36f4006d10 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36f3400640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 0x7f36d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f36f4093e50 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.808+0000 7f36faa00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 0x7f36d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.812+0000 7f36faa00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 0x7f36d8075740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0715c0 tx=0x7f36ec001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:24.812+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36f405e700 con 0x7f36fc0cb5e0 2024-09-17T05:30:24.958 INFO:tasks.ceph.mds.c.smithi043.stderr: -40> 2024-09-17T05:30:20.201+0000 7f7b45000640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-17T05:30:24.958 INFO:tasks.ceph.mds.c.smithi043.stderr: -38> 2024-09-17T05:30:20.201+0000 7f7b45000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:30:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.028+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36fc0af790 con 0x7f36fc0cb5e0 2024-09-17T05:30:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.028+0000 7f36f3400640 1 -- 172.21.15.43:0/2308399381 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3556 (secure 0 0 0) 0x7f36f4063130 con 0x7f36fc0cb5e0 2024-09-17T05:30:25.035 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:25.036 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":113,"btime":"2024-09-17T05:30:24:226077+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:24.226013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 113 2024-09-17T05:30:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 msgr2=0x7f36d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 0x7f36d8075740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0715c0 tx=0x7f36ec001380 comp rx=0 tx=0).stop 2024-09-17T05:30:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 msgr2=0x7f36fc15a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f36f400ba40 tx=0x7f36f400bf00 comp rx=0 tx=0).stop 2024-09-17T05:30:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 shutdown_connections 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36d8073230 0x7f36d8075740 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36fc155d70 0x7f36fc0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc0cb5e0 0x7f36fc15a710 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 --2- 172.21.15.43:0/2308399381 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36fc06f110 0x7f36fc09d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 >> 172.21.15.43:0/2308399381 conn(0x7f36fc0b23d0 msgr2=0x7f36fc06d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 shutdown_connections 2024-09-17T05:30:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.032+0000 7f3701c00640 1 -- 172.21.15.43:0/2308399381 wait complete. 2024-09-17T05:30:25.059 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:25.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:25.080 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- 172.21.15.43:0/2916031958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19580b3af0 msgr2=0x7f19580ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 --2- 172.21.15.43:0/2916031958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19580b3af0 0x7f19580ad0b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1950009c80 tx=0x7f195001cce0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- 172.21.15.43:0/2916031958 shutdown_connections 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 --2- 172.21.15.43:0/2916031958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19580b3af0 0x7f19580ad0b0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 --2- 172.21.15.43:0/2916031958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958155d70 0x7f195815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- 172.21.15.43:0/2916031958 >> 172.21.15.43:0/2916031958 conn(0x7f19580b23d0 msgr2=0x7f19580d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- 172.21.15.43:0/2916031958 shutdown_connections 2024-09-17T05:30:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- 172.21.15.43:0/2916031958 wait complete. 2024-09-17T05:30:25.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 Processor -- start 2024-09-17T05:30:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.216+0000 7f195da00640 1 -- start start 2024-09-17T05:30:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958155d70 0x7f1958074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958090e90 0x7f1958164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1958168c80 con 0x7f195806f110 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1958168b00 con 0x7f1958155d70 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1958168e00 con 0x7f1958090e90 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57208/0 (socket says 172.21.15.43:57208) 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 -- 172.21.15.43:0/3911692508 learned_addr learned my addr 172.21.15.43:0/3911692508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1957400640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958090e90 0x7f1958164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f194e000640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958155d70 0x7f1958074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 -- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958090e90 msgr2=0x7f1958164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958090e90 0x7f1958164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 -- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958155d70 msgr2=0x7f1958074ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958155d70 0x7f1958074ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 -- 172.21.15.43:0/3911692508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581fbe20 con 0x7f195806f110 2024-09-17T05:30:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f1956a00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f194800d7a0 tx=0x7f194800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948017070 con 0x7f195806f110 2024-09-17T05:30:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1948009d20 con 0x7f195806f110 2024-09-17T05:30:25.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480106c0 con 0x7f195806f110 2024-09-17T05:30:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19580cadb0 con 0x7f195806f110 2024-09-17T05:30:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.220+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581fc3a0 con 0x7f195806f110 2024-09-17T05:30:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.224+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19580caf80 con 0x7f195806f110 2024-09-17T05:30:25.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.224+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f194801a070 con 0x7f195806f110 2024-09-17T05:30:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.224+0000 7f194f400640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 0x7f1934075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.224+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f1948094180 con 0x7f195806f110 2024-09-17T05:30:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.224+0000 7f194e000640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 0x7f1934075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.228+0000 7f194e000640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 0x7f1934075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1940007fd0 tx=0x7f1940009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.228+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19581fbe20 con 0x7f195806f110 2024-09-17T05:30:25.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.436+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19580ad0b0 con 0x7f195806f110 2024-09-17T05:30:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f194f400640 1 -- 172.21.15.43:0/3911692508 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+4382 (secure 0 0 0) 0x7f194805e9a0 con 0x7f195806f110 2024-09-17T05:30:25.443 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:25.444 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":114,"btime":"2024-09-17T05:30:25:227587+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7710,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3640890143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3640890143},{"type":"v1","addr":"172.21.15.43:6835","nonce":3640890143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:24.226013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 114 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 msgr2=0x7f1934075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 0x7f1934075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1940007fd0 tx=0x7f1940009040 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 msgr2=0x7f1958090860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f194800d7a0 tx=0x7f194800dc60 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 shutdown_connections 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1934073200 0x7f1934075710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1958155d70 0x7f1958074ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1958090e90 0x7f1958164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 --2- 172.21.15.43:0/3911692508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195806f110 0x7f1958090860 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 >> 172.21.15.43:0/3911692508 conn(0x7f19580b23d0 msgr2=0x7f195815d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 shutdown_connections 2024-09-17T05:30:25.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.440+0000 7f195da00640 1 -- 172.21.15.43:0/3911692508 wait complete. 2024-09-17T05:30:25.467 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-09-17T05:30:25.467 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-09-17T05:30:25.467 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:25.467 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:25.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:25.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:25.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:25.520 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:25.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:25.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 -- 172.21.15.43:0/3135809711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c15fe10 msgr2=0x7fce2c0b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 --2- 172.21.15.43:0/3135809711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c15fe10 0x7fce2c0b4b80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0d0190 tx=0x7fce2001c6e0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 -- 172.21.15.43:0/3135809711 shutdown_connections 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 --2- 172.21.15.43:0/3135809711 >> v2:172.21.15.62:3300/0 conn(0x7fce2c06fc90 0x7fce2c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 --2- 172.21.15.43:0/3135809711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c15fe10 0x7fce2c0b4b80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 -- 172.21.15.43:0/3135809711 >> 172.21.15.43:0/3135809711 conn(0x7fce2c0b23d0 msgr2=0x7fce2c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 -- 172.21.15.43:0/3135809711 shutdown_connections 2024-09-17T05:30:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.772+0000 7fce34000640 1 -- 172.21.15.43:0/3135809711 wait complete. 2024-09-17T05:30:25.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 Processor -- start 2024-09-17T05:30:25.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 -- start start 2024-09-17T05:30:25.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c06fc90 0x7fce2c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c15fe10 0x7fce2c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c16a020 con 0x7fce2c0730b0 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c169ea0 con 0x7fce2c06fc90 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce34000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c16a1a0 con 0x7fce2c15fe10 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57228/0 (socket says 172.21.15.43:57228) 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 -- 172.21.15.43:0/3770236246 learned_addr learned my addr 172.21.15.43:0/3770236246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce31600640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c06fc90 0x7fce2c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce32000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c15fe10 0x7fce2c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 -- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c15fe10 msgr2=0x7fce2c152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c15fe10 0x7fce2c152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 -- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c06fc90 msgr2=0x7fce2c1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c06fc90 0x7fce2c1f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 -- 172.21.15.43:0/3770236246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c155ef0 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.776+0000 7fce30c00640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fce18005d10 tx=0x7fce1800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18017070 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce18009b20 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c1561d0 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18005020 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c1f7180 con 0x7fce2c0730b0 2024-09-17T05:30:25.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c0700a0 con 0x7fce2c0730b0 2024-09-17T05:30:25.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce180051c0 con 0x7fce2c0730b0 2024-09-17T05:30:25.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7fce18094550 con 0x7fce2c0730b0 2024-09-17T05:30:25.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.780+0000 7fce31600640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.784+0000 7fce31600640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 0x7fce0c075710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fce200af000 tx=0x7fce2001af20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.784+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce1805ec50 con 0x7fce2c0730b0 2024-09-17T05:30:25.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:25.996+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fce2c0b4b60 con 0x7fce2c0730b0 2024-09-17T05:30:26.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.236+0000 7fce29600640 1 -- 172.21.15.43:0/3770236246 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7fce18063680 con 0x7fce2c0730b0 2024-09-17T05:30:26.318 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 msgr2=0x7fce0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 0x7fce0c075710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fce200af000 tx=0x7fce2001af20 comp rx=0 tx=0).stop 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 msgr2=0x7fce2c150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fce18005d10 tx=0x7fce1800dbb0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 shutdown_connections 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fce2c15fe10 0x7fce2c152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c0730b0 0x7fce2c150970 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 --2- 172.21.15.43:0/3770236246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce2c06fc90 0x7fce2c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 >> 172.21.15.43:0/3770236246 conn(0x7fce2c0b23d0 msgr2=0x7fce2c0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:26.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 shutdown_connections 2024-09-17T05:30:26.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.240+0000 7fce34000640 1 -- 172.21.15.43:0/3770236246 wait complete. 2024-09-17T05:30:26.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 -- 172.21.15.43:0/3604850400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b006f940 msgr2=0x7fb9b0071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/3604850400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b006f940 0x7fb9b0071150 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0162580 tx=0x7fb9a001c770 comp rx=0 tx=0).stop 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 -- 172.21.15.43:0/3604850400 shutdown_connections 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/3604850400 >> v2:172.21.15.62:3300/0 conn(0x7fb9b00b4570 0x7fb9b00b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/3604850400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b006f940 0x7fb9b0071150 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 -- 172.21.15.43:0/3604850400 >> 172.21.15.43:0/3604850400 conn(0x7fb9b00b2810 msgr2=0x7fb9b00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 -- 172.21.15.43:0/3604850400 shutdown_connections 2024-09-17T05:30:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.476+0000 7fb9b5c00640 1 -- 172.21.15.43:0/3604850400 wait complete. 2024-09-17T05:30:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 Processor -- start 2024-09-17T05:30:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 -- start start 2024-09-17T05:30:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb9b006f940 0x7fb9b0157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b0160210 0x7fb9b0158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b016c020 con 0x7fb9b00b4570 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b016bea0 con 0x7fb9b0160210 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9b5c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b016c1a0 con 0x7fb9b006f940 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57248/0 (socket says 172.21.15.43:57248) 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 -- 172.21.15.43:0/4057950692 learned_addr learned my addr 172.21.15.43:0/4057950692 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9aea00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb9b006f940 0x7fb9b0157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9af400640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b0160210 0x7fb9b0158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 -- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb9b006f940 msgr2=0x7fb9b0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb9b006f940 0x7fb9b0157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 -- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b0160210 msgr2=0x7fb9b0158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b0160210 0x7fb9b0158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.480+0000 7fb9a6000640 1 -- 172.21.15.43:0/4057950692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b006a8d0 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9a6000640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb998005bd0 tx=0x7fb99800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998004230 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9980044c0 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b014c040 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998009a80 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b014c2e0 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b0071150 con 0x7fb9b00b4570 2024-09-17T05:30:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.484+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb998003600 con 0x7fb9b00b4570 2024-09-17T05:30:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.488+0000 7fb9a7400640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 0x7fb98c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.488+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7fb9980948a0 con 0x7fb9b00b4570 2024-09-17T05:30:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.488+0000 7fb9aea00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 0x7fb98c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.488+0000 7fb9aea00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 0x7fb98c075740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00af000 tx=0x7fb9a00b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.492+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb998017020 con 0x7fb9b00b4570 2024-09-17T05:30:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:26.728+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb9b0160b10 con 0x7fb9b00b4570 2024-09-17T05:30:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9a7400640 1 -- 172.21.15.43:0/4057950692 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v116) ==== 101+0+0 (secure 0 0 0) 0x7fb99805f020 con 0x7fb9b00b4570 2024-09-17T05:30:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:30:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 msgr2=0x7fb98c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 0x7fb98c075740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00af000 tx=0x7fb9a00b2040 comp rx=0 tx=0).stop 2024-09-17T05:30:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 msgr2=0x7fb9b0158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb998005bd0 tx=0x7fb99800db90 comp rx=0 tx=0).stop 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 shutdown_connections 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb98c073230 0x7fb98c075740 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b0160210 0x7fb9b0158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b4570 0x7fb9b0158520 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 --2- 172.21.15.43:0/4057950692 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb9b006f940 0x7fb9b0157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 >> 172.21.15.43:0/4057950692 conn(0x7fb9b00b2810 msgr2=0x7fb9b00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 shutdown_connections 2024-09-17T05:30:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.240+0000 7fb9b5c00640 1 -- 172.21.15.43:0/4057950692 wait complete. 2024-09-17T05:30:27.271 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 Processor -- start 2024-09-17T05:30:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 -- start start 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eded015b0 0x555edec8ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec8bf40 0x555edec4a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555edeb98a00 con 0x555edec4f9c0 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555edebf6f40 con 0x555eded015b0 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f325c7ad080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555edec90e50 con 0x555edec8bf40 2024-09-17T05:30:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f3255a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f3255a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57262/0 (socket says 172.21.15.43:57262) 2024-09-17T05:30:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f3255a00640 1 -- 172.21.15.43:0/1393549026 learned_addr learned my addr 172.21.15.43:0/1393549026 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f3256400640 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec8bf40 0x555edec4a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.328+0000 7f3255000640 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eded015b0 0x555edec8ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 -- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec8bf40 msgr2=0x555edec4a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec8bf40 0x555edec4a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 -- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eded015b0 msgr2=0x555edec8ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eded015b0 0x555edec8ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 -- 172.21.15.43:0/1393549026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555edeb71c10 con 0x555edec4f9c0 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f3255a00640 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f324800c670 tx=0x7f324800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c60f837d332795 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f324f400640 1 -- 172.21.15.43:0/1393549026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324800add0 con 0x555edec4f9c0 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f324f400640 1 -- 172.21.15.43:0/1393549026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3248008450 con 0x555edec4f9c0 2024-09-17T05:30:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f324f400640 1 -- 172.21.15.43:0/1393549026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32480088c0 con 0x555edec4f9c0 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- 172.21.15.43:0/1393549026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 msgr2=0x555eded00810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f324800c670 tx=0x7f324800cb30 comp rx=0 tx=0).stop 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- 172.21.15.43:0/1393549026 shutdown_connections 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eded015b0 0x555edec8ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec8bf40 0x555edec4a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 --2- 172.21.15.43:0/1393549026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555edec4f9c0 0x555eded00810 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- 172.21.15.43:0/1393549026 >> 172.21.15.43:0/1393549026 conn(0x555edec36d80 msgr2=0x555eded20b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- 172.21.15.43:0/1393549026 shutdown_connections 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- 172.21.15.43:0/1393549026 wait complete. 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 Processor -- start 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.332+0000 7f325c7ad080 1 -- start start 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec4f9c0 0x555eded1eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555edec8bf40 0x555edecfeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555eded1e4f0 con 0x555eded015b0 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555eded002d0 con 0x555edec8bf40 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555eded00570 con 0x555edec4f9c0 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57278/0 (socket says 172.21.15.43:57278) 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 learned_addr learned my addr 172.21.15.43:0/3119792966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec4f9c0 0x555eded1eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555edec8bf40 0x555edecfeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec4f9c0 msgr2=0x555eded1eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555edec4f9c0 0x555eded1eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555edec8bf40 msgr2=0x555edecfeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555edec8bf40 0x555edecfeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ededbe800 con 0x555eded015b0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f3256400640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f325000a600 tx=0x7f32500aa490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f324e000640 1 -- 172.21.15.43:0/3119792966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32500ba030 con 0x555eded015b0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f324e000640 1 -- 172.21.15.43:0/3119792966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32500aaaf0 con 0x555eded015b0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f324e000640 1 -- 172.21.15.43:0/3119792966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32500af040 con 0x555eded015b0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555ededbe560 con 0x555eded015b0 2024-09-17T05:30:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.336+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ededbc510 con 0x555eded015b0 2024-09-17T05:30:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f324e000640 1 -- 172.21.15.43:0/3119792966 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32500bd030 con 0x555eded015b0 2024-09-17T05:30:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f324e000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 0x7f32380740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f324e000640 1 -- 172.21.15.43:0/3119792966 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f325012d9d0 con 0x555eded015b0 2024-09-17T05:30:27.345 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 0x7f32380740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555ededbde30 0x555ededbc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededbcef0 con 0x555ededbde30 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555ededbde30 0x555ededbc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 0x7f32380740f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f324800c670 tx=0x7f3248002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.340+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555ededbde30 0x555ededbc9b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v94'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededbd2c0 con 0x555ededbde30 2024-09-17T05:30:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v94'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc6a90 con 0x555ededbde30 2024-09-17T05:30:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v94'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.351 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.344+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc7e40 con 0x555ededbde30 2024-09-17T05:30:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.348+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v94'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.348+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc81e0 con 0x555ededbde30 2024-09-17T05:30:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.348+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v94'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.348+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc6e30 con 0x555ededbde30 2024-09-17T05:30:27.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v94'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc7220 con 0x555ededbde30 2024-09-17T05:30:27.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v94'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededc7610 con 0x555ededbde30 2024-09-17T05:30:27.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v94'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.352+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc7610 con 0x555ededbde30 2024-09-17T05:30:27.360 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:27.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v94'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v94'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v94'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.356+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v94'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v94'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v94'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.360+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.368 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.364+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v94'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.364+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.364+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v94'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.364+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v94'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v94'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v94'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.368+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v94'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v94'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.378 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v94'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.372+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.376+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v94'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.376+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.376+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v94'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.376+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v94'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v94'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v94'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.380+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.388 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.384+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v94'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.384+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.384+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v94'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.384+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v94'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededc75c0 con 0x555ededbde30 2024-09-17T05:30:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v94'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555ededc79c0 0x555ededcaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcafe0 con 0x555ededc79c0 2024-09-17T05:30:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555ededc79c0 0x555ededcaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.388+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555ededc79c0 0x555ededcaaa0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v94'108 uv90 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededcb380 con 0x555ededc79c0 2024-09-17T05:30:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v94'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcb720 con 0x555ededc79c0 2024-09-17T05:30:27.398 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v94'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.392+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcbf70 con 0x555ededc79c0 2024-09-17T05:30:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v94'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcc310 con 0x555ededc79c0 2024-09-17T05:30:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v94'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcc6b0 con 0x555ededc79c0 2024-09-17T05:30:27.402 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v94'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.396+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcca50 con 0x555ededc79c0 2024-09-17T05:30:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v94'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcce40 con 0x555ededc79c0 2024-09-17T05:30:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v94'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcce40 con 0x555ededc79c0 2024-09-17T05:30:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000001 [create] v94'116 uv100 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.400+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.404+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000001 [delete] v94'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.404+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.409 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.404+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v94'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.404+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v94'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v94'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 606.00000000 [create] v94'121 uv106 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.408+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 606.00000000 [delete] v94'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.417 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v94'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcde00 con 0x555ededc79c0 2024-09-17T05:30:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v94'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.412+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededcde00 con 0x555ededc79c0 2024-09-17T05:30:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v94'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededccdf0 con 0x555ededc79c0 2024-09-17T05:30:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v94'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededc79c0 2024-09-17T05:30:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555ededcd4c0 0x555ededcd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd01d0 con 0x555ededcd4c0 2024-09-17T05:30:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555ededcd4c0 0x555ededcd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.416+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555ededcd4c0 0x555ededcd8d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(51 607.00000000 [create] v94'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd01d0 con 0x555ededcd4c0 2024-09-17T05:30:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(52 607.00000000 [delete] v94'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd01d0 con 0x555ededcd4c0 2024-09-17T05:30:27.426 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(53 607.00000000 [create] v94'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.420+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd15d0 con 0x555ededcd4c0 2024-09-17T05:30:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v94'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd15d0 con 0x555ededcd4c0 2024-09-17T05:30:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v94'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd05c0 con 0x555ededcd4c0 2024-09-17T05:30:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v94'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededcd4c0 2024-09-17T05:30:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.424+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd05c0 con 0x555ededbde30 2024-09-17T05:30:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.428+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v94'21 uv20 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.428+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd05c0 con 0x555ededbde30 2024-09-17T05:30:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.428+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v94'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.428+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd05c0 con 0x555ededbde30 2024-09-17T05:30:27.436 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.432+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v94'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.432+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd05c0 con 0x555ededbde30 2024-09-17T05:30:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.432+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v94'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3244010660 con 0x555ededbde30 2024-09-17T05:30:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.432+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555ededd0ad0 0x555ededd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.432+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd34c0 con 0x555ededd0ad0 2024-09-17T05:30:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f3256400640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555ededd0ad0 0x555ededd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f3256400640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555ededd0ad0 0x555ededd2ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(61 608.00000000 [create] v94'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd39b0 con 0x555ededd0ad0 2024-09-17T05:30:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(62 608.00000000 [delete] v94'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.436+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd3e90 con 0x555ededd0ad0 2024-09-17T05:30:27.444 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(63 608.00000000 [create] v94'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd5740 con 0x555ededd0ad0 2024-09-17T05:30:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v94'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd5ae0 con 0x555ededd0ad0 2024-09-17T05:30:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v94'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.440+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4730 con 0x555ededd0ad0 2024-09-17T05:30:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v94'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555ededd5030 0x555ededd7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd83d0 con 0x555ededd5030 2024-09-17T05:30:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555ededd5030 0x555ededd7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f3255a00640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555ededd5030 0x555ededd7e90 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(67 604.00000000 [create] v94'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.444+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd83d0 con 0x555ededd5030 2024-09-17T05:30:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(68 604.00000000 [delete] v94'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd83d0 con 0x555ededd5030 2024-09-17T05:30:27.453 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(69 604.00000000 [create] v94'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd97d0 con 0x555ededd5030 2024-09-17T05:30:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v94'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.448+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd97d0 con 0x555ededd5030 2024-09-17T05:30:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v94'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v94'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(73 500.00000000 [create] v94'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.452+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(74 500.00000000 [delete] v94'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.461 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(75 500.00000000 [create] v94'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v94'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.456+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd87c0 con 0x555ededd5030 2024-09-17T05:30:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f3255a00640 1 -- 172.21.15.43:0/3119792966 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v94'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f324801d070 con 0x555ededd5030 2024-09-17T05:30:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555ededd8cd0 0x555ededdb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdb760 con 0x555ededd8cd0 2024-09-17T05:30:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555ededd8cd0 0x555ededdb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f3255000640 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555ededd8cd0 0x555ededdb0f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(78 mds_snaptable [create] v94'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.460+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.464+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(79 mds_snaptable [delete] v94'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.464+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.464+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(80 mds_snaptable [create] v94'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.470 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.464+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v94'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v94'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededbde30 2024-09-17T05:30:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(83 605.00000000 [create] v94'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.468+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededbde30 2024-09-17T05:30:27.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(84 605.00000000 [delete] v94'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededbde30 2024-09-17T05:30:27.477 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(85 605.00000000 [create] v94'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdcb40 con 0x555ededbde30 2024-09-17T05:30:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v94'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.472+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdcb40 con 0x555ededbde30 2024-09-17T05:30:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v94'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededbde30 2024-09-17T05:30:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v94'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededbde30 2024-09-17T05:30:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededcd4c0 2024-09-17T05:30:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v94'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.476+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededcd4c0 2024-09-17T05:30:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.480+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v94'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.480+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededcd4c0 2024-09-17T05:30:27.485 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.480+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v94'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.480+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v94'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v94'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v94'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.484+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.488+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v94'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.488+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.488+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v94'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.494 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:27.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.488+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededd4320 con 0x555ededcd4c0 2024-09-17T05:30:27.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.488+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v94'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.492+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdea10 con 0x555ededcd4c0 2024-09-17T05:30:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.492+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v94'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededcd4c0 2024-09-17T05:30:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.492+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededde9d0 con 0x555ededd0ad0 2024-09-17T05:30:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.492+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v94'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.492+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.496+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v94'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.496+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.504 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.500+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v94'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.500+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.500+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v94'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.500+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.504+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v94'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.504+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.504+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v94'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.504+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.511 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v94'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdcff0 con 0x555ededd0ad0 2024-09-17T05:30:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v94'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v94'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.508+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd0ad0 2024-09-17T05:30:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.512+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v94'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f32500c6070 con 0x555ededd0ad0 2024-09-17T05:30:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.512+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.512+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v94'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.512+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v94'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.521 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v94'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v94'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.516+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v94'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v94'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v94'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.520+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.528 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:27.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v94'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v94'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v94'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.524+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v94'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v94'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.534 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:27.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v94'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.528+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v94'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v94'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdbb30 con 0x555ededd8cd0 2024-09-17T05:30:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v94'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.532+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x555ededdc3a0 con 0x555ededd8cd0 2024-09-17T05:30:27.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f3255000640 1 -- 172.21.15.43:0/3119792966 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v94'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3244027040 con 0x555ededd8cd0 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555ededc79c0 msgr2=0x555ededcaaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x555ededc79c0 0x555ededcaaa0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555ededd5030 msgr2=0x555ededd7e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x555ededd5030 0x555ededd7e90 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555ededd8cd0 msgr2=0x555ededdb0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x555ededd8cd0 0x555ededdb0f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555ededcd4c0 msgr2=0x555ededcd8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555ededcd4c0 0x555ededcd8d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555ededbde30 msgr2=0x555ededbc9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x555ededbde30 0x555ededbc9b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 reap_dead start 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555ededd0ad0 msgr2=0x555ededd2ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x555ededd0ad0 0x555ededd2ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f3256400640 1 -- 172.21.15.43:0/3119792966 reap_dead start 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 msgr2=0x7f32380740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 0x7f32380740f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f324800c670 tx=0x7f3248002fc0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 msgr2=0x555edecff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f325000a600 tx=0x7f32500aa490 comp rx=0 tx=0).stop 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 shutdown_connections 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3238071be0 0x7f32380740f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 --2- 172.21.15.43:0/3119792966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555eded015b0 0x555edecff060 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 >> 172.21.15.43:0/3119792966 conn(0x555edec36d80 msgr2=0x555eded281d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 shutdown_connections 2024-09-17T05:30:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.536+0000 7f325c7ad080 1 -- 172.21.15.43:0/3119792966 wait complete. 2024-09-17T05:30:27.545 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-09-17T05:30:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 Processor -- start 2024-09-17T05:30:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 -- start start 2024-09-17T05:30:27.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6fd1e40 0x5639f6f60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5639f6ec7f40 con 0x5639f6f1e4d0 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5639f6ed4d00 con 0x5639f6fd1e40 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d3b216080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5639f6e69a00 con 0x5639f6f4acb0 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57298/0 (socket says 172.21.15.43:57298) 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 -- 172.21.15.43:0/3682599219 learned_addr learned my addr 172.21.15.43:0/3682599219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34e00640 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6fd1e40 0x5639f6f60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d33a00640 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 -- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 msgr2=0x5639f6fef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 -- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6fd1e40 msgr2=0x5639f6f60b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:27.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6fd1e40 0x5639f6f60b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 -- 172.21.15.43:0/3682599219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f6ff0870 con 0x5639f6f1e4d0 2024-09-17T05:30:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d34400640 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b200 tx=0x7f5d2400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f659e60936b72a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d32600640 1 -- 172.21.15.43:0/3682599219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24016070 con 0x5639f6f1e4d0 2024-09-17T05:30:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d32600640 1 -- 172.21.15.43:0/3682599219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d2400d4c0 con 0x5639f6f1e4d0 2024-09-17T05:30:27.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.640+0000 7f5d32600640 1 -- 172.21.15.43:0/3682599219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400d710 con 0x5639f6f1e4d0 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- 172.21.15.43:0/3682599219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 msgr2=0x5639f6ff14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b200 tx=0x7f5d2400b6c0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- 172.21.15.43:0/3682599219 shutdown_connections 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6fd1e40 0x5639f6f60b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- 172.21.15.43:0/3682599219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6f1e4d0 0x5639f6ff14f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- 172.21.15.43:0/3682599219 >> 172.21.15.43:0/3682599219 conn(0x5639f6f07d80 msgr2=0x5639f6feea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- 172.21.15.43:0/3682599219 shutdown_connections 2024-09-17T05:30:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- 172.21.15.43:0/3682599219 wait complete. 2024-09-17T05:30:27.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 Processor -- start 2024-09-17T05:30:27.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- start start 2024-09-17T05:30:27.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6f1e4d0 0x5639f6feea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fd1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6fd1e40 0x5639f6f41d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5639f6ff0b10 con 0x5639f6fd1e40 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5639f6ff0db0 con 0x5639f6f1e4d0 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d3b216080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5639f6ff1050 con 0x5639f6f4acb0 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6f1e4d0 0x5639f6feea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6fd1e40 0x5639f6f41d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6f1e4d0 0x5639f6feea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:38746/0 (socket says 172.21.15.43:38746) 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34400640 1 -- 172.21.15.43:0/1737739940 learned_addr learned my addr 172.21.15.43:0/1737739940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d33a00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fd1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 -- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 msgr2=0x5639f6fd1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5639f6f4acb0 0x5639f6fd1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 -- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6f1e4d0 msgr2=0x5639f6feea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639f6f1e4d0 0x5639f6feea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.644+0000 7f5d34e00640 1 -- 172.21.15.43:0/1737739940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f6fef0b0 con 0x5639f6fd1e40 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d34e00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6fd1e40 0x5639f6f41d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ad000 tx=0x7f5d2c0afc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 -- 172.21.15.43:0/1737739940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0afe10 con 0x5639f6fd1e40 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 -- 172.21.15.43:0/1737739940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d2c0a7990 con 0x5639f6fd1e40 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639f6fef350 con 0x5639f6fd1e40 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 -- 172.21.15.43:0/1737739940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c004020 con 0x5639f6fd1e40 2024-09-17T05:30:27.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f6f3e2f0 con 0x5639f6fd1e40 2024-09-17T05:30:27.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 -- 172.21.15.43:0/1737739940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d2c0c5020 con 0x5639f6fd1e40 2024-09-17T05:30:27.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d14071be0 0x7f5d140740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d31200640 1 -- 172.21.15.43:0/1737739940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f5d2c1314c0 con 0x5639f6fd1e40 2024-09-17T05:30:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d3b216080 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5639f6fd3710 0x5639f6f3e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x5639f6f3ecd0 con 0x5639f6fd3710 2024-09-17T05:30:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d34400640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d14071be0 0x7f5d140740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.648+0000 7f5d33a00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5639f6fd3710 0x5639f6f3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d33a00640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5639f6fd3710 0x5639f6f3e790 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d34400640 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d14071be0 0x7f5d140740f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5d24014040 tx=0x7f5d24012040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d33a00640 1 -- 172.21.15.43:0/1737739940 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [delete] v94'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d2000d040 con 0x5639f6fd3710 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5639f6fd3710 msgr2=0x5639f6f3e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5639f6fd3710 0x5639f6f3e790 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d14071be0 msgr2=0x7f5d140740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d14071be0 0x7f5d140740f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5d24014040 tx=0x7f5d24012040 comp rx=0 tx=0).stop 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6fd1e40 msgr2=0x5639f6f41d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 --2- 172.21.15.43:0/1737739940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639f6fd1e40 0x5639f6f41d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ad000 tx=0x7f5d2c0afc20 comp rx=0 tx=0).stop 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d34e00640 1 -- 172.21.15.43:0/1737739940 reap_dead start 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 shutdown_connections 2024-09-17T05:30:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 >> 172.21.15.43:0/1737739940 conn(0x5639f6f07d80 msgr2=0x5639f6f1cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 shutdown_connections 2024-09-17T05:30:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.652+0000 7f5d3b216080 1 -- 172.21.15.43:0/1737739940 wait complete. 2024-09-17T05:30:27.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- 172.21.15.43:0/749856289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00737c0 msgr2=0x7f76b00700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 --2- 172.21.15.43:0/749856289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00737c0 0x7f76b00700a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009bf0 tx=0x7f76a801ca80 comp rx=0 tx=0).stop 2024-09-17T05:30:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- 172.21.15.43:0/749856289 shutdown_connections 2024-09-17T05:30:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 --2- 172.21.15.43:0/749856289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00737c0 0x7f76b00700a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- 172.21.15.43:0/749856289 >> 172.21.15.43:0/749856289 conn(0x7f76b00b23d0 msgr2=0x7f76b00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- 172.21.15.43:0/749856289 shutdown_connections 2024-09-17T05:30:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- 172.21.15.43:0/749856289 wait complete. 2024-09-17T05:30:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 Processor -- start 2024-09-17T05:30:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.820+0000 7f76b8000640 1 -- start start 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b006fc90 0x7f76b0156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76b015fe10 0x7f76b0099390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76b016b640 con 0x7f76b00ad820 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76b016b4c0 con 0x7f76b006fc90 2024-09-17T05:30:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76b016b7c0 con 0x7f76b015fe10 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b006fc90 0x7f76b0156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57324/0 (socket says 172.21.15.43:57324) 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 -- 172.21.15.43:0/582703462 learned_addr learned my addr 172.21.15.43:0/582703462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b4c00640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76b015fe10 0x7f76b0099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 -- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76b015fe10 msgr2=0x7f76b0099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76b015fe10 0x7f76b0099390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 -- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b006fc90 msgr2=0x7f76b0156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b006fc90 0x7f76b0156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 -- 172.21.15.43:0/582703462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b0155550 con 0x7f76b00ad820 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b6000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f76a801c4f0 tx=0x7f76a8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a80b2040 con 0x7f76b00ad820 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76a80bb070 con 0x7f76b00ad820 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004530 con 0x7f76b00ad820 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b015bb00 con 0x7f76b00ad820 2024-09-17T05:30:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01f91c0 con 0x7f76b00ad820 2024-09-17T05:30:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.824+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b01f9450 con 0x7f76b00ad820 2024-09-17T05:30:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.828+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76a80046d0 con 0x7f76b00ad820 2024-09-17T05:30:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.828+0000 7f76ad600640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 0x7f7690075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.828+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6139+0+0 (secure 0 0 0) 0x7f76a812ec50 con 0x7f76b00ad820 2024-09-17T05:30:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.828+0000 7f76b5600640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 0x7f7690075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.828+0000 7f76b5600640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 0x7f7690075740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001870 tx=0x7f76a4001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:27.832+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76a8134020 con 0x7f76b00ad820 2024-09-17T05:30:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.044+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f76b00adc90 con 0x7f76b00ad820 2024-09-17T05:30:28.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76ad600640 1 -- 172.21.15.43:0/582703462 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v117) ==== 162+0+0 (secure 0 0 0) 0x7f76a8016790 con 0x7f76b00ad820 2024-09-17T05:30:28.219 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 msgr2=0x7f7690075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 0x7f7690075740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001870 tx=0x7f76a4001380 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 msgr2=0x7f76b00afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f76a801c4f0 tx=0x7f76a8005a60 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 shutdown_connections 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7690073230 0x7f7690075740 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76b00ad820 0x7f76b00afd40 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76b015fe10 0x7f76b0099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 --2- 172.21.15.43:0/582703462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b006fc90 0x7f76b0156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 >> 172.21.15.43:0/582703462 conn(0x7f76b00b23d0 msgr2=0x7f76b0074d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 shutdown_connections 2024-09-17T05:30:28.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.216+0000 7f76b8000640 1 -- 172.21.15.43:0/582703462 wait complete. 2024-09-17T05:30:28.228 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:28.224+0000 7fa5e3a00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-17T05:30:28.229 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:28.224+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:28.229 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:28.224+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:28.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:28.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 -- 172.21.15.43:0/1129198401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cdc004810 msgr2=0x7f8cdc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 --2- 172.21.15.43:0/1129198401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cdc004810 0x7f8cdc004c20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c2e0 tx=0x7f8ce001cf60 comp rx=0 tx=0).stop 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 -- 172.21.15.43:0/1129198401 shutdown_connections 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 --2- 172.21.15.43:0/1129198401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cdc004810 0x7f8cdc004c20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 -- 172.21.15.43:0/1129198401 >> 172.21.15.43:0/1129198401 conn(0x7f8cf80b23d0 msgr2=0x7f8cf80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 -- 172.21.15.43:0/1129198401 shutdown_connections 2024-09-17T05:30:28.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.400+0000 7f8cff800640 1 -- 172.21.15.43:0/1129198401 wait complete. 2024-09-17T05:30:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 Processor -- start 2024-09-17T05:30:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 -- start start 2024-09-17T05:30:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc004810 0x7f8cf809dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf80c99c0 0x7f8cf80c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf80cb860 con 0x7f8cf80740e0 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf80b3470 con 0x7f8cf80c99c0 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cff800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80cb9e0 con 0x7f8cdc004810 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57354/0 (socket says 172.21.15.43:57354) 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 -- 172.21.15.43:0/3837235762 learned_addr learned my addr 172.21.15.43:0/3837235762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:28.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cfce00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc004810 0x7f8cf809dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cfd800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf80c99c0 0x7f8cf80c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 -- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc004810 msgr2=0x7f8cf809dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc004810 0x7f8cf809dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 -- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf80c99c0 msgr2=0x7f8cf80c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf80c99c0 0x7f8cf80c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 -- 172.21.15.43:0/3837235762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf80c5570 con 0x7f8cf80740e0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.404+0000 7f8cefe00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c10 tx=0x7f8ce0006d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00014c0 con 0x7f8cf80740e0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce0001660 con 0x7f8cf80740e0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf80c5850 con 0x7f8cf80740e0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0006330 con 0x7f8cf80740e0 2024-09-17T05:30:28.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf80c5e10 con 0x7f8cf80740e0 2024-09-17T05:30:28.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf80cbf40 con 0x7f8cf80740e0 2024-09-17T05:30:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ce00a5cb0 con 0x7f8cf80740e0 2024-09-17T05:30:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.408+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f8ce012df70 con 0x7f8cf80740e0 2024-09-17T05:30:28.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.412+0000 7f8cfce00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.412+0000 7f8cfce00640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 0x7f8cd4075740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009c20 tx=0x7f8ce80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.416+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ce00bf060 con 0x7f8cf80740e0 2024-09-17T05:30:28.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf80af0c0 con 0x7f8cf80740e0 2024-09-17T05:30:28.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cf5600640 1 -- 172.21.15.43:0/3837235762 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+3556 (secure 0 0 0) 0x7f8ce0016d10 con 0x7f8cf80740e0 2024-09-17T05:30:28.629 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:28.630 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"btime":"2024-09-17T05:30:28:229878+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:28.229812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:28.630 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 118 2024-09-17T05:30:28.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 msgr2=0x7f8cd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 0x7f8cd4075740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009c20 tx=0x7f8ce80091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 msgr2=0x7f8cf80c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c10 tx=0x7f8ce0006d30 comp rx=0 tx=0).stop 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 shutdown_connections 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8cd4073230 0x7f8cd4075740 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cf80c99c0 0x7f8cf80c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80740e0 0x7f8cf80c6610 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 --2- 172.21.15.43:0/3837235762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8cdc004810 0x7f8cf809dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 >> 172.21.15.43:0/3837235762 conn(0x7f8cf80b23d0 msgr2=0x7f8cf80b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 shutdown_connections 2024-09-17T05:30:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.624+0000 7f8cff800640 1 -- 172.21.15.43:0/3837235762 wait complete. 2024-09-17T05:30:28.651 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:28.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 -- 172.21.15.43:0/3632225401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40cd580 msgr2=0x7fdbe4167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 --2- 172.21.15.43:0/3632225401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40cd580 0x7fdbe4167c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009bf0 tx=0x7fdbdc01cb00 comp rx=0 tx=0).stop 2024-09-17T05:30:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 -- 172.21.15.43:0/3632225401 shutdown_connections 2024-09-17T05:30:28.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 --2- 172.21.15.43:0/3632225401 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40cd580 0x7fdbe4167c80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 -- 172.21.15.43:0/3632225401 >> 172.21.15.43:0/3632225401 conn(0x7fdbe40cee80 msgr2=0x7fdbe40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 -- 172.21.15.43:0/3632225401 shutdown_connections 2024-09-17T05:30:28.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.804+0000 7fdbec200640 1 -- 172.21.15.43:0/3632225401 wait complete. 2024-09-17T05:30:28.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 Processor -- start 2024-09-17T05:30:28.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 -- start start 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40b7ce0 0x7fdbe40b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe40cd580 0x7fdbe4167820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe416af80 con 0x7fdbe40d0f50 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe416ae00 con 0x7fdbe40cd580 2024-09-17T05:30:28.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbec200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe416b100 con 0x7fdbe40b7ce0 2024-09-17T05:30:28.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57368/0 (socket says 172.21.15.43:57368) 2024-09-17T05:30:28.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 -- 172.21.15.43:0/3907792942 learned_addr learned my addr 172.21.15.43:0/3907792942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:28.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbe9800640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40b7ce0 0x7fdbe40b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbe8e00640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe40cd580 0x7fdbe4167820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 -- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40b7ce0 msgr2=0x7fdbe40b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40b7ce0 0x7fdbe40b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 -- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe40cd580 msgr2=0x7fdbe4167820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe40cd580 0x7fdbe4167820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 -- 172.21.15.43:0/3907792942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe4167d60 con 0x7fdbe40d0f50 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.808+0000 7fdbea200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01a730 tx=0x7fdbdc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0bc070 con 0x7fdbe40d0f50 2024-09-17T05:30:28.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbdc003cf0 con 0x7fdbe40d0f50 2024-09-17T05:30:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0b2040 con 0x7fdbe40d0f50 2024-09-17T05:30:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe414c490 con 0x7fdbe40d0f50 2024-09-17T05:30:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe415cd00 con 0x7fdbe40d0f50 2024-09-17T05:30:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe40d0840 con 0x7fdbe40d0f50 2024-09-17T05:30:28.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdbdc0bf070 con 0x7fdbe40d0f50 2024-09-17T05:30:28.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 0x7fdbc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.812+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fdbdc12dd20 con 0x7fdbe40d0f50 2024-09-17T05:30:28.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.816+0000 7fdbe9800640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 0x7fdbc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.816+0000 7fdbe9800640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 0x7fdbc4075710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001180 tx=0x7fdbd8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:28.820+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdbdc0166f0 con 0x7fdbe40d0f50 2024-09-17T05:30:28.980 INFO:tasks.ceph.mds.d.smithi062.stderr: -38> 2024-09-17T05:30:24.226+0000 7f4623200640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-17T05:30:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbe40afde0 con 0x7fdbe40d0f50 2024-09-17T05:30:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbe1600640 1 -- 172.21.15.43:0/3907792942 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+3556 (secure 0 0 0) 0x7fdbdc0f84e0 con 0x7fdbe40d0f50 2024-09-17T05:30:29.033 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:29.033 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"btime":"2024-09-17T05:30:28:229878+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:28.229812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 118 2024-09-17T05:30:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 msgr2=0x7fdbc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 0x7fdbc4075710 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8001180 tx=0x7fdbd8009040 comp rx=0 tx=0).stop 2024-09-17T05:30:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 msgr2=0x7fdbe414bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01a730 tx=0x7fdbdc009200 comp rx=0 tx=0).stop 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 shutdown_connections 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbc4073200 0x7fdbc4075710 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe40cd580 0x7fdbe4167820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe40d0f50 0x7fdbe414bf50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 --2- 172.21.15.43:0/3907792942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe40b7ce0 0x7fdbe40b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 >> 172.21.15.43:0/3907792942 conn(0x7fdbe40cee80 msgr2=0x7fdbe40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 shutdown_connections 2024-09-17T05:30:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.028+0000 7fdbec200640 1 -- 172.21.15.43:0/3907792942 wait complete. 2024-09-17T05:30:29.054 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:29.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:29.096 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:30:29.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 -- 172.21.15.43:0/1313508713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 msgr2=0x7fcc0c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 --2- 172.21.15.43:0/1313508713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c06f110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009bc0 tx=0x7fcc0401cb10 comp rx=0 tx=0).stop 2024-09-17T05:30:29.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 -- 172.21.15.43:0/1313508713 shutdown_connections 2024-09-17T05:30:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 --2- 172.21.15.43:0/1313508713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c06f110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 -- 172.21.15.43:0/1313508713 >> 172.21.15.43:0/1313508713 conn(0x7fcc0c0b23d0 msgr2=0x7fcc0c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 -- 172.21.15.43:0/1313508713 shutdown_connections 2024-09-17T05:30:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 -- 172.21.15.43:0/1313508713 wait complete. 2024-09-17T05:30:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.208+0000 7fcc13800640 1 Processor -- start 2024-09-17T05:30:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- start start 2024-09-17T05:30:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c1fc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 0x7fcc0c0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c169dc0 con 0x7fcc0c155d70 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c169c40 con 0x7fcc0c15c7e0 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c169f40 con 0x7fcc0c09dc80 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57372/0 (socket says 172.21.15.43:57372) 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 -- 172.21.15.43:0/4205349791 learned_addr learned my addr 172.21.15.43:0/4205349791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc03e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c1fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc11800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 0x7fcc0c0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 -- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 msgr2=0x7fcc0c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 0x7fcc0c0ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 -- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 msgr2=0x7fcc0c1fc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c1fc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 -- 172.21.15.43:0/4205349791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c1fcd50 con 0x7fcc0c155d70 2024-09-17T05:30:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc11800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 0x7fcc0c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc03e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c1fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc10e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c3f0 tx=0x7fcbfc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc011790 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbfc017070 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc011cc0 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c0cab30 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c0cb0f0 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.212+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c06f110 con 0x7fcc0c155d70 2024-09-17T05:30:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.216+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcbfc01c460 con 0x7fcc0c155d70 2024-09-17T05:30:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.216+0000 7fcc09600640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 0x7fcbe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.216+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fcbfc094c80 con 0x7fcc0c155d70 2024-09-17T05:30:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.216+0000 7fcc03e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 0x7fcbe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.216+0000 7fcc03e00640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 0x7fcbe8075710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c09e110 tx=0x7fcbf4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.220+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcbfc05fb00 con 0x7fcc0c155d70 2024-09-17T05:30:29.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.432+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0c0b0370 con 0x7fcc0c155d70 2024-09-17T05:30:29.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.432+0000 7fcc09600640 1 -- 172.21.15.43:0/4205349791 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+4382 (secure 0 0 0) 0x7fcbfc01ae40 con 0x7fcc0c155d70 2024-09-17T05:30:29.438 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:29.438 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":119,"btime":"2024-09-17T05:30:29:258878+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7393,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3277842592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3277842592},{"type":"v1","addr":"172.21.15.62:6835","nonce":3277842592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:28.229812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 119 2024-09-17T05:30:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 msgr2=0x7fcbe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 0x7fcbe8075710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c09e110 tx=0x7fcbf4009040 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 msgr2=0x7fcc0c153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c3f0 tx=0x7fcbfc00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 shutdown_connections 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbe8073200 0x7fcbe8075710 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc0c15c7e0 0x7fcc0c1fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcc0c09dc80 0x7fcc0c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 --2- 172.21.15.43:0/4205349791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c155d70 0x7fcc0c153d50 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 >> 172.21.15.43:0/4205349791 conn(0x7fcc0c0b23d0 msgr2=0x7fcc0c166910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 shutdown_connections 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.436+0000 7fcc13800640 1 -- 172.21.15.43:0/4205349791 wait complete. 2024-09-17T05:30:29.460 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-09-17T05:30:29.460 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-09-17T05:30:29.461 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:29.461 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:29.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:29.470 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:29.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:29.470 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:29.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:29.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 -- 172.21.15.43:0/4020561037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a40b3af0 msgr2=0x7f49a415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 --2- 172.21.15.43:0/4020561037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a40b3af0 0x7f49a415c7e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4990009840 tx=0x7f499001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 -- 172.21.15.43:0/4020561037 shutdown_connections 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 --2- 172.21.15.43:0/4020561037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a40b3af0 0x7f49a415c7e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 -- 172.21.15.43:0/4020561037 >> 172.21.15.43:0/4020561037 conn(0x7f49a40b23d0 msgr2=0x7f49a40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 -- 172.21.15.43:0/4020561037 shutdown_connections 2024-09-17T05:30:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.720+0000 7f49ac000640 1 -- 172.21.15.43:0/4020561037 wait complete. 2024-09-17T05:30:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 Processor -- start 2024-09-17T05:30:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 -- start start 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a4155d70 0x7f49a4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a4157250 0x7f49a40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4169dc0 con 0x7f49a406f110 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4169c40 con 0x7f49a4155d70 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49ac000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f49a4169f40 con 0x7f49a4157250 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49a9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49a9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57380/0 (socket says 172.21.15.43:57380) 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49a9600640 1 -- 172.21.15.43:0/1074527687 learned_addr learned my addr 172.21.15.43:0/1074527687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49a8c00640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a4155d70 0x7f49a4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.724+0000 7f49aa000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a4157250 0x7f49a40cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 -- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a4157250 msgr2=0x7f49a40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a4157250 0x7f49a40cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 -- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a4155d70 msgr2=0x7f49a4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a4155d70 0x7f49a4156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 -- 172.21.15.43:0/1074527687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a415a450 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a9600640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f499800c460 tx=0x7f499800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998017070 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49980053e0 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a4159e90 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003860 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a409c860 con 0x7f49a406f110 2024-09-17T05:30:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a415c7e0 con 0x7f49a406f110 2024-09-17T05:30:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.728+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f499801a070 con 0x7f49a406f110 2024-09-17T05:30:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.732+0000 7f49a1600640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 0x7f4984075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.732+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f4998093740 con 0x7f49a406f110 2024-09-17T05:30:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.732+0000 7f49a8c00640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 0x7f4984075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.732+0000 7f49a8c00640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 0x7f4984075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f49900ae000 tx=0x7f499001aef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.736+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f499805df00 con 0x7f49a406f110 2024-09-17T05:30:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:29.944+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49a40af690 con 0x7f49a406f110 2024-09-17T05:30:30.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.268+0000 7f49a1600640 1 -- 172.21.15.43:0/1074527687 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) ==== 161+0+0 (secure 0 0 0) 0x7f4998062930 con 0x7f49a406f110 2024-09-17T05:30:30.274 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 msgr2=0x7f4984075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 0x7f4984075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f49900ae000 tx=0x7f499001aef0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 msgr2=0x7f49a4166da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f499800c460 tx=0x7f499800c920 comp rx=0 tx=0).stop 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 shutdown_connections 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4984073200 0x7f4984075710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49a4155d70 0x7f49a4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49a4157250 0x7f49a40cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 --2- 172.21.15.43:0/1074527687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49a406f110 0x7f49a4166da0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 >> 172.21.15.43:0/1074527687 conn(0x7f49a40b23d0 msgr2=0x7f49a40b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 shutdown_connections 2024-09-17T05:30:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.272+0000 7f49ac000640 1 -- 172.21.15.43:0/1074527687 wait complete. 2024-09-17T05:30:30.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 -- 172.21.15.43:0/1135537592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9020003af0 msgr2=0x7f9020003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 --2- 172.21.15.43:0/1135537592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9020003af0 0x7f9020003f60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f902c00c2e0 tx=0x7f902c01cf60 comp rx=0 tx=0).stop 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f903b200640 1 -- 172.21.15.43:0/1135537592 reap_dead start 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 -- 172.21.15.43:0/1135537592 shutdown_connections 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 -- 172.21.15.43:0/1135537592 >> 172.21.15.43:0/1135537592 conn(0x7f903c0cee70 msgr2=0x7f903c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 -- 172.21.15.43:0/1135537592 shutdown_connections 2024-09-17T05:30:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.456+0000 7f9041400640 1 -- 172.21.15.43:0/1135537592 wait complete. 2024-09-17T05:30:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 Processor -- start 2024-09-17T05:30:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 -- start start 2024-09-17T05:30:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9020003af0 0x7f903c1f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f903c14ef70 0x7f903c14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f903c0cdb90 con 0x7f903c14faa0 2024-09-17T05:30:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f903c0b2800 con 0x7f9020003af0 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9041400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f903c0cdd10 con 0x7f903c14ef70 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9031e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9031e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57390/0 (socket says 172.21.15.43:57390) 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f9031e00640 1 -- 172.21.15.43:0/1575506431 learned_addr learned my addr 172.21.15.43:0/1575506431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f903b200640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f903c14ef70 0x7f903c14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.460+0000 7f903a800640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9020003af0 0x7f903c1f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 -- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f903c14ef70 msgr2=0x7f903c14f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f903c14ef70 0x7f903c14f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 -- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9020003af0 msgr2=0x7f903c1f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9020003af0 0x7f903c1f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 -- 172.21.15.43:0/1575506431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c0ca4f0 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9031e00640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9024011800 tx=0x7f9024011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024004460 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9024016070 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902400fcd0 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c0ca7d0 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c0cc810 con 0x7f903c14faa0 2024-09-17T05:30:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903c0cfa90 con 0x7f903c14faa0 2024-09-17T05:30:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.464+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9024018030 con 0x7f903c14faa0 2024-09-17T05:30:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.468+0000 7f9033400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 0x7f9018075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.468+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f9024094c90 con 0x7f903c14faa0 2024-09-17T05:30:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.468+0000 7f903a800640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 0x7f9018075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.468+0000 7f903a800640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 0x7f9018075710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f902c0ac000 tx=0x7f9020005840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.472+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f902405fa00 con 0x7f903c14faa0 2024-09-17T05:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:30.712+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f903c0b3cc0 con 0x7f903c14faa0 2024-09-17T05:30:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.276+0000 7f9033400640 1 -- 172.21.15.43:0/1575506431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v121) ==== 101+0+0 (secure 0 0 0) 0x7f902405f300 con 0x7f903c14faa0 2024-09-17T05:30:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:30:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 msgr2=0x7f9018075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 0x7f9018075710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f902c0ac000 tx=0x7f9020005840 comp rx=0 tx=0).stop 2024-09-17T05:30:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 msgr2=0x7f903c0cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9024011800 tx=0x7f9024011cc0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 shutdown_connections 2024-09-17T05:30:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9018073200 0x7f9018075710 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f903c14ef70 0x7f903c14f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f903c14faa0 0x7f903c0cc2d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 --2- 172.21.15.43:0/1575506431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9020003af0 0x7f903c1f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 >> 172.21.15.43:0/1575506431 conn(0x7f903c0cee70 msgr2=0x7f903c15c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 shutdown_connections 2024-09-17T05:30:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.280+0000 7f9041400640 1 -- 172.21.15.43:0/1575506431 wait complete. 2024-09-17T05:30:31.309 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 Processor -- start 2024-09-17T05:30:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 -- start start 2024-09-17T05:30:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649025c74d0 0x564902697e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564902570f40 con 0x56490267ae40 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56490257dd00 con 0x5649025c74d0 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91b00dc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564902512a00 con 0x5649025f3cb0 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57398/0 (socket says 172.21.15.43:57398) 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 -- 172.21.15.43:0/3890346001 learned_addr learned my addr 172.21.15.43:0/3890346001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a3e00640 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9200640 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649025c74d0 0x564902697e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 -- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025f3cb0 msgr2=0x564902609970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 -- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649025c74d0 msgr2=0x564902697e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649025c74d0 0x564902697e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 -- 172.21.15.43:0/3890346001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56490260ba20 con 0x56490267ae40 2024-09-17T05:30:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a9c00640 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f91a40015d0 tx=0x7f91a4006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac83621a9930a94e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a2a00640 1 -- 172.21.15.43:0/3890346001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4017070 con 0x56490267ae40 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a2a00640 1 -- 172.21.15.43:0/3890346001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a4002790 con 0x56490267ae40 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.368+0000 7f91a2a00640 1 -- 172.21.15.43:0/3890346001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4002c20 con 0x56490267ae40 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- 172.21.15.43:0/3890346001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 msgr2=0x564902609eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f91a40015d0 tx=0x7f91a4006f90 comp rx=0 tx=0).stop 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- 172.21.15.43:0/3890346001 shutdown_connections 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- 172.21.15.43:0/3890346001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649025c74d0 0x564902697e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- 172.21.15.43:0/3890346001 >> 172.21.15.43:0/3890346001 conn(0x5649025b0d80 msgr2=0x5649025f33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- 172.21.15.43:0/3890346001 shutdown_connections 2024-09-17T05:30:31.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- 172.21.15.43:0/3890346001 wait complete. 2024-09-17T05:30:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 Processor -- start 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 -- start start 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025c74d0 0x5649026964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.372+0000 7f91b00dc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91b00dc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5649025c2e70 con 0x5649025f3cb0 2024-09-17T05:30:31.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91b00dc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5649025c2bd0 con 0x56490267ae40 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91b00dc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5649025c3110 con 0x5649025c74d0 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a9200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025c74d0 0x5649026964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a9200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025c74d0 0x5649026964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:37176/0 (socket says 172.21.15.43:37176) 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 learned_addr learned my addr 172.21.15.43:0/294298176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a9c00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025c74d0 msgr2=0x5649026964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5649025c74d0 0x5649026964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490267ae40 msgr2=0x564902609eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56490267ae40 0x564902609eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564902723550 con 0x5649025f3cb0 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 0x564902609970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x56490267ad10 tx=0x7f91940a7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a1600640 1 -- 172.21.15.43:0/294298176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940b0680 con 0x5649025f3cb0 2024-09-17T05:30:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a1600640 1 -- 172.21.15.43:0/294298176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91940b0820 con 0x5649025f3cb0 2024-09-17T05:30:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91a1600640 1 -- 172.21.15.43:0/294298176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91940b0c70 con 0x5649025f3cb0 2024-09-17T05:30:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5649027237f0 con 0x5649025f3cb0 2024-09-17T05:30:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.376+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564902723d30 con 0x5649025f3cb0 2024-09-17T05:30:31.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a1600640 1 -- 172.21.15.43:0/294298176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91940c3020 con 0x5649025f3cb0 2024-09-17T05:30:31.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a1600640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 0x7f9188074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a1600640 1 -- 172.21.15.43:0/294298176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f9194130e30 con 0x5649025f3cb0 2024-09-17T05:30:31.387 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 0x5649027246e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902724c20 con 0x56490260b020 2024-09-17T05:30:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 0x7f9188074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a9c00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 0x5649027246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 0x7f9188074120 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f919c002ef0 tx=0x7f919c002ac0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.380+0000 7f91a9c00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 0x5649027246e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.384+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v95'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.384+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902724ff0 con 0x56490260b020 2024-09-17T05:30:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.384+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v95'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.384+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273d870 con 0x56490260b020 2024-09-17T05:30:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.388+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v95'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.392 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:31.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.388+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273ec20 con 0x56490260b020 2024-09-17T05:30:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.388+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v95'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.388+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273efc0 con 0x56490260b020 2024-09-17T05:30:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v95'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dc10 con 0x56490260b020 2024-09-17T05:30:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v95'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e000 con 0x56490260b020 2024-09-17T05:30:31.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v95'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.392+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490273e000 con 0x56490260b020 2024-09-17T05:30:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.396+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v95'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.396+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.396+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v95'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.402 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.396+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v95'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v95'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v95'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.400+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.404+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v95'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.404+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.404+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v95'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.404+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v95'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.412 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:31.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v95'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v95'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.408+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273dfb0 con 0x56490260b020 2024-09-17T05:30:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.412+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v95'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.412+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e3e0 con 0x56490260b020 2024-09-17T05:30:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.412+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v95'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.412+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e830 con 0x56490260b020 2024-09-17T05:30:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v95'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490273e830 con 0x56490260b020 2024-09-17T05:30:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v95'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e390 con 0x56490260b020 2024-09-17T05:30:31.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v95'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.423 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.416+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e780 con 0x56490260b020 2024-09-17T05:30:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.420+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v95'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.420+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e780 con 0x56490260b020 2024-09-17T05:30:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.420+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v95'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.420+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v95'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v95'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v95'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.424+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v95'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.432 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:31.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v95'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v95'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.428+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490273e230 con 0x56490260b020 2024-09-17T05:30:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v95'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56490273f360 0x564902741780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902741cc0 con 0x56490273f360 2024-09-17T05:30:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56490273f360 0x564902741780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56490273f360 0x564902741780 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v95'128 uv112 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.432+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902742060 con 0x56490273f360 2024-09-17T05:30:31.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.436+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v95'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.436+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902742400 con 0x56490273f360 2024-09-17T05:30:31.442 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:31.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.436+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v95'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.436+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902742c50 con 0x56490273f360 2024-09-17T05:30:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v95'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902742ff0 con 0x56490273f360 2024-09-17T05:30:31.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v95'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743390 con 0x56490273f360 2024-09-17T05:30:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v95'133 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.440+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v95'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.449 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v95'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v95'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.444+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.448+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v95'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.448+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.448+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v95'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.454 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.448+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902747900 con 0x56490273f360 2024-09-17T05:30:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v95'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902747900 con 0x56490273f360 2024-09-17T05:30:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v95'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 606.00000000 [create] v95'141 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.452+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 606.00000000 [delete] v95'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v95'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.461 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902744740 con 0x56490273f360 2024-09-17T05:30:31.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v95'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902744740 con 0x56490273f360 2024-09-17T05:30:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.456+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v95'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902743730 con 0x56490273f360 2024-09-17T05:30:31.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v95'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490273f360 2024-09-17T05:30:31.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564902743db0 0x5649027461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027467a0 con 0x564902743db0 2024-09-17T05:30:31.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564902743db0 0x5649027461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564902743db0 0x5649027461d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(51 607.00000000 [create] v95'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.460+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902746c60 con 0x564902743db0 2024-09-17T05:30:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(52 607.00000000 [delete] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902747140 con 0x564902743db0 2024-09-17T05:30:31.469 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(53 607.00000000 [create] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027489f0 con 0x564902743db0 2024-09-17T05:30:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.464+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902748d90 con 0x564902743db0 2024-09-17T05:30:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.468+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.468+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027479e0 con 0x564902743db0 2024-09-17T05:30:31.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.468+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.468+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027479e0 con 0x56490260b020 2024-09-17T05:30:31.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v95'25 uv24 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027479e0 con 0x56490260b020 2024-09-17T05:30:31.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v95'26 uv26 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027479e0 con 0x56490260b020 2024-09-17T05:30:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v95'27 uv27 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.480 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.472+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027479e0 con 0x56490260b020 2024-09-17T05:30:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.476+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v95'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.476+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 0x56490274b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.476+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274b680 con 0x5649027482e0 2024-09-17T05:30:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.476+0000 7f91a9c00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 0x56490274b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.476+0000 7f91a9c00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 0x56490274b140 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(61 608.00000000 [create] v95'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490274b680 con 0x5649027482e0 2024-09-17T05:30:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(62 608.00000000 [delete] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274b680 con 0x5649027482e0 2024-09-17T05:30:31.486 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:31.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(63 608.00000000 [create] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.480+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274c690 con 0x5649027482e0 2024-09-17T05:30:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274c690 con 0x5649027482e0 2024-09-17T05:30:31.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274b680 con 0x5649027482e0 2024-09-17T05:30:31.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56490274bd00 0x56490274e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274e6a0 con 0x56490274bd00 2024-09-17T05:30:31.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.484+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56490274bd00 0x56490274e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.488+0000 7f91a9200640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56490274bd00 0x56490274e120 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.488+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(67 604.00000000 [create] v95'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.488+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.488+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(68 604.00000000 [delete] v95'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.488+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(69 604.00000000 [create] v95'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.496 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274fa80 con 0x56490274bd00 2024-09-17T05:30:31.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v95'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274fa80 con 0x56490274bd00 2024-09-17T05:30:31.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v95'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.492+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v95'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(73 500.00000000 [create] v95'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(74 500.00000000 [delete] v95'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.496+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.503 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:31.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(75 500.00000000 [create] v95'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v95'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x56490274ea70 con 0x56490274bd00 2024-09-17T05:30:31.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91a9200640 1 -- 172.21.15.43:0/294298176 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v95'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f919c01b5d0 con 0x56490274bd00 2024-09-17T05:30:31.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56490274f370 0x564902751790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902751cd0 con 0x56490274f370 2024-09-17T05:30:31.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.500+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56490274f370 0x564902751790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.504+0000 7f91a3e00640 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56490274f370 0x564902751790 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.504+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(78 mds_snaptable [create] v95'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.504+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902751cd0 con 0x56490274f370 2024-09-17T05:30:31.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.504+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(79 mds_snaptable [delete] v95'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.504+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490274f370 2024-09-17T05:30:31.511 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:31.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(80 mds_snaptable [create] v95'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490274f370 2024-09-17T05:30:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v95'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490274f370 2024-09-17T05:30:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v95'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.508+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490260b020 2024-09-17T05:30:31.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(83 605.00000000 [create] v95'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902752120 con 0x56490260b020 2024-09-17T05:30:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(84 605.00000000 [delete] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490260b020 2024-09-17T05:30:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(85 605.00000000 [create] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.518 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.512+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902753130 con 0x56490260b020 2024-09-17T05:30:31.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902753130 con 0x56490260b020 2024-09-17T05:30:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x56490260b020 2024-09-17T05:30:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x56490260b020 2024-09-17T05:30:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.516+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.520+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v95'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.520+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.520+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v95'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.520+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.520+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v95'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.524+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.524+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v95'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.524+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.524+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v95'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.524+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.528+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v95'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.528+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.528+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v95'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.528+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:31.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.532+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v95'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.532+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752120 con 0x564902743db0 2024-09-17T05:30:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.532+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v95'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.532+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902754f30 con 0x564902743db0 2024-09-17T05:30:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.532+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v95'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x564902743db0 2024-09-17T05:30:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.536+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.536+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v95'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.536+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.536+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v95'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.536+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.544 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:31.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.540+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v95'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.540+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.540+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v95'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.540+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v95'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.551 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.544+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027535e0 con 0x5649027482e0 2024-09-17T05:30:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.548+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.548+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027535e0 con 0x5649027482e0 2024-09-17T05:30:31.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.548+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v95'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.548+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x5649027482e0 2024-09-17T05:30:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v95'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91a400f070 con 0x5649027482e0 2024-09-17T05:30:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v95'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v95'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.552+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v95'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.560 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:31.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v95'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v95'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.556+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v95'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v95'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v95'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.567 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.560+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v95'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v95'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v95'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.564+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v95'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.573 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v95'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027535e0 con 0x56490274f370 2024-09-17T05:30:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v95'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.568+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027535e0 con 0x56490274f370 2024-09-17T05:30:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.572+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v95'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.572+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5649027525d0 con 0x56490274f370 2024-09-17T05:30:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.572+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v95'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.572+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x564902752840 con 0x56490274f370 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91a3e00640 1 -- 172.21.15.43:0/294298176 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v95'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f91940f78e0 con 0x56490274f370 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56490273f360 msgr2=0x564902741780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56490273f360 0x564902741780 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56490274bd00 msgr2=0x56490274e120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56490274bd00 0x56490274e120 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56490274f370 msgr2=0x564902751790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56490274f370 0x564902751790 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564902743db0 msgr2=0x5649027461d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564902743db0 0x5649027461d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91a9c00640 1 -- 172.21.15.43:0/294298176 reap_dead start 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 msgr2=0x5649027246e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 0x5649027246e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 msgr2=0x56490274b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 0x56490274b140 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 msgr2=0x7f9188074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 0x7f9188074120 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f919c002ef0 tx=0x7f919c002ac0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 msgr2=0x564902609970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 0x564902609970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x56490267ad10 tx=0x7f91940a7780 comp rx=0 tx=0).stop 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 shutdown_connections 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5649027482e0 0x56490274b140 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9188071c10 0x7f9188074120 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56490260b020 0x5649027246e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 --2- 172.21.15.43:0/294298176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5649025f3cb0 0x564902609970 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 >> 172.21.15.43:0/294298176 conn(0x5649025b0d80 msgr2=0x5649025f34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 shutdown_connections 2024-09-17T05:30:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.576+0000 7f91b00dc080 1 -- 172.21.15.43:0/294298176 wait complete. 2024-09-17T05:30:31.585 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-09-17T05:30:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 Processor -- start 2024-09-17T05:30:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 -- start start 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 0x560f5e85f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e740150 0x560f5e85f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f5e6a1f40 con 0x7fe390003100 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f5e6aed00 con 0x560f5e7abbe0 2024-09-17T05:30:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3b1001080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f5e643a00 con 0x560f5e740150 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57442/0 (socket says 172.21.15.43:57442) 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.684+0000 7fe3aa200640 1 -- 172.21.15.43:0/564213031 learned_addr learned my addr 172.21.15.43:0/564213031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3a9800640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 0x560f5e85f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aac00640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e740150 0x560f5e85f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 -- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e740150 msgr2=0x560f5e85f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e740150 0x560f5e85f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 -- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 msgr2=0x560f5e85f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 0x560f5e85f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 -- 172.21.15.43:0/564213031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5e736d30 con 0x7fe390003100 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3a9800640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 0x560f5e85f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3aa200640 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b750 tx=0x7fe39c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d01c524ca8cc90 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3a3e00640 1 -- 172.21.15.43:0/564213031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ac20 con 0x7fe390003100 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3a3e00640 1 -- 172.21.15.43:0/564213031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39c00adc0 con 0x7fe390003100 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3a3e00640 1 -- 172.21.15.43:0/564213031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0086a0 con 0x7fe390003100 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- 172.21.15.43:0/564213031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 msgr2=0x560f5e7ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b750 tx=0x7fe39c004890 comp rx=0 tx=0).stop 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- 172.21.15.43:0/564213031 shutdown_connections 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f5e7abbe0 0x560f5e85f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e740150 0x560f5e85f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 --2- 172.21.15.43:0/564213031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe390003100 0x560f5e7ab6a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- 172.21.15.43:0/564213031 >> 172.21.15.43:0/564213031 conn(0x560f5e6e1d80 msgr2=0x560f5e737e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- 172.21.15.43:0/564213031 shutdown_connections 2024-09-17T05:30:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- 172.21.15.43:0/564213031 wait complete. 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 Processor -- start 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.688+0000 7fe3b1001080 1 -- start start 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 0x560f5e867f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e7abbe0 0x560f5e7a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390003100 0x560f5e7d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f5e736fd0 con 0x560f5e740150 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f5e737270 con 0x7fe390003100 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f5e7c9b50 con 0x560f5e7abbe0 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 0x560f5e867f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 0x560f5e867f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57450/0 (socket says 172.21.15.43:57450) 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 -- 172.21.15.43:0/1163419319 learned_addr learned my addr 172.21.15.43:0/1163419319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3a9800640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e7abbe0 0x560f5e7a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aac00640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390003100 0x560f5e7d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 -- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e7abbe0 msgr2=0x560f5e7a81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f5e7abbe0 0x560f5e7a81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 -- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390003100 msgr2=0x560f5e7d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390003100 0x560f5e7d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 -- 172.21.15.43:0/1163419319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5e7950f0 con 0x560f5e740150 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3aa200640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 0x560f5e867f50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe39c017a50 tx=0x7fe39c017a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3a2a00640 1 -- 172.21.15.43:0/1163419319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c026020 con 0x560f5e740150 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3a2a00640 1 -- 172.21.15.43:0/1163419319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39c022470 con 0x560f5e740150 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3a2a00640 1 -- 172.21.15.43:0/1163419319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0228c0 con 0x560f5e740150 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f5e795390 con 0x560f5e740150 2024-09-17T05:30:31.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.692+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5e7958d0 con 0x560f5e740150 2024-09-17T05:30:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3a2a00640 1 -- 172.21.15.43:0/1163419319 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe39c029020 con 0x560f5e740150 2024-09-17T05:30:31.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3a2a00640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe38c071be0 0x7fe38c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3a2a00640 1 -- 172.21.15.43:0/1163419319 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fe39c095fa0 con 0x560f5e740150 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3a9800640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe38c071be0 0x7fe38c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3b1001080 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x560f5e7d2e80 0x560f5e863430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x560f5e7d13e0 con 0x560f5e7d2e80 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3aac00640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x560f5e7d2e80 0x560f5e863430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3a9800640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe38c071be0 0x7fe38c0740f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ab000 tx=0x7fe3980180a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3aac00640 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x560f5e7d2e80 0x560f5e863430 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3aac00640 1 -- 172.21.15.43:0/1163419319 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v95'29 uv29 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe3a40099c0 con 0x560f5e7d2e80 2024-09-17T05:30:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x560f5e7d2e80 msgr2=0x560f5e863430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.696+0000 7fe3b1001080 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x560f5e7d2e80 0x560f5e863430 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe38c071be0 msgr2=0x7fe38c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe38c071be0 0x7fe38c0740f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ab000 tx=0x7fe3980180a0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 msgr2=0x560f5e867f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 --2- 172.21.15.43:0/1163419319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f5e740150 0x560f5e867f50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe39c017a50 tx=0x7fe39c017a80 comp rx=0 tx=0).stop 2024-09-17T05:30:31.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3aac00640 1 -- 172.21.15.43:0/1163419319 reap_dead start 2024-09-17T05:30:31.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 shutdown_connections 2024-09-17T05:30:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 >> 172.21.15.43:0/1163419319 conn(0x560f5e6e1d80 msgr2=0x560f5e7376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 shutdown_connections 2024-09-17T05:30:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.700+0000 7fe3b1001080 1 -- 172.21.15.43:0/1163419319 wait complete. 2024-09-17T05:30:31.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:31.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 -- 172.21.15.43:0/2882330251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80b3af0 msgr2=0x7f76d815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 --2- 172.21.15.43:0/2882330251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80b3af0 0x7f76d815c7e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c001c820 comp rx=0 tx=0).stop 2024-09-17T05:30:31.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 -- 172.21.15.43:0/2882330251 shutdown_connections 2024-09-17T05:30:31.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 --2- 172.21.15.43:0/2882330251 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 --2- 172.21.15.43:0/2882330251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80b3af0 0x7f76d815c7e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 -- 172.21.15.43:0/2882330251 >> 172.21.15.43:0/2882330251 conn(0x7f76d80b23d0 msgr2=0x7f76d80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 -- 172.21.15.43:0/2882330251 shutdown_connections 2024-09-17T05:30:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.864+0000 7f76df200640 1 -- 172.21.15.43:0/2882330251 wait complete. 2024-09-17T05:30:31.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 Processor -- start 2024-09-17T05:30:31.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 -- start start 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 0x7f76d815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 0x7f76d815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8168c60 con 0x7f76d80cefa0 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8168ae0 con 0x7f76d8155d70 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.868+0000 7f76df200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76d8168de0 con 0x7f76d806f110 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76dd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 0x7f76d815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 0x7f76d815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:37210/0 (socket says 172.21.15.43:37210) 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d7e00640 1 -- 172.21.15.43:0/1530146582 learned_addr learned my addr 172.21.15.43:0/1530146582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 -- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 msgr2=0x7f76d8074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 -- 172.21.15.43:0/1530146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 msgr2=0x7f76d815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 0x7f76d815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 -- 172.21.15.43:0/1530146582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d8073cf0 con 0x7f76d8155d70 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76dd200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 0x7f76d815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:31.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d7e00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76cfe00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 0x7f76d815cde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d0ca0 tx=0x7f76c00a5840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00b2040 con 0x7f76d8155d70 2024-09-17T05:30:31.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c00044d0 con 0x7f76d8155d70 2024-09-17T05:30:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0003860 con 0x7f76d8155d70 2024-09-17T05:30:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d8073fd0 con 0x7f76d8155d70 2024-09-17T05:30:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d80cf640 con 0x7f76d8155d70 2024-09-17T05:30:31.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d80ad0b0 con 0x7f76d8155d70 2024-09-17T05:30:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.872+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76c00bf060 con 0x7f76d8155d70 2024-09-17T05:30:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.876+0000 7f76d4c00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:31.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.876+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f76c012e250 con 0x7f76d8155d70 2024-09-17T05:30:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.876+0000 7f76d7e00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:31.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.876+0000 7f76d7e00640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 0x7f76b4075710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001bc0 tx=0x7f76c8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:31.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:31.880+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76c00164d0 con 0x7f76d8155d70 2024-09-17T05:30:32.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.088+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f76d80afee0 con 0x7f76d8155d70 2024-09-17T05:30:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.284+0000 7f76d4c00640 1 -- 172.21.15.43:0/1530146582 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v122) ==== 162+0+0 (secure 0 0 0) 0x7f76c00f8a80 con 0x7f76d8155d70 2024-09-17T05:30:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 msgr2=0x7f76b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 0x7f76b4075710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001bc0 tx=0x7f76c8009040 comp rx=0 tx=0).stop 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 msgr2=0x7f76d815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 0x7f76d815cde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d0ca0 tx=0x7f76c00a5840 comp rx=0 tx=0).stop 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 shutdown_connections 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d8155d70 0x7f76d815cde0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d80cefa0 0x7f76d815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 --2- 172.21.15.43:0/1530146582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76d806f110 0x7f76d8074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 >> 172.21.15.43:0/1530146582 conn(0x7f76d80b23d0 msgr2=0x7f76d80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 shutdown_connections 2024-09-17T05:30:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.288+0000 7f76df200640 1 -- 172.21.15.43:0/1530146582 wait complete. 2024-09-17T05:30:32.297 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:32.297+0000 7fc0e1e00640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-17T05:30:32.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- 172.21.15.43:0/2768125927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f464809ba60 msgr2=0x7f46480b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 --2- 172.21.15.43:0/2768125927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f464809ba60 0x7f46480b3130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f463c009870 tx=0x7f463c01c860 comp rx=0 tx=0).stop 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- 172.21.15.43:0/2768125927 shutdown_connections 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 --2- 172.21.15.43:0/2768125927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f464809ba60 0x7f46480b3130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- 172.21.15.43:0/2768125927 >> 172.21.15.43:0/2768125927 conn(0x7f46480cee60 msgr2=0x7f46480cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- 172.21.15.43:0/2768125927 shutdown_connections 2024-09-17T05:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- 172.21.15.43:0/2768125927 wait complete. 2024-09-17T05:30:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 Processor -- start 2024-09-17T05:30:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.876+0000 7f4650600640 1 -- start start 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46480d0240 0x7f464815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f464816b2e0 con 0x7f46480d0650 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f464816b160 con 0x7f464809ba60 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f464816b460 con 0x7f46480d0240 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46480d0240 0x7f464815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:38930/0 (socket says 172.21.15.43:38930) 2024-09-17T05:30:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57470/0 (socket says 172.21.15.43:57470) 2024-09-17T05:30:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464dc00640 1 -- 172.21.15.43:0/936408923 learned_addr learned my addr 172.21.15.43:0/936408923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 -- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46480d0240 msgr2=0x7f464815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46480d0240 0x7f464815f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 -- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 msgr2=0x7f4648154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 -- 172.21.15.43:0/936408923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464815b7e0 con 0x7f46480d0650 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464dc00640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f464e600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f464000ca70 tx=0x7f464000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640016020 con 0x7f46480d0650 2024-09-17T05:30:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464815bce0 con 0x7f46480d0650 2024-09-17T05:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f464000a950 con 0x7f46480d0650 2024-09-17T05:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464000ada0 con 0x7f46480d0650 2024-09-17T05:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464815c2a0 con 0x7f46480d0650 2024-09-17T05:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.880+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46480b3130 con 0x7f46480d0650 2024-09-17T05:30:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.884+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4640019020 con 0x7f46480d0650 2024-09-17T05:30:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.884+0000 7f4645600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 0x7f4628075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.884+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f4640094590 con 0x7f46480d0650 2024-09-17T05:30:32.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.884+0000 7f464dc00640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 0x7f4628075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:32.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.884+0000 7f464dc00640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 0x7f4628075710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f463c009870 tx=0x7f463c01b0a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:32.888+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4640020020 con 0x7f46480d0650 2024-09-17T05:30:32.983 INFO:tasks.ceph.mds.c.smithi043.stderr: -39> 2024-09-17T05:30:28.224+0000 7fa5e3a00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-17T05:30:33.103 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:30:33.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.100+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46480c8ed0 con 0x7f46480d0650 2024-09-17T05:30:33.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.100+0000 7f4645600640 1 -- 172.21.15.43:0/936408923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+4391 (secure 0 0 0) 0x7f464005ecc0 con 0x7f46480d0650 2024-09-17T05:30:33.107 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:33.107 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":123,"btime":"2024-09-17T05:30:32:290245+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:32.290231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7393":{"gid":7393,"name":"d","rank":0,"incarnation":123,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/3277842592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3277842592},{"type":"v1","addr":"172.21.15.62:6835","nonce":3277842592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:33.107 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 123 2024-09-17T05:30:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 msgr2=0x7f4628075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 0x7f4628075710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f463c009870 tx=0x7f463c01b0a0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 msgr2=0x7f46480cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f464000ca70 tx=0x7f464000cb60 comp rx=0 tx=0).stop 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 shutdown_connections 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4628073200 0x7f4628075710 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46480d0650 0x7f46480cb4a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f46480d0240 0x7f464815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 --2- 172.21.15.43:0/936408923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f464809ba60 0x7f4648154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 >> 172.21.15.43:0/936408923 conn(0x7f46480cee60 msgr2=0x7f46480b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 shutdown_connections 2024-09-17T05:30:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:33.104+0000 7f4650600640 1 -- 172.21.15.43:0/936408923 wait complete. 2024-09-17T05:30:33.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:30:32.290231+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7393': {'gid': 7393, 'name': 'd', 'rank': 0, 'incarnation': 123, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/3277842592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3277842592}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3277842592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:30:33.129 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7393:up:replay 2024-09-17T05:30:33.129 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:30:38.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:38.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 -- 172.21.15.43:0/3416234457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 msgr2=0x7f0a3c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 --2- 172.21.15.43:0/3416234457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c15c7e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0af0a0 tx=0x7f0a3001c740 comp rx=0 tx=0).stop 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 -- 172.21.15.43:0/3416234457 shutdown_connections 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 --2- 172.21.15.43:0/3416234457 >> v2:172.21.15.62:3300/0 conn(0x7f0a3c06f110 0x7f0a3c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 --2- 172.21.15.43:0/3416234457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c15c7e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 -- 172.21.15.43:0/3416234457 >> 172.21.15.43:0/3416234457 conn(0x7f0a3c0b23d0 msgr2=0x7f0a3c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.696+0000 7f0a44000640 1 -- 172.21.15.43:0/3416234457 shutdown_connections 2024-09-17T05:30:38.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 -- 172.21.15.43:0/3416234457 wait complete. 2024-09-17T05:30:38.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 Processor -- start 2024-09-17T05:30:38.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 -- start start 2024-09-17T05:30:38.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c06f110 0x7f0a3c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c0b3920 0x7f0a3c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1689c0 con 0x7f0a3c155d70 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c168840 con 0x7f0a3c06f110 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a44000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3c168b40 con 0x7f0a3c0b3920 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.700+0000 7f0a42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49696/0 (socket says 172.21.15.43:49696) 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 -- 172.21.15.43:0/2665412806 learned_addr learned my addr 172.21.15.43:0/2665412806 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a41600640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c06f110 0x7f0a3c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a40c00640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c0b3920 0x7f0a3c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 -- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c0b3920 msgr2=0x7f0a3c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c0b3920 0x7f0a3c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 -- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c06f110 msgr2=0x7f0a3c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c06f110 0x7f0a3c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 -- 172.21.15.43:0/2665412806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c165bb0 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a42000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c760 tx=0x7f0a3400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003b80 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a34003d40 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340052d0 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c166020 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1fc2d0 con 0x7f0a3c155d70 2024-09-17T05:30:38.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c0b33e0 con 0x7f0a3c155d70 2024-09-17T05:30:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.704+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a34005470 con 0x7f0a3c155d70 2024-09-17T05:30:38.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.708+0000 7f0a39600640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 0x7f0a1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.708+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f0a340940d0 con 0x7f0a3c155d70 2024-09-17T05:30:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.708+0000 7f0a41600640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 0x7f0a1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.708+0000 7f0a41600640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 0x7f0a1c075740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0a300af000 tx=0x7f0a3001af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.712+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0a3c0b3d30 con 0x7f0a3c155d70 2024-09-17T05:30:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3c165d80 con 0x7f0a3c155d70 2024-09-17T05:30:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a39600640 1 -- 172.21.15.43:0/2665412806 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5221 (secure 0 0 0) 0x7f0a3405e880 con 0x7f0a3c155d70 2024-09-17T05:30:38.928 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:38.928 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":126,"btime":"2024-09-17T05:30:35:305172+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/352682704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":352682704},{"type":"v1","addr":"172.21.15.43:6835","nonce":352682704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:35.305155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7393":{"gid":7393,"name":"d","rank":0,"incarnation":123,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/3277842592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3277842592},{"type":"v1","addr":"172.21.15.62:6835","nonce":3277842592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7393,"qdb_cluster":[7393]},"id":6}]} 2024-09-17T05:30:38.929 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 126 2024-09-17T05:30:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 msgr2=0x7f0a1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 0x7f0a1c075740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0a300af000 tx=0x7f0a3001af80 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 msgr2=0x7f0a3c0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c760 tx=0x7f0a3400cc20 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 shutdown_connections 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a1c073230 0x7f0a1c075740 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3c155d70 0x7f0a3c0d0300 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a3c0b3920 0x7f0a3c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 --2- 172.21.15.43:0/2665412806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a3c06f110 0x7f0a3c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 >> 172.21.15.43:0/2665412806 conn(0x7f0a3c0b23d0 msgr2=0x7f0a3c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 shutdown_connections 2024-09-17T05:30:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:38.924+0000 7f0a44000640 1 -- 172.21.15.43:0/2665412806 wait complete. 2024-09-17T05:30:38.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:30:35.305155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7393': {'gid': 7393, 'name': 'd', 'rank': 0, 'incarnation': 123, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6835/3277842592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 3277842592}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3277842592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7393, 'qdb_cluster': [7393]} 2024-09-17T05:30:38.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:30:38.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 -- 172.21.15.43:0/3434895590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c0b3af0 msgr2=0x7fd54c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 --2- 172.21.15.43:0/3434895590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c0b3af0 0x7fd54c15c7e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd538009840 tx=0x7fd53801c6b0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 -- 172.21.15.43:0/3434895590 shutdown_connections 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 --2- 172.21.15.43:0/3434895590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c0b3af0 0x7fd54c15c7e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 -- 172.21.15.43:0/3434895590 >> 172.21.15.43:0/3434895590 conn(0x7fd54c0b23d0 msgr2=0x7fd54c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 -- 172.21.15.43:0/3434895590 shutdown_connections 2024-09-17T05:30:39.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.104+0000 7fd553c00640 1 -- 172.21.15.43:0/3434895590 wait complete. 2024-09-17T05:30:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 Processor -- start 2024-09-17T05:30:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 -- start start 2024-09-17T05:30:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 0x7fd54c09da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd54c155d70 0x7fd54c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c169dc0 con 0x7fd54c157980 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c169c40 con 0x7fd54c06f110 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd553c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c169f40 con 0x7fd54c155d70 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd551c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd551c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49698/0 (socket says 172.21.15.43:49698) 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.108+0000 7fd551c00640 1 -- 172.21.15.43:0/3075616483 learned_addr learned my addr 172.21.15.43:0/3075616483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551200640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 0x7fd54c09da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd54be00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd54c155d70 0x7fd54c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 -- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd54c155d70 msgr2=0x7fd54c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd54c155d70 0x7fd54c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 -- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 msgr2=0x7fd54c09da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 0x7fd54c09da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 -- 172.21.15.43:0/3075616483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c09df90 con 0x7fd54c157980 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551200640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 0x7fd54c09da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd551c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd54400cb90 tx=0x7fd54400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd548c00640 1 -- 172.21.15.43:0/3075616483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544016020 con 0x7fd54c157980 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1599e0 con 0x7fd54c157980 2024-09-17T05:30:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd548c00640 1 -- 172.21.15.43:0/3075616483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd54400ab40 con 0x7fd54c157980 2024-09-17T05:30:39.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd548c00640 1 -- 172.21.15.43:0/3075616483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544012420 con 0x7fd54c157980 2024-09-17T05:30:39.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1586e0 con 0x7fd54c157980 2024-09-17T05:30:39.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.112+0000 7fd548c00640 1 -- 172.21.15.43:0/3075616483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd544019020 con 0x7fd54c157980 2024-09-17T05:30:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd548c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 0x7fd52c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd548c00640 1 -- 172.21.15.43:0/3075616483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fd5440948f0 con 0x7fd54c157980 2024-09-17T05:30:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 0x7fd52c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 Processor -- start 2024-09-17T05:30:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 -- start start 2024-09-17T05:30:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 0x7fd52c075740 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd540001130 tx=0x7fd540009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa52b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa52b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560eaa3fb410 con 0x560ea9fe5460 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560eaa338b40 con 0x560eaa3beca0 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560eaa327f70 con 0x560eaa3d5950 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49714/0 (socket says 172.21.15.43:49714) 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 -- 172.21.15.43:0/4140471189 learned_addr learned my addr 172.21.15.43:0/4140471189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd54be00640 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa52b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551c00640 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa52b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 -- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 msgr2=0x560eaa52b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa52b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 -- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 msgr2=0x560eaa52b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa52b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 -- 172.21.15.43:0/4140471189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eaa2ef5b0 con 0x560ea9fe5460 2024-09-17T05:30:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd551200640 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd5400050d0 tx=0x7fd540011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df4e65b9ab307f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd53d600640 1 -- 172.21.15.43:0/4140471189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540003600 con 0x560ea9fe5460 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd53d600640 1 -- 172.21.15.43:0/4140471189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd540003890 con 0x560ea9fe5460 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.116+0000 7fd53d600640 1 -- 172.21.15.43:0/4140471189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540003c40 con 0x560ea9fe5460 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- 172.21.15.43:0/4140471189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 msgr2=0x560eaa52ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd5400050d0 tx=0x7fd540011000 comp rx=0 tx=0).stop 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- 172.21.15.43:0/4140471189 shutdown_connections 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa52b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa52b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/4140471189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa52ab00 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- 172.21.15.43:0/4140471189 >> 172.21.15.43:0/4140471189 conn(0x560eaa4abd90 msgr2=0x560eaa4db690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- 172.21.15.43:0/4140471189 shutdown_connections 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- 172.21.15.43:0/4140471189 wait complete. 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- start start 2024-09-17T05:30:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 Processor -- start 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa519120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa519ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560eaa2c25f0 con 0x560eaa3beca0 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560eaa32bd80 con 0x560ea9fe5460 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560eaa2ce520 con 0x560eaa3d5950 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd55b3f5000 10 client.0 _ll_register_callbacks cb 0x560eaa2d2830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd54be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd54be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49730/0 (socket says 172.21.15.43:49730) 2024-09-17T05:30:39.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd54be00640 1 -- 172.21.15.43:0/3825046398 learned_addr learned my addr 172.21.15.43:0/3825046398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd551c00640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa519ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.120+0000 7fd551200640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa519120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 -- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 msgr2=0x560eaa519ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa519ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 -- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 msgr2=0x560eaa519120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa519120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 -- 172.21.15.43:0/3825046398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eaa2c6190 con 0x560eaa3beca0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd54be00640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd53801a2e0 tx=0x7fd538005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97136d399c8465a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5380ac040 con 0x560eaa3beca0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fd5380042f0 con 0x560eaa3beca0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560eaa2c0110 con 0x560eaa3beca0 2024-09-17T05:30:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7fd5380044f0 con 0x560eaa3beca0 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 10 client.7863 fetch_fsmap learned FSMap version 126 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) -- 0x560eaa3acf10 con 0x560eaa3beca0 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560eaa2c0110 con 0x560eaa3beca0 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 126) ==== 3502+0+0 (secure 0 0 0) 0x7fd538005040 con 0x560eaa3beca0 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 10 client.7863 fetch_fsmap finished waiting for FSMap version 126 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 10 client.7863 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 20 client.7863 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fca7d0 0x560eaa51a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 4 client.7863 mds_command: new command op to 7393 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:30:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 --> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eaa33aa90 con 0x560ea9fca7d0 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd551200640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fca7d0 0x560eaa51a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd551200640 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fca7d0 0x560eaa51a5b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b31f873ee9691ac server_cookie=327be1177bfece60 in_seq=0 out_seq=0 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 10 client.7863 ms_handle_connect on v2:172.21.15.62:6834/3277842592 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 1 -- 172.21.15.43:0/3825046398 <== mds.0 v2:172.21.15.62:6834/3277842592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560eaa33aa90 con 0x560ea9fca7d0 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.124+0000 7fd532000640 10 client.7863 handle_command_reply: tid=0 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 client.7863 shutdown 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 msgr2=0x560eaa519660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd53801a2e0 tx=0x7fd538005d10 comp rx=0 tx=0).stop 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 shutdown_connections 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fca7d0 0x560eaa51a5b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa519ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3beca0 0x560eaa519660 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3825046398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa519120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd532000640 0 client.7863 ms_handle_reset on v2:172.21.15.62:6834/3277842592 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 >> 172.21.15.43:0/3825046398 conn(0x560eaa4abd90 msgr2=0x560eaa4e3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 shutdown_connections 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3825046398 wait complete. 2024-09-17T05:30:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.128+0000 7fd55b3f5000 20 client.7863 trim_cache size 0 max 16384 2024-09-17T05:30:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.144+0000 7fd55b3f5000 1 Processor -- start 2024-09-17T05:30:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.144+0000 7fd55b3f5000 1 -- start start 2024-09-17T05:30:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa4e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa541110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c2011b0 con 0x560ea9fe5460 2024-09-17T05:30:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c074730 con 0x560eaa3beca0 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c156c90 con 0x560eaa3d5950 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49740/0 (socket says 172.21.15.43:49740) 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 -- 172.21.15.43:0/3613149927 learned_addr learned my addr 172.21.15.43:0/3613149927 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd54be00640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa4e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551c00640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa541110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 -- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 msgr2=0x560eaa541110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa541110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 -- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 msgr2=0x560eaa4e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa4e86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 -- 172.21.15.43:0/3613149927 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eaa383db0 con 0x560ea9fe5460 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd54be00640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa4e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd551200640 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd540005040 tx=0x7fd540012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5a68bc2bbd26e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd53cc00640 1 -- 172.21.15.43:0/3613149927 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400039d0 con 0x560ea9fe5460 2024-09-17T05:30:39.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd53cc00640 1 -- 172.21.15.43:0/3613149927 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd540003b70 con 0x560ea9fe5460 2024-09-17T05:30:39.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd53cc00640 1 -- 172.21.15.43:0/3613149927 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540021450 con 0x560ea9fe5460 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3613149927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 msgr2=0x560eaa4e8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd540005040 tx=0x7fd540012000 comp rx=0 tx=0).stop 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3613149927 shutdown_connections 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eaa3beca0 0x560eaa4e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3d5950 0x560eaa541110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/3613149927 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ea9fe5460 0x560eaa4e8190 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3613149927 >> 172.21.15.43:0/3613149927 conn(0x560eaa4abd90 msgr2=0x560eaa4e7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3613149927 shutdown_connections 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.148+0000 7fd55b3f5000 1 -- 172.21.15.43:0/3613149927 wait complete. 2024-09-17T05:30:39.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- start start 2024-09-17T05:30:39.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 Processor -- start 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa5a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3beca0 0x560eaa5a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560eaa2f3ee0 con 0x560eaa3d5950 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c074110 con 0x560ea9fe5460 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560eaa2e11e0 con 0x560eaa3beca0 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 10 client.0 _ll_register_callbacks cb 0x7fd534003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49756/0 (socket says 172.21.15.43:49756) 2024-09-17T05:30:39.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 -- 172.21.15.43:0/503370663 learned_addr learned my addr 172.21.15.43:0/503370663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551200640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa5a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd54be00640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3beca0 0x560eaa5a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 -- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3beca0 msgr2=0x560eaa5a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3beca0 0x560eaa5a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 -- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 msgr2=0x560eaa5a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa5a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 -- 172.21.15.43:0/503370663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eaa2eeab0 con 0x560eaa3d5950 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd551c00640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd54405aec0 tx=0x7fd544014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1c227f43cc9c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544080040 con 0x560eaa3d5950 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fd54407acf0 con 0x560eaa3d5950 2024-09-17T05:30:39.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd540003b70 con 0x560eaa3d5950 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7fd54405ef70 con 0x560eaa3d5950 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 10 client.7869 fetch_fsmap learned FSMap version 126 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) -- 0x560eaa2cd070 con 0x560eaa3d5950 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd540003b70 con 0x560eaa3d5950 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 126) ==== 3502+0+0 (secure 0 0 0) 0x7fd544068b50 con 0x560eaa3d5950 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 10 client.7869 fetch_fsmap finished waiting for FSMap version 126 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 10 client.7869 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 20 client.7869 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fb8830 0x560eaa53d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 4 client.7869 mds_command: new command op to 7393 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:30:39.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.152+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 --> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] -- command(tid 0: {"prefix": "status"}) -- 0x560eaa336540 con 0x560ea9fb8830 2024-09-17T05:30:39.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd551200640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fb8830 0x560eaa53d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd551200640 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fb8830 0x560eaa53d930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2596eaa517e00eb server_cookie=44e71ddcaf17c483 in_seq=0 out_seq=0 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd532000640 10 client.7869 ms_handle_connect on v2:172.21.15.62:6834/3277842592 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd532000640 1 -- 172.21.15.43:0/503370663 <== mds.0 v2:172.21.15.62:6834/3277842592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560eaa336540 con 0x560ea9fb8830 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd532000640 10 client.7869 handle_command_reply: tid=0 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 client.7869 shutdown 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 msgr2=0x560eaa5a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd54405aec0 tx=0x7fd544014040 comp rx=0 tx=0).stop 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 shutdown_connections 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:6834/3277842592,v1:172.21.15.62:6835/3277842592] conn(0x560ea9fb8830 0x560eaa53d930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560eaa3d5950 0x560eaa5a0da0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560eaa3beca0 0x560eaa5a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 --2- 172.21.15.43:0/503370663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ea9fe5460 0x560eaa5a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 >> 172.21.15.43:0/503370663 conn(0x560eaa4abd90 msgr2=0x560eaa4e1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd532000640 0 client.7869 ms_handle_reset on v2:172.21.15.62:6834/3277842592 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 shutdown_connections 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 1 -- 172.21.15.43:0/503370663 wait complete. 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd55b3f5000 20 client.7869 trim_cache size 0 max 16384 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7393, 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:30:39.164 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8680780779999999, 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 126, 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 95, 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 95, 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.060424721 2024-09-17T05:30:39.165 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 msgr2=0x7fd52c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 0x7fd52c075740 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd540001130 tx=0x7fd540009040 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 msgr2=0x7fd54c08bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd54400cb90 tx=0x7fd54400cc80 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 shutdown_connections 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd52c073230 0x7fd52c075740 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c157980 0x7fd54c08bfe0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd54c155d70 0x7fd54c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 --2- 172.21.15.43:0/3075616483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c06f110 0x7fd54c09da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 >> 172.21.15.43:0/3075616483 conn(0x7fd54c0b23d0 msgr2=0x7fd54c0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 shutdown_connections 2024-09-17T05:30:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.156+0000 7fd553c00640 1 -- 172.21.15.43:0/3075616483 wait complete. 2024-09-17T05:30:39.181 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:30:39.181 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-09-17T05:30:39.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:30:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 -- 172.21.15.43:0/1612304697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e3415ca80 msgr2=0x7f6e3415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 --2- 172.21.15.43:0/1612304697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e3415ca80 0x7f6e3415ce90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 -- 172.21.15.43:0/1612304697 shutdown_connections 2024-09-17T05:30:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 --2- 172.21.15.43:0/1612304697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e3415ca80 0x7f6e3415ce90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 --2- 172.21.15.43:0/1612304697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e340d1cb0 0x7f6e340ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 -- 172.21.15.43:0/1612304697 >> 172.21.15.43:0/1612304697 conn(0x7f6e34068fe0 msgr2=0x7f6e34069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 -- 172.21.15.43:0/1612304697 shutdown_connections 2024-09-17T05:30:39.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 -- 172.21.15.43:0/1612304697 wait complete. 2024-09-17T05:30:39.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.336+0000 7f6e3b200640 1 Processor -- start 2024-09-17T05:30:39.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- start start 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e3415ca80 0x7f6e3414b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e341605e0 0x7f6e3414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34153680 con 0x7f6e340d1cb0 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34153500 con 0x7f6e341605e0 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6e34153800 con 0x7f6e3415ca80 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49784/0 (socket says 172.21.15.43:49784) 2024-09-17T05:30:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 -- 172.21.15.43:0/542889288 learned_addr learned my addr 172.21.15.43:0/542889288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3a800640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e341605e0 0x7f6e3414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39400640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e3415ca80 0x7f6e3414b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 -- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e3415ca80 msgr2=0x7f6e3414b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e3415ca80 0x7f6e3414b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 -- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e341605e0 msgr2=0x7f6e3414f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e341605e0 0x7f6e3414f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 -- 172.21.15.43:0/542889288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34151920 con 0x7f6e340d1cb0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e39e00640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d750 tx=0x7f6e2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e340d1cb0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e28009d20 con 0x7f6e340d1cb0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3406eab0 con 0x7f6e340d1cb0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28005280 con 0x7f6e340d1cb0 2024-09-17T05:30:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.340+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3406ed50 con 0x7f6e340d1cb0 2024-09-17T05:30:39.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e340ccc40 con 0x7f6e340d1cb0 2024-09-17T05:30:39.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e28005420 con 0x7f6e340d1cb0 2024-09-17T05:30:39.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e2e000640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 0x7f6e18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f6e280945b0 con 0x7f6e340d1cb0 2024-09-17T05:30:39.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e39400640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 0x7f6e18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.344+0000 7f6e39400640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 0x7f6e18075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e240af000 tx=0x7f6e2401b0f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.348+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e2800de60 con 0x7f6e340d1cb0 2024-09-17T05:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.568+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e3406a1e0 con 0x7f6e340d1cb0 2024-09-17T05:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.568+0000 7f6e2e000640 1 -- 172.21.15.43:0/542889288 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v126) ==== 53+0+73 (secure 0 0 0) 0x7f6e2805f3b0 con 0x7f6e340d1cb0 2024-09-17T05:30:39.575 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 msgr2=0x7f6e18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 0x7f6e18075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e240af000 tx=0x7f6e2401b0f0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 msgr2=0x7f6e340c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d750 tx=0x7f6e2800dc10 comp rx=0 tx=0).stop 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 shutdown_connections 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6e18073200 0x7f6e18075710 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e341605e0 0x7f6e3414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6e3415ca80 0x7f6e3414b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 --2- 172.21.15.43:0/542889288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e340d1cb0 0x7f6e340c6dc0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 >> 172.21.15.43:0/542889288 conn(0x7f6e34068fe0 msgr2=0x7f6e340cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 shutdown_connections 2024-09-17T05:30:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39.572+0000 7f6e3b200640 1 -- 172.21.15.43:0/542889288 wait complete. 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:39.597 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:30:39.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:30:39.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:30:39.598 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:39.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.598 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:30:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.608 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42004sec preferred_lft 42004sec 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.609 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:30:39.610 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:39.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.611 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:30:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:30:39.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.671 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:30:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:30:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:30:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:30:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:30:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:30:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:30:39.725 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:30:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:30:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:39.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.863 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:30:39.871 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:30:39.872 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:30:39.873 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:39.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:30:39.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:39.935 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:39.987 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:39.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:39.987 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:40.039 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:30:40.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:40.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:40.148 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:40.148 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:40.148 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:40.166 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:40.166 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:40.167 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:40.167 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:40.167 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:30:40.179 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:30:40.182 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:30:40.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:40.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:40.202 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:30:40.268 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:40.269+0000 7fc0e6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:30:40.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:client6872 2024-09-17T05:30:40.306 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:40.306 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:40.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:40.315 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:30:40.362 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:30:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:30:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:30:40.371 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:30:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:30:40.372 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-09-17T05:30:40.373 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-09-17T05:30:40.373 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:40.373 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:40.424 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:40.424 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:40.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:30:40.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 -- 172.21.15.43:0/2592154600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 msgr2=0x7f1f34003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:40.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 --2- 172.21.15.43:0/2592154600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f34003fb0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f400016f0 tx=0x7f1f4001c710 comp rx=0 tx=0).stop 2024-09-17T05:30:40.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 -- 172.21.15.43:0/2592154600 shutdown_connections 2024-09-17T05:30:40.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 --2- 172.21.15.43:0/2592154600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f34003fb0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 -- 172.21.15.43:0/2592154600 >> 172.21.15.43:0/2592154600 conn(0x7f1f500cee80 msgr2=0x7f1f500cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:40.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 -- 172.21.15.43:0/2592154600 shutdown_connections 2024-09-17T05:30:40.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.672+0000 7f1f55400640 1 -- 172.21.15.43:0/2592154600 wait complete. 2024-09-17T05:30:40.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 Processor -- start 2024-09-17T05:30:40.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 -- start start 2024-09-17T05:30:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:40.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f5009e670 0x7f1f5015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 0x7f1f501f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500b37b0 con 0x7f1f34003b40 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500b3630 con 0x7f1f501f9950 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f55400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500b2440 con 0x7f1f5009e670 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49806/0 (socket says 172.21.15.43:49806) 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 -- 172.21.15.43:0/3484619337 learned_addr learned my addr 172.21.15.43:0/3484619337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f45e00640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f5009e670 0x7f1f5015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4f200640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 0x7f1f501f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 -- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f5009e670 msgr2=0x7f1f5015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f5009e670 0x7f1f5015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 -- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 msgr2=0x7f1f501f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 0x7f1f501f9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 -- 172.21.15.43:0/3484619337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f501fa2a0 con 0x7f1f34003b40 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4f200640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 0x7f1f501f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.676+0000 7f1f4e800640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f4001cbf0 tx=0x7f1f40003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f400bd070 con 0x7f1f34003b40 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f40006500 con 0x7f1f34003b40 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40001340 con 0x7f1f34003b40 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f501f9470 con 0x7f1f34003b40 2024-09-17T05:30:40.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f501fcef0 con 0x7f1f34003b40 2024-09-17T05:30:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f500b2930 con 0x7f1f34003b40 2024-09-17T05:30:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f400c0070 con 0x7f1f34003b40 2024-09-17T05:30:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 0x7f1f2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:40.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7f1f4012dee0 con 0x7f1f34003b40 2024-09-17T05:30:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.680+0000 7f1f45e00640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 0x7f1f2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:40.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.684+0000 7f1f45e00640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 0x7f1f2c075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1f38001250 tx=0x7f1f38006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.684+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f400b4070 con 0x7f1f34003b40 2024-09-17T05:30:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f500cc930 con 0x7f1f34003b40 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f47400640 1 -- 172.21.15.43:0/3484619337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v95) ==== 81+0+728 (secure 0 0 0) 0x7f1f40016470 con 0x7f1f34003b40 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:30:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:30:40.878 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:30:40.878 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:40.878 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:30:40.878 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:listed 12 entries 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 msgr2=0x7f1f2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 0x7f1f2c075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1f38001250 tx=0x7f1f38006040 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 msgr2=0x7f1f5009cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f4001cbf0 tx=0x7f1f40003c90 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 shutdown_connections 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073200 0x7f1f2c075710 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f501f9950 0x7f1f501f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f5009e670 0x7f1f5015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 --2- 172.21.15.43:0/3484619337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f34003b40 0x7f1f5009cdf0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 >> 172.21.15.43:0/3484619337 conn(0x7f1f500cee80 msgr2=0x7f1f5015b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 shutdown_connections 2024-09-17T05:30:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:40.872+0000 7f1f55400640 1 -- 172.21.15.43:0/3484619337 wait complete. 2024-09-17T05:30:40.899 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:30:40.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:40.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:30:40.922 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:40.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:40.922 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:40.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 -- 172.21.15.43:0/2347701095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc155d70 msgr2=0x7fd9cc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/2347701095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc155d70 0x7fd9cc15c7e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0d0c40 tx=0x7fd9bc01c800 comp rx=0 tx=0).stop 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 -- 172.21.15.43:0/2347701095 shutdown_connections 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/2347701095 >> v2:172.21.15.62:3300/0 conn(0x7fd9cc0b3af0 0x7fd9cc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/2347701095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc155d70 0x7fd9cc15c7e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 -- 172.21.15.43:0/2347701095 >> 172.21.15.43:0/2347701095 conn(0x7fd9cc0b23d0 msgr2=0x7fd9cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 -- 172.21.15.43:0/2347701095 shutdown_connections 2024-09-17T05:30:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.168+0000 7fd9d0a00640 1 -- 172.21.15.43:0/2347701095 wait complete. 2024-09-17T05:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 Processor -- start 2024-09-17T05:30:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- start start 2024-09-17T05:30:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 0x7fd9cc09c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9cc155d70 0x7fd9cc154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 0x7fd9cc0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc168c00 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc168a80 con 0x7fd9cc06f110 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc168d80 con 0x7fd9cc155d70 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 0x7fd9cc0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 0x7fd9cc09c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9cc155d70 0x7fd9cc154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 0x7fd9cc09c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47734/0 (socket says 172.21.15.43:47734) 2024-09-17T05:30:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c9e00640 1 -- 172.21.15.43:0/3707915717 learned_addr learned my addr 172.21.15.43:0/3707915717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 -- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9cc155d70 msgr2=0x7fd9cc154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9cc155d70 0x7fd9cc154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 -- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 msgr2=0x7fd9cc09c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 0x7fd9cc09c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 -- 172.21.15.43:0/3707915717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0cfff0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9ca800640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 0x7fd9cc0cfab0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400e990 tx=0x7fd9c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004d30 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9c400caa0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc09dbe0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4005020 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.172+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1553c0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc09de30 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd9c40051c0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9c2000640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 0x7fd9a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6225+0+0 (secure 0 0 0) 0x7fd9c4094ce0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9c9e00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 0x7fd9a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.176+0000 7fd9c9e00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 0x7fd9a8075740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009230 tx=0x7fd9bc01b040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.180+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd9cc0cfff0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.388+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd9cc0ad0b0 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.395 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:41.392+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.392+0000 7fd9c2000640 1 -- 172.21.15.43:0/3707915717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v127) ==== 161+0+0 (secure 0 0 0) 0x7fd9c4068020 con 0x7fd9cc0cf6a0 2024-09-17T05:30:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 msgr2=0x7fd9a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 0x7fd9a8075740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009230 tx=0x7fd9bc01b040 comp rx=0 tx=0).stop 2024-09-17T05:30:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 msgr2=0x7fd9cc0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 0x7fd9cc0cfab0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400e990 tx=0x7fd9c400ee50 comp rx=0 tx=0).stop 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 shutdown_connections 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9a8073230 0x7fd9a8075740 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009230 tx=0x7fd9bc01b040 comp rx=0 tx=0).stop 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0cf6a0 0x7fd9cc0cfab0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9cc155d70 0x7fd9cc154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 --2- 172.21.15.43:0/3707915717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9cc06f110 0x7fd9cc09c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 >> 172.21.15.43:0/3707915717 conn(0x7fd9cc0b23d0 msgr2=0x7fd9cc06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 shutdown_connections 2024-09-17T05:30:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.396+0000 7fd9d0a00640 1 -- 172.21.15.43:0/3707915717 wait complete. 2024-09-17T05:30:41.402 INFO:tasks.ceph.mds.d.smithi062.stderr: -1876> 2024-09-17T05:30:32.297+0000 7fc0e1e00640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-17T05:30:41.407 INFO:tasks.ceph.mds.d.smithi062.stderr: -401> 2024-09-17T05:30:40.269+0000 7fc0e6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:41.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:41.531 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:30:41.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 -- 172.21.15.43:0/2993505400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe63415fa70 msgr2=0x7fe63415fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 --2- 172.21.15.43:0/2993505400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe63415fa70 0x7fe63415fe80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009840 tx=0x7fe61c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 -- 172.21.15.43:0/2993505400 shutdown_connections 2024-09-17T05:30:41.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 --2- 172.21.15.43:0/2993505400 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe63415fa70 0x7fe63415fe80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 --2- 172.21.15.43:0/2993505400 >> v2:172.21.15.62:3300/0 conn(0x7fe6340b5fe0 0x7fe63415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 -- 172.21.15.43:0/2993505400 >> 172.21.15.43:0/2993505400 conn(0x7fe6340b2e80 msgr2=0x7fe6340b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:41.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 -- 172.21.15.43:0/2993505400 shutdown_connections 2024-09-17T05:30:41.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.576+0000 7fe63b000640 1 -- 172.21.15.43:0/2993505400 wait complete. 2024-09-17T05:30:41.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 Processor -- start 2024-09-17T05:30:41.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 -- start start 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 0x7fe63406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 0x7fe63406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6341690c0 con 0x7fe63415fa70 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe634168f40 con 0x7fe6340b5fe0 2024-09-17T05:30:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe634169240 con 0x7fe6340b0c20 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe639000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 0x7fe63406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe633e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 0x7fe63406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe633e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:35274/0 (socket says 172.21.15.43:35274) 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe633e00640 1 -- 172.21.15.43:0/2560461846 learned_addr learned my addr 172.21.15.43:0/2560461846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 -- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 msgr2=0x7fe634158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 -- 172.21.15.43:0/2560461846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 msgr2=0x7fe63406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 0x7fe63406d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 -- 172.21.15.43:0/2560461846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe634158da0 con 0x7fe6340b5fe0 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe633e00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:41.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe639000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 0x7fe63406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe62be00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 0x7fe63406ce60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe6340b24c0 tx=0x7fe61c0a5690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b2040 con 0x7fe6340b5fe0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe61c004470 con 0x7fe6340b5fe0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6340ceac0 con 0x7fe6340b5fe0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.580+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c005280 con 0x7fe6340b5fe0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341fc6c0 con 0x7fe6340b5fe0 2024-09-17T05:30:41.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63415f530 con 0x7fe6340b5fe0 2024-09-17T05:30:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe61c005420 con 0x7fe6340b5fe0 2024-09-17T05:30:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe630c00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 0x7fe610075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7fe61c12de60 con 0x7fe6340b5fe0 2024-09-17T05:30:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.584+0000 7fe633e00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 0x7fe610075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.588+0000 7fe633e00640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 0x7fe610075740 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe624009680 tx=0x7fe6240091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:41.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.588+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe61c0bc060 con 0x7fe6340b5fe0 2024-09-17T05:30:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:41.824+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe6340cf380 con 0x7fe6340b5fe0 2024-09-17T05:30:42.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.408+0000 7fe630c00640 1 -- 172.21.15.43:0/2560461846 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v128) ==== 107+0+0 (secure 0 0 0) 0x7fe61c016410 con 0x7fe6340b5fe0 2024-09-17T05:30:42.415 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 msgr2=0x7fe610075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 0x7fe610075740 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe624009680 tx=0x7fe6240091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 msgr2=0x7fe63406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 0x7fe63406ce60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe6340b24c0 tx=0x7fe61c0a5690 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 shutdown_connections 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe610073230 0x7fe610075740 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6340b5fe0 0x7fe63406ce60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe63415fa70 0x7fe63406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 --2- 172.21.15.43:0/2560461846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6340b0c20 0x7fe634158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 >> 172.21.15.43:0/2560461846 conn(0x7fe6340b2e80 msgr2=0x7fe6340b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 shutdown_connections 2024-09-17T05:30:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.412+0000 7fe63b000640 1 -- 172.21.15.43:0/2560461846 wait complete. 2024-09-17T05:30:42.439 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.492+0000 7f99db77e080 1 Processor -- start 2024-09-17T05:30:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.492+0000 7f99db77e080 1 -- start start 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b723a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b71aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643b70b6a00 con 0x5643b7197cb0 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5643b7114f40 con 0x5643b721ee40 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5643b721f890 con 0x5643b7168d70 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49848/0 (socket says 172.21.15.43:49848) 2024-09-17T05:30:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 -- 172.21.15.43:0/2508390251 learned_addr learned my addr 172.21.15.43:0/2508390251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99d5400640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b71aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b723a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 -- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 msgr2=0x5643b723a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b723a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 -- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 msgr2=0x5643b71aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b71aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 -- 172.21.15.43:0/2508390251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b708fc10 con 0x5643b7197cb0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99d5400640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b71aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b220 tx=0x7f99c000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c215d59b46a16760 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cea00640 1 -- 172.21.15.43:0/2508390251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c000a040 con 0x5643b7197cb0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cea00640 1 -- 172.21.15.43:0/2508390251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99c000f620 con 0x5643b7197cb0 2024-09-17T05:30:42.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99cea00640 1 -- 172.21.15.43:0/2508390251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c000fa70 con 0x5643b7197cb0 2024-09-17T05:30:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- 172.21.15.43:0/2508390251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 msgr2=0x5643b71aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b220 tx=0x7f99c000b6e0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- 172.21.15.43:0/2508390251 shutdown_connections 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b71aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71aa830 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 --2- 172.21.15.43:0/2508390251 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b723a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- 172.21.15.43:0/2508390251 >> 172.21.15.43:0/2508390251 conn(0x5643b7154d80 msgr2=0x5643b7166890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- 172.21.15.43:0/2508390251 shutdown_connections 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- 172.21.15.43:0/2508390251 wait complete. 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 Processor -- start 2024-09-17T05:30:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.496+0000 7f99db77e080 1 -- start start 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b7166da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b7237520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643b71975f0 con 0x5643b7197cb0 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5643b71aa0f0 con 0x5643b721ee40 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5643b71aee90 con 0x5643b7168d70 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99d4a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b7166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49862/0 (socket says 172.21.15.43:49862) 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 learned_addr learned my addr 172.21.15.43:0/2666491898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99d5400640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b7237520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 msgr2=0x5643b7166da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5643b7168d70 0x5643b7166da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 msgr2=0x5643b7237520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643b721ee40 0x5643b7237520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b71af0f0 con 0x5643b7197cb0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f99c000bbe0 tx=0x7f99c000fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cd600640 1 -- 172.21.15.43:0/2666491898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c001f070 con 0x5643b7197cb0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cd600640 1 -- 172.21.15.43:0/2666491898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99c0019660 con 0x5643b7197cb0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99cd600640 1 -- 172.21.15.43:0/2666491898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0019ab0 con 0x5643b7197cb0 2024-09-17T05:30:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5643b71af390 con 0x5643b7197cb0 2024-09-17T05:30:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.500+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643b71af8d0 con 0x5643b7197cb0 2024-09-17T05:30:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99cd600640 1 -- 172.21.15.43:0/2666491898 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99c0022070 con 0x5643b7197cb0 2024-09-17T05:30:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99cd600640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 0x7f99b40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99cd600640 1 -- 172.21.15.43:0/2666491898 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f99c0094540 con 0x5643b7197cb0 2024-09-17T05:30:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 0x7f99b40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.509 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5643b72d8d20 0x5643b71afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72c8840 con 0x5643b72d8d20 2024-09-17T05:30:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99d5400640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5643b72d8d20 0x5643b71afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 0x7f99b40740f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c670 tx=0x7f99c8004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.504+0000 7f99d5400640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5643b72d8d20 0x5643b71afd70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.508+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v96'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.508+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b71b02b0 con 0x5643b72d8d20 2024-09-17T05:30:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v96'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e1980 con 0x5643b72d8d20 2024-09-17T05:30:42.517 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v96'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2d30 con 0x5643b72d8d20 2024-09-17T05:30:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.512+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e30d0 con 0x5643b72d8d20 2024-09-17T05:30:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.516+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.516+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e1d20 con 0x5643b72d8d20 2024-09-17T05:30:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.516+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.516+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e20c0 con 0x5643b72d8d20 2024-09-17T05:30:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v96'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e24b0 con 0x5643b72d8d20 2024-09-17T05:30:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v96'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e24b0 con 0x5643b72d8d20 2024-09-17T05:30:42.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v96'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:42.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.520+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.524+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.528+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v96'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.528+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.528+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v96'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.528+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.536 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v96'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v96'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v96'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.532+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.536+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v96'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.536+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.536+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v96'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.536+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v96'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v96'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v96'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.547 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.540+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.544+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v96'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.544+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.544+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v96'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.544+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.548+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v96'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.548+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.548+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v96'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.548+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v96'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.557 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:42.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v96'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v96'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.552+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v96'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e2460 con 0x5643b72d8d20 2024-09-17T05:30:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v96'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5643b72e2830 0x5643b72e5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e5ed0 con 0x5643b72e2830 2024-09-17T05:30:42.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5643b72e2830 0x5643b72e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.556+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5643b72e2830 0x5643b72e5990 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.560+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v96'151 uv132 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.560+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e6270 con 0x5643b72e2830 2024-09-17T05:30:42.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.560+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v96'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.560+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e6610 con 0x5643b72e2830 2024-09-17T05:30:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:42.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v96'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e6e60 con 0x5643b72e2830 2024-09-17T05:30:42.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v96'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7200 con 0x5643b72e2830 2024-09-17T05:30:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v96'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.564+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e75a0 con 0x5643b72e2830 2024-09-17T05:30:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v96'156 uv149 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e7940 con 0x5643b72e2830 2024-09-17T05:30:42.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v96'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.574 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:42.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v96'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.568+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v96'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v96'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v96'161 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v96'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.572+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.580 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:42.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.576+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v96'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.576+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ebeb0 con 0x5643b72e2830 2024-09-17T05:30:42.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.576+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v96'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.576+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ebeb0 con 0x5643b72e2830 2024-09-17T05:30:42.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v96'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v96'166 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v96'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.580+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.587 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:42.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v96'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e8cf0 con 0x5643b72e2830 2024-09-17T05:30:42.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v96'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e8cf0 con 0x5643b72e2830 2024-09-17T05:30:42.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v96'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.584+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72e7ce0 con 0x5643b72e2830 2024-09-17T05:30:42.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v96'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72e2830 2024-09-17T05:30:42.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5643b72e8360 0x5643b72ea780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72eade0 con 0x5643b72e8360 2024-09-17T05:30:42.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5643b72e8360 0x5643b72ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5643b72e8360 0x5643b72ea780 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v96'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.588+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72eb2a0 con 0x5643b72e8360 2024-09-17T05:30:42.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72eb780 con 0x5643b72e8360 2024-09-17T05:30:42.596 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:42.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ed030 con 0x5643b72e8360 2024-09-17T05:30:42.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.592+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ed3d0 con 0x5643b72e8360 2024-09-17T05:30:42.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v96'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ec020 con 0x5643b72e8360 2024-09-17T05:30:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ec410 con 0x5643b72d8d20 2024-09-17T05:30:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'31 uv30 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.596+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72ec410 con 0x5643b72d8d20 2024-09-17T05:30:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v96'32 uv32 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ec410 con 0x5643b72d8d20 2024-09-17T05:30:42.605 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v96'33 uv33 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72ec410 con 0x5643b72d8d20 2024-09-17T05:30:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v96'34 uv34 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 0x5643b72efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f00d0 con 0x5643b72ed770 2024-09-17T05:30:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.600+0000 7f99d5400640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 0x5643b72efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.604+0000 7f99d5400640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 0x5643b72efb90 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.604+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v96'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.604+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f00d0 con 0x5643b72ed770 2024-09-17T05:30:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f00d0 con 0x5643b72ed770 2024-09-17T05:30:42.614 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f10e0 con 0x5643b72ed770 2024-09-17T05:30:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.608+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f10e0 con 0x5643b72ed770 2024-09-17T05:30:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v96'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f00d0 con 0x5643b72ed770 2024-09-17T05:30:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5643b72f05e0 0x5643b72f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f30c0 con 0x5643b72f05e0 2024-09-17T05:30:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5643b72f05e0 0x5643b72f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.612+0000 7f99d4a00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5643b72f05e0 0x5643b72f2a00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.616+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v96'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.616+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v96'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v96'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.625 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f44a0 con 0x5643b72f05e0 2024-09-17T05:30:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v96'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.620+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f44a0 con 0x5643b72f05e0 2024-09-17T05:30:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v96'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v96'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v96'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.624+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v96'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v96'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.632 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:42.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v96'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f3490 con 0x5643b72f05e0 2024-09-17T05:30:42.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.628+0000 7f99d4a00640 1 -- 172.21.15.43:0/2666491898 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v96'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c800aae0 con 0x5643b72f05e0 2024-09-17T05:30:42.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5643b72f3db0 0x5643b72f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6870 con 0x5643b72f3db0 2024-09-17T05:30:42.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5643b72f3db0 0x5643b72f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99cfe00640 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5643b72f3db0 0x5643b72f41c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v96'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f6870 con 0x5643b72f3db0 2024-09-17T05:30:42.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v96'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.632+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6870 con 0x5643b72f3db0 2024-09-17T05:30:42.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v96'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.640 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72f3db0 2024-09-17T05:30:42.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v96'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72f3db0 2024-09-17T05:30:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v96'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.636+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72d8d20 2024-09-17T05:30:42.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v96'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72d8d20 2024-09-17T05:30:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72d8d20 2024-09-17T05:30:42.646 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.640+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7c70 con 0x5643b72d8d20 2024-09-17T05:30:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7c70 con 0x5643b72d8d20 2024-09-17T05:30:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v96'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72d8d20 2024-09-17T05:30:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72d8d20 2024-09-17T05:30:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.644+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v96'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v96'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.655 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v96'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.648+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.652+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v96'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.652+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.652+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v96'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.652+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v96'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v96'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.662 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:42.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v96'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.656+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f6c60 con 0x5643b72e8360 2024-09-17T05:30:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.660+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v96'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.660+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f9a70 con 0x5643b72e8360 2024-09-17T05:30:42.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.660+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v96'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72e8360 2024-09-17T05:30:42.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.660+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v96'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v96'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.670 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v96'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.664+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v96'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v96'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.668+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.676 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v96'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.672+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72ed770 2024-09-17T05:30:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.676+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99d0002050 con 0x5643b72ed770 2024-09-17T05:30:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.676+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.676+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v96'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.676+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v96'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v96'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.685 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:42.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v96'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.680+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v96'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v96'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v96'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.684+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.691 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:42.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v96'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v96'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v96'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.688+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v96'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v96'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.698 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v96'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.692+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v96'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v96'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f7110 con 0x5643b72f3db0 2024-09-17T05:30:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v96'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.696+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5643b72f74d0 con 0x5643b72f3db0 2024-09-17T05:30:42.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99cfe00640 1 -- 172.21.15.43:0/2666491898 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v96'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f99c002b070 con 0x5643b72f3db0 2024-09-17T05:30:42.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5643b72e2830 msgr2=0x5643b72e5990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5643b72e2830 0x5643b72e5990 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5643b72f05e0 msgr2=0x5643b72f2a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5643b72f05e0 0x5643b72f2a00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5643b72f3db0 msgr2=0x5643b72f41c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5643b72f3db0 0x5643b72f41c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5643b72e8360 msgr2=0x5643b72ea780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5643b72e8360 0x5643b72ea780 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 reap_dead start 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5643b72d8d20 msgr2=0x5643b71afd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5643b72d8d20 0x5643b71afd70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99d5400640 1 -- 172.21.15.43:0/2666491898 reap_dead start 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 msgr2=0x5643b72efb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 0x5643b72efb90 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 msgr2=0x7f99b40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 0x7f99b40740f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c670 tx=0x7f99c8004890 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 msgr2=0x5643b71b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f99c000bbe0 tx=0x7f99c000fea0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 shutdown_connections 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5643b72ed770 0x5643b72efb90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f99b4071be0 0x7f99b40740f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 --2- 172.21.15.43:0/2666491898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643b7197cb0 0x5643b71b0a80 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 >> 172.21.15.43:0/2666491898 conn(0x5643b7154d80 msgr2=0x5643b7239dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 shutdown_connections 2024-09-17T05:30:42.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.700+0000 7f99db77e080 1 -- 172.21.15.43:0/2666491898 wait complete. 2024-09-17T05:30:42.709 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-09-17T05:30:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 Processor -- start 2024-09-17T05:30:42.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 -- start start 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce7618d70 0x55fce765dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce76cee40 0x55fce765f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fce75c4f40 con 0x55fce7647cb0 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fce75d1d00 con 0x55fce76cee40 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc68b363080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55fce7566a00 con 0x55fce7618d70 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49874/0 (socket says 172.21.15.43:49874) 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 -- 172.21.15.43:0/1678622910 learned_addr learned my addr 172.21.15.43:0/1678622910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc684e00640 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce76cee40 0x55fce765f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc684400640 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce7618d70 0x55fce765dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 -- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce7618d70 msgr2=0x55fce765dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce7618d70 0x55fce765dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 -- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce76cee40 msgr2=0x55fce765f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.804+0000 7fc67be00640 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce76cee40 0x55fce765f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc67be00640 1 -- 172.21.15.43:0/1678622910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce7661330 con 0x55fce7647cb0 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc67be00640 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc66c001e80 tx=0x7fc66c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4290d8b95d5bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc683000640 1 -- 172.21.15.43:0/1678622910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00e840 con 0x55fce7647cb0 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc683000640 1 -- 172.21.15.43:0/1678622910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc66c00e9e0 con 0x55fce7647cb0 2024-09-17T05:30:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc683000640 1 -- 172.21.15.43:0/1678622910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00ee30 con 0x55fce7647cb0 2024-09-17T05:30:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- 172.21.15.43:0/1678622910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 msgr2=0x55fce765ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc66c001e80 tx=0x7fc66c00b740 comp rx=0 tx=0).stop 2024-09-17T05:30:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- 172.21.15.43:0/1678622910 shutdown_connections 2024-09-17T05:30:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce76cee40 0x55fce765f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce765ee90 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 --2- 172.21.15.43:0/1678622910 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce7618d70 0x55fce765dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- 172.21.15.43:0/1678622910 >> 172.21.15.43:0/1678622910 conn(0x55fce7604d80 msgr2=0x55fce7616890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- 172.21.15.43:0/1678622910 shutdown_connections 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- 172.21.15.43:0/1678622910 wait complete. 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 Processor -- start 2024-09-17T05:30:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 -- start start 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.808+0000 7fc68b363080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce7618d70 0x55fce76eae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce7777750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce76cee40 0x55fce777a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fce76608b0 con 0x55fce7647cb0 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fce7607320 con 0x55fce7618d70 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55fce7660b50 con 0x55fce76cee40 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce7777750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc684400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce7618d70 0x55fce76eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce7777750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49890/0 (socket says 172.21.15.43:49890) 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 -- 172.21.15.43:0/2360831259 learned_addr learned my addr 172.21.15.43:0/2360831259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc684e00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce76cee40 0x55fce777a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 -- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce76cee40 msgr2=0x55fce777a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55fce76cee40 0x55fce777a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 -- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce7618d70 msgr2=0x55fce76eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce7618d70 0x55fce76eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 -- 172.21.15.43:0/2360831259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce7778020 con 0x55fce7647cb0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc684400640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fce7618d70 0x55fce76eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc67be00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce7777750 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc66c017660 tx=0x7fc66c017690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc681c00640 1 -- 172.21.15.43:0/2360831259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c017a40 con 0x55fce7647cb0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc681c00640 1 -- 172.21.15.43:0/2360831259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc66c017be0 con 0x55fce7647cb0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc681c00640 1 -- 172.21.15.43:0/2360831259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c017df0 con 0x55fce7647cb0 2024-09-17T05:30:42.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fce77782c0 con 0x55fce7647cb0 2024-09-17T05:30:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce7778800 con 0x55fce7647cb0 2024-09-17T05:30:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.812+0000 7fc681c00640 1 -- 172.21.15.43:0/2360831259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc66c020400 con 0x55fce7647cb0 2024-09-17T05:30:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc681c00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc660071be0 0x7fc6600740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc681c00640 1 -- 172.21.15.43:0/2360831259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7fc66c094710 con 0x55fce7647cb0 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684400640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc660071be0 0x7fc6600740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55fce7616ea0 0x55fce7776270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x55fce77767b0 con 0x55fce7616ea0 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684e00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55fce7616ea0 0x55fce7776270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684400640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc660071be0 0x7fc6600740f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc6740af000 tx=0x7fc67401c250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684e00640 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55fce7616ea0 0x55fce7776270 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684e00640 1 -- 172.21.15.43:0/2360831259 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [delete] v96'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc67c002770 con 0x55fce7616ea0 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55fce7616ea0 msgr2=0x55fce7776270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55fce7616ea0 0x55fce7776270 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc660071be0 msgr2=0x7fc6600740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc660071be0 0x7fc6600740f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc6740af000 tx=0x7fc67401c250 comp rx=0 tx=0).stop 2024-09-17T05:30:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 msgr2=0x55fce7777750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 --2- 172.21.15.43:0/2360831259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fce7647cb0 0x55fce7777750 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fc66c017660 tx=0x7fc66c017690 comp rx=0 tx=0).stop 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc684e00640 1 -- 172.21.15.43:0/2360831259 reap_dead start 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 shutdown_connections 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 >> 172.21.15.43:0/2360831259 conn(0x55fce7604d80 msgr2=0x55fce76edcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 shutdown_connections 2024-09-17T05:30:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.816+0000 7fc68b363080 1 -- 172.21.15.43:0/2360831259 wait complete. 2024-09-17T05:30:42.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 -- 172.21.15.43:0/723765759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015c7e0 msgr2=0x7f414006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 --2- 172.21.15.43:0/723765759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015c7e0 0x7f414006f110 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4138009bc0 tx=0x7f413801cb10 comp rx=0 tx=0).stop 2024-09-17T05:30:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 -- 172.21.15.43:0/723765759 shutdown_connections 2024-09-17T05:30:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 --2- 172.21.15.43:0/723765759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015c7e0 0x7f414006f110 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 -- 172.21.15.43:0/723765759 >> 172.21.15.43:0/723765759 conn(0x7f41400b23d0 msgr2=0x7f41400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 -- 172.21.15.43:0/723765759 shutdown_connections 2024-09-17T05:30:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.984+0000 7f4147200640 1 -- 172.21.15.43:0/723765759 wait complete. 2024-09-17T05:30:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 Processor -- start 2024-09-17T05:30:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- start start 2024-09-17T05:30:42.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f414015c7e0 0x7f414015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4140169dc0 con 0x7f414015e540 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4140169c40 con 0x7f4140155d70 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4140169f40 con 0x7f414015c7e0 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f413fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f413f400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f414015c7e0 0x7f414015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49910/0 (socket says 172.21.15.43:49910) 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f413fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:47838/0 (socket says 172.21.15.43:47838) 2024-09-17T05:30:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 -- 172.21.15.43:0/2071126201 learned_addr learned my addr 172.21.15.43:0/2071126201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 -- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f414015c7e0 msgr2=0x7f414015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f414015c7e0 0x7f414015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 -- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 msgr2=0x7f41401fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 -- 172.21.15.43:0/2071126201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4140074a30 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f413fe00640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4145200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41380b0040 tx=0x7f4138005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380b2040 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41380041f0 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004600 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4140074d10 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.988+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4140160360 con 0x7f414015e540 2024-09-17T05:30:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414006f110 con 0x7f414015e540 2024-09-17T05:30:42.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f41380bd030 con 0x7f414015e540 2024-09-17T05:30:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f4137e00640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 0x7f4120075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f413812e5a0 con 0x7f414015e540 2024-09-17T05:30:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f413fe00640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 0x7f4120075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.992+0000 7f413fe00640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 0x7f4120075710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4130007fd0 tx=0x7f4130009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:42.996+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41380c4060 con 0x7f414015e540 2024-09-17T05:30:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.212+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f41401606d0 con 0x7f414015e540 2024-09-17T05:30:43.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.216+0000 7f4137e00640 1 -- 172.21.15.43:0/2071126201 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v129) ==== 162+0+0 (secure 0 0 0) 0x7f4138016790 con 0x7f414015e540 2024-09-17T05:30:43.222 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 msgr2=0x7f4120075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 0x7f4120075710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4130007fd0 tx=0x7f4130009040 comp rx=0 tx=0).stop 2024-09-17T05:30:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 msgr2=0x7f4140154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41380b0040 tx=0x7f4138005ae0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 shutdown_connections 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4120073200 0x7f4120075710 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f414015e540 0x7f4140154eb0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f414015c7e0 0x7f414015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 --2- 172.21.15.43:0/2071126201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4140155d70 0x7f41401fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 >> 172.21.15.43:0/2071126201 conn(0x7f41400b23d0 msgr2=0x7f414006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 shutdown_connections 2024-09-17T05:30:43.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.220+0000 7f4147200640 1 -- 172.21.15.43:0/2071126201 wait complete. 2024-09-17T05:30:43.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- 172.21.15.43:0/1555187887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb468070620 msgr2=0x7fb4680b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1555187887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb468070620 0x7fb4680b30d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb4680cfc10 tx=0x7fb45801c810 comp rx=0 tx=0).stop 2024-09-17T05:30:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- 172.21.15.43:0/1555187887 shutdown_connections 2024-09-17T05:30:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1555187887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb4680cb860 0x7fb4680cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1555187887 >> v2:172.21.15.62:3300/0 conn(0x7fb46806ce60 0x7fb46806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1555187887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb468070620 0x7fb4680b30d0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- 172.21.15.43:0/1555187887 >> 172.21.15.43:0/1555187887 conn(0x7fb4680b23d0 msgr2=0x7fb4680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- 172.21.15.43:0/1555187887 shutdown_connections 2024-09-17T05:30:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- 172.21.15.43:0/1555187887 wait complete. 2024-09-17T05:30:43.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 Processor -- start 2024-09-17T05:30:43.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.812+0000 7fb46ee00640 1 -- start start 2024-09-17T05:30:43.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 0x7fb4680c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb468070620 0x7fb46814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680ce490 con 0x7fb4680cb860 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680ce310 con 0x7fb468070620 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb4680ce610 con 0x7fb46806ce60 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb467e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 0x7fb4680c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb45fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb468070620 0x7fb46814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43354/0 (socket says 172.21.15.43:43354) 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb467e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 0x7fb4680c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:57736/0 (socket says 172.21.15.43:57736) 2024-09-17T05:30:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 -- 172.21.15.43:0/1925897691 learned_addr learned my addr 172.21.15.43:0/1925897691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 -- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 msgr2=0x7fb4680c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 0x7fb4680c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 -- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb468070620 msgr2=0x7fb46814b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb468070620 0x7fb46814b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 -- 172.21.15.43:0/1925897691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681fc6c0 con 0x7fb4680cb860 2024-09-17T05:30:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ce00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb46000cab0 tx=0x7fb46000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000ae20 con 0x7fb4680cb860 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb46000f560 con 0x7fb4680cb860 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000f9b0 con 0x7fb4680cb860 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681fc9a0 con 0x7fb4680cb860 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4680c4cd0 con 0x7fb4680cb860 2024-09-17T05:30:43.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.816+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4680c4f30 con 0x7fb4680cb860 2024-09-17T05:30:43.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.820+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb46000fb50 con 0x7fb4680cb860 2024-09-17T05:30:43.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.820+0000 7fb464c00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 0x7fb444075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.820+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7fb460094d80 con 0x7fb4680cb860 2024-09-17T05:30:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.820+0000 7fb467e00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 0x7fb444075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:43.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.820+0000 7fb467e00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 0x7fb444075710 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae000 tx=0x7fb45801b050 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:43.824+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb460014030 con 0x7fb4680cb860 2024-09-17T05:30:44.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4680c9de0 con 0x7fb4680cb860 2024-09-17T05:30:44.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb464c00640 1 -- 172.21.15.43:0/1925897691 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) ==== 95+0+5214 (secure 0 0 0) 0x7fb46005f570 con 0x7fb4680cb860 2024-09-17T05:30:44.040 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:44.041 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"btime":"2024-09-17T05:30:43:221445+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/352682704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":352682704},{"type":"v1","addr":"172.21.15.43:6835","nonce":352682704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:43.221417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"d","rank":0,"incarnation":129,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/2775198885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2775198885},{"type":"v1","addr":"172.21.15.62:6835","nonce":2775198885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:44.041 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 129 2024-09-17T05:30:44.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 msgr2=0x7fb444075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 0x7fb444075710 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae000 tx=0x7fb45801b050 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 msgr2=0x7fb46815a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fb46000cab0 tx=0x7fb46000cf70 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 shutdown_connections 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb444073200 0x7fb444075710 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb468070620 0x7fb46814b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4680cb860 0x7fb46815a150 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 --2- 172.21.15.43:0/1925897691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb46806ce60 0x7fb4680c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 >> 172.21.15.43:0/1925897691 conn(0x7fb4680b23d0 msgr2=0x7fb4680b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 shutdown_connections 2024-09-17T05:30:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:44.036+0000 7fb46ee00640 1 -- 172.21.15.43:0/1925897691 wait complete. 2024-09-17T05:30:44.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:30:43.221417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'd', 'rank': 0, 'incarnation': 129, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/2775198885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2775198885}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2775198885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:30:44.063 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7896:up:replay 2024-09-17T05:30:44.063 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:30:47.414 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:47.413+0000 7fbcd5000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:49.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- 172.21.15.43:0/419257198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540b7ce0 msgr2=0x7ff7540d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 --2- 172.21.15.43:0/419257198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540b7ce0 0x7ff7540d0840 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff7440098f0 tx=0x7ff74401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- 172.21.15.43:0/419257198 shutdown_connections 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 --2- 172.21.15.43:0/419257198 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540d0d80 0x7ff7540cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 --2- 172.21.15.43:0/419257198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540b7ce0 0x7ff7540d0840 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- 172.21.15.43:0/419257198 >> 172.21.15.43:0/419257198 conn(0x7ff7540cee80 msgr2=0x7ff7540cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- 172.21.15.43:0/419257198 shutdown_connections 2024-09-17T05:30:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- 172.21.15.43:0/419257198 wait complete. 2024-09-17T05:30:49.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 Processor -- start 2024-09-17T05:30:49.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.636+0000 7ff75ba00640 1 -- start start 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7540b7ce0 0x7ff7540cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540cd3b0 0x7ff754155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff754169e10 con 0x7ff7540d0d80 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff754169c90 con 0x7ff7540b7ce0 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff754169f90 con 0x7ff7540cd3b0 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff753e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540cd3b0 0x7ff754155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43364/0 (socket says 172.21.15.43:43364) 2024-09-17T05:30:49.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7540b7ce0 0x7ff7540cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:49.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 -- 172.21.15.43:0/761299398 learned_addr learned my addr 172.21.15.43:0/761299398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 -- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540cd3b0 msgr2=0x7ff754155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540cd3b0 0x7ff754155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 -- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7540b7ce0 msgr2=0x7ff7540cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7540b7ce0 0x7ff7540cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 -- 172.21.15.43:0/761299398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75415cff0 con 0x7ff7540d0d80 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff759a00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c730 tx=0x7ff74c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c003b80 con 0x7ff7540d0d80 2024-09-17T05:30:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff74c003d40 con 0x7ff7540d0d80 2024-09-17T05:30:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0052d0 con 0x7ff7540d0d80 2024-09-17T05:30:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75415cd10 con 0x7ff7540d0d80 2024-09-17T05:30:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75406f5a0 con 0x7ff7540d0d80 2024-09-17T05:30:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.640+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7540d0840 con 0x7ff7540d0d80 2024-09-17T05:30:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.644+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff74c005470 con 0x7ff7540d0d80 2024-09-17T05:30:49.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.644+0000 7ff750c00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 0x7ff734075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:49.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.644+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7ff74c094140 con 0x7ff7540d0d80 2024-09-17T05:30:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.644+0000 7ff759000640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 0x7ff734075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.644+0000 7ff759000640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 0x7ff734075740 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7440af000 tx=0x7ff74401b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:49.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.648+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff75415cff0 con 0x7ff7540d0d80 2024-09-17T05:30:49.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7540b0990 con 0x7ff7540d0d80 2024-09-17T05:30:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff750c00640 1 -- 172.21.15.43:0/761299398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5221 (secure 0 0 0) 0x7ff74c05e8a0 con 0x7ff7540d0d80 2024-09-17T05:30:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:49.860 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":132,"btime":"2024-09-17T05:30:46:227643+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/352682704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":352682704},{"type":"v1","addr":"172.21.15.43:6835","nonce":352682704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:46.227634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"d","rank":0,"incarnation":129,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2775198885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2775198885},{"type":"v1","addr":"172.21.15.62:6835","nonce":2775198885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":6}]} 2024-09-17T05:30:49.860 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 132 2024-09-17T05:30:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 msgr2=0x7ff734075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 0x7ff734075740 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7440af000 tx=0x7ff74401b120 comp rx=0 tx=0).stop 2024-09-17T05:30:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 msgr2=0x7ff754155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c730 tx=0x7ff74c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 shutdown_connections 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff734073230 0x7ff734075740 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7540d0d80 0x7ff754155db0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7540cd3b0 0x7ff754155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 --2- 172.21.15.43:0/761299398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7540b7ce0 0x7ff7540cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 >> 172.21.15.43:0/761299398 conn(0x7ff7540cee80 msgr2=0x7ff7540726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 shutdown_connections 2024-09-17T05:30:49.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:49.856+0000 7ff75ba00640 1 -- 172.21.15.43:0/761299398 wait complete. 2024-09-17T05:30:49.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:30:46.227634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'd', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6835/2775198885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2775198885}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2775198885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7896, 'qdb_cluster': [7896]} 2024-09-17T05:30:49.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:30:49.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 -- 172.21.15.43:0/1593189409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0650 msgr2=0x7f9d1c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 --2- 172.21.15.43:0/1593189409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0cd750 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009bc0 tx=0x7f9d1401cac0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 -- 172.21.15.43:0/1593189409 shutdown_connections 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 --2- 172.21.15.43:0/1593189409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0cd750 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 -- 172.21.15.43:0/1593189409 >> 172.21.15.43:0/1593189409 conn(0x7f9d1c0cee60 msgr2=0x7f9d1c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 -- 172.21.15.43:0/1593189409 shutdown_connections 2024-09-17T05:30:50.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.040+0000 7f9d23400640 1 -- 172.21.15.43:0/1593189409 wait complete. 2024-09-17T05:30:50.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 Processor -- start 2024-09-17T05:30:50.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 -- start start 2024-09-17T05:30:50.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d1c09ba60 0x7f9d1c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 0x7f9d1c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c16aa80 con 0x7f9d1c0d0650 2024-09-17T05:30:50.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c16a900 con 0x7f9d1c0d0240 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d23400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c16ac00 con 0x7f9d1c09ba60 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43374/0 (socket says 172.21.15.43:43374) 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 -- 172.21.15.43:0/2192841234 learned_addr learned my addr 172.21.15.43:0/2192841234 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d20a00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d1c09ba60 0x7f9d1c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 0x7f9d1c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 -- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d1c09ba60 msgr2=0x7f9d1c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d1c09ba60 0x7f9d1c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 -- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 msgr2=0x7f9d1c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 0x7f9d1c15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 -- 172.21.15.43:0/2192841234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c15e2e0 con 0x7f9d1c0d0650 2024-09-17T05:30:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 0x7f9d1c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d21400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b10 tx=0x7f9d14003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.044+0000 7f9d18c00640 1 -- 172.21.15.43:0/2192841234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140bb070 con 0x7f9d1c0d0650 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d18c00640 1 -- 172.21.15.43:0/2192841234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d14004d40 con 0x7f9d1c0d0650 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c15e5c0 con 0x7f9d1c0d0650 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d18c00640 1 -- 172.21.15.43:0/2192841234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140b2040 con 0x7f9d1c0d0650 2024-09-17T05:30:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c15eb80 con 0x7f9d1c0d0650 2024-09-17T05:30:50.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d18c00640 1 -- 172.21.15.43:0/2192841234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9d140be070 con 0x7f9d1c0d0650 2024-09-17T05:30:50.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.048+0000 7f9d2a98f000 1 Processor -- start 2024-09-17T05:30:50.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 -- start start 2024-09-17T05:30:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d18c00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 0x7f9cfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d18c00640 1 -- 172.21.15.43:0/2192841234 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f9d1412e1a0 con 0x7f9d1c0d0650 2024-09-17T05:30:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 0x7f9cfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19fe486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19fe48be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c19fd1d410 con 0x55c19fce0ca0 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c19fc5ab40 con 0x55c19f907460 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c19fc49f70 con 0x55c19fcf7950 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43382/0 (socket says 172.21.15.43:43382) 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 -- 172.21.15.43:0/2197992632 learned_addr learned my addr 172.21.15.43:0/2197992632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d20a00640 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19fe486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 0x7f9cfc075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001230 tx=0x7f9d08009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19fe48be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 -- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 msgr2=0x55c19fe48be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19fe48be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 -- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 msgr2=0x55c19fe486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19fe486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 -- 172.21.15.43:0/2197992632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19fc115b0 con 0x55c19fce0ca0 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d21400640 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9d140f87f0 tx=0x7f9d140f8630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baaf4b49aa4d05c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d0d200640 1 -- 172.21.15.43:0/2197992632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140f8370 con 0x55c19fce0ca0 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d0d200640 1 -- 172.21.15.43:0/2197992632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d140f8f60 con 0x55c19fce0ca0 2024-09-17T05:30:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.052+0000 7f9d0d200640 1 -- 172.21.15.43:0/2197992632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1411a0d0 con 0x55c19fce0ca0 2024-09-17T05:30:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2197992632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 msgr2=0x55c19fe49120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9d140f87f0 tx=0x7f9d140f8630 comp rx=0 tx=0).stop 2024-09-17T05:30:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2197992632 shutdown_connections 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19fe49120 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19fe48be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2197992632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19fe486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2197992632 >> 172.21.15.43:0/2197992632 conn(0x55c19fdcdd90 msgr2=0x55c19fdfd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2197992632 shutdown_connections 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2197992632 wait complete. 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- start start 2024-09-17T05:30:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 Processor -- start 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19fe4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19fe503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c19fbe45f0 con 0x55c19f907460 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c19fc4dd80 con 0x55c19fcf7950 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c19fbf0520 con 0x55c19fce0ca0 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d2a98f000 10 client.0 _ll_register_callbacks cb 0x55c19fbf4830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43398/0 (socket says 172.21.15.43:43398) 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 -- 172.21.15.43:0/1182381457 learned_addr learned my addr 172.21.15.43:0/1182381457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d1be00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19fe4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d21400640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19fe503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 -- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 msgr2=0x55c19fe4fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19fe4fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 -- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 msgr2=0x55c19fe503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19fe503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d20a00640 1 -- 172.21.15.43:0/1182381457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19fbe8190 con 0x55c19f907460 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.056+0000 7f9d21400640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19fe503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d20a00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9d10001e30 tx=0x7f9d100116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8d2a6e878591e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100052e0 con 0x55c19f907460 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f9d10003600 con 0x55c19f907460 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c19fbe2110 con 0x55c19f907460 2024-09-17T05:30:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7f9d10022050 con 0x55c19f907460 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 10 client.7956 fetch_fsmap learned FSMap version 132 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) -- 0x55c19fccef10 con 0x55c19f907460 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c19fbe2110 con 0x55c19f907460 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 132) ==== 3502+0+0 (secure 0 0 0) 0x7f9d10025020 con 0x55c19f907460 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 10 client.7956 fetch_fsmap finished waiting for FSMap version 132 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 10 client.7956 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 20 client.7956 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8ec7d0 0x55c19fe59d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 4 client.7956 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:30:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 --> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c19fc5ca90 con 0x55c19f8ec7d0 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d1be00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8ec7d0 0x55c19fe59d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d1be00640 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8ec7d0 0x55c19fe59d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de68abef5a40b65b server_cookie=63408026b23a6cae in_seq=0 out_seq=0 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 10 client.7956 ms_handle_connect on v2:172.21.15.62:6834/2775198885 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 1 -- 172.21.15.43:0/1182381457 <== mds.0 v2:172.21.15.62:6834/2775198885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c19fc5ca90 con 0x55c19f8ec7d0 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d01600640 10 client.7956 handle_command_reply: tid=0 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 client.7956 shutdown 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 msgr2=0x55c19fe4f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.060+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9d10001e30 tx=0x7f9d100116e0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 shutdown_connections 2024-09-17T05:30:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8ec7d0 0x55c19fe59d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19fe503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19fe4fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/1182381457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19fe4f920 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d01600640 0 client.7956 ms_handle_reset on v2:172.21.15.62:6834/2775198885 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 >> 172.21.15.43:0/1182381457 conn(0x55c19fdcdd90 msgr2=0x55c19fe04440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 shutdown_connections 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 1 -- 172.21.15.43:0/1182381457 wait complete. 2024-09-17T05:30:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.064+0000 7f9d2a98f000 20 client.7956 trim_cache size 0 max 16384 2024-09-17T05:30:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.080+0000 7f9d2a98f000 1 Processor -- start 2024-09-17T05:30:50.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.080+0000 7f9d2a98f000 1 -- start start 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19feba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19febb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c205310 con 0x55c19fce0ca0 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c204310 con 0x55c19f907460 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c0c84c0 con 0x55c19fcf7950 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43412/0 (socket says 172.21.15.43:43412) 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 -- 172.21.15.43:0/2090510761 learned_addr learned my addr 172.21.15.43:0/2090510761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d21400640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19febb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d20a00640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19feba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 -- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 msgr2=0x55c19febb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19febb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 -- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 msgr2=0x55c19feba620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19feba620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 -- 172.21.15.43:0/2090510761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19fca5db0 con 0x55c19fce0ca0 2024-09-17T05:30:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d20a00640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19feba620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d1be00640 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001a10 tx=0x7f9d08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6599013da9de84bc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d03e00640 1 -- 172.21.15.43:0/2090510761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800ee50 con 0x55c19fce0ca0 2024-09-17T05:30:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d03e00640 1 -- 172.21.15.43:0/2090510761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d08015880 con 0x55c19fce0ca0 2024-09-17T05:30:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d03e00640 1 -- 172.21.15.43:0/2090510761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08015ab0 con 0x55c19fce0ca0 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2090510761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 msgr2=0x55c19febab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001a10 tx=0x7f9d08014000 comp rx=0 tx=0).stop 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2090510761 shutdown_connections 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fcf7950 0x55c19febb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19fce0ca0 0x55c19febab60 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/2090510761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19f907460 0x55c19feba620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2090510761 >> 172.21.15.43:0/2090510761 conn(0x55c19fdcdd90 msgr2=0x55c19fe46620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2090510761 shutdown_connections 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.084+0000 7f9d2a98f000 1 -- 172.21.15.43:0/2090510761 wait complete. 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- start start 2024-09-17T05:30:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 Processor -- start 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19febe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19febee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c19fc15ee0 con 0x55c19f907460 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c19fd09750 con 0x55c19fcf7950 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c19fccea80 con 0x55c19fce0ca0 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 10 client.0 _ll_register_callbacks cb 0x7f9d10001240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43420/0 (socket says 172.21.15.43:43420) 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 -- 172.21.15.43:0/3830057002 learned_addr learned my addr 172.21.15.43:0/3830057002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d1be00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19febe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d21400640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19febee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 -- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 msgr2=0x55c19febe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19febe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 -- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 msgr2=0x55c19febee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19febee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 -- 172.21.15.43:0/3830057002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19fc10ab0 con 0x55c19f907460 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d20a00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9d10001ec0 tx=0x7f9d10003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a834736b763c5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:30:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100052e0 con 0x55c19f907460 2024-09-17T05:30:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f9d1001d560 con 0x55c19f907460 2024-09-17T05:30:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d08015880 con 0x55c19f907460 2024-09-17T05:30:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7f9d10035050 con 0x55c19f907460 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 10 client.7962 fetch_fsmap learned FSMap version 132 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) -- 0x55c19fbef070 con 0x55c19f907460 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d08015880 con 0x55c19f907460 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 132) ==== 3502+0+0 (secure 0 0 0) 0x7f9d10038020 con 0x55c19f907460 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 10 client.7962 fetch_fsmap finished waiting for FSMap version 132 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 10 client.7962 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 20 client.7962 populate_metadata read hostname 'smithi043' 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8da830 0x55c19fe7a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 4 client.7962 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.088+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 --> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] -- command(tid 0: {"prefix": "status"}) -- 0x55c19fc58540 con 0x55c19f8da830 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d1be00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8da830 0x55c19fe7a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d1be00640 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8da830 0x55c19fe7a810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85863132442ba7c server_cookie=15fb1e45db3269 in_seq=0 out_seq=0 2024-09-17T05:30:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d01600640 10 client.7962 ms_handle_connect on v2:172.21.15.62:6834/2775198885 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d01600640 1 -- 172.21.15.43:0/3830057002 <== mds.0 v2:172.21.15.62:6834/2775198885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c19fc58540 con 0x55c19f8da830 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d01600640 10 client.7962 handle_command_reply: tid=0 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 client.7962 shutdown 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 msgr2=0x55c19febe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9d10001ec0 tx=0x7f9d10003f50 comp rx=0 tx=0).stop 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 shutdown_connections 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:6834/2775198885,v1:172.21.15.62:6835/2775198885] conn(0x55c19f8da830 0x55c19fe7a810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c19fcf7950 0x55c19febee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c19fce0ca0 0x55c19febe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 --2- 172.21.15.43:0/3830057002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19f907460 0x55c19febe390 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d01600640 0 client.7962 ms_handle_reset on v2:172.21.15.62:6834/2775198885 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 >> 172.21.15.43:0/3830057002 conn(0x55c19fdcdd90 msgr2=0x55c19fe46fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 shutdown_connections 2024-09-17T05:30:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 1 -- 172.21.15.43:0/3830057002 wait complete. 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d2a98f000 20 client.7962 trim_cache size 0 max 16384 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7896, 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8733574759999998, 2024-09-17T05:30:50.100 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 132, 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 96, 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 96, 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 8.5624593870000005 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 msgr2=0x7f9cfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 0x7f9cfc075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001230 tx=0x7f9d08009040 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 msgr2=0x7f9d1c0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009b10 tx=0x7f9d14003bb0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 shutdown_connections 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9cfc073200 0x7f9cfc075710 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d1c0d0240 0x7f9d1c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d1c0d0650 0x7f9d1c0b1d40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 --2- 172.21.15.43:0/2192841234 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9d1c09ba60 0x7f9d1c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 >> 172.21.15.43:0/2192841234 conn(0x7f9d1c0cee60 msgr2=0x7f9d1c0b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 shutdown_connections 2024-09-17T05:30:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.092+0000 7f9d23400640 1 -- 172.21.15.43:0/2192841234 wait complete. 2024-09-17T05:30:50.117 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:30:50.117 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-09-17T05:30:50.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:30:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- 172.21.15.43:0/3327718765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f006fc90 msgr2=0x7fd6f00700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 --2- 172.21.15.43:0/3327718765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f006fc90 0x7fd6f00700a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00ccb60 tx=0x7fd6d801c7a0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- 172.21.15.43:0/3327718765 shutdown_connections 2024-09-17T05:30:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 --2- 172.21.15.43:0/3327718765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f006fc90 0x7fd6f00700a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 --2- 172.21.15.43:0/3327718765 >> v2:172.21.15.62:3300/0 conn(0x7fd6f015fe10 0x7fd6f00b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- 172.21.15.43:0/3327718765 >> 172.21.15.43:0/3327718765 conn(0x7fd6f00b23d0 msgr2=0x7fd6f00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- 172.21.15.43:0/3327718765 shutdown_connections 2024-09-17T05:30:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- 172.21.15.43:0/3327718765 wait complete. 2024-09-17T05:30:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 Processor -- start 2024-09-17T05:30:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.272+0000 7fd6f6400640 1 -- start start 2024-09-17T05:30:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 0x7fd6f015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6f00730b0 0x7fd6f0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 0x7fd6f01537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f016a000 con 0x7fd6f015fe10 2024-09-17T05:30:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0169e80 con 0x7fd6f006fc90 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f016a180 con 0x7fd6f00730b0 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 0x7fd6f01537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 0x7fd6f015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 0x7fd6f015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:46760/0 (socket says 172.21.15.43:46760) 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6ef400640 1 -- 172.21.15.43:0/2819744407 learned_addr learned my addr 172.21.15.43:0/2819744407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6e7e00640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6f00730b0 0x7fd6f0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 -- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6f00730b0 msgr2=0x7fd6f0153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6f00730b0 0x7fd6f0153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 -- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 msgr2=0x7fd6f015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 0x7fd6f015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 -- 172.21.15.43:0/2819744407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0153d00 con 0x7fd6f015fe10 2024-09-17T05:30:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6efe00640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 0x7fd6f01537c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c730 tx=0x7fd6e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8017070 con 0x7fd6f015fe10 2024-09-17T05:30:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e8003c70 con 0x7fd6f015fe10 2024-09-17T05:30:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8003e60 con 0x7fd6f015fe10 2024-09-17T05:30:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f0099940 con 0x7fd6f015fe10 2024-09-17T05:30:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f0156dd0 con 0x7fd6f015fe10 2024-09-17T05:30:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f00700a0 con 0x7fd6f015fe10 2024-09-17T05:30:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.276+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6e801a070 con 0x7fd6f015fe10 2024-09-17T05:30:50.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.280+0000 7fd6e7400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 0x7fd6cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:50.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.280+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7fd6e8093920 con 0x7fd6f015fe10 2024-09-17T05:30:50.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.280+0000 7fd6ef400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 0x7fd6cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.280+0000 7fd6ef400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 0x7fd6cc075710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0005fd0 tx=0x7fd6e000b410 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.284+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6e805e080 con 0x7fd6f015fe10 2024-09-17T05:30:50.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.492+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd6f00c9de0 con 0x7fd6f015fe10 2024-09-17T05:30:50.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.492+0000 7fd6e7400640 1 -- 172.21.15.43:0/2819744407 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) ==== 53+0+73 (secure 0 0 0) 0x7fd6e8062ab0 con 0x7fd6f015fe10 2024-09-17T05:30:50.498 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 msgr2=0x7fd6cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 0x7fd6cc075710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0005fd0 tx=0x7fd6e000b410 comp rx=0 tx=0).stop 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 msgr2=0x7fd6f01537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 0x7fd6f01537c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c730 tx=0x7fd6e800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 shutdown_connections 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd6cc073200 0x7fd6cc075710 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6f015fe10 0x7fd6f01537c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6f00730b0 0x7fd6f0153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 --2- 172.21.15.43:0/2819744407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f006fc90 0x7fd6f015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:50.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 >> 172.21.15.43:0/2819744407 conn(0x7fd6f00b23d0 msgr2=0x7fd6f0073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:50.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 shutdown_connections 2024-09-17T05:30:50.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50.496+0000 7fd6f6400640 1 -- 172.21.15.43:0/2819744407 wait complete. 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:30:50.521 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:50.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.522 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:30:50.531 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:30:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:30:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:30:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.531 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41993sec preferred_lft 41993sec 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.532 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:30:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:50.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.535 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:30:50.592 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:30:50.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.595 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:30:50.644 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:30:50.644 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:30:50.645 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:30:50.645 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:30:50.645 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:30:50.645 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:30:50.645 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:30:50.645 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:30:50.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:30:50.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:30:50.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:30:50.646 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:30:50.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:30:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:50.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.784 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:30:50.793 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:30:50.793 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:30:50.793 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:50.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:30:50.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:50.859 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:50.915 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:50.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:50.915 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:50.966 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:30:50.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:50.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:51.069 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:51.069 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:51.069 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:51.087 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:51.087 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:51.087 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:51.088 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:51.088 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:51.099 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:51.099 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:51.099 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:30:51.099 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:30:51.100 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:51.100 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:30:51.100 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:30:51.100 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:30:51.103 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:30:51.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:51.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:51.123 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:30:51.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:30:51.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:51.225 INFO:teuthology.orchestra.run.smithi043.stdout:client7579 2024-09-17T05:30:51.225 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:51.225 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:51.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:51.235 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:30:51.285 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:30:51.294 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:30:51.294 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:30:51.294 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:51.294 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:30:51.294 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:30:51.295 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-09-17T05:30:51.295 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-09-17T05:30:51.295 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:51.296 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:51.347 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:51.347 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:51.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 -- 172.21.15.43:0/1672323664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3680b3af0 msgr2=0x7fe3680ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 --2- 172.21.15.43:0/1672323664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3680b3af0 0x7fe3680ad0b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe360009900 tx=0x7fe36001c820 comp rx=0 tx=0).stop 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 -- 172.21.15.43:0/1672323664 shutdown_connections 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 --2- 172.21.15.43:0/1672323664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3680b3af0 0x7fe3680ad0b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 --2- 172.21.15.43:0/1672323664 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368155d70 0x7fe36815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 -- 172.21.15.43:0/1672323664 >> 172.21.15.43:0/1672323664 conn(0x7fe3680b23d0 msgr2=0x7fe3680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 -- 172.21.15.43:0/1672323664 shutdown_connections 2024-09-17T05:30:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.592+0000 7fe36e800640 1 -- 172.21.15.43:0/1672323664 wait complete. 2024-09-17T05:30:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 Processor -- start 2024-09-17T05:30:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- start start 2024-09-17T05:30:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 0x7fe368074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368164770 0x7fe368164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe368168c80 con 0x7fe368155d70 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe368168b00 con 0x7fe36806f110 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe368168e00 con 0x7fe368164770 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43464/0 (socket says 172.21.15.43:43464) 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 -- 172.21.15.43:0/1373186835 learned_addr learned my addr 172.21.15.43:0/1373186835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe367400640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 0x7fe368074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe367e00640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368164770 0x7fe368164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 -- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368164770 msgr2=0x7fe368164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368164770 0x7fe368164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 -- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 msgr2=0x7fe368074a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 0x7fe368074a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 -- 172.21.15.43:0/1373186835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3681650c0 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe367400640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 0x7fe368074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35ea00640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c3c0 tx=0x7fe35000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350017070 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3500053c0 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350003860 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe368074f60 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.596+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe368073510 con 0x7fe368155d70 2024-09-17T05:30:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3680ad0b0 con 0x7fe368155d70 2024-09-17T05:30:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe35001a070 con 0x7fe368155d70 2024-09-17T05:30:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe35fe00640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 0x7fe344075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7fe3500937a0 con 0x7fe368155d70 2024-09-17T05:30:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe367400640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 0x7fe344075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.600+0000 7fe367400640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 0x7fe344075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3680715c0 tx=0x7fe358009850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:51.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.604+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3681650c0 con 0x7fe368155d70 2024-09-17T05:30:51.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:51.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.788+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3680ca2f0 con 0x7fe368155d70 2024-09-17T05:30:51.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe35fe00640 1 -- 172.21.15.43:0/1373186835 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v96) ==== 81+0+850 (secure 0 0 0) 0x7fe35005df00 con 0x7fe368155d70 2024-09-17T05:30:51.796 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:30:51.796 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:30:51.796 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:30:51.797 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:30:51.798 INFO:teuthology.orchestra.run.smithi043.stderr:listed 14 entries 2024-09-17T05:30:51.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 msgr2=0x7fe344075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:51.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 0x7fe344075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3680715c0 tx=0x7fe358009850 comp rx=0 tx=0).stop 2024-09-17T05:30:51.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 msgr2=0x7fe368155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe35000c3c0 tx=0x7fe35000c880 comp rx=0 tx=0).stop 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 shutdown_connections 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe344073200 0x7fe344075710 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe368164770 0x7fe368164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe368155d70 0x7fe368155220 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 --2- 172.21.15.43:0/1373186835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe36806f110 0x7fe368074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 >> 172.21.15.43:0/1373186835 conn(0x7fe3680b23d0 msgr2=0x7fe36815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 shutdown_connections 2024-09-17T05:30:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:51.792+0000 7fe36e800640 1 -- 172.21.15.43:0/1373186835 wait complete. 2024-09-17T05:30:51.819 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:30:51.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:51.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:30:51.890 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:51.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:51.890 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:51.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:52.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 -- 172.21.15.43:0/20503320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ac004810 msgr2=0x7f48ac004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 --2- 172.21.15.43:0/20503320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ac004810 0x7f48ac004c20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009b40 tx=0x7f48b001d270 comp rx=0 tx=0).stop 2024-09-17T05:30:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cd000640 1 -- 172.21.15.43:0/20503320 reap_dead start 2024-09-17T05:30:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 -- 172.21.15.43:0/20503320 shutdown_connections 2024-09-17T05:30:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 -- 172.21.15.43:0/20503320 >> 172.21.15.43:0/20503320 conn(0x7f48c80b23d0 msgr2=0x7f48c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 -- 172.21.15.43:0/20503320 shutdown_connections 2024-09-17T05:30:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.140+0000 7f48cf000640 1 -- 172.21.15.43:0/20503320 wait complete. 2024-09-17T05:30:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 Processor -- start 2024-09-17T05:30:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 -- start start 2024-09-17T05:30:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 0x7f48c80ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80add60 0x7f48c80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48c815c960 con 0x7f48c809d960 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48c815c7e0 con 0x7f48c80add60 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cf000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f48c815cae0 con 0x7f48ac004810 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48c7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 0x7f48c80ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43482/0 (socket says 172.21.15.43:43482) 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 -- 172.21.15.43:0/2637547138 learned_addr learned my addr 172.21.15.43:0/2637547138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48cd000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80add60 0x7f48c80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 -- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 msgr2=0x7f48c80ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 0x7f48c80ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 -- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80add60 msgr2=0x7f48c80ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80add60 0x7f48c80ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48bfe00640 1 -- 172.21.15.43:0/2637547138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c80cae20 con 0x7f48c809d960 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.144+0000 7f48c7e00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 0x7f48c80ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48bfe00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f48b001d0a0 tx=0x7f48b00050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00bb030 con 0x7f48c809d960 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b0004840 con 0x7f48c809d960 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0004c70 con 0x7f48c809d960 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c80c81a0 con 0x7f48c809d960 2024-09-17T05:30:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c80c7630 con 0x7f48c809d960 2024-09-17T05:30:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c80ad980 con 0x7f48c809d960 2024-09-17T05:30:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48b00be030 con 0x7f48c809d960 2024-09-17T05:30:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 0x7f48a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6311+0+0 (secure 0 0 0) 0x7f48b012fa10 con 0x7f48c809d960 2024-09-17T05:30:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.148+0000 7f48c7e00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 0x7f48a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.152+0000 7f48c7e00640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 0x7f48a4075710 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f48b8001250 tx=0x7f48b80068b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:52.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.152+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48b0016f80 con 0x7f48c809d960 2024-09-17T05:30:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.360+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f48c815a8b0 con 0x7f48c809d960 2024-09-17T05:30:52.367 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:52.360+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.364+0000 7f48c4c00640 1 -- 172.21.15.43:0/2637547138 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) ==== 161+0+0 (secure 0 0 0) 0x7f48b01010d0 con 0x7f48c809d960 2024-09-17T05:30:52.369 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 msgr2=0x7f48a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 0x7f48a4075710 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f48b8001250 tx=0x7f48b80068b0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 msgr2=0x7f48c815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f48b001d0a0 tx=0x7f48b00050a0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 shutdown_connections 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48a4073200 0x7f48a4075710 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c80add60 0x7f48c80ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c809d960 0x7f48c815acd0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 --2- 172.21.15.43:0/2637547138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f48ac004810 0x7f48c80ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 >> 172.21.15.43:0/2637547138 conn(0x7f48c80b23d0 msgr2=0x7f48c80b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 shutdown_connections 2024-09-17T05:30:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.368+0000 7f48cf000640 1 -- 172.21.15.43:0/2637547138 wait complete. 2024-09-17T05:30:52.379 INFO:tasks.ceph.mds.d.smithi062.stderr: -677> 2024-09-17T05:30:47.413+0000 7fbcd5000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:52.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:52.504 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 -- 172.21.15.43:0/187781745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c80b3af0 msgr2=0x7ff0c815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 --2- 172.21.15.43:0/187781745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c80b3af0 0x7ff0c815c7e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009870 tx=0x7ff0b001c5a0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 -- 172.21.15.43:0/187781745 shutdown_connections 2024-09-17T05:30:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 --2- 172.21.15.43:0/187781745 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c80b3af0 0x7ff0c815c7e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 -- 172.21.15.43:0/187781745 >> 172.21.15.43:0/187781745 conn(0x7ff0c80b23d0 msgr2=0x7ff0c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 -- 172.21.15.43:0/187781745 shutdown_connections 2024-09-17T05:30:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.548+0000 7ff0cd000640 1 -- 172.21.15.43:0/187781745 wait complete. 2024-09-17T05:30:52.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 Processor -- start 2024-09-17T05:30:52.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 -- start start 2024-09-17T05:30:52.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 0x7ff0c8155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c8155d70 0x7ff0c809d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8169e50 con 0x7ff0c809da00 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8169cd0 con 0x7ff0c806f110 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0cd000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c8169fd0 con 0x7ff0c8155d70 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43496/0 (socket says 172.21.15.43:43496) 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.552+0000 7ff0c6800640 1 -- 172.21.15.43:0/761489253 learned_addr learned my addr 172.21.15.43:0/761489253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0bd400640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c8155d70 0x7ff0c809d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c5e00640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 0x7ff0c8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 -- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c8155d70 msgr2=0x7ff0c809d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c8155d70 0x7ff0c809d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 -- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 msgr2=0x7ff0c8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 0x7ff0c8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 -- 172.21.15.43:0/761489253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c80caa10 con 0x7ff0c809da00 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c5e00640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 0x7ff0c8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0c6800640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0007fd0 tx=0x7ff0c000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0017070 con 0x7ff0c809da00 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0c0003c90 con 0x7ff0c809da00 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00052d0 con 0x7ff0c809da00 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c80cacf0 con 0x7ff0c809da00 2024-09-17T05:30:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c80c8880 con 0x7ff0c809da00 2024-09-17T05:30:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c80d1150 con 0x7ff0c809da00 2024-09-17T05:30:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.556+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff0c001a070 con 0x7ff0c809da00 2024-09-17T05:30:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.560+0000 7ff0bea00640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 0x7ff0a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.560+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6397+0+0 (secure 0 0 0) 0x7ff0c0093a30 con 0x7ff0c809da00 2024-09-17T05:30:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.560+0000 7ff0c5e00640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 0x7ff0a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.560+0000 7ff0c5e00640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 0x7ff0a4075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8000f70 tx=0x7ff0b8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.564+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff0c80d1150 con 0x7ff0c809da00 2024-09-17T05:30:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:52.796+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff0c815c7e0 con 0x7ff0c809da00 2024-09-17T05:30:53.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0bea00640 1 -- 172.21.15.43:0/761489253 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v134) ==== 107+0+0 (secure 0 0 0) 0x7ff0c005e1d0 con 0x7ff0c809da00 2024-09-17T05:30:53.444 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 msgr2=0x7ff0a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 0x7ff0a4075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8000f70 tx=0x7ff0b8009040 comp rx=0 tx=0).stop 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 msgr2=0x7ff0c80c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0007fd0 tx=0x7ff0c000cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 shutdown_connections 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff0a4073200 0x7ff0a4075710 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c809da00 0x7ff0c80c5f00 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff0c8155d70 0x7ff0c809d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 --2- 172.21.15.43:0/761489253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0c806f110 0x7ff0c8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 >> 172.21.15.43:0/761489253 conn(0x7ff0c80b23d0 msgr2=0x7ff0c806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 shutdown_connections 2024-09-17T05:30:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.440+0000 7ff0cd000640 1 -- 172.21.15.43:0/761489253 wait complete. 2024-09-17T05:30:53.472 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.528+0000 7f7ddf676080 1 Processor -- start 2024-09-17T05:30:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.528+0000 7f7ddf676080 1 -- start start 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad90e2cb0 0x558ad916a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916b010 0x558ad916a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ad9001a00 con 0x558ad916c8b0 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558ad905ff40 con 0x558ad90e2cb0 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558ad90b65b0 con 0x558ad916b010 2024-09-17T05:30:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43512/0 (socket says 172.21.15.43:43512) 2024-09-17T05:30:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd8800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad90e2cb0 0x558ad916a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 -- 172.21.15.43:0/1980675470 learned_addr learned my addr 172.21.15.43:0/1980675470 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916b010 0x558ad916a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 -- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916b010 msgr2=0x558ad916a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916b010 0x558ad916a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 -- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad90e2cb0 msgr2=0x558ad916a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad90e2cb0 0x558ad916a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 -- 172.21.15.43:0/1980675470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad8fdac10 con 0x558ad916c8b0 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd9200640 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0001600 tx=0x7f7dd0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b79c944833a6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd6a00640 1 -- 172.21.15.43:0/1980675470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0016070 con 0x558ad916c8b0 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd6a00640 1 -- 172.21.15.43:0/1980675470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dd000f800 con 0x558ad916c8b0 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7dd6a00640 1 -- 172.21.15.43:0/1980675470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000fc30 con 0x558ad916c8b0 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- 172.21.15.43:0/1980675470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 msgr2=0x558ad90fa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0001600 tx=0x7f7dd0001ac0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- 172.21.15.43:0/1980675470 shutdown_connections 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad916c8b0 0x558ad90fa280 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916b010 0x558ad916a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 --2- 172.21.15.43:0/1980675470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad90e2cb0 0x558ad916a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- 172.21.15.43:0/1980675470 >> 172.21.15.43:0/1980675470 conn(0x558ad909fd80 msgr2=0x558ad9186e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- 172.21.15.43:0/1980675470 shutdown_connections 2024-09-17T05:30:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- 172.21.15.43:0/1980675470 wait complete. 2024-09-17T05:30:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 Processor -- start 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.532+0000 7f7ddf676080 1 -- start start 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 0x558ad9223b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad916b010 0x558ad9185ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916c8b0 0x558ad9191100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558ad90b4d50 con 0x558ad90e2cb0 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558ad90b4ab0 con 0x558ad916b010 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558ad90b4ff0 con 0x558ad916c8b0 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 0x558ad9223b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 0x558ad9223b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43516/0 (socket says 172.21.15.43:43516) 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 learned_addr learned my addr 172.21.15.43:0/4087109213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd9200640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916c8b0 0x558ad9191100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad916b010 0x558ad9185ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916c8b0 msgr2=0x558ad9191100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558ad916c8b0 0x558ad9191100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad916b010 msgr2=0x558ad9185ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad916b010 0x558ad9185ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad9191640 con 0x558ad90e2cb0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ad916b010 0x558ad9185ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 0x558ad9223b80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0087b0 tx=0x7f7dcc008c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd5600640 1 -- 172.21.15.43:0/4087109213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00abc0 con 0x558ad90e2cb0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd5600640 1 -- 172.21.15.43:0/4087109213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dcc00ad60 con 0x558ad90e2cb0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7dd5600640 1 -- 172.21.15.43:0/4087109213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc015680 con 0x558ad90e2cb0 2024-09-17T05:30:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558ad9190480 con 0x558ad90e2cb0 2024-09-17T05:30:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.536+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad91909c0 con 0x558ad90e2cb0 2024-09-17T05:30:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd5600640 1 -- 172.21.15.43:0/4087109213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7dcc015820 con 0x558ad90e2cb0 2024-09-17T05:30:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd5600640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7db8071be0 0x7f7db80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd5600640 1 -- 172.21.15.43:0/4087109213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6397+0+0 (secure 0 0 0) 0x7f7dcc0936d0 con 0x558ad90e2cb0 2024-09-17T05:30:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7db8071be0 0x7f7db80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.546 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x558ad918fcd0 0x558ad9224230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9224770 con 0x558ad918fcd0 2024-09-17T05:30:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd9200640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x558ad918fcd0 0x558ad9224230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7db8071be0 0x7f7db80740f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40ab000 tx=0x7f7dc4002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.540+0000 7f7dd9200640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x558ad918fcd0 0x558ad9224230 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.544+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v97'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.544+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9224b40 con 0x558ad918fcd0 2024-09-17T05:30:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.544+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v97'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.544+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad922f740 con 0x558ad918fcd0 2024-09-17T05:30:53.552 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v97'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230af0 con 0x558ad918fcd0 2024-09-17T05:30:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230e90 con 0x558ad918fcd0 2024-09-17T05:30:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.548+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad922fae0 con 0x558ad918fcd0 2024-09-17T05:30:53.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.552+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.552+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad922fed0 con 0x558ad918fcd0 2024-09-17T05:30:53.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.552+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v97'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.552+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v97'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.561 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:53.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v97'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.556+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v97'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.560+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.569 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:53.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v97'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.564+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.568+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.568+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230270 con 0x558ad918fcd0 2024-09-17T05:30:53.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.568+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.568+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230670 con 0x558ad918fcd0 2024-09-17T05:30:53.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9230630 con 0x558ad918fcd0 2024-09-17T05:30:53.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v97'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v97'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.572+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.580 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:53.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v97'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v97'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v97'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.576+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.580+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v97'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.580+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.580+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v97'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.580+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v97'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.589 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v97'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v97'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.584+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v97'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad918fcd0 2024-09-17T05:30:53.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v97'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558ad9231740 0x558ad9233b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92304f0 con 0x558ad9231740 2024-09-17T05:30:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558ad9231740 0x558ad9233b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.588+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558ad9231740 0x558ad9233b60 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v97'176 uv155 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92340a0 con 0x558ad9231740 2024-09-17T05:30:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v97'177 uv177 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9234440 con 0x558ad9231740 2024-09-17T05:30:53.598 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:53.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v97'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.592+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9234c90 con 0x558ad9231740 2024-09-17T05:30:53.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v97'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9235030 con 0x558ad9231740 2024-09-17T05:30:53.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v97'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92353d0 con 0x558ad9231740 2024-09-17T05:30:53.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v97'181 uv174 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.596+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9235770 con 0x558ad9231740 2024-09-17T05:30:53.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.600+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v97'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.600+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9235b60 con 0x558ad9231740 2024-09-17T05:30:53.606 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:53.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.600+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v97'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.600+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v97'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v97'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v97'186 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.604+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v97'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.613 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v97'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v97'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.608+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.612+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v97'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.612+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.612+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v97'191 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.612+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v97'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.621 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v97'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v97'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.616+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v97'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92348c0 con 0x558ad9231740 2024-09-17T05:30:53.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v97'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad9231740 2024-09-17T05:30:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x558ad92364d0 0x558ad92388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9238e30 con 0x558ad92364d0 2024-09-17T05:30:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x558ad92364d0 0x558ad92388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.620+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x558ad92364d0 0x558ad92388f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v97'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9239250 con 0x558ad92364d0 2024-09-17T05:30:53.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v97'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239730 con 0x558ad92364d0 2024-09-17T05:30:53.631 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v97'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.624+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923afe0 con 0x558ad92364d0 2024-09-17T05:30:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.628+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.628+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923b380 con 0x558ad92364d0 2024-09-17T05:30:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.628+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.628+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239fd0 con 0x558ad92364d0 2024-09-17T05:30:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.628+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.632+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad918fcd0 2024-09-17T05:30:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.632+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'36 uv35 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.632+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad918fcd0 2024-09-17T05:30:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.632+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v97'37 uv37 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.632+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad918fcd0 2024-09-17T05:30:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v97'38 uv38 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad918fcd0 2024-09-17T05:30:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v97'39 uv39 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x558ad923b720 0x558ad923db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad923b720 2024-09-17T05:30:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7dd9200640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x558ad923b720 0x558ad923db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.636+0000 7f7dd9200640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x558ad923b720 0x558ad923db40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v97'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad923b720 2024-09-17T05:30:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v97'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923a3c0 con 0x558ad923b720 2024-09-17T05:30:53.647 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:53.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v97'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.640+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923f0e0 con 0x558ad923b720 2024-09-17T05:30:53.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.644+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.644+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923f0e0 con 0x558ad923b720 2024-09-17T05:30:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.644+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.644+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad923e0d0 con 0x558ad923b720 2024-09-17T05:30:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.644+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x558ad923e5e0 0x558ad9240a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241060 con 0x558ad923e5e0 2024-09-17T05:30:53.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x558ad923e5e0 0x558ad9240a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7dd8800640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x558ad923e5e0 0x558ad9240a00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v97'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9241120 con 0x558ad923e5e0 2024-09-17T05:30:53.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v97'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.648+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v97'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.656 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:53.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9242550 con 0x558ad923e5e0 2024-09-17T05:30:53.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v97'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9242550 con 0x558ad923e5e0 2024-09-17T05:30:53.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v97'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.652+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v97'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v97'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v97'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.656+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v97'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.663 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v97'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9241540 con 0x558ad923e5e0 2024-09-17T05:30:53.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7dd8800640 1 -- 172.21.15.43:0/4087109213 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v97'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dcc05a030 con 0x558ad923e5e0 2024-09-17T05:30:53.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558ad9241bc0 0x558ad9243fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92445f0 con 0x558ad9241bc0 2024-09-17T05:30:53.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558ad9241bc0 0x558ad9243fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.660+0000 7f7dd7e00640 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558ad9241bc0 0x558ad9243fe0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v97'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad92449c0 con 0x558ad9241bc0 2024-09-17T05:30:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v97'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad9241bc0 2024-09-17T05:30:53.670 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v97'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.664+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad9241bc0 2024-09-17T05:30:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v97'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad9241bc0 2024-09-17T05:30:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v97'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad918fcd0 2024-09-17T05:30:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v97'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.668+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad918fcd0 2024-09-17T05:30:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v97'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad918fcd0 2024-09-17T05:30:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v97'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.677 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245df0 con 0x558ad918fcd0 2024-09-17T05:30:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.672+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245df0 con 0x558ad918fcd0 2024-09-17T05:30:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad918fcd0 2024-09-17T05:30:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad918fcd0 2024-09-17T05:30:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad92364d0 2024-09-17T05:30:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v97'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.676+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad92364d0 2024-09-17T05:30:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v97'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9244de0 con 0x558ad92364d0 2024-09-17T05:30:53.685 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v97'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v97'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.680+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.684+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v97'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.684+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.684+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v97'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.684+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.688+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v97'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.688+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.693 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.688+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v97'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.688+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9239bc0 con 0x558ad92364d0 2024-09-17T05:30:53.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.688+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v97'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.692+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9247c50 con 0x558ad92364d0 2024-09-17T05:30:53.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.692+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v97'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad92364d0 2024-09-17T05:30:53.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.692+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.692+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v97'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.692+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.696+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v97'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.696+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.696+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v97'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.702 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.696+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v97'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v97'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v97'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.700+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v97'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.708 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:53.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v97'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v97'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.704+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad923b720 2024-09-17T05:30:53.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dd000f070 con 0x558ad923b720 2024-09-17T05:30:53.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v97'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v97'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.708+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.716 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:53.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.712+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v97'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.712+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.712+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v97'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.712+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v97'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v97'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v97'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.723 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v97'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.716+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v97'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v97'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v97'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.720+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v97'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.729 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v97'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92462a0 con 0x558ad9241bc0 2024-09-17T05:30:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v97'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.724+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad92462a0 con 0x558ad9241bc0 2024-09-17T05:30:53.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v97'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245290 con 0x558ad9241bc0 2024-09-17T05:30:53.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v97'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x558ad9245500 con 0x558ad9241bc0 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7dd7e00640 1 -- 172.21.15.43:0/4087109213 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v97'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7dc40add90 con 0x558ad9241bc0 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558ad9231740 msgr2=0x558ad9233b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558ad9231740 0x558ad9233b60 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x558ad923e5e0 msgr2=0x558ad9240a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x558ad923e5e0 0x558ad9240a00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558ad9241bc0 msgr2=0x558ad9243fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x558ad9241bc0 0x558ad9243fe0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x558ad92364d0 msgr2=0x558ad92388f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 reap_dead start 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x558ad92364d0 0x558ad92388f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x558ad918fcd0 msgr2=0x558ad9224230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x558ad918fcd0 0x558ad9224230 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x558ad923b720 msgr2=0x558ad923db40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x558ad923b720 0x558ad923db40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7db8071be0 msgr2=0x7f7db80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7db8071be0 0x7f7db80740f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40ab000 tx=0x7f7dc4002040 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 msgr2=0x558ad9223b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7ddf676080 1 --2- 172.21.15.43:0/4087109213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558ad90e2cb0 0x558ad9223b80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0087b0 tx=0x7f7dcc008c70 comp rx=0 tx=0).stop 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.728+0000 7f7dd9200640 1 -- 172.21.15.43:0/4087109213 reap_dead start 2024-09-17T05:30:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.732+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 shutdown_connections 2024-09-17T05:30:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.732+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 >> 172.21.15.43:0/4087109213 conn(0x558ad909fd80 msgr2=0x558ad9186590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.732+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 shutdown_connections 2024-09-17T05:30:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.732+0000 7f7ddf676080 1 -- 172.21.15.43:0/4087109213 wait complete. 2024-09-17T05:30:53.740 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-09-17T05:30:53.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 Processor -- start 2024-09-17T05:30:53.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- start start 2024-09-17T05:30:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c9070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c905acb0 0x5631c9071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631c8f79a00 con 0x5631c90e1e40 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631c8fd7f40 con 0x5631c905acb0 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5631c90e2b30 con 0x5631c902e4d0 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59412/0 (socket says 172.21.15.43:59412) 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 -- 172.21.15.43:0/3087354131 learned_addr learned my addr 172.21.15.43:0/3087354131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f180fe00640 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c905acb0 0x5631c9071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1814a00640 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c9070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 -- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 msgr2=0x5631c9070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c9070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 -- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c905acb0 msgr2=0x5631c9071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c905acb0 0x5631c9071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 -- 172.21.15.43:0/3087354131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c8f52c10 con 0x5631c90e1e40 2024-09-17T05:30:53.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f1815400640 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f181000c6f0 tx=0x7f181000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e261d1bb7ea1eac server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f180ea00640 1 -- 172.21.15.43:0/3087354131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810004020 con 0x5631c90e1e40 2024-09-17T05:30:53.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f180ea00640 1 -- 172.21.15.43:0/3087354131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18100042b0 con 0x5631c90e1e40 2024-09-17T05:30:53.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f180ea00640 1 -- 172.21.15.43:0/3087354131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000aa80 con 0x5631c90e1e40 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- 172.21.15.43:0/3087354131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 msgr2=0x5631c90723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f181000c6f0 tx=0x7f181000cbb0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- 172.21.15.43:0/3087354131 shutdown_connections 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c905acb0 0x5631c9071e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c90e1e40 0x5631c90723d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 --2- 172.21.15.43:0/3087354131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c9070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- 172.21.15.43:0/3087354131 >> 172.21.15.43:0/3087354131 conn(0x5631c9017d80 msgr2=0x5631c9029890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:53.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- 172.21.15.43:0/3087354131 shutdown_connections 2024-09-17T05:30:53.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.836+0000 7f181c449080 1 -- 172.21.15.43:0/3087354131 wait complete. 2024-09-17T05:30:53.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 Processor -- start 2024-09-17T05:30:53.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 -- start start 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c90f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 0x5631c9187a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 0x5631c9187f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5631c9073950 con 0x5631c905acb0 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631c905a7f0 con 0x5631c90e1e40 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f181c449080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5631c9073bf0 con 0x5631c902e4d0 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f180fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 0x5631c9187a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1814a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c90f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 0x5631c9187f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f180fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 0x5631c9187a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59426/0 (socket says 172.21.15.43:59426) 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 0x5631c9187f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:37482/0 (socket says 172.21.15.43:37482) 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f180fe00640 1 -- 172.21.15.43:0/49230772 learned_addr learned my addr 172.21.15.43:0/49230772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 -- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 msgr2=0x5631c90f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5631c902e4d0 0x5631c90f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 -- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 msgr2=0x5631c9187a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 0x5631c9187a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 -- 172.21.15.43:0/49230772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631c919cad0 con 0x5631c90e1e40 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f180fe00640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5631c905acb0 0x5631c9187a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f1815400640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 0x5631c9187f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f181000c6c0 tx=0x7f1810004090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.840+0000 7f180d600640 1 -- 172.21.15.43:0/49230772 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000f6f0 con 0x5631c90e1e40 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180d600640 1 -- 172.21.15.43:0/49230772 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f181000f890 con 0x5631c90e1e40 2024-09-17T05:30:53.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631c91885a0 con 0x5631c90e1e40 2024-09-17T05:30:53.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180d600640 1 -- 172.21.15.43:0/49230772 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181000fce0 con 0x5631c90e1e40 2024-09-17T05:30:53.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631c9188ae0 con 0x5631c90e1e40 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180d600640 1 -- 172.21.15.43:0/49230772 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1810003020 con 0x5631c90e1e40 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180d600640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f17f4071c10 0x7f17f4074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180d600640 1 -- 172.21.15.43:0/49230772 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6397+0+0 (secure 0 0 0) 0x7f1810094dd0 con 0x5631c90e1e40 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f1814a00640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f17f4071c10 0x7f17f4074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f181c449080 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5631c91894c0 0x5631c91997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:53.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5631c9199d10 con 0x5631c91894c0 2024-09-17T05:30:53.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.844+0000 7f180fe00640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5631c91894c0 0x5631c91997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:53.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f1814a00640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f17f4071c10 0x7f17f4074120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f180800c660 tx=0x7f180800e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f180fe00640 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5631c91894c0 0x5631c91997d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f180fe00640 1 -- 172.21.15.43:0/49230772 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [delete] v97'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18000b0470 con 0x5631c91894c0 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5631c91894c0 msgr2=0x5631c91997d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5631c91894c0 0x5631c91997d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f17f4071c10 msgr2=0x7f17f4074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f17f4071c10 0x7f17f4074120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f180800c660 tx=0x7f180800e040 comp rx=0 tx=0).stop 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 msgr2=0x5631c9187f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:53.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 --2- 172.21.15.43:0/49230772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631c90e1e40 0x5631c9187f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f181000c6c0 tx=0x7f1810004090 comp rx=0 tx=0).stop 2024-09-17T05:30:53.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f1815400640 1 -- 172.21.15.43:0/49230772 reap_dead start 2024-09-17T05:30:53.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 shutdown_connections 2024-09-17T05:30:53.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 >> 172.21.15.43:0/49230772 conn(0x5631c9017d80 msgr2=0x5631c90e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:53.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 shutdown_connections 2024-09-17T05:30:53.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:53.848+0000 7f181c449080 1 -- 172.21.15.43:0/49230772 wait complete. 2024-09-17T05:30:53.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- 172.21.15.43:0/2810152300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c0737c0 msgr2=0x7f6c5c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 --2- 172.21.15.43:0/2810152300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c0737c0 0x7f6c5c0700a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009bf0 tx=0x7f6c5401ca80 comp rx=0 tx=0).stop 2024-09-17T05:30:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- 172.21.15.43:0/2810152300 shutdown_connections 2024-09-17T05:30:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 --2- 172.21.15.43:0/2810152300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c0737c0 0x7f6c5c0700a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- 172.21.15.43:0/2810152300 >> 172.21.15.43:0/2810152300 conn(0x7f6c5c0b23d0 msgr2=0x7f6c5c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:54.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- 172.21.15.43:0/2810152300 shutdown_connections 2024-09-17T05:30:54.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- 172.21.15.43:0/2810152300 wait complete. 2024-09-17T05:30:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 Processor -- start 2024-09-17T05:30:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.016+0000 7f6c64200640 1 -- start start 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 0x7f6c5c0747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c15fe10 0x7f6c5c074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c16b610 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c16b490 con 0x7f6c5c15fe10 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c16b790 con 0x7f6c5c06fc90 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59436/0 (socket says 172.21.15.43:59436) 2024-09-17T05:30:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 -- 172.21.15.43:0/2826765979 learned_addr learned my addr 172.21.15.43:0/2826765979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c60e00640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c15fe10 0x7f6c5c074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c61800640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 0x7f6c5c0747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 -- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 msgr2=0x7f6c5c0747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 0x7f6c5c0747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 -- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c15fe10 msgr2=0x7f6c5c074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c15fe10 0x7f6c5c074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 -- 172.21.15.43:0/2826765979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c15d640 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c61800640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 0x7f6c5c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c62200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009b10 tx=0x7f6c54009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540bb070 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c54004d40 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c15d920 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c540b2040 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c15dee0 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.020+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c0700a0 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.024+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c540be070 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.024+0000 7f6c59600640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 0x7f6c3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.024+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6397+0+0 (secure 0 0 0) 0x7f6c5412e510 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.024+0000 7f6c61800640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 0x7f6c3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.024+0000 7f6c61800640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 0x7f6c3c075740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6c50007a60 tx=0x7f6c50009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:54.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.028+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6c54016790 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.236+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6c5c0ca140 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c59600640 1 -- 172.21.15.43:0/2826765979 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v135) ==== 162+0+0 (secure 0 0 0) 0x7f6c540f8ca0 con 0x7f6c5c15d0a0 2024-09-17T05:30:54.456 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 msgr2=0x7f6c3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 0x7f6c3c075740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6c50007a60 tx=0x7f6c50009040 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 msgr2=0x7f6c5c158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009b10 tx=0x7f6c54009200 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 shutdown_connections 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c3c073230 0x7f6c3c075740 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c5c15fe10 0x7f6c5c074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15d0a0 0x7f6c5c158b70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 --2- 172.21.15.43:0/2826765979 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c5c06fc90 0x7f6c5c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 >> 172.21.15.43:0/2826765979 conn(0x7f6c5c0b23d0 msgr2=0x7f6c5c0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 shutdown_connections 2024-09-17T05:30:54.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.452+0000 7f6c64200640 1 -- 172.21.15.43:0/2826765979 wait complete. 2024-09-17T05:30:54.466 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:54.460+0000 7f924dc00640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-17T05:30:54.466 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:30:54.460+0000 7f924dc00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-17T05:30:54.466 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:54.460+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:54.466 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:54.460+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:54.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 -- 172.21.15.43:0/932381217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00d34f0 msgr2=0x7f75a00b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 --2- 172.21.15.43:0/932381217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00d34f0 0x7f75a00b3f40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7588005fd0 tx=0x7f758801c790 comp rx=0 tx=0).stop 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 -- 172.21.15.43:0/932381217 shutdown_connections 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 --2- 172.21.15.43:0/932381217 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00b4480 0x7f75a00b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 --2- 172.21.15.43:0/932381217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00d34f0 0x7f75a00b3f40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 -- 172.21.15.43:0/932381217 >> 172.21.15.43:0/932381217 conn(0x7f75a00d45e0 msgr2=0x7f75a00d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 -- 172.21.15.43:0/932381217 shutdown_connections 2024-09-17T05:30:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.636+0000 7f75a4a00640 1 -- 172.21.15.43:0/932381217 wait complete. 2024-09-17T05:30:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 Processor -- start 2024-09-17T05:30:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 -- start start 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00d0fd0 0x7f75a015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a00d34f0 0x7f75a015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a016c440 con 0x7f75a00b4480 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f75a016c2c0 con 0x7f75a00d34f0 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f75a4a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f75a016c5c0 con 0x7f75a00d0fd0 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59448/0 (socket says 172.21.15.43:59448) 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 -- 172.21.15.43:0/2939199768 learned_addr learned my addr 172.21.15.43:0/2939199768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f7597e00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00d0fd0 0x7f75a015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759e800640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a00d34f0 0x7f75a015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 -- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00d0fd0 msgr2=0x7f75a015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00d0fd0 0x7f75a015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 -- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a00d34f0 msgr2=0x7f75a015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a00d34f0 0x7f75a015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.640+0000 7f759de00640 1 -- 172.21.15.43:0/2939199768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a0146e60 con 0x7f75a00b4480 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f759de00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590017070 con 0x7f75a00b4480 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7590003600 con 0x7f75a00b4480 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590003a50 con 0x7f75a00b4480 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a0147360 con 0x7f75a00b4480 2024-09-17T05:30:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a00cb9f0 con 0x7f75a00b4480 2024-09-17T05:30:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a00b4890 con 0x7f75a00b4480 2024-09-17T05:30:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f759001a070 con 0x7f75a00b4480 2024-09-17T05:30:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.644+0000 7f7596000640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 0x7f757c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.648+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f75900937f0 con 0x7f75a00b4480 2024-09-17T05:30:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.648+0000 7f7597e00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 0x7f757c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.648+0000 7f7597e00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 0x7f757c075710 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75880ad000 tx=0x7f75880a9040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.652+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f759005dea0 con 0x7f75a00b4480 2024-09-17T05:30:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.864+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75a00b8a90 con 0x7f75a00b4480 2024-09-17T05:30:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.864+0000 7f7596000640 1 -- 172.21.15.43:0/2939199768 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+4382 (secure 0 0 0) 0x7f75900628d0 con 0x7f75a00b4480 2024-09-17T05:30:54.870 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:54.870 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":137,"btime":"2024-09-17T05:30:54:467437+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3644279875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3644279875},{"type":"v1","addr":"172.21.15.62:6835","nonce":3644279875}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:54.467373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 137 2024-09-17T05:30:54.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 msgr2=0x7f757c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 0x7f757c075710 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75880ad000 tx=0x7f75880a9040 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 msgr2=0x7f75a00ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 shutdown_connections 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f757c073200 0x7f757c075710 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a00d34f0 0x7f75a015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75a00d0fd0 0x7f75a015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 --2- 172.21.15.43:0/2939199768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75a00b4480 0x7f75a00ca9b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 >> 172.21.15.43:0/2939199768 conn(0x7f75a00d45e0 msgr2=0x7f75a00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 shutdown_connections 2024-09-17T05:30:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:54.868+0000 7f75a4a00640 1 -- 172.21.15.43:0/2939199768 wait complete. 2024-09-17T05:30:54.893 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:54.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- 172.21.15.43:0/2093119883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a5015adc0 msgr2=0x7f5a5006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2093119883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a5015adc0 0x7f5a5006ce60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5a500c7260 tx=0x7f5a4001dbf0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- 172.21.15.43:0/2093119883 shutdown_connections 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2093119883 >> v2:172.21.15.62:3300/0 conn(0x7f5a500cc160 0x7f5a5006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2093119883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a5015adc0 0x7f5a5006ce60 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- 172.21.15.43:0/2093119883 >> 172.21.15.43:0/2093119883 conn(0x7f5a500b23d0 msgr2=0x7f5a500d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- 172.21.15.43:0/2093119883 shutdown_connections 2024-09-17T05:30:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- 172.21.15.43:0/2093119883 wait complete. 2024-09-17T05:30:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 Processor -- start 2024-09-17T05:30:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.048+0000 7f5a54c00640 1 -- start start 2024-09-17T05:30:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 0x7f5a5015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5015adc0 0x7f5a5015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a501681a0 con 0x7f5a500cc160 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a50168020 con 0x7f5a500cc6d0 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5a50168320 con 0x7f5a5015adc0 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59480/0 (socket says 172.21.15.43:59480) 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 -- 172.21.15.43:0/2700101608 learned_addr learned my addr 172.21.15.43:0/2700101608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a45400640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 0x7f5a5015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4e800640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5015adc0 0x7f5a5015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 -- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5015adc0 msgr2=0x7f5a5015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5015adc0 0x7f5a5015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 -- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 msgr2=0x7f5a5015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 0x7f5a5015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 -- 172.21.15.43:0/2700101608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5015d4e0 con 0x7f5a500cc160 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a45400640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 0x7f5a5015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a4de00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5a400b1040 tx=0x7f5a40004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400b3070 con 0x7f5a500cc160 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a40005530 con 0x7f5a500cc160 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40004930 con 0x7f5a500cc160 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5015d7c0 con 0x7f5a500cc160 2024-09-17T05:30:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5015dd80 con 0x7f5a500cc160 2024-09-17T05:30:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.052+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a500ce100 con 0x7f5a500cc160 2024-09-17T05:30:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.056+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a400be060 con 0x7f5a500cc160 2024-09-17T05:30:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.056+0000 7f5a46a00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 0x7f5a2c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.056+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f5a4012e780 con 0x7f5a500cc160 2024-09-17T05:30:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.056+0000 7f5a45400640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 0x7f5a2c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.056+0000 7f5a45400640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 0x7f5a2c075740 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a500736a0 tx=0x7f5a38009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.060+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a400177e0 con 0x7f5a500cc160 2024-09-17T05:30:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.272+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a50074050 con 0x7f5a500cc160 2024-09-17T05:30:55.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.272+0000 7f5a46a00640 1 -- 172.21.15.43:0/2700101608 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+4382 (secure 0 0 0) 0x7f5a400f9490 con 0x7f5a500cc160 2024-09-17T05:30:55.278 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:55.278 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":137,"btime":"2024-09-17T05:30:54:467437+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3644279875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3644279875},{"type":"v1","addr":"172.21.15.62:6835","nonce":3644279875}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:54.467373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:55.279 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 137 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 msgr2=0x7f5a2c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 0x7f5a2c075740 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a500736a0 tx=0x7f5a38009040 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 msgr2=0x7f5a5009db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5a400b1040 tx=0x7f5a40004600 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 shutdown_connections 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a2c073230 0x7f5a2c075740 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a500cc6d0 0x7f5a5015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a5015adc0 0x7f5a5015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 --2- 172.21.15.43:0/2700101608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a500cc160 0x7f5a5009db60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 >> 172.21.15.43:0/2700101608 conn(0x7f5a500b23d0 msgr2=0x7f5a5015a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 shutdown_connections 2024-09-17T05:30:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.276+0000 7f5a54c00640 1 -- 172.21.15.43:0/2700101608 wait complete. 2024-09-17T05:30:55.301 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:55.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- 172.21.15.43:0/4039711972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 msgr2=0x7f3d900b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 --2- 172.21.15.43:0/4039711972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d900b1e80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009c80 tx=0x7f3d8801cce0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- 172.21.15.43:0/4039711972 shutdown_connections 2024-09-17T05:30:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 --2- 172.21.15.43:0/4039711972 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d900b1e80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 --2- 172.21.15.43:0/4039711972 >> v2:172.21.15.62:3300/0 conn(0x7f3d90070d60 0x7f3d900b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- 172.21.15.43:0/4039711972 >> 172.21.15.43:0/4039711972 conn(0x7f3d900b24c0 msgr2=0x7f3d900b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- 172.21.15.43:0/4039711972 shutdown_connections 2024-09-17T05:30:55.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- 172.21.15.43:0/4039711972 wait complete. 2024-09-17T05:30:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 Processor -- start 2024-09-17T05:30:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.460+0000 7f3d95200640 1 -- start start 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d9013a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d9015fbb0 0x7f3d900b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9013faf0 con 0x7f3d90151ab0 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9013f970 con 0x7f3d9015fbb0 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9013fc70 con 0x7f3d900b1a70 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59494/0 (socket says 172.21.15.43:59494) 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 -- 172.21.15.43:0/1812719337 learned_addr learned my addr 172.21.15.43:0/1812719337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d87e00640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d9015fbb0 0x7f3d900b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8de00640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d9013a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 -- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 msgr2=0x7f3d9013a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d9013a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 -- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d9015fbb0 msgr2=0x7f3d900b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d9015fbb0 0x7f3d900b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 -- 172.21.15.43:0/1812719337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d90155830 con 0x7f3d90151ab0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d8e800640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009bd0 tx=0x7f3d88005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d880bc070 con 0x7f3d90151ab0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d900b41f0 con 0x7f3d90151ab0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d88004130 con 0x7f3d90151ab0 2024-09-17T05:30:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d880b2040 con 0x7f3d90151ab0 2024-09-17T05:30:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d900b47b0 con 0x7f3d90151ab0 2024-09-17T05:30:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.464+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d901555a0 con 0x7f3d90151ab0 2024-09-17T05:30:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.468+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d88004560 con 0x7f3d90151ab0 2024-09-17T05:30:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.468+0000 7f3d86000640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 0x7f3d6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.468+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f3d8812e530 con 0x7f3d90151ab0 2024-09-17T05:30:55.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.468+0000 7f3d8de00640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 0x7f3d6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.468+0000 7f3d8de00640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 0x7f3d6c075710 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.472+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d88016990 con 0x7f3d90151ab0 2024-09-17T05:30:55.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.684+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d90141350 con 0x7f3d90151ab0 2024-09-17T05:30:55.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.684+0000 7f3d86000640 1 -- 172.21.15.43:0/1812719337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) ==== 95+0+4382 (secure 0 0 0) 0x7f3d880f8b60 con 0x7f3d90151ab0 2024-09-17T05:30:55.690 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:55.691 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":137,"btime":"2024-09-17T05:30:54:467437+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7597,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3644279875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3644279875},{"type":"v1","addr":"172.21.15.62:6835","nonce":3644279875}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:54.467373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:55.691 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 137 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 msgr2=0x7f3d6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 0x7f3d6c075710 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 msgr2=0x7f3d900b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009bd0 tx=0x7f3d88005ae0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 shutdown_connections 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d6c073200 0x7f3d6c075710 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d9015fbb0 0x7f3d900b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d90151ab0 0x7f3d900b54f0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 --2- 172.21.15.43:0/1812719337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d900b1a70 0x7f3d9013a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 >> 172.21.15.43:0/1812719337 conn(0x7f3d900b24c0 msgr2=0x7f3d9006f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 shutdown_connections 2024-09-17T05:30:55.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.688+0000 7f3d95200640 1 -- 172.21.15.43:0/1812719337 wait complete. 2024-09-17T05:30:55.712 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-09-17T05:30:55.713 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-09-17T05:30:55.713 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:55.713 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:55.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:55.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:55.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:55.722 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:55.770 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:55.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:55.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 -- 172.21.15.43:0/11502706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43800b3af0 msgr2=0x7f43800ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 --2- 172.21.15.43:0/11502706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43800b3af0 0x7f43800ad0b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f4378009bc0 tx=0x7f437801cae0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 -- 172.21.15.43:0/11502706 shutdown_connections 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 --2- 172.21.15.43:0/11502706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43800b3af0 0x7f43800ad0b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 --2- 172.21.15.43:0/11502706 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4380155d70 0x7f438015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 -- 172.21.15.43:0/11502706 >> 172.21.15.43:0/11502706 conn(0x7f43800b23d0 msgr2=0x7f43800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:55.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 -- 172.21.15.43:0/11502706 shutdown_connections 2024-09-17T05:30:55.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.968+0000 7f4386600640 1 -- 172.21.15.43:0/11502706 wait complete. 2024-09-17T05:30:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 Processor -- start 2024-09-17T05:30:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- start start 2024-09-17T05:30:55.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438006f110 0x7f438009d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4380155d70 0x7f43800cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168c10 con 0x7f4380165340 2024-09-17T05:30:55.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168a90 con 0x7f4380155d70 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4380168d90 con 0x7f438006f110 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59524/0 (socket says 172.21.15.43:59524) 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 -- 172.21.15.43:0/252628744 learned_addr learned my addr 172.21.15.43:0/252628744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437f400640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438006f110 0x7f438009d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 -- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438006f110 msgr2=0x7f438009d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438006f110 0x7f438009d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 -- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4380155d70 msgr2=0x7f43800cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4380155d70 0x7f43800cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 -- 172.21.15.43:0/252628744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43800cb2f0 con 0x7f4380165340 2024-09-17T05:30:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f437fe00640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f43780b0040 tx=0x7f4378005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780b2040 con 0x7f4380165340 2024-09-17T05:30:55.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43780bb070 con 0x7f4380165340 2024-09-17T05:30:55.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43780045b0 con 0x7f4380165340 2024-09-17T05:30:55.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43800cb5d0 con 0x7f4380165340 2024-09-17T05:30:55.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.972+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438009ddf0 con 0x7f4380165340 2024-09-17T05:30:55.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.976+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43800ad0b0 con 0x7f4380165340 2024-09-17T05:30:55.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.976+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43780c0460 con 0x7f4380165340 2024-09-17T05:30:55.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.976+0000 7f4377400640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 0x7f435c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:55.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.976+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f437812fa40 con 0x7f4380165340 2024-09-17T05:30:55.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.976+0000 7f437f400640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 0x7f435c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:55.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.980+0000 7f437f400640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 0x7f435c075710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f43800715c0 tx=0x7f43700091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:55.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:55.980+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4378016790 con 0x7f4380165340 2024-09-17T05:30:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.188+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f438015eea0 con 0x7f4380165340 2024-09-17T05:30:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4377400640 1 -- 172.21.15.43:0/252628744 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v138) ==== 161+0+0 (secure 0 0 0) 0x7f43780fa7c0 con 0x7f4380165340 2024-09-17T05:30:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 msgr2=0x7f435c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 0x7f435c075710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f43800715c0 tx=0x7f43700091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 msgr2=0x7f43800cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f43780b0040 tx=0x7f4378005a60 comp rx=0 tx=0).stop 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 shutdown_connections 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f435c073200 0x7f435c075710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4380155d70 0x7f43800cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4380165340 0x7f43800cf690 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 --2- 172.21.15.43:0/252628744 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f438006f110 0x7f438009d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 >> 172.21.15.43:0/252628744 conn(0x7f43800b23d0 msgr2=0x7f43800af410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:56.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 shutdown_connections 2024-09-17T05:30:56.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.460+0000 7f4386600640 1 -- 172.21.15.43:0/252628744 wait complete. 2024-09-17T05:30:56.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:30:56.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 -- 172.21.15.43:0/3834427200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 msgr2=0x7f5cdc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:56.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 --2- 172.21.15.43:0/3834427200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cdc005160 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009ec0 tx=0x7f5cf001d5f0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfa800640 1 -- 172.21.15.43:0/3834427200 reap_dead start 2024-09-17T05:30:56.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 -- 172.21.15.43:0/3834427200 shutdown_connections 2024-09-17T05:30:56.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 -- 172.21.15.43:0/3834427200 >> 172.21.15.43:0/3834427200 conn(0x7f5cf40b23d0 msgr2=0x7f5cf40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:56.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 -- 172.21.15.43:0/3834427200 shutdown_connections 2024-09-17T05:30:56.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.644+0000 7f5cfc800640 1 -- 172.21.15.43:0/3834427200 wait complete. 2024-09-17T05:30:56.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 Processor -- start 2024-09-17T05:30:56.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- start start 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 0x7f5cf40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cf40ccff0 0x7f5cf41f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf40ad440 con 0x7f5cdc004d50 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf40ad2c0 con 0x7f5cf409d9f0 2024-09-17T05:30:56.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf40ad5c0 con 0x7f5cf40ccff0 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59544/0 (socket says 172.21.15.43:59544) 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 -- 172.21.15.43:0/1813671351 learned_addr learned my addr 172.21.15.43:0/1813671351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9400640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 0x7f5cf40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 -- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cf40ccff0 msgr2=0x7f5cf41f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfa800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cf40ccff0 0x7f5cf41f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cf40ccff0 0x7f5cf41f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 -- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 msgr2=0x7f5cf40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 0x7f5cf40ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 -- 172.21.15.43:0/1813671351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf40cd400 con 0x7f5cdc004d50 2024-09-17T05:30:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9400640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 0x7f5cf40ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cf9e00640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80126c0 tx=0x7f5ce8012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8010ae0 con 0x7f5cdc004d50 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ce8017030 con 0x7f5cdc004d50 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8013450 con 0x7f5cdc004d50 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf40c3d30 con 0x7f5cdc004d50 2024-09-17T05:30:56.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.648+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf40c4510 con 0x7f5cdc004d50 2024-09-17T05:30:56.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.652+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf406f7f0 con 0x7f5cdc004d50 2024-09-17T05:30:56.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.652+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ce80135f0 con 0x7f5cdc004d50 2024-09-17T05:30:56.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.656+0000 7f5cee000640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.656+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f5ce8098720 con 0x7f5cdc004d50 2024-09-17T05:30:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.656+0000 7f5cf9400640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.656+0000 7f5cf9400640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 0x7f5cd8075710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5cf41f6a80 tx=0x7f5ce40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.656+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5cf40cd400 con 0x7f5cdc004d50 2024-09-17T05:30:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:56.900+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5cf40b12e0 con 0x7f5cdc004d50 2024-09-17T05:30:57.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.460+0000 7f5cee000640 1 -- 172.21.15.43:0/1813671351 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v139) ==== 101+0+0 (secure 0 0 0) 0x7f5ce8063410 con 0x7f5cdc004d50 2024-09-17T05:30:57.466 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:30:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 msgr2=0x7f5cd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 0x7f5cd8075710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5cf41f6a80 tx=0x7f5ce40091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 msgr2=0x7f5cf40959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80126c0 tx=0x7f5ce8012b80 comp rx=0 tx=0).stop 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 shutdown_connections 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cf409d9f0 0x7f5cf40ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5cf40ccff0 0x7f5cf41f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 --2- 172.21.15.43:0/1813671351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cdc004d50 0x7f5cf40959e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 >> 172.21.15.43:0/1813671351 conn(0x7f5cf40b23d0 msgr2=0x7f5cf40af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 shutdown_connections 2024-09-17T05:30:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.464+0000 7f5cfc800640 1 -- 172.21.15.43:0/1813671351 wait complete. 2024-09-17T05:30:57.494 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:30:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.552+0000 7fc8baa4d080 1 Processor -- start 2024-09-17T05:30:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.552+0000 7fc8baa4d080 1 -- start start 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 0x5569afff3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569afffda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569afecaf40 con 0x5569affd55b0 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5569afed7d00 con 0x5569aff239c0 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5569afe6ca00 con 0x5569afff4260 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59556/0 (socket says 172.21.15.43:59556) 2024-09-17T05:30:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 -- 172.21.15.43:0/576891413 learned_addr learned my addr 172.21.15.43:0/576891413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b3000640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 0x5569afff3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569afffda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 -- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 msgr2=0x5569afffda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569afffda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 -- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 msgr2=0x5569afff3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 0x5569afff3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 -- 172.21.15.43:0/576891413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569aff1efa0 con 0x5569affd55b0 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b3000640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 0x5569afff3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b2600640 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b2d0 tx=0x7fc8a000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7309d64b49babd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b1200640 1 -- 172.21.15.43:0/576891413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000a040 con 0x5569affd55b0 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b1200640 1 -- 172.21.15.43:0/576891413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a000f620 con 0x5569affd55b0 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8b1200640 1 -- 172.21.15.43:0/576891413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000fa70 con 0x5569affd55b0 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- 172.21.15.43:0/576891413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 msgr2=0x5569afffd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b2d0 tx=0x7fc8a000b790 comp rx=0 tx=0).stop 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- 172.21.15.43:0/576891413 shutdown_connections 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569afffda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569affd55b0 0x5569afffd550 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/576891413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569aff239c0 0x5569afff3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- 172.21.15.43:0/576891413 >> 172.21.15.43:0/576891413 conn(0x5569aff0ad80 msgr2=0x5569aff60a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- 172.21.15.43:0/576891413 shutdown_connections 2024-09-17T05:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 -- 172.21.15.43:0/576891413 wait complete. 2024-09-17T05:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.556+0000 7fc8baa4d080 1 Processor -- start 2024-09-17T05:30:57.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- start start 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569affd55b0 0x5569aff60980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569affd3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5569afff4cc0 con 0x5569aff239c0 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5569afff4f60 con 0x5569affd55b0 2024-09-17T05:30:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5569afffb750 con 0x5569afff4260 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59568/0 (socket says 172.21.15.43:59568) 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 learned_addr learned my addr 172.21.15.43:0/3990122397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569affd3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569affd55b0 0x5569aff60980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 msgr2=0x5569affd3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5569afff4260 0x5569affd3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569affd55b0 msgr2=0x5569aff60980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569affd55b0 0x5569aff60980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569affd4760 con 0x5569aff239c0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc8a401b6e0 tx=0x7fc8a40ab750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8ab400640 1 -- 172.21.15.43:0/3990122397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40ba020 con 0x5569aff239c0 2024-09-17T05:30:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8ab400640 1 -- 172.21.15.43:0/3990122397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a40b6450 con 0x5569aff239c0 2024-09-17T05:30:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5569affd4a00 con 0x5569aff239c0 2024-09-17T05:30:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8ab400640 1 -- 172.21.15.43:0/3990122397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40b6820 con 0x5569aff239c0 2024-09-17T05:30:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.560+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569affd2df0 con 0x5569aff239c0 2024-09-17T05:30:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8ab400640 1 -- 172.21.15.43:0/3990122397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8a40bd020 con 0x5569aff239c0 2024-09-17T05:30:57.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8ab400640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 0x7fc894074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8ab400640 1 -- 172.21.15.43:0/3990122397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7fc8a412e500 con 0x5569aff239c0 2024-09-17T05:30:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 0x7fc894074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.570 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:30:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5569aff60ec0 0x5569b0099b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009a0a0 con 0x5569aff60ec0 2024-09-17T05:30:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5569aff60ec0 0x5569b0099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.564+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 0x7fc894074120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b270 tx=0x7fc8a0001090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.568+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5569aff60ec0 0x5569b0099b60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.568+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v98'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.568+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009a440 con 0x5569aff60ec0 2024-09-17T05:30:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.568+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v98'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.568+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009a7e0 con 0x5569aff60ec0 2024-09-17T05:30:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.572+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v98'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.577 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:30:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.572+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009bb90 con 0x5569aff60ec0 2024-09-17T05:30:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.572+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.572+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009bf30 con 0x5569aff60ec0 2024-09-17T05:30:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009ab80 con 0x5569aff60ec0 2024-09-17T05:30:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009af70 con 0x5569aff60ec0 2024-09-17T05:30:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v98'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.576+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009b360 con 0x5569aff60ec0 2024-09-17T05:30:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.580+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v98'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.580+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b360 con 0x5569aff60ec0 2024-09-17T05:30:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.580+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v98'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.587 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:30:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.580+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.584+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.584+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.584+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.584+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v98'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v98'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.588+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.592+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v98'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.597 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:30:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.592+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.592+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v98'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.592+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b790 con 0x5569aff60ec0 2024-09-17T05:30:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.596+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v98'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.596+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b790 con 0x5569aff60ec0 2024-09-17T05:30:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.596+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v98'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.596+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.600+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v98'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.600+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.600+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v98'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.600+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.604+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v98'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.604+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.610 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:30:57.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.604+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v98'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.604+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v98'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v98'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v98'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.608+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.612+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v98'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.612+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.612+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v98'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.612+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.619 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:30:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.616+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v98'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.616+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.616+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v98'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.616+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.616+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v98'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009b310 con 0x5569aff60ec0 2024-09-17T05:30:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v98'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5569b009c7e0 0x5569b009ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009f140 con 0x5569b009c7e0 2024-09-17T05:30:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5569b009c7e0 0x5569b009ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.620+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5569b009c7e0 0x5569b009ec00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v98'197 uv180 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b009f4e0 con 0x5569b009c7e0 2024-09-17T05:30:57.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v98'198 uv198 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b009f880 con 0x5569b009c7e0 2024-09-17T05:30:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v98'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.631 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:30:57.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.624+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0120 con 0x5569b009c7e0 2024-09-17T05:30:57.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.628+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v98'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.628+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a04c0 con 0x5569b009c7e0 2024-09-17T05:30:57.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.628+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v98'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.628+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0860 con 0x5569b009c7e0 2024-09-17T05:30:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v98'202 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a0c00 con 0x5569b009c7e0 2024-09-17T05:30:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v98'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.638 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:30:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v98'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.632+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v98'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v98'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v98'207 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.636+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.640+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v98'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.640+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.646 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:30:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.640+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v98'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.640+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5170 con 0x5569b009c7e0 2024-09-17T05:30:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v98'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5170 con 0x5569b009c7e0 2024-09-17T05:30:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v98'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v98'212 uv196 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.644+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v98'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.653 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:30:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v98'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a1fb0 con 0x5569b009c7e0 2024-09-17T05:30:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v98'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.648+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a1fb0 con 0x5569b009c7e0 2024-09-17T05:30:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v98'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a0fa0 con 0x5569b009c7e0 2024-09-17T05:30:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v98'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b009c7e0 2024-09-17T05:30:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5569b00a1870 0x5569b00a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a41d0 con 0x5569b00a1870 2024-09-17T05:30:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5569b00a1870 0x5569b00a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.652+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5569b00a1870 0x5569b00a3c90 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.656+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v98'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.656+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a45f0 con 0x5569b00a1870 2024-09-17T05:30:57.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.656+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.656+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a4aa0 con 0x5569b00a1870 2024-09-17T05:30:57.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.663 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:30:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a6350 con 0x5569b00a1870 2024-09-17T05:30:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a66f0 con 0x5569b00a1870 2024-09-17T05:30:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v98'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.660+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5340 con 0x5569b00a1870 2024-09-17T05:30:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569aff60ec0 2024-09-17T05:30:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'40 uv39 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569aff60ec0 2024-09-17T05:30:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v98'41 uv41 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.664+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569aff60ec0 2024-09-17T05:30:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v98'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.672 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:30:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569aff60ec0 2024-09-17T05:30:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v98'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 0x5569b00a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569b00a6a90 2024-09-17T05:30:57.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 0x5569b00a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.668+0000 7fc8b3a00640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 0x5569b00a8eb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v98'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569b00a6a90 2024-09-17T05:30:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a5730 con 0x5569b00a6a90 2024-09-17T05:30:57.678 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:30:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.672+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00aa400 con 0x5569b00a6a90 2024-09-17T05:30:57.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00aa400 con 0x5569b00a6a90 2024-09-17T05:30:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v98'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00a93f0 con 0x5569b00a6a90 2024-09-17T05:30:57.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5569b00a9900 0x5569b00abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac330 con 0x5569b00a9900 2024-09-17T05:30:57.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.676+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5569b00a9900 0x5569b00abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8b3000640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5569b00a9900 0x5569b00abd20 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v98'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00ac330 con 0x5569b00a9900 2024-09-17T05:30:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v98'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v98'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.687 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:30:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.680+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ad7e0 con 0x5569b00a9900 2024-09-17T05:30:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v98'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ad7e0 con 0x5569b00a9900 2024-09-17T05:30:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v98'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v98'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.684+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v98'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v98'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v98'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.695 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:30:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.688+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v98'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00ac7d0 con 0x5569b00a9900 2024-09-17T05:30:57.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8b3000640 1 -- 172.21.15.43:0/3990122397 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v98'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a40f4e20 con 0x5569b00a9900 2024-09-17T05:30:57.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5569b00acce0 0x5569b00af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00af8a0 con 0x5569b00acce0 2024-09-17T05:30:57.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5569b00acce0 0x5569b00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.692+0000 7fc8b2600640 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5569b00acce0 0x5569b00af190 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v98'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.700 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:30:57.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569b00acce0 2024-09-17T05:30:57.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569b00acce0 2024-09-17T05:30:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.696+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569aff60ec0 2024-09-17T05:30:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(83 605.00000000 [create] v98'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569aff60ec0 2024-09-17T05:30:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(84 605.00000000 [delete] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569aff60ec0 2024-09-17T05:30:57.707 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:30:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(85 605.00000000 [create] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.700+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0c80 con 0x5569aff60ec0 2024-09-17T05:30:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0c80 con 0x5569aff60ec0 2024-09-17T05:30:57.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569aff60ec0 2024-09-17T05:30:57.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569aff60ec0 2024-09-17T05:30:57.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.704+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569b00a1870 2024-09-17T05:30:57.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(89 mds0_inotable [create] v98'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569b00a1870 2024-09-17T05:30:57.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(90 mds0_inotable [delete] v98'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00afc70 con 0x5569b00a1870 2024-09-17T05:30:57.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(91 mds0_inotable [create] v98'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.715 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:30:57.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.708+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0080 con 0x5569b00a1870 2024-09-17T05:30:57.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.712+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.712+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0080 con 0x5569b00a1870 2024-09-17T05:30:57.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.712+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.712+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0030 con 0x5569b00a1870 2024-09-17T05:30:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(94 100.00000000 [create] v98'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0030 con 0x5569b00a1870 2024-09-17T05:30:57.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(95 100.00000000 [delete] v98'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0030 con 0x5569b00a1870 2024-09-17T05:30:57.722 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:30:57.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(96 100.00000000 [create] v98'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.716+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0030 con 0x5569b00a1870 2024-09-17T05:30:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.720+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.720+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a1870 2024-09-17T05:30:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.720+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00a1870 2024-09-17T05:30:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.720+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0b00 con 0x5569b00a6a90 2024-09-17T05:30:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v98'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v98'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.731 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:30:57.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.724+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v98'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.728+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.737 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:30:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.732+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00a6a90 2024-09-17T05:30:57.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.736+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8ac002040 con 0x5569b00a6a90 2024-09-17T05:30:57.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.736+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0440 con 0x5569b00acce0 2024-09-17T05:30:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.736+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(109 400.00000000 [create] v98'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.736+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0130 con 0x5569b00acce0 2024-09-17T05:30:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(110 400.00000000 [delete] v98'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0130 con 0x5569b00acce0 2024-09-17T05:30:57.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v98'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.745 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:30:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0130 con 0x5569b00acce0 2024-09-17T05:30:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.740+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0130 con 0x5569b00acce0 2024-09-17T05:30:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 100.00000000.inode [create] v98'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 100.00000000.inode [delete] v98'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.744+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v98'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.752 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:30:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v98'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.748+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 1.00000000 [create] v98'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 1.00000000 [delete] v98'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v98'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:30:57.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.752+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.756+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.756+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.756+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.756+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5569b00b0110 con 0x5569b00acce0 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8b2600640 1 -- 172.21.15.43:0/3990122397 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc8a000d040 con 0x5569b00acce0 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5569b009c7e0 msgr2=0x5569b009ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5569b009c7e0 0x5569b009ec00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5569b00a9900 msgr2=0x5569b00abd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5569b00a9900 0x5569b00abd20 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5569b00acce0 msgr2=0x5569b00af190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5569b00acce0 0x5569b00af190 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5569b00a1870 msgr2=0x5569b00a3c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5569b00a1870 0x5569b00a3c90 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 reap_dead start 2024-09-17T05:30:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5569aff60ec0 msgr2=0x5569b0099b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5569aff60ec0 0x5569b0099b60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8b3a00640 1 -- 172.21.15.43:0/3990122397 reap_dead start 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 msgr2=0x5569b00a8eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 0x5569b00a8eb0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 msgr2=0x7fc894074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 0x7fc894074120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000b270 tx=0x7fc8a0001090 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 msgr2=0x5569afffb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc8a401b6e0 tx=0x7fc8a40ab750 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 shutdown_connections 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5569b00a6a90 0x5569b00a8eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc894071c10 0x7fc894074120 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 --2- 172.21.15.43:0/3990122397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5569aff239c0 0x5569afffb210 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 >> 172.21.15.43:0/3990122397 conn(0x5569aff0ad80 msgr2=0x5569aff61db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:57.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 shutdown_connections 2024-09-17T05:30:57.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.760+0000 7fc8baa4d080 1 -- 172.21.15.43:0/3990122397 wait complete. 2024-09-17T05:30:57.770 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-09-17T05:30:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 Processor -- start 2024-09-17T05:30:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 -- start start 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x5627965e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 0x5627965e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627964f6a00 con 0x5627965d7cb0 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562796554f40 con 0x56279665ee40 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56279665f890 con 0x5627965a8d70 2024-09-17T05:30:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcef600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x5627965e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 0x5627965e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:60740/0 (socket says 172.21.15.43:60740) 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 -- 172.21.15.43:0/4062195119 learned_addr learned my addr 172.21.15.43:0/4062195119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 -- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 msgr2=0x5627965e9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 0x5627965e9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 -- 172.21.15.43:0/4062195119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 msgr2=0x5627965e9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x5627965e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0000640 1 -- 172.21.15.43:0/4062195119 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627964cfc10 con 0x5627965a8d70 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.864+0000 7fdcf0a00640 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 0x5627965e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf0000640 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdce001a6e0 tx=0x7fdce001a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571b483f23dcc0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcee200640 1 -- 172.21.15.43:0/4062195119 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00a7810 con 0x5627965a8d70 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcee200640 1 -- 172.21.15.43:0/4062195119 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce00a7b10 con 0x5627965a8d70 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcee200640 1 -- 172.21.15.43:0/4062195119 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00b0470 con 0x5627965a8d70 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 msgr2=0x5627965edb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdce001a6e0 tx=0x7fdce001a870 comp rx=0 tx=0).stop 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/4062195119 shutdown_connections 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56279665ee40 0x5627965e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x5627965e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/4062195119 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5627965a8d70 0x5627965edb20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/4062195119 >> 172.21.15.43:0/4062195119 conn(0x562796594d80 msgr2=0x5627965a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/4062195119 shutdown_connections 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/4062195119 wait complete. 2024-09-17T05:30:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 Processor -- start 2024-09-17T05:30:57.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- start start 2024-09-17T05:30:57.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627965a8d70 0x562796712bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x562796678550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56279665ee40 0x562796676c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5627965ee5f0 con 0x5627965d7cb0 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56279667b790 con 0x5627965a8d70 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcf6ec8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5627965d7540 con 0x56279665ee40 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcef600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x562796678550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcef600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x562796678550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59580/0 (socket says 172.21.15.43:59580) 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.868+0000 7fdcef600640 1 -- 172.21.15.43:0/3481932548 learned_addr learned my addr 172.21.15.43:0/3481932548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcf0000640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627965a8d70 0x562796712bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcf0a00640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56279665ee40 0x562796676c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 -- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56279665ee40 msgr2=0x562796676c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56279665ee40 0x562796676c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 -- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627965a8d70 msgr2=0x562796712bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627965a8d70 0x562796712bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 -- 172.21.15.43:0/3481932548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562796678c60 con 0x5627965d7cb0 2024-09-17T05:30:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcf0000640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627965a8d70 0x562796712bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcef600640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x562796678550 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ef30 tx=0x7fdcdc00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcece00640 1 -- 172.21.15.43:0/3481932548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00cea0 con 0x5627965d7cb0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcece00640 1 -- 172.21.15.43:0/3481932548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcdc001bf0 con 0x5627965d7cb0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcece00640 1 -- 172.21.15.43:0/3481932548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc001e40 con 0x5627965d7cb0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562796678f00 con 0x5627965d7cb0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562796679440 con 0x5627965d7cb0 2024-09-17T05:30:57.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.872+0000 7fdcece00640 1 -- 172.21.15.43:0/3481932548 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdcdc00f040 con 0x5627965d7cb0 2024-09-17T05:30:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcece00640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdcd0071be0 0x7fdcd00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcece00640 1 -- 172.21.15.43:0/3481932548 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7fdcdc092f30 con 0x5627965d7cb0 2024-09-17T05:30:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0000640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdcd0071be0 0x7fdcd00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562796679b80 0x562796716200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x562796716740 con 0x562796679b80 2024-09-17T05:30:57.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0a00640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562796679b80 0x562796716200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0000640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdcd0071be0 0x7fdcd00740f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdce00ba000 tx=0x7fdce00b4040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0a00640 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562796679b80 0x562796716200 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0a00640 1 -- 172.21.15.43:0/3481932548 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [delete] v98'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdce8002040 con 0x562796679b80 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562796679b80 msgr2=0x562796716200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562796679b80 0x562796716200 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdcd0071be0 msgr2=0x7fdcd00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdcd0071be0 0x7fdcd00740f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdce00ba000 tx=0x7fdce00b4040 comp rx=0 tx=0).stop 2024-09-17T05:30:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 msgr2=0x562796678550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 --2- 172.21.15.43:0/3481932548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5627965d7cb0 0x562796678550 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00ef30 tx=0x7fdcdc00c6d0 comp rx=0 tx=0).stop 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf0a00640 1 -- 172.21.15.43:0/3481932548 reap_dead start 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 shutdown_connections 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 >> 172.21.15.43:0/3481932548 conn(0x562796594d80 msgr2=0x56279667ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 shutdown_connections 2024-09-17T05:30:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:57.876+0000 7fdcf6ec8080 1 -- 172.21.15.43:0/3481932548 wait complete. 2024-09-17T05:30:57.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:30:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 -- 172.21.15.43:0/1025636930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d815c510 msgr2=0x7f31d80ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 --2- 172.21.15.43:0/1025636930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d815c510 0x7f31d80ae0b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009840 tx=0x7f31c401c820 comp rx=0 tx=0).stop 2024-09-17T05:30:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 -- 172.21.15.43:0/1025636930 shutdown_connections 2024-09-17T05:30:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 --2- 172.21.15.43:0/1025636930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d815c510 0x7f31d80ae0b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 -- 172.21.15.43:0/1025636930 >> 172.21.15.43:0/1025636930 conn(0x7f31d806d9f0 msgr2=0x7f31d80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 -- 172.21.15.43:0/1025636930 shutdown_connections 2024-09-17T05:30:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.044+0000 7f31dea00640 1 -- 172.21.15.43:0/1025636930 wait complete. 2024-09-17T05:30:58.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 Processor -- start 2024-09-17T05:30:58.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 -- start start 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 0x7f31d8154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31d815c510 0x7f31d809c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31d816a0b0 con 0x7f31d815b9f0 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8169f30 con 0x7f31d80ae5f0 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f31d816a230 con 0x7f31d815c510 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 0x7f31d8154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59594/0 (socket says 172.21.15.43:59594) 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 -- 172.21.15.43:0/3690373154 learned_addr learned my addr 172.21.15.43:0/3690373154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31de000640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31d815c510 0x7f31d809c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 -- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31d815c510 msgr2=0x7f31d809c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31d815c510 0x7f31d809c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 -- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 msgr2=0x7f31d8154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 0x7f31d8154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 -- 172.21.15.43:0/3690373154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d815e4b0 con 0x7f31d815b9f0 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dd600640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 0x7f31d8154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dcc00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f31c4019430 tx=0x7f31c40a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4004100 con 0x7f31d815b9f0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31c4004390 con 0x7f31d815b9f0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.048+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d815d6f0 con 0x7f31d815b9f0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4005030 con 0x7f31d815b9f0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d815dcb0 con 0x7f31d815b9f0 2024-09-17T05:30:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d80ae0b0 con 0x7f31d815b9f0 2024-09-17T05:30:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31c40bf030 con 0x7f31d815b9f0 2024-09-17T05:30:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31d5600640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 0x7f31b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6483+0+0 (secure 0 0 0) 0x7f31c412e400 con 0x7f31d815b9f0 2024-09-17T05:30:58.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.052+0000 7f31dd600640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 0x7f31b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.056+0000 7f31dd600640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 0x7f31b8075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:58.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.056+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31c40c6060 con 0x7f31d815b9f0 2024-09-17T05:30:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.268+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f31d80cf670 con 0x7f31d815b9f0 2024-09-17T05:30:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.476+0000 7f31d5600640 1 -- 172.21.15.43:0/3690373154 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v140) ==== 162+0+0 (secure 0 0 0) 0x7f31c4016410 con 0x7f31d815b9f0 2024-09-17T05:30:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 msgr2=0x7f31b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 0x7f31b8075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc009040 comp rx=0 tx=0).stop 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 msgr2=0x7f31d809be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f31c4019430 tx=0x7f31c40a66d0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 shutdown_connections 2024-09-17T05:30:58.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31b8073200 0x7f31b8075710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31d815c510 0x7f31d809c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31d815b9f0 0x7f31d809be50 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 --2- 172.21.15.43:0/3690373154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d80ae5f0 0x7f31d8154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 >> 172.21.15.43:0/3690373154 conn(0x7f31d806d9f0 msgr2=0x7f31d80aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 shutdown_connections 2024-09-17T05:30:58.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.480+0000 7f31dea00640 1 -- 172.21.15.43:0/3690373154 wait complete. 2024-09-17T05:30:58.495 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:58.493+0000 7f307e200640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-17T05:30:58.495 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:30:58.493+0000 7f307e200640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-17T05:30:58.496 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:58.492+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:58.496 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:30:58.492+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:30:58.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:58.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- 172.21.15.43:0/1160892317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc0b3920 msgr2=0x7ffadc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1160892317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc0b3920 0x7ffadc0ad0b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffadc072b60 tx=0x7ffad401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- 172.21.15.43:0/1160892317 shutdown_connections 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1160892317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc0b3920 0x7ffadc0ad0b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1160892317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc06f110 0x7ffadc0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1160892317 >> v2:172.21.15.62:3300/0 conn(0x7ffadc155d70 0x7ffadc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- 172.21.15.43:0/1160892317 >> 172.21.15.43:0/1160892317 conn(0x7ffadc0b23d0 msgr2=0x7ffadc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- 172.21.15.43:0/1160892317 shutdown_connections 2024-09-17T05:30:58.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- 172.21.15.43:0/1160892317 wait complete. 2024-09-17T05:30:58.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 Processor -- start 2024-09-17T05:30:58.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.664+0000 7ffae1e00640 1 -- start start 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffadc06f110 0x7ffadc155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc0b3920 0x7ffadc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc15f8d0 con 0x7ffadc155d70 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc15f750 con 0x7ffadc06f110 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc15fa50 con 0x7ffadc0b3920 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59608/0 (socket says 172.21.15.43:59608) 2024-09-17T05:30:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 -- 172.21.15.43:0/1426939386 learned_addr learned my addr 172.21.15.43:0/1426939386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadaa00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffadc06f110 0x7ffadc155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffad3e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc0b3920 0x7ffadc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 -- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc0b3920 msgr2=0x7ffadc157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc0b3920 0x7ffadc157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 -- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffadc06f110 msgr2=0x7ffadc155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffadc06f110 0x7ffadc155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 -- 172.21.15.43:0/1426939386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc159900 con 0x7ffadc155d70 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffadb400640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffad401b370 tx=0x7ffad4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40bb070 con 0x7ffadc155d70 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad4004b40 con 0x7ffadc155d70 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1559a0 con 0x7ffadc155d70 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40b2040 con 0x7ffadc155d70 2024-09-17T05:30:58.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc09b900 con 0x7ffadc155d70 2024-09-17T05:30:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.668+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc09b390 con 0x7ffadc155d70 2024-09-17T05:30:58.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.672+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffad4003fd0 con 0x7ffadc155d70 2024-09-17T05:30:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.672+0000 7ffad3400640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 0x7ffab8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.672+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7ffad412e2e0 con 0x7ffadc155d70 2024-09-17T05:30:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.672+0000 7ffadaa00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 0x7ffab8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:58.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.672+0000 7ffadaa00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 0x7ffab8075710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009600 tx=0x7ffacc0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:58.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.676+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffad4016970 con 0x7ffadc155d70 2024-09-17T05:30:58.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffadc15ff20 con 0x7ffadc155d70 2024-09-17T05:30:58.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffad3400640 1 -- 172.21.15.43:0/1426939386 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+3556 (secure 0 0 0) 0x7ffad40f89e0 con 0x7ffadc155d70 2024-09-17T05:30:58.889 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:58.889 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":142,"btime":"2024-09-17T05:30:58:497029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:58.496957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:58.889 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 142 2024-09-17T05:30:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 msgr2=0x7ffab8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 0x7ffab8075710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009600 tx=0x7ffacc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 msgr2=0x7ffadc157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffad401b370 tx=0x7ffad4003f60 comp rx=0 tx=0).stop 2024-09-17T05:30:58.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 shutdown_connections 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffab8073200 0x7ffab8075710 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffadc155d70 0x7ffadc157f90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffadc0b3920 0x7ffadc157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 --2- 172.21.15.43:0/1426939386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffadc06f110 0x7ffadc155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 >> 172.21.15.43:0/1426939386 conn(0x7ffadc0b23d0 msgr2=0x7ffadc15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 shutdown_connections 2024-09-17T05:30:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:58.884+0000 7ffae1e00640 1 -- 172.21.15.43:0/1426939386 wait complete. 2024-09-17T05:30:58.912 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:58.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 -- 172.21.15.43:0/1253968589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 msgr2=0x7f9518004d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 --2- 172.21.15.43:0/1253968589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f9518004d50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f951c001860 tx=0x7f951c01f030 comp rx=0 tx=0).stop 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 -- 172.21.15.43:0/1253968589 shutdown_connections 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 --2- 172.21.15.43:0/1253968589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f9518004d50 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 -- 172.21.15.43:0/1253968589 >> 172.21.15.43:0/1253968589 conn(0x7f95340b2e80 msgr2=0x7f95340b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 -- 172.21.15.43:0/1253968589 shutdown_connections 2024-09-17T05:30:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.068+0000 7f953a000640 1 -- 172.21.15.43:0/1253968589 wait complete. 2024-09-17T05:30:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 Processor -- start 2024-09-17T05:30:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- start start 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953414eaf0 0x7f95341f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95341f7780 0x7f95340702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9534160000 con 0x7f9518004880 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f953415fe80 con 0x7f95341f7780 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9534160180 con 0x7f953414eaf0 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59628/0 (socket says 172.21.15.43:59628) 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 -- 172.21.15.43:0/2438084762 learned_addr learned my addr 172.21.15.43:0/2438084762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f952aa00640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953414eaf0 0x7f95341f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533e00640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95341f7780 0x7f95340702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 -- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953414eaf0 msgr2=0x7f95341f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953414eaf0 0x7f95341f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 -- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95341f7780 msgr2=0x7f95340702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95341f7780 0x7f95340702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 -- 172.21.15.43:0/2438084762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340cd770 con 0x7f9518004880 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f9533400640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9524011a40 tx=0x7f9524011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952401e070 con 0x7f9518004880 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9524004460 con 0x7f9518004880 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340cda50 con 0x7f9518004880 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952400fc00 con 0x7f9518004880 2024-09-17T05:30:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.072+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95340ce010 con 0x7f9518004880 2024-09-17T05:30:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95340b4820 con 0x7f9518004880 2024-09-17T05:30:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9524021070 con 0x7f9518004880 2024-09-17T05:30:59.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f952be00640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 0x7f9510075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7f9524094e80 con 0x7f9518004880 2024-09-17T05:30:59.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f952aa00640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 0x7f9510075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.076+0000 7f952aa00640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 0x7f9510075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f951c001830 tx=0x7f951c01be70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.080+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f952405f570 con 0x7f9518004880 2024-09-17T05:30:59.218 INFO:tasks.ceph.mds.c.smithi043.stderr: -55> 2024-09-17T05:30:54.460+0000 7f924dc00640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-17T05:30:59.218 INFO:tasks.ceph.mds.c.smithi043.stderr: -54> 2024-09-17T05:30:54.460+0000 7f924dc00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-17T05:30:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.292+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95340cf380 con 0x7f9518004880 2024-09-17T05:30:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.292+0000 7f952be00640 1 -- 172.21.15.43:0/2438084762 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+3556 (secure 0 0 0) 0x7f9524063fa0 con 0x7f9518004880 2024-09-17T05:30:59.298 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:59.298 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":142,"btime":"2024-09-17T05:30:58:497029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:58.496957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 142 2024-09-17T05:30:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 msgr2=0x7f9510075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 0x7f9510075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f951c001830 tx=0x7f951c01be70 comp rx=0 tx=0).stop 2024-09-17T05:30:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 msgr2=0x7f95340ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9524011a40 tx=0x7f9524011f00 comp rx=0 tx=0).stop 2024-09-17T05:30:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 shutdown_connections 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9510073200 0x7f9510075710 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95341f7780 0x7f95340702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f953414eaf0 0x7f95341f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 --2- 172.21.15.43:0/2438084762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004880 0x7f95340ce810 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 >> 172.21.15.43:0/2438084762 conn(0x7f95340b2e80 msgr2=0x7f95340cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 shutdown_connections 2024-09-17T05:30:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.296+0000 7f953a000640 1 -- 172.21.15.43:0/2438084762 wait complete. 2024-09-17T05:30:59.322 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:30:59.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:59.339 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- 172.21.15.43:0/2124375497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 msgr2=0x7fcd400cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 --2- 172.21.15.43:0/2124375497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd400cd750 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009bc0 tx=0x7fcd3801cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- 172.21.15.43:0/2124375497 shutdown_connections 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 --2- 172.21.15.43:0/2124375497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd400cd750 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- 172.21.15.43:0/2124375497 >> 172.21.15.43:0/2124375497 conn(0x7fcd400cee60 msgr2=0x7fcd400cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- 172.21.15.43:0/2124375497 shutdown_connections 2024-09-17T05:30:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- 172.21.15.43:0/2124375497 wait complete. 2024-09-17T05:30:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 Processor -- start 2024-09-17T05:30:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.480+0000 7fcd47c00640 1 -- start start 2024-09-17T05:30:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd4009ba60 0x7fcd400c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd400d0240 0x7fcd40165f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4016ab30 con 0x7fcd400d0650 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4016a9b0 con 0x7fcd400d0240 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4016acb0 con 0x7fcd4009ba60 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59666/0 (socket says 172.21.15.43:59666) 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 -- 172.21.15.43:0/3354928455 learned_addr learned my addr 172.21.15.43:0/3354928455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd3fe00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd400d0240 0x7fcd40165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45200640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd4009ba60 0x7fcd400c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 -- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd4009ba60 msgr2=0x7fcd400c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd4009ba60 0x7fcd400c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 -- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd400d0240 msgr2=0x7fcd40165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd400d0240 0x7fcd40165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 -- 172.21.15.43:0/3354928455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd40166d00 con 0x7fcd400d0650 2024-09-17T05:30:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd45c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801a7d0 tx=0x7fcd38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380bb070 con 0x7fcd400d0650 2024-09-17T05:30:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd38004d40 con 0x7fcd400d0650 2024-09-17T05:30:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380b2040 con 0x7fcd400d0650 2024-09-17T05:30:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4015e9b0 con 0x7fcd400d0650 2024-09-17T05:30:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4006f0f0 con 0x7fcd400d0650 2024-09-17T05:30:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.484+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd40166a50 con 0x7fcd400d0650 2024-09-17T05:30:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.488+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd380a5d10 con 0x7fcd400d0650 2024-09-17T05:30:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.488+0000 7fcd3cc00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 0x7fcd20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.488+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7fcd3812e560 con 0x7fcd400d0650 2024-09-17T05:30:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.488+0000 7fcd45200640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 0x7fcd20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.488+0000 7fcd45200640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 0x7fcd20075710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd340098e0 tx=0x7fcd34009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:59.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.492+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd38016790 con 0x7fcd400d0650 2024-09-17T05:30:59.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.700+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd400b3130 con 0x7fcd400d0650 2024-09-17T05:30:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.700+0000 7fcd3cc00640 1 -- 172.21.15.43:0/3354928455 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) ==== 95+0+4379 (secure 0 0 0) 0x7fcd380f8c00 con 0x7fcd400d0650 2024-09-17T05:30:59.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:30:59.706 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":143,"btime":"2024-09-17T05:30:59:498709+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/279176341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":279176341},{"type":"v1","addr":"172.21.15.43:6835","nonce":279176341}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":143}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:30:58.496957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:30:59.706 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 143 2024-09-17T05:30:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 msgr2=0x7fcd20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 0x7fcd20075710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd340098e0 tx=0x7fcd34009060 comp rx=0 tx=0).stop 2024-09-17T05:30:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 msgr2=0x7fcd40166480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801a7d0 tx=0x7fcd38004270 comp rx=0 tx=0).stop 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 shutdown_connections 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd20073200 0x7fcd20075710 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd400d0240 0x7fcd40165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd400d0650 0x7fcd40166480 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 --2- 172.21.15.43:0/3354928455 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd4009ba60 0x7fcd400c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 >> 172.21.15.43:0/3354928455 conn(0x7fcd400cee60 msgr2=0x7fcd400ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 shutdown_connections 2024-09-17T05:30:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:30:59.704+0000 7fcd47c00640 1 -- 172.21.15.43:0/3354928455 wait complete. 2024-09-17T05:30:59.728 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-09-17T05:30:59.728 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-09-17T05:30:59.729 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:30:59.729 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:59.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:59.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:30:59.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:59.780 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:30:59.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:30:59.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:00.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 -- 172.21.15.43:0/2904067061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40d0d80 msgr2=0x7f27f40cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2904067061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40d0d80 0x7f27f40cce70 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f27f4075050 tx=0x7f27e001c800 comp rx=0 tx=0).stop 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 -- 172.21.15.43:0/2904067061 shutdown_connections 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2904067061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40d0d80 0x7f27f40cce70 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2904067061 >> v2:172.21.15.62:3300/0 conn(0x7f27f40b7ce0 0x7f27f40d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.036+0000 7f27fbe00640 1 -- 172.21.15.43:0/2904067061 >> 172.21.15.43:0/2904067061 conn(0x7f27f40cee80 msgr2=0x7f27f40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- 172.21.15.43:0/2904067061 shutdown_connections 2024-09-17T05:31:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- 172.21.15.43:0/2904067061 wait complete. 2024-09-17T05:31:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 Processor -- start 2024-09-17T05:31:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- start start 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f40b7ce0 0x7f27f4155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 0x7f27f415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4169e10 con 0x7f27f40cd3b0 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4169c90 con 0x7f27f40b7ce0 2024-09-17T05:31:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27fbe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f27f4169f90 con 0x7f27f40d0d80 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27f8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27f8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59680/0 (socket says 172.21.15.43:59680) 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.040+0000 7f27f8a00640 1 -- 172.21.15.43:0/2176798869 learned_addr learned my addr 172.21.15.43:0/2176798869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f9400640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f40b7ce0 0x7f27f4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f9e00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 0x7f27f415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 -- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 msgr2=0x7f27f415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 0x7f27f415c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 -- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f40b7ce0 msgr2=0x7f27f4155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f40b7ce0 0x7f27f4155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 -- 172.21.15.43:0/2176798869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f409dab0 con 0x7f27f40cd3b0 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f9e00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 0x7f27f415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f8a00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f27e0019410 tx=0x7f27e00a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0004140 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e00be070 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0005030 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f409dd90 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f409e350 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f40d0840 con 0x7f27f40cd3b0 2024-09-17T05:31:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.044+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27e00051d0 con 0x7f27f40cd3b0 2024-09-17T05:31:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.048+0000 7f27f1600640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 0x7f27d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.048+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7f27e012ecb0 con 0x7f27f40cd3b0 2024-09-17T05:31:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.048+0000 7f27f9400640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 0x7f27d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.048+0000 7f27f9400640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 0x7f27d4075710 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f27f4071220 tx=0x7f27e8005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:00.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.052+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27e00c7020 con 0x7f27f40cd3b0 2024-09-17T05:31:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.260+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27f40008e0 con 0x7f27f40cd3b0 2024-09-17T05:31:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.504+0000 7f27f1600640 1 -- 172.21.15.43:0/2176798869 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v144) ==== 161+0+0 (secure 0 0 0) 0x7f27e00163f0 con 0x7f27f40cd3b0 2024-09-17T05:31:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 msgr2=0x7f27d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 0x7f27d4075710 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f27f4071220 tx=0x7f27e8005f40 comp rx=0 tx=0).stop 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 msgr2=0x7f27f4156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f27e0019410 tx=0x7f27e00a6790 comp rx=0 tx=0).stop 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 shutdown_connections 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f27d4073200 0x7f27d4075710 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27f40d0d80 0x7f27f415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f40cd3b0 0x7f27f4156c00 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 --2- 172.21.15.43:0/2176798869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f40b7ce0 0x7f27f4155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 >> 172.21.15.43:0/2176798869 conn(0x7f27f40cee80 msgr2=0x7f27f4072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 shutdown_connections 2024-09-17T05:31:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.508+0000 7f27fbe00640 1 -- 172.21.15.43:0/2176798869 wait complete. 2024-09-17T05:31:00.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- 172.21.15.43:0/22472017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06f110 msgr2=0x7fefec0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 --2- 172.21.15.43:0/22472017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06f110 0x7fefec0b33e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fefec072b60 tx=0x7fefd801c800 comp rx=0 tx=0).stop 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- 172.21.15.43:0/22472017 shutdown_connections 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 --2- 172.21.15.43:0/22472017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec06f110 0x7fefec0b33e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 --2- 172.21.15.43:0/22472017 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec0b3920 0x7fefec0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 --2- 172.21.15.43:0/22472017 >> v2:172.21.15.62:3300/0 conn(0x7fefec155d70 0x7fefec15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- 172.21.15.43:0/22472017 >> 172.21.15.43:0/22472017 conn(0x7fefec0b23d0 msgr2=0x7fefec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- 172.21.15.43:0/22472017 shutdown_connections 2024-09-17T05:31:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- 172.21.15.43:0/22472017 wait complete. 2024-09-17T05:31:00.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 Processor -- start 2024-09-17T05:31:00.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.688+0000 7feff1c00640 1 -- start start 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec06f110 0x7fefec0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec155d70 0x7fefec165870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefec15f7e0 con 0x7fefec0b3920 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefec15f660 con 0x7fefec155d70 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fefec15f960 con 0x7fefec06f110 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59698/0 (socket says 172.21.15.43:59698) 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 -- 172.21.15.43:0/1815073303 learned_addr learned my addr 172.21.15.43:0/1815073303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:00.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefeaa00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec06f110 0x7fefec0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefeb400640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec155d70 0x7fefec165870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 -- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec06f110 msgr2=0x7fefec0c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec06f110 0x7fefec0c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 -- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec155d70 msgr2=0x7fefec165870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec155d70 0x7fefec165870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:00.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 -- 172.21.15.43:0/1815073303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec1f6e20 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefea000640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b0040 tx=0x7fefd80a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80b2040 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd8004430 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8005280 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec1f7100 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1f76c0 con 0x7fefec0b3920 2024-09-17T05:31:00.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.692+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec1f7b00 con 0x7fefec0b3920 2024-09-17T05:31:00.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.696+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefd8005420 con 0x7fefec0b3920 2024-09-17T05:31:00.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.696+0000 7fefe2a00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 0x7fefcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:00.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.696+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7fefd812e680 con 0x7fefec0b3920 2024-09-17T05:31:00.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.696+0000 7fefeaa00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 0x7fefcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:00.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.696+0000 7fefeaa00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 0x7fefcc075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fefec0715c0 tx=0x7fefdc00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.700+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefd80164b0 con 0x7fefec0b3920 2024-09-17T05:31:00.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:00.936+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fefec15fe10 con 0x7fefec0b3920 2024-09-17T05:31:01.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.504+0000 7fefe2a00640 1 -- 172.21.15.43:0/1815073303 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v145) ==== 101+0+0 (secure 0 0 0) 0x7fefd80f8c60 con 0x7fefec0b3920 2024-09-17T05:31:01.511 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 msgr2=0x7fefcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 0x7fefcc075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fefec0715c0 tx=0x7fefdc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 msgr2=0x7fefec1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b0040 tx=0x7fefd80a5790 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 shutdown_connections 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefcc073200 0x7fefcc075710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec155d70 0x7fefec165870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefec0b3920 0x7fefec1552b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 --2- 172.21.15.43:0/1815073303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefec06f110 0x7fefec0c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 >> 172.21.15.43:0/1815073303 conn(0x7fefec0b23d0 msgr2=0x7fefec0cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 shutdown_connections 2024-09-17T05:31:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.508+0000 7feff1c00640 1 -- 172.21.15.43:0/1815073303 wait complete. 2024-09-17T05:31:01.537 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:01.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.592+0000 7f74b5e69080 1 Processor -- start 2024-09-17T05:31:01.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.592+0000 7f74b5e69080 1 -- start start 2024-09-17T05:31:01.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 0x5646693222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646692d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564669226a00 con 0x5646692dd9c0 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564669284f40 con 0x56466938f5b0 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56466931e8c0 con 0x56466938f9c0 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59704/0 (socket says 172.21.15.43:59704) 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 -- 172.21.15.43:0/948563870 learned_addr learned my addr 172.21.15.43:0/948563870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 -- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 msgr2=0x5646692d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646692d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 -- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 msgr2=0x5646693222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74ae600640 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 0x5646693222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 0x5646693222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 -- 172.21.15.43:0/948563870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646691ffc10 con 0x5646692dd9c0 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74ae600640 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 0x5646693222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74af000640 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b220 tx=0x7f74a000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890ea3019ba73b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74ad200640 1 -- 172.21.15.43:0/948563870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000a040 con 0x5646692dd9c0 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74ad200640 1 -- 172.21.15.43:0/948563870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a000f620 con 0x5646692dd9c0 2024-09-17T05:31:01.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74ad200640 1 -- 172.21.15.43:0/948563870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000fa90 con 0x5646692dd9c0 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- 172.21.15.43:0/948563870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 msgr2=0x56466931a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f74a000b220 tx=0x7f74a000b6e0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- 172.21.15.43:0/948563870 shutdown_connections 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56466938f5b0 0x5646693222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646692d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 --2- 172.21.15.43:0/948563870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646692dd9c0 0x56466931a3c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- 172.21.15.43:0/948563870 >> 172.21.15.43:0/948563870 conn(0x5646692c4d80 msgr2=0x56466938e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:01.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- 172.21.15.43:0/948563870 shutdown_connections 2024-09-17T05:31:01.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.596+0000 7f74b5e69080 1 -- 172.21.15.43:0/948563870 wait complete. 2024-09-17T05:31:01.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 Processor -- start 2024-09-17T05:31:01.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- start start 2024-09-17T05:31:01.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646692dd9c0 0x5646693aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646693b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646692d9670 con 0x56466938f5b0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646693b0730 con 0x5646692dd9c0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5646693b00c0 con 0x56466938f9c0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59718/0 (socket says 172.21.15.43:59718) 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 learned_addr learned my addr 172.21.15.43:0/745395118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646692dd9c0 0x5646693aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646693b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 msgr2=0x5646693b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646693b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646692dd9c0 msgr2=0x5646693aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646692dd9c0 0x5646693aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564669322550 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56466938f9c0 0x5646693b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646692dd9c0 0x5646693aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f749c0086d0 tx=0x7f749c008b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74a7400640 1 -- 172.21.15.43:0/745395118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c006920 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74a7400640 1 -- 172.21.15.43:0/745395118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f749c006ac0 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646693227f0 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74a7400640 1 -- 172.21.15.43:0/745395118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c014410 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.600+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564669388f90 con 0x56466938f5b0 2024-09-17T05:31:01.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74a7400640 1 -- 172.21.15.43:0/745395118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f749c0145b0 con 0x56466938f5b0 2024-09-17T05:31:01.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74a7400640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 0x7f74900740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74a7400640 1 -- 172.21.15.43:0/745395118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7f749c0930a0 con 0x56466938f5b0 2024-09-17T05:31:01.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 0x7f74900740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.611 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 0x56466938c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466938cf10 con 0x56466938c5c0 2024-09-17T05:31:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 0x56466938c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.604+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 0x7f74900740f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f74a0003040 tx=0x7f74a0000ee0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.608+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 0x56466938c9d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.608+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v99'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.608+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669319820 con 0x56466938c5c0 2024-09-17T05:31:01.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v99'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454350 con 0x56466938c5c0 2024-09-17T05:31:01.617 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v99'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669455700 con 0x56466938c5c0 2024-09-17T05:31:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.612+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669455aa0 con 0x56466938c5c0 2024-09-17T05:31:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694546f0 con 0x56466938c5c0 2024-09-17T05:31:01.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454ab0 con 0x56466938c5c0 2024-09-17T05:31:01.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v99'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.616+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669454ea0 con 0x56466938c5c0 2024-09-17T05:31:01.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v99'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454ea0 con 0x56466938c5c0 2024-09-17T05:31:01.625 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v99'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.620+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.624+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.633 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:01.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.628+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.632+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.632+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669454e50 con 0x56466938c5c0 2024-09-17T05:31:01.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.632+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.632+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669455270 con 0x56466938c5c0 2024-09-17T05:31:01.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v99'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v99'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.636+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.643 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v99'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v99'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v99'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.640+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v99'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v99'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v99'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.644+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.651 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v99'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v99'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v99'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.648+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x56466938c5c0 2024-09-17T05:31:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v99'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564669456350 0x564669458770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694550d0 con 0x564669456350 2024-09-17T05:31:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564669456350 0x564669458770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564669456350 0x564669458770 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v99'218 uv201 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.652+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669458cb0 con 0x564669456350 2024-09-17T05:31:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v99'219 uv219 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669459050 con 0x564669456350 2024-09-17T05:31:01.665 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v99'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694598a0 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v99'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.656+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669459c40 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v99'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669459fe0 con 0x564669456350 2024-09-17T05:31:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v99'223 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v99'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.660+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.668 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:01.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v99'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v99'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v99'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.664+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v99'228 uv211 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v99'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.674 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v99'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.668+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e550 con 0x564669456350 2024-09-17T05:31:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v99'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e550 con 0x564669456350 2024-09-17T05:31:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v99'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v99'233 uv217 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.672+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v99'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.681 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:01.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v99'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945b390 con 0x564669456350 2024-09-17T05:31:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v99'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.676+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945b390 con 0x564669456350 2024-09-17T05:31:01.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v99'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945a380 con 0x564669456350 2024-09-17T05:31:01.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v99'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669456350 2024-09-17T05:31:01.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56466945aa00 0x56466945ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945d440 con 0x56466945aa00 2024-09-17T05:31:01.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56466945aa00 0x56466945ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.680+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56466945aa00 0x56466945ce20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v99'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945d8e0 con 0x56466945aa00 2024-09-17T05:31:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v99'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ddc0 con 0x56466945aa00 2024-09-17T05:31:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v99'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.691 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.684+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945f670 con 0x56466945aa00 2024-09-17T05:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v99'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945fa10 con 0x56466945aa00 2024-09-17T05:31:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v99'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e660 con 0x56466945aa00 2024-09-17T05:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v99'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.688+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466938c5c0 2024-09-17T05:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'44 uv43 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466938c5c0 2024-09-17T05:31:01.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v99'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466938c5c0 2024-09-17T05:31:01.699 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v99'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.692+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466938c5c0 2024-09-17T05:31:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v99'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 0x5646694621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466945fdb0 2024-09-17T05:31:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 0x5646694621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74afa00640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 0x5646694621d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v99'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.696+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466945fdb0 2024-09-17T05:31:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v99'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945ea50 con 0x56466945fdb0 2024-09-17T05:31:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v99'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.706 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669463720 con 0x56466945fdb0 2024-09-17T05:31:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v99'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.700+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669463720 con 0x56466945fdb0 2024-09-17T05:31:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v99'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669462710 con 0x56466945fdb0 2024-09-17T05:31:01.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v99'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564669462c20 0x564669465040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465650 con 0x564669462c20 2024-09-17T05:31:01.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564669462c20 0x564669465040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.704+0000 7f74af000640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564669462c20 0x564669465040 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v99'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669465710 con 0x564669462c20 2024-09-17T05:31:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v99'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.714 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v99'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.708+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669466b10 con 0x564669462c20 2024-09-17T05:31:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v99'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669466b10 con 0x564669462c20 2024-09-17T05:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v99'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v99'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.712+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v99'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v99'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v99'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.716+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v99'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669465b00 con 0x564669462c20 2024-09-17T05:31:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74af000640 1 -- 172.21.15.43:0/745395118 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v99'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a000d040 con 0x564669462c20 2024-09-17T05:31:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564669466180 0x5646694685a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669468c20 con 0x564669466180 2024-09-17T05:31:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564669466180 0x5646694685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74ae600640 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564669466180 0x5646694685a0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v99'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.720+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669468c20 con 0x564669466180 2024-09-17T05:31:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v99'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x564669466180 2024-09-17T05:31:01.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v99'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.729 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:01.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x564669466180 2024-09-17T05:31:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v99'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.724+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x564669466180 2024-09-17T05:31:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v99'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x56466938c5c0 2024-09-17T05:31:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v99'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469070 con 0x56466938c5c0 2024-09-17T05:31:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v99'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.728+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x56466938c5c0 2024-09-17T05:31:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.732+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v99'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.736 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.732+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466946a080 con 0x56466938c5c0 2024-09-17T05:31:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.732+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v99'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.732+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466946a080 con 0x56466938c5c0 2024-09-17T05:31:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.732+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v99'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.736+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x56466938c5c0 2024-09-17T05:31:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.736+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v99'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466938c5c0 2024-09-17T05:31:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.736+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469070 con 0x56466945aa00 2024-09-17T05:31:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.736+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v99'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.742 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.736+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [write 0~34] v99'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v99'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 100.00000000 [create] v99'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.740+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.744+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 100.00000000 [delete] v99'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.744+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.748 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.744+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v99'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.744+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466945e250 con 0x56466945aa00 2024-09-17T05:31:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.744+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v99'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.748+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466946be80 con 0x56466945aa00 2024-09-17T05:31:01.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.748+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v99'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x56466945aa00 2024-09-17T05:31:01.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.748+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.748+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v99'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v99'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.758 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:01.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v99'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.752+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v99'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.756+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.756+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v99'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.756+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.756+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v99'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.756+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.763 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v99'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466946a530 con 0x56466945fdb0 2024-09-17T05:31:01.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v99'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x56466946a530 con 0x56466945fdb0 2024-09-17T05:31:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v99'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.760+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x56466945fdb0 2024-09-17T05:31:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v99'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f74a80b3040 con 0x56466945fdb0 2024-09-17T05:31:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v99'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v99'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.764+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.772 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v99'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v99'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v99'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.768+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v99'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v99'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v99'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.779 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.772+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v99'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v99'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v99'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.776+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v99'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v99'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.786 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:01.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v99'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.780+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v99'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x564669469520 con 0x564669466180 2024-09-17T05:31:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v99'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x5646694698e0 con 0x564669466180 2024-09-17T05:31:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74ae600640 1 -- 172.21.15.43:0/745395118 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v99'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f749c059950 con 0x564669466180 2024-09-17T05:31:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564669456350 msgr2=0x564669458770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564669456350 0x564669458770 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564669462c20 msgr2=0x564669465040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564669462c20 0x564669465040 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564669466180 msgr2=0x5646694685a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564669466180 0x5646694685a0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56466945aa00 msgr2=0x56466945ce20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56466945aa00 0x56466945ce20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 msgr2=0x56466938c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74afa00640 1 -- 172.21.15.43:0/745395118 reap_dead start 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 0x56466938c9d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 msgr2=0x5646694621d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 0x5646694621d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 msgr2=0x7f74900740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 0x7f74900740f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f74a0003040 tx=0x7f74a0000ee0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 msgr2=0x564669388a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f749c0086d0 tx=0x7f749c008b90 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.784+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 shutdown_connections 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56466945fdb0 0x5646694621d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7490071be0 0x7f74900740f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56466938f5b0 0x564669388a50 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 --2- 172.21.15.43:0/745395118 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56466938c5c0 0x56466938c9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 >> 172.21.15.43:0/745395118 conn(0x5646692c4d80 msgr2=0x5646693af430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 shutdown_connections 2024-09-17T05:31:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.788+0000 7f74b5e69080 1 -- 172.21.15.43:0/745395118 wait complete. 2024-09-17T05:31:01.797 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-09-17T05:31:01.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 Processor -- start 2024-09-17T05:31:01.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 -- start start 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0ee1cb0 0x5586a101b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0f68e40 0x5586a101b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586a0e00a00 con 0x5586a0eb2d70 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5586a0e5ef40 con 0x5586a0ee1cb0 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70fd19a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5586a0f69890 con 0x5586a0f68e40 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59742/0 (socket says 172.21.15.43:59742) 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 -- 172.21.15.43:0/1918274531 learned_addr learned my addr 172.21.15.43:0/1918274531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f5a00640 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0ee1cb0 0x5586a101b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6e00640 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0f68e40 0x5586a101b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 -- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0f68e40 msgr2=0x5586a101b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0f68e40 0x5586a101b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 -- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0ee1cb0 msgr2=0x5586a101b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0ee1cb0 0x5586a101b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 -- 172.21.15.43:0/1918274531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a0dd9c10 con 0x5586a0eb2d70 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70f6400640 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f70e809a320 tx=0x7f70e801a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dbc2338fe9ea44 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70efe00640 1 -- 172.21.15.43:0/1918274531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80a7a90 con 0x5586a0eb2d70 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70efe00640 1 -- 172.21.15.43:0/1918274531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70e80a7c30 con 0x5586a0eb2d70 2024-09-17T05:31:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.892+0000 7f70efe00640 1 -- 172.21.15.43:0/1918274531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80b0470 con 0x5586a0eb2d70 2024-09-17T05:31:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- 172.21.15.43:0/1918274531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 msgr2=0x5586a0ef7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f70e809a320 tx=0x7f70e801a790 comp rx=0 tx=0).stop 2024-09-17T05:31:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- 172.21.15.43:0/1918274531 shutdown_connections 2024-09-17T05:31:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0ee1cb0 0x5586a101b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0f68e40 0x5586a101b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1918274531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0eb2d70 0x5586a0ef7d50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- 172.21.15.43:0/1918274531 >> 172.21.15.43:0/1918274531 conn(0x5586a0e9ed80 msgr2=0x5586a0eb0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- 172.21.15.43:0/1918274531 shutdown_connections 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- 172.21.15.43:0/1918274531 wait complete. 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 Processor -- start 2024-09-17T05:31:01.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- start start 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 0x5586a0f67750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0ee1cb0 0x5586a0f837a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0f68e40 0x5586a0f83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5586a0ee18f0 con 0x5586a0ee1cb0 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5586a0f853c0 con 0x5586a0f68e40 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70fd19a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5586a0efa020 con 0x5586a0eb2d70 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70f6400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 0x5586a0f67750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70f5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0ee1cb0 0x5586a0f837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70f6400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 0x5586a0f67750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:60898/0 (socket says 172.21.15.43:60898) 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70f6400640 1 -- 172.21.15.43:0/1119159652 learned_addr learned my addr 172.21.15.43:0/1119159652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.896+0000 7f70f6e00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0f68e40 0x5586a0f83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 -- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 msgr2=0x5586a0f67750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 0x5586a0f67750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 -- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0f68e40 msgr2=0x5586a0f83ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5586a0f68e40 0x5586a0f83ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 -- 172.21.15.43:0/1119159652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a0f85020 con 0x5586a0ee1cb0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f6400640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5586a0eb2d70 0x5586a0f67750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70f5a00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0ee1cb0 0x5586a0f837a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f70e4003e80 tx=0x7f70e400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 -- 172.21.15.43:0/1119159652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400bea0 con 0x5586a0ee1cb0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 -- 172.21.15.43:0/1119159652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70e4016070 con 0x5586a0ee1cb0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 -- 172.21.15.43:0/1119159652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011470 con 0x5586a0ee1cb0 2024-09-17T05:31:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5586a10172f0 con 0x5586a0ee1cb0 2024-09-17T05:31:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a1017830 con 0x5586a0ee1cb0 2024-09-17T05:31:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 -- 172.21.15.43:0/1119159652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70e400f040 con 0x5586a0ee1cb0 2024-09-17T05:31:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70d8071be0 0x7f70d80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.900+0000 7f70eea00640 1 -- 172.21.15.43:0/1119159652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7f70e4092cb0 con 0x5586a0ee1cb0 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6400640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70d8071be0 0x7f70d80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5586a10200f0 0x5586a1020500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x5586a10156f0 con 0x5586a10200f0 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6e00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5586a10200f0 0x5586a1020500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6400640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70d8071be0 0x7f70d80740f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70e8004240 tx=0x7f70e80b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6e00640 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5586a10200f0 0x5586a1020500 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6e00640 1 -- 172.21.15.43:0/1119159652 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [delete] v99'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70f000b040 con 0x5586a10200f0 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5586a10200f0 msgr2=0x5586a1020500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5586a10200f0 0x5586a1020500 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70d8071be0 msgr2=0x7f70d80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70d8071be0 0x7f70d80740f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70e8004240 tx=0x7f70e80b2040 comp rx=0 tx=0).stop 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0ee1cb0 msgr2=0x5586a0f837a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 --2- 172.21.15.43:0/1119159652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5586a0ee1cb0 0x5586a0f837a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f70e4003e80 tx=0x7f70e400b5a0 comp rx=0 tx=0).stop 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70f6e00640 1 -- 172.21.15.43:0/1119159652 reap_dead start 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 shutdown_connections 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 >> 172.21.15.43:0/1119159652 conn(0x5586a0e9ed80 msgr2=0x5586a0f88050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 shutdown_connections 2024-09-17T05:31:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:01.904+0000 7f70fd19a080 1 -- 172.21.15.43:0/1119159652 wait complete. 2024-09-17T05:31:01.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 -- 172.21.15.43:0/3168034978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e80737c0 msgr2=0x7f18e80700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 --2- 172.21.15.43:0/3168034978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e80737c0 0x7f18e80700a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009840 tx=0x7f18e001c700 comp rx=0 tx=0).stop 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 -- 172.21.15.43:0/3168034978 shutdown_connections 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 --2- 172.21.15.43:0/3168034978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e80737c0 0x7f18e80700a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 --2- 172.21.15.43:0/3168034978 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e815fe10 0x7f18e80b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 -- 172.21.15.43:0/3168034978 >> 172.21.15.43:0/3168034978 conn(0x7f18e80b23d0 msgr2=0x7f18e80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 -- 172.21.15.43:0/3168034978 shutdown_connections 2024-09-17T05:31:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.072+0000 7f18ee800640 1 -- 172.21.15.43:0/3168034978 wait complete. 2024-09-17T05:31:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 Processor -- start 2024-09-17T05:31:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 -- start start 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e815fe10 0x7f18e8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 0x7f18e81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8169f80 con 0x7f18e806fc90 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8169e00 con 0x7f18e815fe10 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18ee800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f18e816a100 con 0x7f18e8099ae0 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59754/0 (socket says 172.21.15.43:59754) 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 -- 172.21.15.43:0/845555981 learned_addr learned my addr 172.21.15.43:0/845555981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7e00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 0x7f18e81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18dea00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e815fe10 0x7f18e8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 -- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 msgr2=0x7f18e81570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 0x7f18e81570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 -- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e815fe10 msgr2=0x7f18e8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e815fe10 0x7f18e8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 -- 172.21.15.43:0/845555981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e81f9b00 con 0x7f18e806fc90 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7e00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 0x7f18e81570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.076+0000 7f18e7400640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f18d800ed60 tx=0x7f18d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:02.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003b80 con 0x7f18e806fc90 2024-09-17T05:31:02.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d8003d20 con 0x7f18e806fc90 2024-09-17T05:31:02.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8004280 con 0x7f18e806fc90 2024-09-17T05:31:02.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e8158dc0 con 0x7f18e806fc90 2024-09-17T05:31:02.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e81f9820 con 0x7f18e806fc90 2024-09-17T05:31:02.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e80700a0 con 0x7f18e806fc90 2024-09-17T05:31:02.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.080+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18d8004420 con 0x7f18e806fc90 2024-09-17T05:31:02.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.084+0000 7f18dfe00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 0x7f18c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.084+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7f18d8095440 con 0x7f18e806fc90 2024-09-17T05:31:02.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.084+0000 7f18dea00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 0x7f18c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.084+0000 7f18dea00640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 0x7f18c4075710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f18e815abb0 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.088+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18d80600f0 con 0x7f18e806fc90 2024-09-17T05:31:02.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.296+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f18e815f7e0 con 0x7f18e806fc90 2024-09-17T05:31:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18dfe00640 1 -- 172.21.15.43:0/845555981 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v146) ==== 162+0+0 (secure 0 0 0) 0x7f18d805f9f0 con 0x7f18e806fc90 2024-09-17T05:31:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 msgr2=0x7f18c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 0x7f18c4075710 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f18e815abb0 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2024-09-17T05:31:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 msgr2=0x7f18e80994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f18d800ed60 tx=0x7f18d800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 shutdown_connections 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f18c4073200 0x7f18c4075710 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18e815fe10 0x7f18e8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f18e8099ae0 0x7f18e81570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 --2- 172.21.15.43:0/845555981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e806fc90 0x7f18e80994b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 >> 172.21.15.43:0/845555981 conn(0x7f18e80b23d0 msgr2=0x7f18e80aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 shutdown_connections 2024-09-17T05:31:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.512+0000 7f18ee800640 1 -- 172.21.15.43:0/845555981 wait complete. 2024-09-17T05:31:02.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 -- 172.21.15.43:0/1536296318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c80d0910 msgr2=0x7fe6c806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 --2- 172.21.15.43:0/1536296318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c80d0910 0x7fe6c806ce60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009840 tx=0x7fe6b001c790 comp rx=0 tx=0).stop 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 -- 172.21.15.43:0/1536296318 shutdown_connections 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 --2- 172.21.15.43:0/1536296318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 0x7fe6c806d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 --2- 172.21.15.43:0/1536296318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c80d0910 0x7fe6c806ce60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 -- 172.21.15.43:0/1536296318 >> 172.21.15.43:0/1536296318 conn(0x7fe6c80b23d0 msgr2=0x7fe6c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 -- 172.21.15.43:0/1536296318 shutdown_connections 2024-09-17T05:31:02.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.696+0000 7fe6cde00640 1 -- 172.21.15.43:0/1536296318 wait complete. 2024-09-17T05:31:02.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 Processor -- start 2024-09-17T05:31:02.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- start start 2024-09-17T05:31:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c80d0910 0x7fe6c80c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 0x7fe6c80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c80ce280 con 0x7fe6c8070620 2024-09-17T05:31:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c80ce100 con 0x7fe6c80d0910 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c80ce400 con 0x7fe6c80d0d20 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59776/0 (socket says 172.21.15.43:59776) 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 -- 172.21.15.43:0/3374570018 learned_addr learned my addr 172.21.15.43:0/3374570018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c7400640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 0x7fe6c80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6be000640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c80d0910 0x7fe6c80c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 -- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 msgr2=0x7fe6c80c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 0x7fe6c80c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 -- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c80d0910 msgr2=0x7fe6c80c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c80d0910 0x7fe6c80c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 -- 172.21.15.43:0/3374570018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c80cacd0 con 0x7fe6c8070620 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6c6a00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c460 tx=0x7fe6b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8016020 con 0x7fe6c8070620 2024-09-17T05:31:02.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6b8012940 con 0x7fe6c8070620 2024-09-17T05:31:02.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8012d90 con 0x7fe6c8070620 2024-09-17T05:31:02.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c80cafb0 con 0x7fe6c8070620 2024-09-17T05:31:02.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.700+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c80c7500 con 0x7fe6c8070620 2024-09-17T05:31:02.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c80ce990 con 0x7fe6c8070620 2024-09-17T05:31:02.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6b8019020 con 0x7fe6c8070620 2024-09-17T05:31:02.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6bfe00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 0x7fe6a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:02.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6569+0+0 (secure 0 0 0) 0x7fe6b80940b0 con 0x7fe6c8070620 2024-09-17T05:31:02.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6be000640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 0x7fe6a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:02.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.704+0000 7fe6be000640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 0x7fe6a4075740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00ae000 tx=0x7fe6b00b3040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:02.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.708+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe6b8020030 con 0x7fe6c8070620 2024-09-17T05:31:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.920+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6c80af4d0 con 0x7fe6c8070620 2024-09-17T05:31:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.920+0000 7fe6bfe00640 1 -- 172.21.15.43:0/3374570018 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) ==== 95+0+4388 (secure 0 0 0) 0x7fe6b805e7e0 con 0x7fe6c8070620 2024-09-17T05:31:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:02.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":147,"btime":"2024-09-17T05:31:02:519354+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:02.519334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8091":{"gid":8091,"name":"c","rank":0,"incarnation":147,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/279176341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":279176341},{"type":"v1","addr":"172.21.15.43:6835","nonce":279176341}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 147 2024-09-17T05:31:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 msgr2=0x7fe6a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 0x7fe6a4075740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00ae000 tx=0x7fe6b00b3040 comp rx=0 tx=0).stop 2024-09-17T05:31:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 msgr2=0x7fe6c809cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c460 tx=0x7fe6b800c920 comp rx=0 tx=0).stop 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 shutdown_connections 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6a4073230 0x7fe6a4075740 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6c80d0910 0x7fe6c80c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6c80d0d20 0x7fe6c80c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3374570018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c8070620 0x7fe6c809cf80 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 >> 172.21.15.43:0/3374570018 conn(0x7fe6c80b23d0 msgr2=0x7fe6c80afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 shutdown_connections 2024-09-17T05:31:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:02.924+0000 7fe6cde00640 1 -- 172.21.15.43:0/3374570018 wait complete. 2024-09-17T05:31:02.948 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:31:03.247 INFO:tasks.ceph.mds.d.smithi062.stderr: -53> 2024-09-17T05:30:58.493+0000 7f307e200640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-17T05:31:03.248 INFO:tasks.ceph.mds.d.smithi062.stderr: -52> 2024-09-17T05:30:58.493+0000 7f307e200640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-17T05:31:03.372 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:31:05.539 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:05.532+0000 7f398ae00640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-17T05:31:05.541 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:05.536+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:05.541 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:05.536+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:07.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- 172.21.15.43:0/3499005390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 msgr2=0x7f34640b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 --2- 172.21.15.43:0/3499005390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34640b1e80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f345c009bc0 tx=0x7f345c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- 172.21.15.43:0/3499005390 shutdown_connections 2024-09-17T05:31:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 --2- 172.21.15.43:0/3499005390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34640b1e80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 --2- 172.21.15.43:0/3499005390 >> v2:172.21.15.62:3300/0 conn(0x7f3464070d60 0x7f34640b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- 172.21.15.43:0/3499005390 >> 172.21.15.43:0/3499005390 conn(0x7f34640b24c0 msgr2=0x7f34640b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- 172.21.15.43:0/3499005390 shutdown_connections 2024-09-17T05:31:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- 172.21.15.43:0/3499005390 wait complete. 2024-09-17T05:31:08.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 Processor -- start 2024-09-17T05:31:08.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.120+0000 7f346c400640 1 -- start start 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34641fc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f346415fbb0 0x7f3464137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f346413faf0 con 0x7f34641518b0 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f346413f970 con 0x7f346415fbb0 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f346413fc70 con 0x7f34640b1a70 2024-09-17T05:31:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38446/0 (socket says 172.21.15.43:38446) 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 -- 172.21.15.43:0/3305999453 learned_addr learned my addr 172.21.15.43:0/3305999453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f3469a00640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34641fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f3469000640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f346415fbb0 0x7f3464137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 -- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 msgr2=0x7f34641fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34641fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 -- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f346415fbb0 msgr2=0x7f3464137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f346415fbb0 0x7f3464137050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 -- 172.21.15.43:0/3305999453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346415af10 con 0x7f34641518b0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346a400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f345c01a710 tx=0x7f345c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0bb070 con 0x7f34641518b0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f345c004d40 con 0x7f34641518b0 2024-09-17T05:31:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0b2040 con 0x7f34641518b0 2024-09-17T05:31:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34641fca80 con 0x7f34641518b0 2024-09-17T05:31:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.124+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346415b490 con 0x7f34641518b0 2024-09-17T05:31:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34641401b0 con 0x7f34641518b0 2024-09-17T05:31:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f345c0be070 con 0x7f34641518b0 2024-09-17T05:31:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f3461600640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 0x7f3444075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f345c12e1b0 con 0x7f34641518b0 2024-09-17T05:31:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f3469a00640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 0x7f3444075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.128+0000 7f3469a00640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 0x7f3444075710 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f34640c6280 tx=0x7f3458009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.132+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f345c016790 con 0x7f34641518b0 2024-09-17T05:31:08.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.340+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346415b6f0 con 0x7f34641518b0 2024-09-17T05:31:08.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.340+0000 7f3461600640 1 -- 172.21.15.43:0/3305999453 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+4383 (secure 0 0 0) 0x7f345c0f8eb0 con 0x7f34641518b0 2024-09-17T05:31:08.346 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:08.347 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":151,"btime":"2024-09-17T05:31:05:541550+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:05.541545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:08.347 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 151 2024-09-17T05:31:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 msgr2=0x7f3444075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 0x7f3444075710 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f34640c6280 tx=0x7f3458009040 comp rx=0 tx=0).stop 2024-09-17T05:31:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 msgr2=0x7f34640cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f345c01a710 tx=0x7f345c003f60 comp rx=0 tx=0).stop 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 shutdown_connections 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3444073200 0x7f3444075710 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f346415fbb0 0x7f3464137050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34641518b0 0x7f34640cd9c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 --2- 172.21.15.43:0/3305999453 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f34640b1a70 0x7f34641fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 >> 172.21.15.43:0/3305999453 conn(0x7f34640b24c0 msgr2=0x7f346406f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 shutdown_connections 2024-09-17T05:31:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.344+0000 7f346c400640 1 -- 172.21.15.43:0/3305999453 wait complete. 2024-09-17T05:31:08.369 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:31:08.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- 172.21.15.43:0/3702953729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc155d70 msgr2=0x7efedc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 --2- 172.21.15.43:0/3702953729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc155d70 0x7efedc15c7e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7efedc0d0c40 tx=0x7efed001c740 comp rx=0 tx=0).stop 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- 172.21.15.43:0/3702953729 shutdown_connections 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 --2- 172.21.15.43:0/3702953729 >> v2:172.21.15.62:3300/0 conn(0x7efedc0b3af0 0x7efedc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 --2- 172.21.15.43:0/3702953729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc155d70 0x7efedc15c7e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- 172.21.15.43:0/3702953729 >> 172.21.15.43:0/3702953729 conn(0x7efedc0b23d0 msgr2=0x7efedc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- 172.21.15.43:0/3702953729 shutdown_connections 2024-09-17T05:31:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- 172.21.15.43:0/3702953729 wait complete. 2024-09-17T05:31:08.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 Processor -- start 2024-09-17T05:31:08.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.524+0000 7efee4000640 1 -- start start 2024-09-17T05:31:08.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc06f110 0x7efedc154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 0x7efedc0992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efedc168c00 con 0x7efedc0af0d0 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efedc168a80 con 0x7efedc155d70 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efedc168d80 con 0x7efedc06f110 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38464/0 (socket says 172.21.15.43:38464) 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 -- 172.21.15.43:0/4215798409 learned_addr learned my addr 172.21.15.43:0/4215798409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee1600640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc06f110 0x7efedc154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee0c00640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 0x7efedc0992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 -- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc06f110 msgr2=0x7efedc154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc06f110 0x7efedc154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 -- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 msgr2=0x7efedc0992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 0x7efedc0992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 -- 172.21.15.43:0/4215798409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc155050 con 0x7efedc0af0d0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee0c00640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 0x7efedc0992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee2000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400cdc0 con 0x7efedc0af0d0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed4004b30 con 0x7efedc0af0d0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc0af6b0 con 0x7efedc0af0d0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4005020 con 0x7efedc0af0d0 2024-09-17T05:31:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc0af970 con 0x7efedc0af0d0 2024-09-17T05:31:08.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.528+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc0ad0b0 con 0x7efedc0af0d0 2024-09-17T05:31:08.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.532+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efed40051c0 con 0x7efedc0af0d0 2024-09-17T05:31:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.532+0000 7efed9600640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 0x7efebc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.532+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7efed4095330 con 0x7efedc0af0d0 2024-09-17T05:31:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.532+0000 7efee1600640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 0x7efebc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.532+0000 7efee1600640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 0x7efebc075710 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efed00af000 tx=0x7efed001af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.536+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efed405fe80 con 0x7efedc0af0d0 2024-09-17T05:31:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.744+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efedc15da50 con 0x7efedc0af0d0 2024-09-17T05:31:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.744+0000 7efed9600640 1 -- 172.21.15.43:0/4215798409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+4383 (secure 0 0 0) 0x7efed405f780 con 0x7efedc0af0d0 2024-09-17T05:31:08.752 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:08.752 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":151,"btime":"2024-09-17T05:31:05:541550+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:05.541545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 151 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 msgr2=0x7efebc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 0x7efebc075710 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efed00af000 tx=0x7efed001af80 comp rx=0 tx=0).stop 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 msgr2=0x7efedc099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed400ef10 comp rx=0 tx=0).stop 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 shutdown_connections 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efebc073200 0x7efebc075710 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efedc155d70 0x7efedc0992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efedc0af0d0 0x7efedc099800 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 --2- 172.21.15.43:0/4215798409 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efedc06f110 0x7efedc154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 >> 172.21.15.43:0/4215798409 conn(0x7efedc0b23d0 msgr2=0x7efedc0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 shutdown_connections 2024-09-17T05:31:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.748+0000 7efee4000640 1 -- 172.21.15.43:0/4215798409 wait complete. 2024-09-17T05:31:08.774 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:08.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:08.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- 172.21.15.43:0/2643858826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c0b3920 msgr2=0x7ff01c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 --2- 172.21.15.43:0/2643858826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c0b3920 0x7ff01c0ad0b0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0711f0 tx=0x7ff01401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- 172.21.15.43:0/2643858826 shutdown_connections 2024-09-17T05:31:08.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 --2- 172.21.15.43:0/2643858826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c0b3920 0x7ff01c0ad0b0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 --2- 172.21.15.43:0/2643858826 >> v2:172.21.15.62:3300/0 conn(0x7ff01c06f110 0x7ff01c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 --2- 172.21.15.43:0/2643858826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c155d70 0x7ff01c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- 172.21.15.43:0/2643858826 >> 172.21.15.43:0/2643858826 conn(0x7ff01c0b23d0 msgr2=0x7ff01c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:08.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- 172.21.15.43:0/2643858826 shutdown_connections 2024-09-17T05:31:08.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- 172.21.15.43:0/2643858826 wait complete. 2024-09-17T05:31:08.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 Processor -- start 2024-09-17T05:31:08.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.928+0000 7ff023a00640 1 -- start start 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c06f110 0x7ff01c09c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 0x7ff01c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c15f830 con 0x7ff01c155d70 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c15f6b0 con 0x7ff01c0b3920 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c15f9b0 con 0x7ff01c06f110 2024-09-17T05:31:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38488/0 (socket says 172.21.15.43:38488) 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 -- 172.21.15.43:0/3321516495 learned_addr learned my addr 172.21.15.43:0/3321516495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021000640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c06f110 0x7ff01c09c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff013e00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 0x7ff01c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 -- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c06f110 msgr2=0x7ff01c09c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c06f110 0x7ff01c09c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 -- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 msgr2=0x7ff01c0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 0x7ff01c0b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 -- 172.21.15.43:0/3321516495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c09d3c0 con 0x7ff01c155d70 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff013e00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 0x7ff01c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff021a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff01401b370 tx=0x7ff014005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0140bc070 con 0x7ff01c155d70 2024-09-17T05:31:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff014004130 con 0x7ff01c155d70 2024-09-17T05:31:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c0cc930 con 0x7ff01c155d70 2024-09-17T05:31:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0140b2040 con 0x7ff01c155d70 2024-09-17T05:31:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1fc1c0 con 0x7ff01c155d70 2024-09-17T05:31:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.932+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c15fe80 con 0x7ff01c155d70 2024-09-17T05:31:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.936+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff014004580 con 0x7ff01c155d70 2024-09-17T05:31:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.936+0000 7ff019600640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 0x7feff8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.936+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7ff01412e4f0 con 0x7ff01c155d70 2024-09-17T05:31:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.936+0000 7ff021000640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 0x7feff8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.936+0000 7ff021000640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 0x7feff8075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0715c0 tx=0x7ff00c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:08.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:08.940+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff014016970 con 0x7ff01c155d70 2024-09-17T05:31:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff01c0d0610 con 0x7ff01c155d70 2024-09-17T05:31:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff019600640 1 -- 172.21.15.43:0/3321516495 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+4383 (secure 0 0 0) 0x7ff0140f90e0 con 0x7ff01c155d70 2024-09-17T05:31:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:09.157 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":151,"btime":"2024-09-17T05:31:05:541550+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:05.541545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:09.157 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 151 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 msgr2=0x7feff8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 0x7feff8075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0715c0 tx=0x7ff00c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 msgr2=0x7ff01c0b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff01401b370 tx=0x7ff014005a60 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 shutdown_connections 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feff8073200 0x7feff8075710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff01c0b3920 0x7ff01c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff01c155d70 0x7ff01c0b1470 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 --2- 172.21.15.43:0/3321516495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff01c06f110 0x7ff01c09c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 >> 172.21.15.43:0/3321516495 conn(0x7ff01c0b23d0 msgr2=0x7ff01c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:09.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 shutdown_connections 2024-09-17T05:31:09.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.152+0000 7ff023a00640 1 -- 172.21.15.43:0/3321516495 wait complete. 2024-09-17T05:31:09.179 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-09-17T05:31:09.179 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-09-17T05:31:09.179 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:09.179 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:09.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:09.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:09.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:09.189 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:09.238 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:09.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:09.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 -- 172.21.15.43:0/1806377275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a40737c0 msgr2=0x7f27a40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 --2- 172.21.15.43:0/1806377275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a40737c0 0x7f27a40700a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f279c009bc0 tx=0x7f279c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 -- 172.21.15.43:0/1806377275 shutdown_connections 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 --2- 172.21.15.43:0/1806377275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a40737c0 0x7f27a40700a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 --2- 172.21.15.43:0/1806377275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a415fe10 0x7f27a40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 -- 172.21.15.43:0/1806377275 >> 172.21.15.43:0/1806377275 conn(0x7f27a40b23d0 msgr2=0x7f27a40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:09.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 -- 172.21.15.43:0/1806377275 shutdown_connections 2024-09-17T05:31:09.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.436+0000 7f27aac00640 1 -- 172.21.15.43:0/1806377275 wait complete. 2024-09-17T05:31:09.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 Processor -- start 2024-09-17T05:31:09.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 -- start start 2024-09-17T05:31:09.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a406fc90 0x7f27a41f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a4157680 0x7f27a415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4169f30 con 0x7f27a415fe10 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27a4169db0 con 0x7f27a406fc90 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27aac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f27a416a0b0 con 0x7f27a4157680 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38516/0 (socket says 172.21.15.43:38516) 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 -- 172.21.15.43:0/3420778003 learned_addr learned my addr 172.21.15.43:0/3420778003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27a3e00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a406fc90 0x7f27a41f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f27a8c00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a4157680 0x7f27a415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:09.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 -- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a4157680 msgr2=0x7f27a415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a4157680 0x7f27a415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 -- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a406fc90 msgr2=0x7f27a41f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a406fc90 0x7f27a41f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.440+0000 7f279be00640 1 -- 172.21.15.43:0/3420778003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a41f9db0 con 0x7f27a415fe10 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f279be00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c430 tx=0x7f278c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011790 con 0x7f27a415fe10 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f278c017070 con 0x7f27a415fe10 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011ce0 con 0x7f27a415fe10 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a41fa050 con 0x7f27a415fe10 2024-09-17T05:31:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a41fa610 con 0x7f27a415fe10 2024-09-17T05:31:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a406a790 con 0x7f27a415fe10 2024-09-17T05:31:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f278c01c460 con 0x7f27a415fe10 2024-09-17T05:31:09.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 0x7f2780075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:09.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.444+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f278c0943b0 con 0x7f27a415fe10 2024-09-17T05:31:09.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.448+0000 7f27a3e00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 0x7f2780075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:09.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.448+0000 7f27a3e00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 0x7f2780075740 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2794009ad0 tx=0x7f27940091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:09.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.448+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f278c05f0d0 con 0x7f27a415fe10 2024-09-17T05:31:09.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:09.660+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27a415cec0 con 0x7f27a415fe10 2024-09-17T05:31:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.020+0000 7f27a0c00640 1 -- 172.21.15.43:0/3420778003 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v152) ==== 161+0+0 (secure 0 0 0) 0x7f278c05e9d0 con 0x7f27a415fe10 2024-09-17T05:31:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:10.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 msgr2=0x7f2780075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:10.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 0x7f2780075740 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2794009ad0 tx=0x7f27940091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 msgr2=0x7f27a4157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c430 tx=0x7f278c00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 shutdown_connections 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2780073230 0x7f2780075740 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f27a4157680 0x7f27a415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27a415fe10 0x7f27a4157140 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 --2- 172.21.15.43:0/3420778003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a406fc90 0x7f27a41f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 >> 172.21.15.43:0/3420778003 conn(0x7f27a40b23d0 msgr2=0x7f27a40ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 shutdown_connections 2024-09-17T05:31:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.024+0000 7f27aac00640 1 -- 172.21.15.43:0/3420778003 wait complete. 2024-09-17T05:31:10.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- 172.21.15.43:0/2291256152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 msgr2=0x7f71cc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 --2- 172.21.15.43:0/2291256152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc15c7e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc01c820 comp rx=0 tx=0).stop 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- 172.21.15.43:0/2291256152 shutdown_connections 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 --2- 172.21.15.43:0/2291256152 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc06f110 0x7f71cc0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 --2- 172.21.15.43:0/2291256152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc15c7e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- 172.21.15.43:0/2291256152 >> 172.21.15.43:0/2291256152 conn(0x7f71cc0b23d0 msgr2=0x7f71cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- 172.21.15.43:0/2291256152 shutdown_connections 2024-09-17T05:31:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- 172.21.15.43:0/2291256152 wait complete. 2024-09-17T05:31:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 Processor -- start 2024-09-17T05:31:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.208+0000 7f71d2c00640 1 -- start start 2024-09-17T05:31:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71cc06f110 0x7f71cc0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc0b3920 0x7f71cc15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc168ab0 con 0x7f71cc155d70 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc168930 con 0x7f71cc06f110 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc168c30 con 0x7f71cc0b3920 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71cc06f110 0x7f71cc0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38538/0 (socket says 172.21.15.43:38538) 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 -- 172.21.15.43:0/3664413644 learned_addr learned my addr 172.21.15.43:0/3664413644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71cb400640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc0b3920 0x7f71cc15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 -- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc0b3920 msgr2=0x7f71cc15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc0b3920 0x7f71cc15a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 -- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71cc06f110 msgr2=0x7f71cc0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71cc06f110 0x7f71cc0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 -- 172.21.15.43:0/3664413644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1f9680 con 0x7f71cc155d70 2024-09-17T05:31:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d0c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c730 tx=0x7f71c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4003b80 con 0x7f71cc155d70 2024-09-17T05:31:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71c4003d20 con 0x7f71cc155d70 2024-09-17T05:31:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1f9960 con 0x7f71cc155d70 2024-09-17T05:31:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c400aab0 con 0x7f71cc155d70 2024-09-17T05:31:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc0c7bc0 con 0x7f71cc155d70 2024-09-17T05:31:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.212+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc0b33e0 con 0x7f71cc155d70 2024-09-17T05:31:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.216+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71c400ac50 con 0x7f71cc155d70 2024-09-17T05:31:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.216+0000 7f71c3e00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 0x7f71ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.216+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f71c4094310 con 0x7f71cc155d70 2024-09-17T05:31:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.216+0000 7f71cbe00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 0x7f71ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.216+0000 7f71cbe00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 0x7f71ac075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0af000 tx=0x7f71bc01b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.220+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71cc0b3d30 con 0x7f71cc155d70 2024-09-17T05:31:10.292 INFO:tasks.ceph.mds.c.smithi043.stderr: -45> 2024-09-17T05:31:05.532+0000 7f398ae00640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-17T05:31:10.412 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:31:10.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:10.464+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f71cc1f9410 con 0x7f71cc155d70 2024-09-17T05:31:11.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.024+0000 7f71c3e00640 1 -- 172.21.15.43:0/3664413644 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v153) ==== 101+0+0 (secure 0 0 0) 0x7f71c405e8a0 con 0x7f71cc155d70 2024-09-17T05:31:11.030 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 msgr2=0x7f71ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 0x7f71ac075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0af000 tx=0x7f71bc01b060 comp rx=0 tx=0).stop 2024-09-17T05:31:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 msgr2=0x7f71cc073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c730 tx=0x7f71c400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 shutdown_connections 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71ac073200 0x7f71ac075710 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71cc155d70 0x7f71cc073800 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f71cc0b3920 0x7f71cc15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 --2- 172.21.15.43:0/3664413644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71cc06f110 0x7f71cc0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 >> 172.21.15.43:0/3664413644 conn(0x7f71cc0b23d0 msgr2=0x7f71cc0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 shutdown_connections 2024-09-17T05:31:11.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.028+0000 7f71d2c00640 1 -- 172.21.15.43:0/3664413644 wait complete. 2024-09-17T05:31:11.057 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 Processor -- start 2024-09-17T05:31:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- start start 2024-09-17T05:31:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18adc5cb0 0x55b18ad94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 0x55b18ae68430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b18ae4d8e0 con 0x55b18ad96d70 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b18ace4a00 con 0x55b18ae4ce40 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b18ae4db30 con 0x55b18adc5cb0 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38566/0 (socket says 172.21.15.43:38566) 2024-09-17T05:31:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 -- 172.21.15.43:0/1991119416 learned_addr learned my addr 172.21.15.43:0/1991119416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1400640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18adc5cb0 0x55b18ad94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a2800640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 0x55b18ae68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 -- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18adc5cb0 msgr2=0x55b18ad94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18adc5cb0 0x55b18ad94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 -- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 msgr2=0x55b18ae68430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 0x55b18ae68430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 -- 172.21.15.43:0/1991119416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18ace3a20 con 0x55b18ad96d70 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a2800640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 0x55b18ae68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a1e00640 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f1194004f60 tx=0x7f119401a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c31697c6f4ac04 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f119be00640 1 -- 172.21.15.43:0/1991119416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940a7b10 con 0x55b18ad96d70 2024-09-17T05:31:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f119be00640 1 -- 172.21.15.43:0/1991119416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11940a7cd0 con 0x55b18ad96d70 2024-09-17T05:31:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f119be00640 1 -- 172.21.15.43:0/1991119416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940b06d0 con 0x55b18ad96d70 2024-09-17T05:31:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- 172.21.15.43:0/1991119416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 msgr2=0x55b18ae693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f1194004f60 tx=0x7f119401a850 comp rx=0 tx=0).stop 2024-09-17T05:31:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- 172.21.15.43:0/1991119416 shutdown_connections 2024-09-17T05:31:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ae4ce40 0x55b18ae68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18adc5cb0 0x55b18ad94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 --2- 172.21.15.43:0/1991119416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18ad96d70 0x55b18ae693a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- 172.21.15.43:0/1991119416 >> 172.21.15.43:0/1991119416 conn(0x55b18ad82d80 msgr2=0x55b18ae6b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.116+0000 7f11a8c69080 1 -- 172.21.15.43:0/1991119416 shutdown_connections 2024-09-17T05:31:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 -- 172.21.15.43:0/1991119416 wait complete. 2024-09-17T05:31:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 Processor -- start 2024-09-17T05:31:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 -- start start 2024-09-17T05:31:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ad96d70 0x55b18ae66920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18ae4ce40 0x55b18ae6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b18addf220 con 0x55b18adc5cb0 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b18ae6afd0 con 0x55b18ad96d70 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a8c69080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b18addf4c0 con 0x55b18ae4ce40 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38578/0 (socket says 172.21.15.43:38578) 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 learned_addr learned my addr 172.21.15.43:0/2790468556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ad96d70 0x55b18ae66920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18ae4ce40 0x55b18ae6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18ae4ce40 msgr2=0x55b18ae6b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18ae4ce40 0x55b18ae6b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ad96d70 msgr2=0x55b18ae66920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b18ad96d70 0x55b18ae66920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18acbdc10 con 0x55b18adc5cb0 2024-09-17T05:31:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b18ae4ce40 0x55b18ae6b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1190008750 tx=0x7f1190008c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.120+0000 7f119aa00640 1 -- 172.21.15.43:0/2790468556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119001a070 con 0x55b18adc5cb0 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f119aa00640 1 -- 172.21.15.43:0/2790468556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f119000abc0 con 0x55b18adc5cb0 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f119aa00640 1 -- 172.21.15.43:0/2790468556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190015420 con 0x55b18adc5cb0 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b18add8cd0 con 0x55b18adc5cb0 2024-09-17T05:31:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18add9210 con 0x55b18adc5cb0 2024-09-17T05:31:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f119aa00640 1 -- 172.21.15.43:0/2790468556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f119001d070 con 0x55b18adc5cb0 2024-09-17T05:31:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f119aa00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 0x7f1184074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f119aa00640 1 -- 172.21.15.43:0/2790468556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f1190093280 con 0x55b18adc5cb0 2024-09-17T05:31:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 0x7f1184074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b18adb86a0 0x55b18adb8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0d2f0 con 0x55b18adb86a0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.124+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b18adb86a0 0x55b18adb8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.128+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 0x7f1184074120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f11940b2000 tx=0x7f11940b4040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.128+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b18adb86a0 0x55b18adb8ab0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.128+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v100'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.128+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af0d690 con 0x55b18adb86a0 2024-09-17T05:31:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v100'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0da30 con 0x55b18adb86a0 2024-09-17T05:31:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v100'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0ede0 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.132+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0f180 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0ddd0 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e1c0 con 0x55b18adb86a0 2024-09-17T05:31:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v100'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.136+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af0e5b0 con 0x55b18adb86a0 2024-09-17T05:31:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v100'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e5b0 con 0x55b18adb86a0 2024-09-17T05:31:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v100'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.140+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.144+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.153 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:11.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.148+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.152+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v100'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v100'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.163 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:11.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v100'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.156+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.160+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v100'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v100'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.171 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:11.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v100'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.164+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18adb86a0 2024-09-17T05:31:11.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b18af0e960 0x55b18af11a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af0e560 con 0x55b18af0e960 2024-09-17T05:31:11.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.168+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b18af0e960 0x55b18af11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.172+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b18af0e960 0x55b18af11a40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.172+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'240 uv222 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.172+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af11f80 con 0x55b18af0e960 2024-09-17T05:31:11.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.172+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'241 uv241 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.172+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af12320 con 0x55b18af0e960 2024-09-17T05:31:11.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.180 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:11.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af12b70 con 0x55b18af0e960 2024-09-17T05:31:11.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v100'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af12f10 con 0x55b18af0e960 2024-09-17T05:31:11.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.176+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af132b0 con 0x55b18af0e960 2024-09-17T05:31:11.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v100'245 uv227 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af132b0 con 0x55b18af0e960 2024-09-17T05:31:11.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v100'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af132b0 con 0x55b18af0e960 2024-09-17T05:31:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v100'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.188 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.180+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v100'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v100'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v100'250 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.184+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.188+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v100'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.188+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.188+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v100'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.194 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.188+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v100'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v100'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v100'255 uv238 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.192+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v100'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.200 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:11.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v100'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v100'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v100'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.196+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af127a0 con 0x55b18af0e960 2024-09-17T05:31:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v100'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af0e960 2024-09-17T05:31:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55b18af13c70 0x55b18af14080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af16910 con 0x55b18af13c70 2024-09-17T05:31:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55b18af13c70 0x55b18af14080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55b18af13c70 0x55b18af14080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v100'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.200+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af16cb0 con 0x55b18af13c70 2024-09-17T05:31:11.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v100'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af170d0 con 0x55b18af13c70 2024-09-17T05:31:11.209 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v100'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af18980 con 0x55b18af13c70 2024-09-17T05:31:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v100'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.204+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af18d20 con 0x55b18af13c70 2024-09-17T05:31:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v100'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af17970 con 0x55b18af13c70 2024-09-17T05:31:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v100'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af17d80 con 0x55b18adb86a0 2024-09-17T05:31:11.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'48 uv47 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.208+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af17d80 con 0x55b18adb86a0 2024-09-17T05:31:11.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.212+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v100'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.212+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af17d80 con 0x55b18adb86a0 2024-09-17T05:31:11.218 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.212+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v100'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.212+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af17d80 con 0x55b18adb86a0 2024-09-17T05:31:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v100'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 0x55b18af1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ba20 con 0x55b18af190c0 2024-09-17T05:31:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 0x55b18af1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a2800640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 0x55b18af1b4e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v100'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af1ba20 con 0x55b18af190c0 2024-09-17T05:31:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v100'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.216+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ba20 con 0x55b18af190c0 2024-09-17T05:31:11.224 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:11.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v100'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ca30 con 0x55b18af190c0 2024-09-17T05:31:11.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v100'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ca30 con 0x55b18af190c0 2024-09-17T05:31:11.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v100'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.220+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ba20 con 0x55b18af190c0 2024-09-17T05:31:11.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v100'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55b18af1c0a0 0x55b18af1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ea90 con 0x55b18af1c0a0 2024-09-17T05:31:11.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55b18af1c0a0 0x55b18af1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a1e00640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55b18af1c0a0 0x55b18af1e4c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v100'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.224+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v100'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v100'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.233 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1fe70 con 0x55b18af1c0a0 2024-09-17T05:31:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v100'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.228+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1fe70 con 0x55b18af1c0a0 2024-09-17T05:31:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v100'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v100'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v100'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.232+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v100'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v100'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.241 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v100'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.236+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af1ee60 con 0x55b18af1c0a0 2024-09-17T05:31:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a1e00640 1 -- 172.21.15.43:0/2790468556 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v100'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11940a7b10 con 0x55b18af1c0a0 2024-09-17T05:31:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55b18af1f780 0x55b18af21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22120 con 0x55b18af1f780 2024-09-17T05:31:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55b18af1f780 0x55b18af21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a1400640 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55b18af1f780 0x55b18af21ba0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v100'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af224f0 con 0x55b18af1f780 2024-09-17T05:31:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v100'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.240+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af1f780 2024-09-17T05:31:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v100'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.248 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af1f780 2024-09-17T05:31:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v100'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af1f780 2024-09-17T05:31:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v100'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.244+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18adb86a0 2024-09-17T05:31:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.248+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v100'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.248+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18adb86a0 2024-09-17T05:31:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.248+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v100'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.248+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18adb86a0 2024-09-17T05:31:11.256 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v100'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af23920 con 0x55b18adb86a0 2024-09-17T05:31:11.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v100'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af23920 con 0x55b18adb86a0 2024-09-17T05:31:11.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v100'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.252+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18adb86a0 2024-09-17T05:31:11.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v100'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18adb86a0 2024-09-17T05:31:11.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v100'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v100'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.256+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.264 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.260+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v100'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.260+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.260+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v100'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.260+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v100'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.268 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:11.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v100'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22910 con 0x55b18af13c70 2024-09-17T05:31:11.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v100'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.264+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af25720 con 0x55b18af13c70 2024-09-17T05:31:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v100'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af13c70 2024-09-17T05:31:11.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(99 mds0_sessionmap [create] v100'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v100'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.276 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:11.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(101 mds0_sessionmap [create] v100'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v100'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.272+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(103 609.00000000 [create] v100'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(104 609.00000000 [delete] v100'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.282 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:11.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(105 609.00000000 [create] v100'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.276+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af23dd0 con 0x55b18af190c0 2024-09-17T05:31:11.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v100'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af23dd0 con 0x55b18af190c0 2024-09-17T05:31:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v100'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af190c0 2024-09-17T05:31:11.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v100'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119c002040 con 0x55b18af190c0 2024-09-17T05:31:11.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.280+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.284+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v100'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.284+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.284+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v100'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.284+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.284+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v100'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.291 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.288+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.288+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v100'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.288+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.288+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v100'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.288+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v100'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v100'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v100'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.298 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.292+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v100'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v100'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v100'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.296+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v100'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v100'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.305 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:11.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v100'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.300+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v100'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af22dc0 con 0x55b18af1f780 2024-09-17T05:31:11.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v100'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55b18af23180 con 0x55b18af1f780 2024-09-17T05:31:11.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a1400640 1 -- 172.21.15.43:0/2790468556 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v100'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f119001c030 con 0x55b18af1f780 2024-09-17T05:31:11.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b18af0e960 msgr2=0x55b18af11a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b18af0e960 0x55b18af11a40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55b18af1c0a0 msgr2=0x55b18af1e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55b18af1c0a0 0x55b18af1e4c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55b18af1f780 msgr2=0x55b18af21ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55b18af1f780 0x55b18af21ba0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55b18af13c70 msgr2=0x55b18af14080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55b18af13c70 0x55b18af14080 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 reap_dead start 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b18adb86a0 msgr2=0x55b18adb8ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55b18adb86a0 0x55b18adb8ab0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a2800640 1 -- 172.21.15.43:0/2790468556 reap_dead start 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 msgr2=0x55b18af1b4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 0x55b18af1b4e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 msgr2=0x7f1184074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 0x7f1184074120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f11940b2000 tx=0x7f11940b4040 comp rx=0 tx=0).stop 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 msgr2=0x55b18ae6b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1190008750 tx=0x7f1190008c10 comp rx=0 tx=0).stop 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 shutdown_connections 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55b18af190c0 0x55b18af1b4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1184071c10 0x7f1184074120 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 --2- 172.21.15.43:0/2790468556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b18adc5cb0 0x55b18ae6b390 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 >> 172.21.15.43:0/2790468556 conn(0x55b18ad82d80 msgr2=0x55b18ae6c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 shutdown_connections 2024-09-17T05:31:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.304+0000 7f11a8c69080 1 -- 172.21.15.43:0/2790468556 wait complete. 2024-09-17T05:31:11.316 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-09-17T05:31:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.412+0000 7f3e4741a080 1 Processor -- start 2024-09-17T05:31:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.412+0000 7f3e4741a080 1 -- start start 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6dd74d0 0x560fa6f31f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e8ae40 0x560fa6f2f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fa6d22a00 con 0x560fa6e03cb0 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560fa6d80f40 con 0x560fa6dd74d0 2024-09-17T05:31:11.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560fa6e8bb80 con 0x560fa6e8ae40 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38604/0 (socket says 172.21.15.43:38604) 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 -- 172.21.15.43:0/2136053038 learned_addr learned my addr 172.21.15.43:0/2136053038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e41000640 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e8ae40 0x560fa6f2f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e40600640 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6dd74d0 0x560fa6f31f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 -- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e8ae40 msgr2=0x560fa6f2f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e8ae40 0x560fa6f2f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 -- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6dd74d0 msgr2=0x560fa6f31f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6dd74d0 0x560fa6f31f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 -- 172.21.15.43:0/2136053038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa6cfbc10 con 0x560fa6e03cb0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b5f0 tx=0x7f3e2c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a859276c48f46317 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3e800640 1 -- 172.21.15.43:0/2136053038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00a950 con 0x560fa6e03cb0 2024-09-17T05:31:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3e800640 1 -- 172.21.15.43:0/2136053038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e2c00aaf0 con 0x560fa6e03cb0 2024-09-17T05:31:11.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e3e800640 1 -- 172.21.15.43:0/2136053038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c015470 con 0x560fa6e03cb0 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- 172.21.15.43:0/2136053038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 msgr2=0x560fa6f32450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b5f0 tx=0x7f3e2c00bab0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- 172.21.15.43:0/2136053038 shutdown_connections 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e8ae40 0x560fa6f2f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6e03cb0 0x560fa6f32450 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 --2- 172.21.15.43:0/2136053038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6dd74d0 0x560fa6f31f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- 172.21.15.43:0/2136053038 >> 172.21.15.43:0/2136053038 conn(0x560fa6dc0d80 msgr2=0x560fa6dd2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- 172.21.15.43:0/2136053038 shutdown_connections 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.416+0000 7f3e4741a080 1 -- 172.21.15.43:0/2136053038 wait complete. 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 Processor -- start 2024-09-17T05:31:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- start start 2024-09-17T05:31:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 0x560fa6e199e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e03cb0 0x560fa6f44650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6e8ae40 0x560fa6f44b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560fa6f313f0 con 0x560fa6dd74d0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560fa6dd4ec0 con 0x560fa6e8ae40 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560fa6f32a90 con 0x560fa6e03cb0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 0x560fa6e199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e41000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6e8ae40 0x560fa6f44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 0x560fa6e199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38610/0 (socket says 172.21.15.43:38610) 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 -- 172.21.15.43:0/3060583059 learned_addr learned my addr 172.21.15.43:0/3060583059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e03cb0 0x560fa6f44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 -- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e03cb0 msgr2=0x560fa6f44650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560fa6e03cb0 0x560fa6f44650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 -- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6e8ae40 msgr2=0x560fa6f44b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fa6e8ae40 0x560fa6f44b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 -- 172.21.15.43:0/3060583059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa6df6510 con 0x560fa6dd74d0 2024-09-17T05:31:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e40600640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 0x560fa6e199e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3e30002fc0 tx=0x7f3e3000c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e3d400640 1 -- 172.21.15.43:0/3060583059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000ce40 con 0x560fa6dd74d0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fa6df67b0 con 0x560fa6dd74d0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e3d400640 1 -- 172.21.15.43:0/3060583059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e30001bf0 con 0x560fa6dd74d0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e3d400640 1 -- 172.21.15.43:0/3060583059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30001e40 con 0x560fa6dd74d0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.420+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa6df6cf0 con 0x560fa6dd74d0 2024-09-17T05:31:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e3d400640 1 -- 172.21.15.43:0/3060583059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e3000f040 con 0x560fa6dd74d0 2024-09-17T05:31:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e3d400640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e20071be0 0x7f3e200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e3d400640 1 -- 172.21.15.43:0/3060583059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f3e30092fd0 con 0x560fa6dd74d0 2024-09-17T05:31:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e4741a080 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560fa6e1b3a0 0x560fa6e1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x560fa6e1bcf0 con 0x560fa6e1b3a0 2024-09-17T05:31:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e20071be0 0x7f3e200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e41000640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560fa6e1b3a0 0x560fa6e1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e3fc00640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e20071be0 0x7f3e200740f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b590 tx=0x7f3e2c002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.424+0000 7f3e41000640 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560fa6e1b3a0 0x560fa6e1b7b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e41000640 1 -- 172.21.15.43:0/3060583059 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v100'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3e38002050 con 0x560fa6e1b3a0 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560fa6e1b3a0 msgr2=0x560fa6e1b7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x560fa6e1b3a0 0x560fa6e1b7b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e20071be0 msgr2=0x7f3e200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3e20071be0 0x7f3e200740f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b590 tx=0x7f3e2c002fc0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 msgr2=0x560fa6e199e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 --2- 172.21.15.43:0/3060583059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560fa6dd74d0 0x560fa6e199e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3e30002fc0 tx=0x7f3e3000c610 comp rx=0 tx=0).stop 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e41000640 1 -- 172.21.15.43:0/3060583059 reap_dead start 2024-09-17T05:31:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 shutdown_connections 2024-09-17T05:31:11.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 >> 172.21.15.43:0/3060583059 conn(0x560fa6dc0d80 msgr2=0x560fa6ea9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 shutdown_connections 2024-09-17T05:31:11.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.428+0000 7f3e4741a080 1 -- 172.21.15.43:0/3060583059 wait complete. 2024-09-17T05:31:11.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:11.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.592+0000 7f7865200640 1 -- 172.21.15.43:0/549451948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b6840 msgr2=0x7f78600ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.592+0000 7f7865200640 1 --2- 172.21.15.43:0/549451948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b6840 0x7f78600ccfa0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7848009840 tx=0x7f784801c820 comp rx=0 tx=0).stop 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- 172.21.15.43:0/549451948 shutdown_connections 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 --2- 172.21.15.43:0/549451948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b6840 0x7f78600ccfa0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- 172.21.15.43:0/549451948 >> 172.21.15.43:0/549451948 conn(0x7f78600b2e80 msgr2=0x7f78600b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- 172.21.15.43:0/549451948 shutdown_connections 2024-09-17T05:31:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- 172.21.15.43:0/549451948 wait complete. 2024-09-17T05:31:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 Processor -- start 2024-09-17T05:31:11.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- start start 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78600b6840 0x7f78601553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78600cd4e0 0x7f78601558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7860169ef0 con 0x7f78600b45a0 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7860169d70 con 0x7f78600cd4e0 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.596+0000 7f7865200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f786016a070 con 0x7f78600b6840 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38620/0 (socket says 172.21.15.43:38620) 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 -- 172.21.15.43:0/775586025 learned_addr learned my addr 172.21.15.43:0/775586025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:11.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7855400640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78600b6840 0x7f78601553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785e800640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78600cd4e0 0x7f78601558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 -- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78600b6840 msgr2=0x7f78601553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78600b6840 0x7f78601553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 -- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78600cd4e0 msgr2=0x7f78601558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78600cd4e0 0x7f78601558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 -- 172.21.15.43:0/775586025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860167390 con 0x7f78600b45a0 2024-09-17T05:31:11.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f785de00640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f785000c420 tx=0x7f785000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850017070 con 0x7f78600b45a0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78500053c0 con 0x7f78600b45a0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7860164bd0 con 0x7f78600b45a0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850003860 con 0x7f78600b45a0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7860166bb0 con 0x7f78600b45a0 2024-09-17T05:31:11.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78600ccfa0 con 0x7f78600b45a0 2024-09-17T05:31:11.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.600+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f785001a070 con 0x7f78600b45a0 2024-09-17T05:31:11.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.604+0000 7f7856a00640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 0x7f783c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:11.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.604+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6655+0+0 (secure 0 0 0) 0x7f7850093890 con 0x7f78600b45a0 2024-09-17T05:31:11.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.604+0000 7f7855400640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 0x7f783c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:11.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.604+0000 7f7855400640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 0x7f783c075740 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78480af000 tx=0x7f784801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.608+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f785005dea0 con 0x7f78600b45a0 2024-09-17T05:31:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:11.816+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f78600b65f0 con 0x7f78600b45a0 2024-09-17T05:31:12.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7856a00640 1 -- 172.21.15.43:0/775586025 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v154) ==== 162+0+0 (secure 0 0 0) 0x7f78500628d0 con 0x7f78600b45a0 2024-09-17T05:31:12.040 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 msgr2=0x7f783c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 0x7f783c075740 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f78480af000 tx=0x7f784801b060 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 msgr2=0x7f78601543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f785000c420 tx=0x7f785000c8e0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 shutdown_connections 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f783c073230 0x7f783c075740 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78600cd4e0 0x7f78601558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78600b6840 0x7f78601553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 --2- 172.21.15.43:0/775586025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78600b45a0 0x7f78601543f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.036+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 >> 172.21.15.43:0/775586025 conn(0x7f78600b2e80 msgr2=0x7f786015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.040+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 shutdown_connections 2024-09-17T05:31:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.040+0000 7f7865200640 1 -- 172.21.15.43:0/775586025 wait complete. 2024-09-17T05:31:12.051 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:12.048+0000 7f0f75a00640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-17T05:31:12.052 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:12.048+0000 7f0f75a00640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-17T05:31:12.052 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:12.048+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:12.052 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:12.048+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:12.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:12.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- 172.21.15.43:0/3042990736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 msgr2=0x7f88f80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 --2- 172.21.15.43:0/3042990736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f80ad0b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009c80 tx=0x7f88f001cce0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- 172.21.15.43:0/3042990736 shutdown_connections 2024-09-17T05:31:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 --2- 172.21.15.43:0/3042990736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f80ad0b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 --2- 172.21.15.43:0/3042990736 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f80b3af0 0x7f88f815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- 172.21.15.43:0/3042990736 >> 172.21.15.43:0/3042990736 conn(0x7f88f80b23d0 msgr2=0x7f88f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- 172.21.15.43:0/3042990736 shutdown_connections 2024-09-17T05:31:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- 172.21.15.43:0/3042990736 wait complete. 2024-09-17T05:31:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 Processor -- start 2024-09-17T05:31:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.224+0000 7f88ffe00640 1 -- start start 2024-09-17T05:31:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f8155d70 0x7f88f8098b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f80cd230 0x7f88f80cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8168c80 con 0x7f88f806f110 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88f8168b00 con 0x7f88f80cd230 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f88f8168e00 con 0x7f88f8155d70 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38628/0 (socket says 172.21.15.43:38628) 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 -- 172.21.15.43:0/429760338 learned_addr learned my addr 172.21.15.43:0/429760338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fca00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f8155d70 0x7f88f8098b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fde00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f80cd230 0x7f88f80cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 -- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f8155d70 msgr2=0x7f88f8098b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f8155d70 0x7f88f8098b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 -- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f80cd230 msgr2=0x7f88f80cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f80cd230 0x7f88f80cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 -- 172.21.15.43:0/429760338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f80cdb80 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88fd400640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c3f0 tx=0x7f88ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec017070 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88ec0053c0 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec003860 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f8160360 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f8160b40 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.228+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f80ad0b0 con 0x7f88f806f110 2024-09-17T05:31:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.232+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f88ec01a070 con 0x7f88f806f110 2024-09-17T05:31:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.232+0000 7f88f5600640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 0x7f88d8075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.232+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f88ec0939c0 con 0x7f88f806f110 2024-09-17T05:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.232+0000 7f88fca00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 0x7f88d8075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.232+0000 7f88fca00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 0x7f88d8075770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88e4004630 tx=0x7f88e400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.236+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f88f80cdb80 con 0x7f88f806f110 2024-09-17T05:31:12.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.448+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f80ca420 con 0x7f88f806f110 2024-09-17T05:31:12.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.448+0000 7f88f5600640 1 -- 172.21.15.43:0/429760338 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+4383 (secure 0 0 0) 0x7f88ec05dee0 con 0x7f88f806f110 2024-09-17T05:31:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:12.454 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":156,"btime":"2024-09-17T05:31:12:053307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:12.053227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:12.455 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 156 2024-09-17T05:31:12.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 msgr2=0x7f88d8075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 0x7f88d8075770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88e4004630 tx=0x7f88e400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 msgr2=0x7f88f809cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c3f0 tx=0x7f88ec00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 shutdown_connections 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88d8073260 0x7f88d8075770 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f80cd230 0x7f88f80cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88f8155d70 0x7f88f8098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 --2- 172.21.15.43:0/429760338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88f806f110 0x7f88f809cd00 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 >> 172.21.15.43:0/429760338 conn(0x7f88f80b23d0 msgr2=0x7f88f815db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 shutdown_connections 2024-09-17T05:31:12.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.452+0000 7f88ffe00640 1 -- 172.21.15.43:0/429760338 wait complete. 2024-09-17T05:31:12.476 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:12.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:12.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 -- 172.21.15.43:0/1458541235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c80b3af0 msgr2=0x7f31c815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 --2- 172.21.15.43:0/1458541235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c80b3af0 0x7f31c815c7e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009840 tx=0x7f31b401c790 comp rx=0 tx=0).stop 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 -- 172.21.15.43:0/1458541235 shutdown_connections 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 --2- 172.21.15.43:0/1458541235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c80b3af0 0x7f31c815c7e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 -- 172.21.15.43:0/1458541235 >> 172.21.15.43:0/1458541235 conn(0x7f31c80b23d0 msgr2=0x7f31c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 -- 172.21.15.43:0/1458541235 shutdown_connections 2024-09-17T05:31:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.628+0000 7f31cfe00640 1 -- 172.21.15.43:0/1458541235 wait complete. 2024-09-17T05:31:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 Processor -- start 2024-09-17T05:31:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 -- start start 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31c8155d70 0x7f31c809be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c81fbfb0 0x7f31c81fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8169d70 con 0x7f31c806f110 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8169bf0 con 0x7f31c81fbfb0 2024-09-17T05:31:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cfe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f31c8169ef0 con 0x7f31c8155d70 2024-09-17T05:31:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38630/0 (socket says 172.21.15.43:38630) 2024-09-17T05:31:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 -- 172.21.15.43:0/110190302 learned_addr learned my addr 172.21.15.43:0/110190302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cde00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c81fbfb0 0x7f31c81fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cca00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31c8155d70 0x7f31c809be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 -- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31c8155d70 msgr2=0x7f31c809be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31c8155d70 0x7f31c809be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 -- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c81fbfb0 msgr2=0x7f31c81fc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c81fbfb0 0x7f31c81fc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 -- 172.21.15.43:0/110190302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c809cad0 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.632+0000 7f31cd400640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00c460 tx=0x7f31bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0052d0 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c809c860 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31bc005470 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc012870 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c80b1480 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c815c7e0 con 0x7f31c806f110 2024-09-17T05:31:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31bc003600 con 0x7f31c806f110 2024-09-17T05:31:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 0x7f31a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.636+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f31bc093ff0 con 0x7f31c806f110 2024-09-17T05:31:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.640+0000 7f31cca00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 0x7f31a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.640+0000 7f31cca00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 0x7f31a8075740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31b40af000 tx=0x7f31b401afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.640+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31bc05e5d0 con 0x7f31c806f110 2024-09-17T05:31:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.852+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c80ca4f0 con 0x7f31c806f110 2024-09-17T05:31:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.852+0000 7f31c5600640 1 -- 172.21.15.43:0/110190302 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+4383 (secure 0 0 0) 0x7f31bc063000 con 0x7f31c806f110 2024-09-17T05:31:12.859 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:12.859 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":156,"btime":"2024-09-17T05:31:12:053307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:12.053227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 156 2024-09-17T05:31:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 msgr2=0x7f31a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 0x7f31a8075740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31b40af000 tx=0x7f31b401afd0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 msgr2=0x7f31c8072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00c460 tx=0x7f31bc00c920 comp rx=0 tx=0).stop 2024-09-17T05:31:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 shutdown_connections 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f31a8073230 0x7f31a8075740 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31c81fbfb0 0x7f31c81fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f31c8155d70 0x7f31c809be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 --2- 172.21.15.43:0/110190302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31c806f110 0x7f31c8072f00 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 >> 172.21.15.43:0/110190302 conn(0x7f31c80b23d0 msgr2=0x7f31c806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 shutdown_connections 2024-09-17T05:31:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:12.856+0000 7f31cfe00640 1 -- 172.21.15.43:0/110190302 wait complete. 2024-09-17T05:31:12.881 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:12.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:13.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 -- 172.21.15.43:0/2017975593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 msgr2=0x7f2f300cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 --2- 172.21.15.43:0/2017975593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f300cdc70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2f300b6d30 tx=0x7f2f2001c890 comp rx=0 tx=0).stop 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 -- 172.21.15.43:0/2017975593 shutdown_connections 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 --2- 172.21.15.43:0/2017975593 >> v2:172.21.15.62:3300/0 conn(0x7f2f300b4620 0x7f2f300b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 --2- 172.21.15.43:0/2017975593 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f300b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 --2- 172.21.15.43:0/2017975593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f300cdc70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 -- 172.21.15.43:0/2017975593 >> 172.21.15.43:0/2017975593 conn(0x7f2f300b2de0 msgr2=0x7f2f300b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 -- 172.21.15.43:0/2017975593 shutdown_connections 2024-09-17T05:31:13.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.032+0000 7f2f36e00640 1 -- 172.21.15.43:0/2017975593 wait complete. 2024-09-17T05:31:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.036+0000 7f2f36e00640 1 Processor -- start 2024-09-17T05:31:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.036+0000 7f2f36e00640 1 -- start start 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f300b4620 0x7f2f300c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f30142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f300cc550 con 0x7f2f300b63b0 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f300cc3d0 con 0x7f2f300b4620 2024-09-17T05:31:13.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2f300cc6d0 con 0x7f2f300ce1b0 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38650/0 (socket says 172.21.15.43:38650) 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 -- 172.21.15.43:0/3332902635 learned_addr learned my addr 172.21.15.43:0/3332902635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f2fe00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f300b4620 0x7f2f300c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f34e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f30142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 -- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 msgr2=0x7f2f30142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f30142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 -- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f300b4620 msgr2=0x7f2f300c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f300b4620 0x7f2f300c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 -- 172.21.15.43:0/3332902635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f300c6bd0 con 0x7f2f300b63b0 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f34e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f30142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f27e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d6a0 tx=0x7f2f1800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18017070 con 0x7f2f300b63b0 2024-09-17T05:31:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f18005310 con 0x7f2f300b63b0 2024-09-17T05:31:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18009820 con 0x7f2f300b63b0 2024-09-17T05:31:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f30143220 con 0x7f2f300b63b0 2024-09-17T05:31:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f300c58d0 con 0x7f2f300b63b0 2024-09-17T05:31:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.040+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f300cdc70 con 0x7f2f300b63b0 2024-09-17T05:31:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.044+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2f180099c0 con 0x7f2f300b63b0 2024-09-17T05:31:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.044+0000 7f2f2cc00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 0x7f2f0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.044+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f2f180947e0 con 0x7f2f300b63b0 2024-09-17T05:31:13.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.044+0000 7f2f2fe00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 0x7f2f0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.044+0000 7f2f2fe00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 0x7f2f0c075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2f200af000 tx=0x7f2f200a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:13.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.048+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2f1805ed00 con 0x7f2f300b63b0 2024-09-17T05:31:13.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f300b34b0 con 0x7f2f300b63b0 2024-09-17T05:31:13.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f2cc00640 1 -- 172.21.15.43:0/3332902635 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) ==== 95+0+4383 (secure 0 0 0) 0x7f2f18063730 con 0x7f2f300b63b0 2024-09-17T05:31:13.261 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:13.261 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":156,"btime":"2024-09-17T05:31:12:053307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7058,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:12.053227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:13.261 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 156 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 msgr2=0x7f2f0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 0x7f2f0c075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2f200af000 tx=0x7f2f200a7040 comp rx=0 tx=0).stop 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 msgr2=0x7f2f30140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d6a0 tx=0x7f2f1800db60 comp rx=0 tx=0).stop 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 shutdown_connections 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f0c073200 0x7f2f0c075710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f300ce1b0 0x7f2f30142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300b63b0 0x7f2f30140390 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 --2- 172.21.15.43:0/3332902635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f300b4620 0x7f2f300c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 >> 172.21.15.43:0/3332902635 conn(0x7f2f300b2de0 msgr2=0x7f2f300718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 shutdown_connections 2024-09-17T05:31:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.256+0000 7f2f36e00640 1 -- 172.21.15.43:0/3332902635 wait complete. 2024-09-17T05:31:13.284 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-09-17T05:31:13.284 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-09-17T05:31:13.284 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:13.284 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:13.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:13.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:13.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:13.293 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:13.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:13.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 -- 172.21.15.43:0/1657114032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c06d570 msgr2=0x7f669c06f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1657114032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c06d570 0x7f669c06f510 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6688009900 tx=0x7f668801c850 comp rx=0 tx=0).stop 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 -- 172.21.15.43:0/1657114032 shutdown_connections 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1657114032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c06fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1657114032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c06d570 0x7f669c06f510 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 -- 172.21.15.43:0/1657114032 >> 172.21.15.43:0/1657114032 conn(0x7f669c0b23d0 msgr2=0x7f669c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 -- 172.21.15.43:0/1657114032 shutdown_connections 2024-09-17T05:31:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.544+0000 7f66a3e00640 1 -- 172.21.15.43:0/1657114032 wait complete. 2024-09-17T05:31:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 Processor -- start 2024-09-17T05:31:13.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- start start 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c1f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f669c1f8d50 0x7f669c0cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f669c16c030 con 0x7f669c0d0c20 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f669c16beb0 con 0x7f669c1f8d50 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f669c16c1b0 con 0x7f669c06fa50 2024-09-17T05:31:13.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38668/0 (socket says 172.21.15.43:38668) 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 -- 172.21.15.43:0/1137390392 learned_addr learned my addr 172.21.15.43:0/1137390392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a1400640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c1f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a1e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f669c1f8d50 0x7f669c0cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 -- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 msgr2=0x7f669c1f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c1f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 -- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f669c1f8d50 msgr2=0x7f669c0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f669c1f8d50 0x7f669c0cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 -- 172.21.15.43:0/1137390392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c099150 con 0x7f669c0d0c20 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a1400640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c1f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a0a00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6688009850 tx=0x7f66880a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66880b2040 con 0x7f669c0d0c20 2024-09-17T05:31:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66880043b0 con 0x7f669c0d0c20 2024-09-17T05:31:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c099430 con 0x7f669c0d0c20 2024-09-17T05:31:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688005270 con 0x7f669c0d0c20 2024-09-17T05:31:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.548+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c0999f0 con 0x7f669c0d0c20 2024-09-17T05:31:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669c098ec0 con 0x7f669c0d0c20 2024-09-17T05:31:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66880bd060 con 0x7f669c0d0c20 2024-09-17T05:31:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f6699600640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 0x7f667c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f668812d9a0 con 0x7f669c0d0c20 2024-09-17T05:31:13.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f66a1400640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 0x7f667c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:13.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.552+0000 7f66a1400640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 0x7f667c075710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f66900016b0 tx=0x7f6690009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:13.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.556+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f66880164d0 con 0x7f669c0d0c20 2024-09-17T05:31:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:13.764+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f669c06f510 con 0x7f669c0d0c20 2024-09-17T05:31:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.044+0000 7f6699600640 1 -- 172.21.15.43:0/1137390392 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v157) ==== 161+0+0 (secure 0 0 0) 0x7f66880f8630 con 0x7f669c0d0c20 2024-09-17T05:31:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 msgr2=0x7f667c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 0x7f667c075710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f66900016b0 tx=0x7f6690009040 comp rx=0 tx=0).stop 2024-09-17T05:31:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 msgr2=0x7f669c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6688009850 tx=0x7f66880a5620 comp rx=0 tx=0).stop 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 shutdown_connections 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f667c073200 0x7f667c075710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f669c1f8d50 0x7f669c0cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0c20 0x7f669c0cb680 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 --2- 172.21.15.43:0/1137390392 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f669c06fa50 0x7f669c1f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 >> 172.21.15.43:0/1137390392 conn(0x7f669c0b23d0 msgr2=0x7f669c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 shutdown_connections 2024-09-17T05:31:14.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.048+0000 7f66a3e00640 1 -- 172.21.15.43:0/1137390392 wait complete. 2024-09-17T05:31:14.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:14.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- 172.21.15.43:0/1486210729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e00003af0 msgr2=0x7f5e00003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 --2- 172.21.15.43:0/1486210729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e00003af0 0x7f5e00003f60 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00c260 tx=0x7f5e0c01cf30 comp rx=0 tx=0).stop 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e1b200640 1 -- 172.21.15.43:0/1486210729 reap_dead start 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- 172.21.15.43:0/1486210729 shutdown_connections 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- 172.21.15.43:0/1486210729 >> 172.21.15.43:0/1486210729 conn(0x7f5e1c0b23d0 msgr2=0x7f5e1c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- 172.21.15.43:0/1486210729 shutdown_connections 2024-09-17T05:31:14.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- 172.21.15.43:0/1486210729 wait complete. 2024-09-17T05:31:14.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 Processor -- start 2024-09-17T05:31:14.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.232+0000 7f5e21400640 1 -- start start 2024-09-17T05:31:14.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 0x7f5e1c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e1c1f5750 0x7f5e1c1fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c15cb50 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c15c9d0 con 0x7f5e00003af0 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c1663b0 con 0x7f5e1c1f5750 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33660/0 (socket says 172.21.15.43:33660) 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 -- 172.21.15.43:0/1007050113 learned_addr learned my addr 172.21.15.43:0/1007050113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1a800640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 0x7f5e1c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e11e00640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e1c1f5750 0x7f5e1c1fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 -- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e1c1f5750 msgr2=0x7f5e1c1fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e1c1f5750 0x7f5e1c1fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 -- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 msgr2=0x7f5e1c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 0x7f5e1c0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 -- 172.21.15.43:0/1007050113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c073fa0 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1a800640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 0x7f5e1c0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1b200640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0c7280 tx=0x7f5e14010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ee70 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e14015070 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14003d10 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c074280 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c074840 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c0ad0b0 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e14004030 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 0x7f5df8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f5e140950f0 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1a800640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 0x7f5df8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:14.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.240+0000 7f5e1a800640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 0x7f5df8075710 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ac000 tx=0x7f5e0c019d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:14.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.244+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e1405fc60 con 0x7f5e1c1f5b60 2024-09-17T05:31:14.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:14.480+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5e1c0c6af0 con 0x7f5e1c1f5b60 2024-09-17T05:31:15.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.056+0000 7f5e13400640 1 -- 172.21.15.43:0/1007050113 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v158) ==== 101+0+0 (secure 0 0 0) 0x7f5e1405f560 con 0x7f5e1c1f5b60 2024-09-17T05:31:15.057 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 msgr2=0x7f5df8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 0x7f5df8075710 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ac000 tx=0x7f5e0c019d70 comp rx=0 tx=0).stop 2024-09-17T05:31:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 msgr2=0x7f5e1c1fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0c7280 tx=0x7f5e14010e80 comp rx=0 tx=0).stop 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 shutdown_connections 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5df8073200 0x7f5df8075710 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e1c1f5b60 0x7f5e1c1fbe70 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e1c1f5750 0x7f5e1c1fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 --2- 172.21.15.43:0/1007050113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e00003af0 0x7f5e1c0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 >> 172.21.15.43:0/1007050113 conn(0x7f5e1c0b23d0 msgr2=0x7f5e1c15b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 shutdown_connections 2024-09-17T05:31:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.060+0000 7f5e21400640 1 -- 172.21.15.43:0/1007050113 wait complete. 2024-09-17T05:31:15.084 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.144+0000 7fb0c9428080 1 Processor -- start 2024-09-17T05:31:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.144+0000 7fb0c9428080 1 -- start start 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af3692c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af36943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564af359ba00 con 0x564af367ccb0 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564af35f9f40 con 0x564af36504d0 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564af3704b80 con 0x564af3703e40 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33664/0 (socket says 172.21.15.43:33664) 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 -- 172.21.15.43:0/117751050 learned_addr learned my addr 172.21.15.43:0/117751050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c2600640 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af3692c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c3000640 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af36943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 -- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 msgr2=0x564af36943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af36943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 -- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 msgr2=0x564af3692c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af3692c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 -- 172.21.15.43:0/117751050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af3574c10 con 0x564af367ccb0 2024-09-17T05:31:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b7a0 tx=0x7fb0b0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af265566becae3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0bbe00640 1 -- 172.21.15.43:0/117751050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000be40 con 0x564af367ccb0 2024-09-17T05:31:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0bbe00640 1 -- 172.21.15.43:0/117751050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b000abd0 con 0x564af367ccb0 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0bbe00640 1 -- 172.21.15.43:0/117751050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0008430 con 0x564af367ccb0 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- 172.21.15.43:0/117751050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 msgr2=0x564af3693e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b7a0 tx=0x7fb0b0004f90 comp rx=0 tx=0).stop 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- 172.21.15.43:0/117751050 shutdown_connections 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af36943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af3693e90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 --2- 172.21.15.43:0/117751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af3692c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- 172.21.15.43:0/117751050 >> 172.21.15.43:0/117751050 conn(0x564af3639d80 msgr2=0x564af364b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- 172.21.15.43:0/117751050 shutdown_connections 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.148+0000 7fb0c9428080 1 -- 172.21.15.43:0/117751050 wait complete. 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 Processor -- start 2024-09-17T05:31:15.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- start start 2024-09-17T05:31:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af37be150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af37ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564af3697150 con 0x564af367ccb0 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564af364dfa0 con 0x564af36504d0 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564af3720600 con 0x564af3703e40 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33670/0 (socket says 172.21.15.43:33670) 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 learned_addr learned my addr 172.21.15.43:0/1208695773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c3000640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af37ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af37be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 msgr2=0x564af37ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564af3703e40 0x564af37ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 msgr2=0x564af37be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af37be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af3720070 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564af36504d0 0x564af37be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000a7b0 tx=0x7fb0b00089d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0baa00640 1 -- 172.21.15.43:0/1208695773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0008e00 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0baa00640 1 -- 172.21.15.43:0/1208695773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b0015420 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0baa00640 1 -- 172.21.15.43:0/1208695773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0015670 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564af3723700 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.152+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af37bda10 con 0x564af367ccb0 2024-09-17T05:31:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0baa00640 1 -- 172.21.15.43:0/1208695773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0b0015810 con 0x564af367ccb0 2024-09-17T05:31:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0baa00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 0x7fb0a40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0baa00640 1 -- 172.21.15.43:0/1208695773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fb0b0094990 con 0x564af367ccb0 2024-09-17T05:31:15.158 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564af37b0da0 0x564af37b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 0x7fb0a40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37b16f0 con 0x564af37b0da0 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c3000640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564af37b0da0 0x564af37b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 0x7fb0a40740f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40105c0 tx=0x7fb0b4002930 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.156+0000 7fb0c3000640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564af37b0da0 0x564af37b11b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.160+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v101'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.160+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37c6840 con 0x564af37b0da0 2024-09-17T05:31:15.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.160+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v101'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.160+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c6be0 con 0x564af37b0da0 2024-09-17T05:31:15.165 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:15.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.164+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v101'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.164+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7f90 con 0x564af37b0da0 2024-09-17T05:31:15.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.164+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.164+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c8330 con 0x564af37b0da0 2024-09-17T05:31:15.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c6f80 con 0x564af37b0da0 2024-09-17T05:31:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7370 con 0x564af37b0da0 2024-09-17T05:31:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v101'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.168+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37c7760 con 0x564af37b0da0 2024-09-17T05:31:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.172+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v101'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.172+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7760 con 0x564af37b0da0 2024-09-17T05:31:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:15.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.172+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v101'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.172+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.176+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.180+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.180+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.180+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.180+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.183 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.184+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v101'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.188+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v101'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.193 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v101'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.192+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.196+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.196+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.196+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.196+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v101'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v101'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.202 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:15.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v101'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.200+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37b0da0 2024-09-17T05:31:15.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564af37c7b00 0x564af37cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.204+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37c7710 con 0x564af37c7b00 2024-09-17T05:31:15.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564af37c7b00 0x564af37cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564af37c7b00 0x564af37cabf0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'261 uv244 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37cb130 con 0x564af37c7b00 2024-09-17T05:31:15.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'262 uv262 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.208+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb4d0 con 0x564af37c7b00 2024-09-17T05:31:15.213 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:15.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.212+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.212+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cbd70 con 0x564af37c7b00 2024-09-17T05:31:15.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.212+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v101'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.212+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cc110 con 0x564af37c7b00 2024-09-17T05:31:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cc4b0 con 0x564af37c7b00 2024-09-17T05:31:15.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v101'266 uv249 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37cc850 con 0x564af37c7b00 2024-09-17T05:31:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v101'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.216+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37ccbf0 con 0x564af37c7b00 2024-09-17T05:31:15.220 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.220+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v101'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.220+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.220+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v101'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.220+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v101'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v101'271 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v101'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.224+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.228 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.228+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v101'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.228+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.228+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v101'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.228+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v101'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v101'276 uv260 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v101'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.232+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:15.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v101'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v101'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v101'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.236+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cb9a0 con 0x564af37c7b00 2024-09-17T05:31:15.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v101'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37c7b00 2024-09-17T05:31:15.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564af37cd5b0 0x564af37cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37cff10 con 0x564af37cd5b0 2024-09-17T05:31:15.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564af37cd5b0 0x564af37cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564af37cd5b0 0x564af37cf9d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v101'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.240+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d02b0 con 0x564af37cd5b0 2024-09-17T05:31:15.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v101'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0720 con 0x564af37cd5b0 2024-09-17T05:31:15.245 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v101'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d1fd0 con 0x564af37cd5b0 2024-09-17T05:31:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v101'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.244+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d2370 con 0x564af37cd5b0 2024-09-17T05:31:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v101'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0fc0 con 0x564af37cd5b0 2024-09-17T05:31:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v101'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0fc0 con 0x564af37b0da0 2024-09-17T05:31:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'52 uv51 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.248+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d0fc0 con 0x564af37b0da0 2024-09-17T05:31:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.252+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v101'53 uv53 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.252+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0fc0 con 0x564af37b0da0 2024-09-17T05:31:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.252+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v101'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.254 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:15.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.252+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0fc0 con 0x564af37b0da0 2024-09-17T05:31:15.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v101'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 0x564af37d4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d4c60 con 0x564af37d18c0 2024-09-17T05:31:15.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c3000640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 0x564af37d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c3000640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 0x564af37d4720 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v101'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d4c60 con 0x564af37d18c0 2024-09-17T05:31:15.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v101'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.256+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d4c60 con 0x564af37d18c0 2024-09-17T05:31:15.262 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:15.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v101'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d5c70 con 0x564af37d18c0 2024-09-17T05:31:15.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v101'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d5c70 con 0x564af37d18c0 2024-09-17T05:31:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v101'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.260+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d4c60 con 0x564af37d18c0 2024-09-17T05:31:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v101'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564af37d52e0 0x564af37d7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d7cd0 con 0x564af37d52e0 2024-09-17T05:31:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564af37d52e0 0x564af37d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c2600640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564af37d52e0 0x564af37d7700 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v101'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.264+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d7d90 con 0x564af37d52e0 2024-09-17T05:31:15.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v101'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.269 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:15.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v101'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d9190 con 0x564af37d52e0 2024-09-17T05:31:15.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v101'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.268+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d9190 con 0x564af37d52e0 2024-09-17T05:31:15.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v101'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v101'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v101'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.272+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v101'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v101'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.277 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:15.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v101'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.276+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d8180 con 0x564af37d52e0 2024-09-17T05:31:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c2600640 1 -- 172.21.15.43:0/1208695773 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v101'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b400e8d0 con 0x564af37d52e0 2024-09-17T05:31:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564af37d8800 0x564af37dac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db2e0 con 0x564af37d8800 2024-09-17T05:31:15.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564af37d8800 0x564af37dac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c1c00640 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564af37d8800 0x564af37dac20 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v101'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37db2e0 con 0x564af37d8800 2024-09-17T05:31:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v101'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.280+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37d8800 2024-09-17T05:31:15.284 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.284+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v101'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.284+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37d8800 2024-09-17T05:31:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.284+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v101'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.284+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37d8800 2024-09-17T05:31:15.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v101'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37b0da0 2024-09-17T05:31:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v101'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37b0da0 2024-09-17T05:31:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v101'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.288+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37b0da0 2024-09-17T05:31:15.291 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v101'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dc740 con 0x564af37b0da0 2024-09-17T05:31:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v101'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dc740 con 0x564af37b0da0 2024-09-17T05:31:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v101'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.292+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37b0da0 2024-09-17T05:31:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v101'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37b0da0 2024-09-17T05:31:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37cd5b0 2024-09-17T05:31:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v101'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37cd5b0 2024-09-17T05:31:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v101'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.296+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37db730 con 0x564af37cd5b0 2024-09-17T05:31:15.300 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v101'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v101'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v101'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.300+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v101'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v101'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v101'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.307 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.304+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37d0bb0 con 0x564af37cd5b0 2024-09-17T05:31:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.308+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v101'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.308+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37de540 con 0x564af37cd5b0 2024-09-17T05:31:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.308+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v101'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37cd5b0 2024-09-17T05:31:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.308+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.312+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v101'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.312 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:15.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.312+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.312+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.312+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.316+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 609.00000000 [create] v101'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.316+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.316+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 609.00000000 [delete] v101'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.316+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.320+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v101'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.323 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.324+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dcbf0 con 0x564af37d18c0 2024-09-17T05:31:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.324+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.324+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dcbf0 con 0x564af37d18c0 2024-09-17T05:31:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.324+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.324+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d18c0 2024-09-17T05:31:15.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0bc002050 con 0x564af37d18c0 2024-09-17T05:31:15.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(109 400.00000000 [create] v101'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(110 400.00000000 [delete] v101'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.328+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(111 400.00000000 [create] v101'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.332 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.332+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(114 100.00000000.inode [create] v101'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(115 100.00000000.inode [delete] v101'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(116 100.00000000.inode [create] v101'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.339 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:15.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.336+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(119 1.00000000 [create] v101'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.340+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(120 1.00000000 [delete] v101'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(121 1.00000000 [create] v101'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.345 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dcbf0 con 0x564af37d8800 2024-09-17T05:31:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.344+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dcbf0 con 0x564af37d8800 2024-09-17T05:31:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbbe0 con 0x564af37d8800 2024-09-17T05:31:15.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x564af37dbfa0 con 0x564af37d8800 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c1c00640 1 -- 172.21.15.43:0/1208695773 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb0b005b190 con 0x564af37d8800 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564af37c7b00 msgr2=0x564af37cabf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564af37c7b00 0x564af37cabf0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564af37d52e0 msgr2=0x564af37d7700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564af37d52e0 0x564af37d7700 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564af37d8800 msgr2=0x564af37dac20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564af37d8800 0x564af37dac20 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564af37cd5b0 msgr2=0x564af37cf9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564af37cd5b0 0x564af37cf9d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 reap_dead start 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564af37b0da0 msgr2=0x564af37b11b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564af37b0da0 0x564af37b11b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c3000640 1 -- 172.21.15.43:0/1208695773 reap_dead start 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 msgr2=0x564af37d4720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 0x564af37d4720 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 msgr2=0x7fb0a40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 0x7fb0a40740f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40105c0 tx=0x7fb0b4002930 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 msgr2=0x564af371fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000a7b0 tx=0x7fb0b00089d0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 shutdown_connections 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564af37d18c0 0x564af37d4720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0a4071be0 0x7fb0a40740f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 --2- 172.21.15.43:0/1208695773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564af367ccb0 0x564af371fb30 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 >> 172.21.15.43:0/1208695773 conn(0x564af3639d80 msgr2=0x564af3706470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 shutdown_connections 2024-09-17T05:31:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.348+0000 7fb0c9428080 1 -- 172.21.15.43:0/1208695773 wait complete. 2024-09-17T05:31:15.356 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-09-17T05:31:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 Processor -- start 2024-09-17T05:31:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 -- start start 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c7ad010 0x55796c7ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c73ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55796c7351a0 con 0x55796c724cb0 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55796c643a00 con 0x55796c7ad010 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55796c735320 con 0x55796c7ae8b0 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33688/0 (socket says 172.21.15.43:33688) 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 -- 172.21.15.43:0/808072713 learned_addr learned my addr 172.21.15.43:0/808072713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:15.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a7fe00640 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c7ad010 0x55796c7ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a85400640 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c73ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 -- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 msgr2=0x55796c73ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c73ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 -- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c7ad010 msgr2=0x55796c7ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.456+0000 7f3a84a00640 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c7ad010 0x55796c7ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a84a00640 1 -- 172.21.15.43:0/808072713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796c642a20 con 0x55796c724cb0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a84a00640 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b5f0 tx=0x7f3a7800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b55695d9bb7e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a7ea00640 1 -- 172.21.15.43:0/808072713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800a8f0 con 0x55796c724cb0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a7ea00640 1 -- 172.21.15.43:0/808072713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a7800aa90 con 0x55796c724cb0 2024-09-17T05:31:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a7ea00640 1 -- 172.21.15.43:0/808072713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78015470 con 0x55796c724cb0 2024-09-17T05:31:15.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/808072713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 msgr2=0x55796c7ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b5f0 tx=0x7f3a7800bab0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/808072713 shutdown_connections 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c7ad010 0x55796c7ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c73ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/808072713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c724cb0 0x55796c7ccb10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/808072713 >> 172.21.15.43:0/808072713 conn(0x55796c6e1d80 msgr2=0x55796c6f8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/808072713 shutdown_connections 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/808072713 wait complete. 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 Processor -- start 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- start start 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c724cb0 0x55796c7c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 0x55796c7aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c7c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55796c7c9100 con 0x55796c7ad010 2024-09-17T05:31:15.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55796c7ab680 con 0x55796c724cb0 2024-09-17T05:31:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.460+0000 7f3a8b8e2080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55796c7ab920 con 0x55796c7ae8b0 2024-09-17T05:31:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 0x55796c7aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 0x55796c7aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33690/0 (socket says 172.21.15.43:33690) 2024-09-17T05:31:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 -- 172.21.15.43:0/2377965328 learned_addr learned my addr 172.21.15.43:0/2377965328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a84a00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c724cb0 0x55796c7c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a85400640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c7c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 -- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 msgr2=0x55796c7c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55796c7ae8b0 0x55796c7c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 -- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c724cb0 msgr2=0x55796c7c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55796c724cb0 0x55796c7c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 -- 172.21.15.43:0/2377965328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796c61cc10 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7fe00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 0x55796c7aad70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000a700 tx=0x7f3a7000abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7d600640 1 -- 172.21.15.43:0/2377965328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7001c070 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7d600640 1 -- 172.21.15.43:0/2377965328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a70008d50 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55796c7abbc0 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7d600640 1 -- 172.21.15.43:0/2377965328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700176e0 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796c7ac100 con 0x55796c7ad010 2024-09-17T05:31:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.464+0000 7f3a7d600640 1 -- 172.21.15.43:0/2377965328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a7001f070 con 0x55796c7ad010 2024-09-17T05:31:15.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a7d600640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a64071be0 0x7f3a640740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a7d600640 1 -- 172.21.15.43:0/2377965328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f3a700937c0 con 0x55796c7ad010 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a84a00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a64071be0 0x7f3a640740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55796c7b06a0 0x55796c7b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x55796c7ac990 con 0x55796c7b06a0 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a85400640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55796c7b06a0 0x55796c7b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a84a00640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a64071be0 0x7f3a640740f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b5f0 tx=0x7f3a78002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a85400640 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55796c7b06a0 0x55796c7b0ab0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a85400640 1 -- 172.21.15.43:0/2377965328 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [delete] v101'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3a80002050 con 0x55796c7b06a0 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55796c7b06a0 msgr2=0x55796c7b0ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55796c7b06a0 0x55796c7b0ab0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a64071be0 msgr2=0x7f3a640740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a64071be0 0x7f3a640740f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b5f0 tx=0x7f3a78002fc0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 msgr2=0x55796c7aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 --2- 172.21.15.43:0/2377965328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55796c7ad010 0x55796c7aad70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000a700 tx=0x7f3a7000abc0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a85400640 1 -- 172.21.15.43:0/2377965328 reap_dead start 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 shutdown_connections 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 >> 172.21.15.43:0/2377965328 conn(0x55796c6e1d80 msgr2=0x55796c6f8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 shutdown_connections 2024-09-17T05:31:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.468+0000 7f3a8b8e2080 1 -- 172.21.15.43:0/2377965328 wait complete. 2024-09-17T05:31:15.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 -- 172.21.15.43:0/2120176298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67940d0390 msgr2=0x7f67940703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 --2- 172.21.15.43:0/2120176298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67940d0390 0x7f67940703f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f678c009bc0 tx=0x7f678c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 -- 172.21.15.43:0/2120176298 shutdown_connections 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 --2- 172.21.15.43:0/2120176298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67940d0390 0x7f67940703f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 -- 172.21.15.43:0/2120176298 >> 172.21.15.43:0/2120176298 conn(0x7f67940b2810 msgr2=0x7f67940b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 -- 172.21.15.43:0/2120176298 shutdown_connections 2024-09-17T05:31:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.636+0000 7f679b200640 1 -- 172.21.15.43:0/2120176298 wait complete. 2024-09-17T05:31:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 Processor -- start 2024-09-17T05:31:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 -- start start 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67940d0390 0x7f6794155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f679414ea10 0x7f679415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6794169e30 con 0x7f679415f470 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6794169cb0 con 0x7f67940d0390 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f679b200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6794169fb0 con 0x7f679414ea10 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33706/0 (socket says 172.21.15.43:33706) 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 -- 172.21.15.43:0/391031585 learned_addr learned my addr 172.21.15.43:0/391031585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f6799200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f679414ea10 0x7f679415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f6793e00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67940d0390 0x7f6794155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 -- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f679414ea10 msgr2=0x7f679415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f679414ea10 0x7f679415a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 -- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67940d0390 msgr2=0x7f6794155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67940d0390 0x7f6794155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 -- 172.21.15.43:0/391031585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941fbbe0 con 0x7f679415f470 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.640+0000 7f678be00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c460 tx=0x7f677c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0052d0 con 0x7f679415f470 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f677c005490 con 0x7f679415f470 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679415eb20 con 0x7f679415f470 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c012930 con 0x7f679415f470 2024-09-17T05:31:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6794166600 con 0x7f679415f470 2024-09-17T05:31:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679415e890 con 0x7f679415f470 2024-09-17T05:31:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f677c012ad0 con 0x7f679415f470 2024-09-17T05:31:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 0x7f6770075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:15.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f677c093fc0 con 0x7f679415f470 2024-09-17T05:31:15.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.644+0000 7f6793e00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 0x7f6770075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:15.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.648+0000 7f6793e00640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 0x7f6770075710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6784001180 tx=0x7f6784001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.648+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f677c05e570 con 0x7f679415f470 2024-09-17T05:31:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:15.860+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f679415f040 con 0x7f679415f470 2024-09-17T05:31:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.060+0000 7f6790c00640 1 -- 172.21.15.43:0/391031585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v159) ==== 162+0+0 (secure 0 0 0) 0x7f677c062fa0 con 0x7f679415f470 2024-09-17T05:31:16.063 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:16.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 msgr2=0x7f6770075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 0x7f6770075710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6784001180 tx=0x7f6784001110 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 msgr2=0x7f679414e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c460 tx=0x7f677c00c920 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 shutdown_connections 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6770073200 0x7f6770075710 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f679414ea10 0x7f679415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f679415f470 0x7f679414e1c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 --2- 172.21.15.43:0/391031585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67940d0390 0x7f6794155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 >> 172.21.15.43:0/391031585 conn(0x7f67940b2810 msgr2=0x7f6794075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 shutdown_connections 2024-09-17T05:31:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.064+0000 7f679b200640 1 -- 172.21.15.43:0/391031585 wait complete. 2024-09-17T05:31:16.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:16.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 -- 172.21.15.43:0/2423729552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600cd4e0 msgr2=0x7fd1600cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 --2- 172.21.15.43:0/2423729552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600cd4e0 0x7fd1600cd8f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd158009c80 tx=0x7fd15801cda0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 -- 172.21.15.43:0/2423729552 shutdown_connections 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 --2- 172.21.15.43:0/2423729552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600cd4e0 0x7fd1600cd8f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 --2- 172.21.15.43:0/2423729552 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b6840 0x7fd1600ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 -- 172.21.15.43:0/2423729552 >> 172.21.15.43:0/2423729552 conn(0x7fd1600b2e80 msgr2=0x7fd1600b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 -- 172.21.15.43:0/2423729552 shutdown_connections 2024-09-17T05:31:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.648+0000 7fd167000640 1 -- 172.21.15.43:0/2423729552 wait complete. 2024-09-17T05:31:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 Processor -- start 2024-09-17T05:31:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 -- start start 2024-09-17T05:31:16.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b45a0 0x7fd160153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:16.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:16.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 0x7fd16015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd160168e70 con 0x7fd1600b6840 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd160168cf0 con 0x7fd1600cd4e0 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd167000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd160168ff0 con 0x7fd1600b45a0 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33720/0 (socket says 172.21.15.43:33720) 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 -- 172.21.15.43:0/1687423029 learned_addr learned my addr 172.21.15.43:0/1687423029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd15fe00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b45a0 0x7fd160153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd165000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 0x7fd16015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 -- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b45a0 msgr2=0x7fd160153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b45a0 0x7fd160153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 -- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 msgr2=0x7fd16015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.652+0000 7fd157e00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 0x7fd16015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd157e00640 1 -- 172.21.15.43:0/1687423029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160166300 con 0x7fd1600b6840 2024-09-17T05:31:16.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd165000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 0x7fd16015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:16.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd157e00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd14800e710 tx=0x7fd14800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14800cab0 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd14800cc50 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd160154880 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148005280 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd160154e40 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1600d0090 con 0x7fd1600b6840 2024-09-17T05:31:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1480047b0 con 0x7fd1600b6840 2024-09-17T05:31:16.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.656+0000 7fd15cc00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 0x7fd13c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:16.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.660+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fd148095170 con 0x7fd1600b6840 2024-09-17T05:31:16.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.660+0000 7fd15fe00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 0x7fd13c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:16.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.660+0000 7fd15fe00640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 0x7fd13c075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd150007fd0 tx=0x7fd150009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:16.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.664+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1600d0090 con 0x7fd1600b6840 2024-09-17T05:31:16.804 INFO:tasks.ceph.mds.c.smithi043.stderr: -52> 2024-09-17T05:31:12.048+0000 7f0f75a00640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-17T05:31:16.804 INFO:tasks.ceph.mds.c.smithi043.stderr: -51> 2024-09-17T05:31:12.048+0000 7f0f75a00640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-17T05:31:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.872+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd160154610 con 0x7fd1600b6840 2024-09-17T05:31:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.872+0000 7fd15cc00640 1 -- 172.21.15.43:0/1687423029 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+4392 (secure 0 0 0) 0x7fd14805fce0 con 0x7fd1600b6840 2024-09-17T05:31:16.873 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:16.873 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":160,"btime":"2024-09-17T05:31:16:065844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:16.065823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7058":{"gid":7058,"name":"d","rank":0,"incarnation":160,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 160 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 msgr2=0x7fd13c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 0x7fd13c075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd150007fd0 tx=0x7fd150009040 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 msgr2=0x7fd160165dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd14800e710 tx=0x7fd14800ebd0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 shutdown_connections 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd13c073200 0x7fd13c075710 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1600cd4e0 0x7fd16015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1600b6840 0x7fd160165dc0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 --2- 172.21.15.43:0/1687423029 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd1600b45a0 0x7fd160153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 >> 172.21.15.43:0/1687423029 conn(0x7fd1600b2e80 msgr2=0x7fd1600b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 shutdown_connections 2024-09-17T05:31:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:16.876+0000 7fd167000640 1 -- 172.21.15.43:0/1687423029 wait complete. 2024-09-17T05:31:16.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 160, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:31:16.065823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7058}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7058': {'gid': 7058, 'name': 'd', 'rank': 0, 'incarnation': 160, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/2994345600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2994345600}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2994345600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:31:16.897 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7058:up:replay 2024-09-17T05:31:16.897 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:31:16.921 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:31:21.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:21.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:22.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:22.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 -- 172.21.15.43:0/4158376639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c0155d70 msgr2=0x7f08c015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 --2- 172.21.15.43:0/4158376639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c0155d70 0x7f08c015c7e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f08c00af0a0 tx=0x7f08b401c740 comp rx=0 tx=0).stop 2024-09-17T05:31:22.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 -- 172.21.15.43:0/4158376639 shutdown_connections 2024-09-17T05:31:22.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 --2- 172.21.15.43:0/4158376639 >> v2:172.21.15.62:3300/0 conn(0x7f08c006f110 0x7f08c00b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 --2- 172.21.15.43:0/4158376639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c0155d70 0x7f08c015c7e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 -- 172.21.15.43:0/4158376639 >> 172.21.15.43:0/4158376639 conn(0x7f08c00b23d0 msgr2=0x7f08c00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 -- 172.21.15.43:0/4158376639 shutdown_connections 2024-09-17T05:31:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 -- 172.21.15.43:0/4158376639 wait complete. 2024-09-17T05:31:22.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.472+0000 7f08c7400640 1 Processor -- start 2024-09-17T05:31:22.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- start start 2024-09-17T05:31:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c006f110 0x7f08c0153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 0x7f08c015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0168990 con 0x7f08c00b3920 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0168810 con 0x7f08c006f110 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f08c0168b10 con 0x7f08c0155d70 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33748/0 (socket says 172.21.15.43:33748) 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 -- 172.21.15.43:0/653534821 learned_addr learned my addr 172.21.15.43:0/653534821 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c4a00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c006f110 0x7f08c0153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c5400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 0x7f08c015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 -- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 msgr2=0x7f08c015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 0x7f08c015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 -- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c006f110 msgr2=0x7f08c0153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c006f110 0x7f08c0153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 -- 172.21.15.43:0/653534821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c01539d0 con 0x7f08c00b3920 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c5400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 0x7f08c015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bfe00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00e7f0 tx=0x7f08ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00cc60 con 0x7f08c00b3920 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08ac00ce00 con 0x7f08c00b3920 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c0153ee0 con 0x7f08c00b3920 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac005280 con 0x7f08c00b3920 2024-09-17T05:31:22.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c01544a0 con 0x7f08c00b3920 2024-09-17T05:31:22.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.476+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c0153c50 con 0x7f08c00b3920 2024-09-17T05:31:22.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.480+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08ac005420 con 0x7f08c00b3920 2024-09-17T05:31:22.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.480+0000 7f08bcc00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 0x7f08a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.480+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f08ac095040 con 0x7f08c00b3920 2024-09-17T05:31:22.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.480+0000 7f08c4a00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 0x7f08a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.480+0000 7f08c4a00640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 0x7f08a0075740 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f08b40af000 tx=0x7f08b401af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.484+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08ac05fbb0 con 0x7f08c00b3920 2024-09-17T05:31:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.696+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08c00b33e0 con 0x7f08c00b3920 2024-09-17T05:31:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.696+0000 7f08bcc00640 1 -- 172.21.15.43:0/653534821 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) ==== 95+0+5225 (secure 0 0 0) 0x7f08ac05f4b0 con 0x7f08c00b3920 2024-09-17T05:31:22.699 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:22.699 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":163,"btime":"2024-09-17T05:31:19:073363+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8292,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1578591246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1578591246},{"type":"v1","addr":"172.21.15.43:6835","nonce":1578591246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:19.073351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7058":{"gid":7058,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6835/2994345600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2994345600},{"type":"v1","addr":"172.21.15.62:6835","nonce":2994345600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7058,"qdb_cluster":[7058]},"id":6}]} 2024-09-17T05:31:22.699 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 163 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 msgr2=0x7f08a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 0x7f08a0075740 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f08b40af000 tx=0x7f08b401af80 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 msgr2=0x7f08c0159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00e7f0 tx=0x7f08ac00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 shutdown_connections 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f08a0073230 0x7f08a0075740 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f08c0155d70 0x7f08c015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08c00b3920 0x7f08c0159ad0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 --2- 172.21.15.43:0/653534821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c006f110 0x7f08c0153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 >> 172.21.15.43:0/653534821 conn(0x7f08c00b23d0 msgr2=0x7f08c015f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 shutdown_connections 2024-09-17T05:31:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.700+0000 7f08c7400640 1 -- 172.21.15.43:0/653534821 wait complete. 2024-09-17T05:31:22.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 163, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:31:19.073351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7058}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7058': {'gid': 7058, 'name': 'd', 'rank': 0, 'incarnation': 160, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.62:6835/2994345600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2994345600}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2994345600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7058, 'qdb_cluster': [7058]} 2024-09-17T05:31:22.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:31:22.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- 172.21.15.43:0/1445026144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40b3af0 msgr2=0x7fe1e40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 --2- 172.21.15.43:0/1445026144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40b3af0 0x7fe1e40ad0b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009c80 tx=0x7fe1dc01cce0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- 172.21.15.43:0/1445026144 shutdown_connections 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 --2- 172.21.15.43:0/1445026144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40b3af0 0x7fe1e40ad0b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 --2- 172.21.15.43:0/1445026144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e4155d70 0x7fe1e415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- 172.21.15.43:0/1445026144 >> 172.21.15.43:0/1445026144 conn(0x7fe1e40b23d0 msgr2=0x7fe1e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- 172.21.15.43:0/1445026144 shutdown_connections 2024-09-17T05:31:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- 172.21.15.43:0/1445026144 wait complete. 2024-09-17T05:31:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 Processor -- start 2024-09-17T05:31:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.880+0000 7fe1ec000640 1 -- start start 2024-09-17T05:31:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 0x7fe1e40cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40cca90 0x7fe1e4154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4168c10 con 0x7fe1e4155d70 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4168a90 con 0x7fe1e40cca90 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e4168d90 con 0x7fe1e406f110 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 0x7fe1e40cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:45370/0 (socket says 172.21.15.43:45370) 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 0x7fe1e40cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33750/0 (socket says 172.21.15.43:33750) 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 -- 172.21.15.43:0/3877728275 learned_addr learned my addr 172.21.15.43:0/3877728275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40cca90 0x7fe1e4154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 -- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40cca90 msgr2=0x7fe1e4154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40cca90 0x7fe1e4154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 -- 172.21.15.43:0/3877728275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 msgr2=0x7fe1e40cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 0x7fe1e40cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 -- 172.21.15.43:0/3877728275 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e41f8680 con 0x7fe1e406f110 2024-09-17T05:31:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800e7a0 tx=0x7fe1d800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e1600640 1 -- 172.21.15.43:0/3877728275 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800ccc0 con 0x7fe1e406f110 2024-09-17T05:31:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e1600640 1 -- 172.21.15.43:0/3877728275 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d800ce60 con 0x7fe1e406f110 2024-09-17T05:31:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e41f8960 con 0x7fe1e406f110 2024-09-17T05:31:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1e1600640 1 -- 172.21.15.43:0/3877728275 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8004280 con 0x7fe1e406f110 2024-09-17T05:31:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.884+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e41f8f20 con 0x7fe1e406f110 2024-09-17T05:31:22.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1e1600640 1 -- 172.21.15.43:0/3877728275 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe1d8003600 con 0x7fe1e406f110 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1e1600640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 0x7fe1c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1f374e000 1 Processor -- start 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1e1600640 1 -- 172.21.15.43:0/3877728275 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fe1d8094ed0 con 0x7fe1e406f110 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1f374e000 1 -- start start 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 0x7fe1c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.888+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 0x7fe1c4075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1e41fcc20 tx=0x7fe1d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded4725be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded4726120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ded45fa410 con 0x55ded41e4460 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ded4537b40 con 0x55ded45bdca0 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ded4526f70 con 0x55ded45d4950 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33760/0 (socket says 172.21.15.43:33760) 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 -- 172.21.15.43:0/1279478515 learned_addr learned my addr 172.21.15.43:0/1279478515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded4725be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1ea000640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded4726120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 -- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 msgr2=0x55ded4725be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded4725be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 -- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 msgr2=0x55ded4726120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded4726120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 -- 172.21.15.43:0/1279478515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded44ee5b0 con 0x55ded41e4460 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1ea000640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded4726120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe1d805f420 tx=0x7fe1d805eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07da1ffee3bb3e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1d5c00640 1 -- 172.21.15.43:0/1279478515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d805f490 con 0x55ded41e4460 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1d5c00640 1 -- 172.21.15.43:0/1279478515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d8068d70 con 0x55ded41e4460 2024-09-17T05:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1d5c00640 1 -- 172.21.15.43:0/1279478515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80688e0 con 0x55ded41e4460 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- 172.21.15.43:0/1279478515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 msgr2=0x55ded47256a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe1d805f420 tx=0x7fe1d805eea0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- 172.21.15.43:0/1279478515 shutdown_connections 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded4726120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded4725be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1279478515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded47256a0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- 172.21.15.43:0/1279478515 >> 172.21.15.43:0/1279478515 conn(0x55ded46aad90 msgr2=0x55ded46da690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- 172.21.15.43:0/1279478515 shutdown_connections 2024-09-17T05:31:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- 172.21.15.43:0/1279478515 wait complete. 2024-09-17T05:31:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 -- start start 2024-09-17T05:31:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.892+0000 7fe1f374e000 1 Processor -- start 2024-09-17T05:31:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45bdca0 0x55ded472ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 0x55ded472d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ded44c15f0 con 0x55ded41e4460 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ded452ad80 con 0x55ded45d4950 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ded44cd520 con 0x55ded45bdca0 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 10 client.0 _ll_register_callbacks cb 0x55ded44d1830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33770/0 (socket says 172.21.15.43:33770) 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 -- 172.21.15.43:0/3796213696 learned_addr learned my addr 172.21.15.43:0/3796213696 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45bdca0 0x55ded472ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ea000640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 0x55ded472d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 -- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45bdca0 msgr2=0x55ded472ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45bdca0 0x55ded472ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 -- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 msgr2=0x55ded472d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 0x55ded472d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 -- 172.21.15.43:0/3796213696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded44c5190 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ea000640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 0x55ded472d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e9600640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800c6a0 tx=0x7fe1d8092f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2de490b73e6ad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d805f490 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fe1d807f120 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ded44bf110 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7fe1d807f320 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 10 client.8310 fetch_fsmap learned FSMap version 163 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=163,monmap=2+}) -- 0x55ded45abf10 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ded44bf110 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 163) ==== 3502+0+0 (secure 0 0 0) 0x7fe1d805e7e0 con 0x55ded41e4460 2024-09-17T05:31:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 10 client.8310 fetch_fsmap finished waiting for FSMap version 163 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 10 client.8310 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 20 client.8310 populate_metadata read hostname 'smithi043' 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41c97d0 0x55ded4736d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 4 client.8310 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 --> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded4539a90 con 0x55ded41c97d0 2024-09-17T05:31:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41c97d0 0x55ded4736d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1ca000640 10 client.8310 ms_handle_connect on v2:172.21.15.62:6834/2994345600 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.896+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41c97d0 0x55ded4736d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b0131620ad8105 server_cookie=2bda9b728abd36e1 in_seq=0 out_seq=0 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1ca000640 1 -- 172.21.15.43:0/3796213696 <== mds.0 v2:172.21.15.62:6834/2994345600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ded4539a90 con 0x55ded41c97d0 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1ca000640 10 client.8310 handle_command_reply: tid=0 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 client.8310 shutdown 2024-09-17T05:31:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 msgr2=0x55ded472c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800c6a0 tx=0x7fe1d8092f60 comp rx=0 tx=0).stop 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 shutdown_connections 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41c97d0 0x55ded4736d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45d4950 0x55ded472d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45bdca0 0x55ded472ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1ca000640 0 client.8310 ms_handle_reset on v2:172.21.15.62:6834/2994345600 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 --2- 172.21.15.43:0/3796213696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded472c920 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 >> 172.21.15.43:0/3796213696 conn(0x55ded46aad90 msgr2=0x55ded46e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 shutdown_connections 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 1 -- 172.21.15.43:0/3796213696 wait complete. 2024-09-17T05:31:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.900+0000 7fe1f374e000 20 client.8310 trim_cache size 0 max 16384 2024-09-17T05:31:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 Processor -- start 2024-09-17T05:31:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 -- start start 2024-09-17T05:31:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded46db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded46dbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e42059e0 con 0x55ded41e4460 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4204a60 con 0x55ded45bdca0 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e4203ef0 con 0x55ded45d4950 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33776/0 (socket says 172.21.15.43:33776) 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 -- 172.21.15.43:0/1284282555 learned_addr learned my addr 172.21.15.43:0/1284282555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1ea000640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded46dbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded46db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 -- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 msgr2=0x55ded46dbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded46dbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 -- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 msgr2=0x55ded46db930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded46db930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 -- 172.21.15.43:0/1284282555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded4582db0 con 0x55ded41e4460 2024-09-17T05:31:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded46db930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8066080 tx=0x7fe1d8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea8d847c37dcb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1d5200640 1 -- 172.21.15.43:0/1284282555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8075070 con 0x55ded41e4460 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1d5200640 1 -- 172.21.15.43:0/1284282555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d8063ec0 con 0x55ded41e4460 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.920+0000 7fe1d5200640 1 -- 172.21.15.43:0/1284282555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8075210 con 0x55ded41e4460 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- 172.21.15.43:0/1284282555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 msgr2=0x55ded46db3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8066080 tx=0x7fe1d8000bd0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- 172.21.15.43:0/1284282555 shutdown_connections 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded46db930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded46dbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1284282555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded46db3f0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- 172.21.15.43:0/1284282555 >> 172.21.15.43:0/1284282555 conn(0x55ded46aad90 msgr2=0x55ded46d9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- 172.21.15.43:0/1284282555 shutdown_connections 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- 172.21.15.43:0/1284282555 wait complete. 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- start start 2024-09-17T05:31:22.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 Processor -- start 2024-09-17T05:31:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded479b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded479bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ded452ad80 con 0x55ded41e4460 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4204410 con 0x55ded45bdca0 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ded44c15f0 con 0x55ded45d4950 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1f374e000 10 client.0 _ll_register_callbacks cb 0x7fe1d8080f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33790/0 (socket says 172.21.15.43:33790) 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e9600640 1 -- 172.21.15.43:0/1011460994 learned_addr learned my addr 172.21.15.43:0/1011460994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded479b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1ea000640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded479bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e9600640 1 -- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 msgr2=0x55ded479bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.924+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded479bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e9600640 1 -- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 msgr2=0x55ded479b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:22.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded479b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e9600640 1 -- 172.21.15.43:0/1011460994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded44edab0 con 0x55ded41e4460 2024-09-17T05:31:22.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e9600640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800e770 tx=0x7fe1d8009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f601a313f2f48a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:22.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8075040 con 0x55ded41e4460 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fe1d805e6f0 con 0x55ded41e4460 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d8063ec0 con 0x55ded41e4460 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=163) ==== 24+0+0 (secure 0 0 0) 0x7fe1d805f490 con 0x55ded41e4460 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 10 client.8316 fetch_fsmap learned FSMap version 163 2024-09-17T05:31:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=163,monmap=2+}) -- 0x55ded44cc070 con 0x55ded41e4460 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d8063ec0 con 0x55ded41e4460 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 163) ==== 3502+0+0 (secure 0 0 0) 0x7fe1d805ade0 con 0x55ded41e4460 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 10 client.8316 fetch_fsmap finished waiting for FSMap version 163 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 10 client.8316 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 20 client.8316 populate_metadata read hostname 'smithi043' 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41b7830 0x55ded47566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 4 client.8316 mds_command: new command op to 7058 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 --> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] -- command(tid 0: {"prefix": "status"}) -- 0x55ded4535540 con 0x55ded41b7830 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41b7830 0x55ded47566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1e8c00640 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41b7830 0x55ded47566b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01c005a3c49f071 server_cookie=1d72c13fbc0ffc5c in_seq=0 out_seq=0 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 10 client.8316 ms_handle_connect on v2:172.21.15.62:6834/2994345600 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 1 -- 172.21.15.43:0/1011460994 <== mds.0 v2:172.21.15.62:6834/2994345600 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ded4535540 con 0x55ded41b7830 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 10 client.8316 handle_command_reply: tid=0 2024-09-17T05:31:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 client.8316 shutdown 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 msgr2=0x55ded479b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800e770 tx=0x7fe1d8009aa0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 shutdown_connections 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:6834/2994345600,v1:172.21.15.62:6835/2994345600] conn(0x55ded41b7830 0x55ded47566b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ded45bdca0 0x55ded479b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ded45d4950 0x55ded479bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ca000640 0 client.8316 ms_handle_reset on v2:172.21.15.62:6834/2994345600 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 --2- 172.21.15.43:0/1011460994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ded41e4460 0x55ded479b240 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 >> 172.21.15.43:0/1011460994 conn(0x55ded46aad90 msgr2=0x55ded47366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 shutdown_connections 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 1 -- 172.21.15.43:0/1011460994 wait complete. 2024-09-17T05:31:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1f374e000 20 client.8316 trim_cache size 0 max 16384 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7058, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8568262190000002, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 163, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 101, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 101, 2024-09-17T05:31:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 19.554400437999998 2024-09-17T05:31:22.935 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:31:22.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 msgr2=0x7fe1c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 0x7fe1c4075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1e41fcc20 tx=0x7fe1d0009040 comp rx=0 tx=0).stop 2024-09-17T05:31:22.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 msgr2=0x7fe1e415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800e7a0 tx=0x7fe1d800ec60 comp rx=0 tx=0).stop 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 shutdown_connections 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1c4073200 0x7fe1c4075710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1e40cca90 0x7fe1e4154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e4155d70 0x7fe1e40cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 --2- 172.21.15.43:0/3877728275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe1e406f110 0x7fe1e415a9a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.928+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 >> 172.21.15.43:0/3877728275 conn(0x7fe1e40b23d0 msgr2=0x7fe1e40af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.932+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 shutdown_connections 2024-09-17T05:31:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:22.932+0000 7fe1ec000640 1 -- 172.21.15.43:0/3877728275 wait complete. 2024-09-17T05:31:22.950 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:31:22.950 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-09-17T05:31:22.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.108+0000 7f7d22600640 1 -- 172.21.15.43:0/933805194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0240 msgr2=0x7f7d1c0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.108+0000 7f7d22600640 1 --2- 172.21.15.43:0/933805194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0240 0x7f7d1c0cd040 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0751a0 tx=0x7f7d0401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 -- 172.21.15.43:0/933805194 shutdown_connections 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 --2- 172.21.15.43:0/933805194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0240 0x7f7d1c0cd040 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 --2- 172.21.15.43:0/933805194 >> v2:172.21.15.62:3300/0 conn(0x7f7d1c0d0650 0x7f7d1c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 --2- 172.21.15.43:0/933805194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c09ba60 0x7f7d1c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 -- 172.21.15.43:0/933805194 >> 172.21.15.43:0/933805194 conn(0x7f7d1c0cee60 msgr2=0x7f7d1c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 -- 172.21.15.43:0/933805194 shutdown_connections 2024-09-17T05:31:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 -- 172.21.15.43:0/933805194 wait complete. 2024-09-17T05:31:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 Processor -- start 2024-09-17T05:31:23.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.112+0000 7f7d22600640 1 -- start start 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d1c09ba60 0x7f7d1c1f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c0d0240 0x7f7d1c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c161950 con 0x7f7d1c0d0650 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1617d0 con 0x7f7d1c09ba60 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c0b1d40 con 0x7f7d1c0d0240 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33804/0 (socket says 172.21.15.43:33804) 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 -- 172.21.15.43:0/409156318 learned_addr learned my addr 172.21.15.43:0/409156318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1b400640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d1c09ba60 0x7f7d1c1f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d12a00640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c0d0240 0x7f7d1c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 -- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c0d0240 msgr2=0x7f7d1c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c0d0240 0x7f7d1c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 -- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d1c09ba60 msgr2=0x7f7d1c1f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d1c09ba60 0x7f7d1c1f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 -- 172.21.15.43:0/409156318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c06e010 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d1be00640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400d9d0 tx=0x7f7d1400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14017070 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d14004b30 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c1668c0 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400be60 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c06e280 con 0x7f7d1c0d0650 2024-09-17T05:31:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c0b3130 con 0x7f7d1c0d0650 2024-09-17T05:31:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.116+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d14003980 con 0x7f7d1c0d0650 2024-09-17T05:31:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.120+0000 7f7d13e00640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 0x7f7cf8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.120+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f7d14094d50 con 0x7f7d1c0d0650 2024-09-17T05:31:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.120+0000 7f7d1b400640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 0x7f7cf8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:23.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.120+0000 7f7d1b400640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 0x7f7cf8075740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009ac0 tx=0x7f7d0c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:23.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.124+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d14020020 con 0x7f7d1c0d0650 2024-09-17T05:31:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.336+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d1c072290 con 0x7f7d1c0d0650 2024-09-17T05:31:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d13e00640 1 -- 172.21.15.43:0/409156318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v163) ==== 53+0+73 (secure 0 0 0) 0x7f7d14003c70 con 0x7f7d1c0d0650 2024-09-17T05:31:23.340 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 msgr2=0x7f7cf8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 0x7f7cf8075740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009ac0 tx=0x7f7d0c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 msgr2=0x7f7d1c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400d9d0 tx=0x7f7d1400de90 comp rx=0 tx=0).stop 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 shutdown_connections 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7cf8073230 0x7f7cf8075740 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d1c0d0650 0x7f7d1c1577c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7d1c0d0240 0x7f7d1c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 --2- 172.21.15.43:0/409156318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d1c09ba60 0x7f7d1c1f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 >> 172.21.15.43:0/409156318 conn(0x7f7d1c0cee60 msgr2=0x7f7d1c0b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 shutdown_connections 2024-09-17T05:31:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23.340+0000 7f7d22600640 1 -- 172.21.15.43:0/409156318 wait complete. 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:31:23.362 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:31:23.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:31:23.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:31:23.363 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:31:23.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.363 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:31:23.373 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41960sec preferred_lft 41960sec 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.374 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.375 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:31:23.376 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:31:23.376 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:31:23.376 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:31:23.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.376 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:31:23.438 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:31:23.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.440 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:31:23.496 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:31:23.497 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:31:23.497 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:31:23.497 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:31:23.497 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:31:23.497 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:31:23.497 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:31:23.497 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:31:23.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:31:23.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:31:23.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:31:23.498 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:31:23.531 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:23.533+0000 7f3f57600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:31:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:31:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:31:23.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.636 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:31:23.687 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:31:23.688 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:31:23.688 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:31:23.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:31:23.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:31:23.755 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:31:23.811 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:31:23.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:31:23.811 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:31:23.866 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:31:23.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:31:23.968 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:31:23.969 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:31:23.969 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:31:23.987 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:31:23.987 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:31:23.987 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:31:23.987 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:31:23.988 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:31:23.999 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:31:24.000 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:31:24.004 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:31:24.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:24.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:31:24.024 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:31:24.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:31:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:31:24.131 INFO:teuthology.orchestra.run.smithi043.stdout:client7915 2024-09-17T05:31:24.131 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:24.131 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:24.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:24.141 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:31:24.185 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:31:24.194 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:31:24.194 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:31:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:24.194 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:31:24.194 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:31:24.195 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-09-17T05:31:24.195 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-09-17T05:31:24.195 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:24.195 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:24.247 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:24.247 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:24.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 -- 172.21.15.43:0/2658986793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a415c7e0 msgr2=0x7f91a406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 --2- 172.21.15.43:0/2658986793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a415c7e0 0x7f91a406f110 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f919c009bc0 tx=0x7f919c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 -- 172.21.15.43:0/2658986793 shutdown_connections 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 --2- 172.21.15.43:0/2658986793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a415c7e0 0x7f91a406f110 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 -- 172.21.15.43:0/2658986793 >> 172.21.15.43:0/2658986793 conn(0x7f91a40b23d0 msgr2=0x7f91a40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 -- 172.21.15.43:0/2658986793 shutdown_connections 2024-09-17T05:31:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.500+0000 7f91a9800640 1 -- 172.21.15.43:0/2658986793 wait complete. 2024-09-17T05:31:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 Processor -- start 2024-09-17T05:31:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 -- start start 2024-09-17T05:31:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a415c7e0 0x7f91a409be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91a40cbb10 0x7f91a415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4169dc0 con 0x7f91a4155d70 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4169c40 con 0x7f91a415c7e0 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a9800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f91a4169f40 con 0x7f91a40cbb10 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53242/0 (socket says 172.21.15.43:53242) 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 -- 172.21.15.43:0/3991374551 learned_addr learned my addr 172.21.15.43:0/3991374551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f919a000640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a415c7e0 0x7f91a409be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a3400640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91a40cbb10 0x7f91a415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:24.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 -- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91a40cbb10 msgr2=0x7f91a415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:24.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91a40cbb10 0x7f91a415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 -- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a415c7e0 msgr2=0x7f91a409be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:24.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a415c7e0 0x7f91a409be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.504+0000 7f91a2a00640 1 -- 172.21.15.43:0/3991374551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a409cb30 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f91a2a00640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f919400c410 tx=0x7f9194001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194011790 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9194017070 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a409ce10 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194011ce0 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a409a7d0 con 0x7f91a4155d70 2024-09-17T05:31:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a406f110 con 0x7f91a4155d70 2024-09-17T05:31:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f919401c460 con 0x7f91a4155d70 2024-09-17T05:31:24.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 0x7f9180075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:24.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.508+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7f91940942d0 con 0x7f91a4155d70 2024-09-17T05:31:24.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.512+0000 7f919a000640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 0x7f9180075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:24.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.512+0000 7f919a000640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 0x7f9180075740 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f91a40af8b0 tx=0x7f918c0013a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:24.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.516+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f919405e880 con 0x7f91a4155d70 2024-09-17T05:31:24.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91a415b960 con 0x7f91a4155d70 2024-09-17T05:31:24.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f919b400640 1 -- 172.21.15.43:0/3991374551 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v101) ==== 81+0+1456 (secure 0 0 0) 0x7f91940940d0 con 0x7f91a4155d70 2024-09-17T05:31:24.704 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:31:24.704 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:31:24.705 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:31:24.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:listed 24 entries 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 msgr2=0x7f9180075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 0x7f9180075740 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f91a40af8b0 tx=0x7f918c0013a0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 msgr2=0x7f91a4155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f919400c410 tx=0x7f9194001e80 comp rx=0 tx=0).stop 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 shutdown_connections 2024-09-17T05:31:24.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9180073230 0x7f9180075740 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a415c7e0 0x7f91a409be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f91a40cbb10 0x7f91a415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 --2- 172.21.15.43:0/3991374551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a4155d70 0x7f91a4155340 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 >> 172.21.15.43:0/3991374551 conn(0x7f91a40b23d0 msgr2=0x7f91a40ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 shutdown_connections 2024-09-17T05:31:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:24.704+0000 7f91a9800640 1 -- 172.21.15.43:0/3991374551 wait complete. 2024-09-17T05:31:24.727 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:31:24.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:24.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:31:24.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:24.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:24.751 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:24.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 -- 172.21.15.43:0/2807476810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 msgr2=0x7fefe00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2807476810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00ad0b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fefe00af0a0 tx=0x7fefd801cac0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 -- 172.21.15.43:0/2807476810 shutdown_connections 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2807476810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00ad0b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2807476810 >> v2:172.21.15.62:3300/0 conn(0x7fefe00b3af0 0x7fefe015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 -- 172.21.15.43:0/2807476810 >> 172.21.15.43:0/2807476810 conn(0x7fefe00b23d0 msgr2=0x7fefe00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 -- 172.21.15.43:0/2807476810 shutdown_connections 2024-09-17T05:31:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.008+0000 7fefe6c00640 1 -- 172.21.15.43:0/2807476810 wait complete. 2024-09-17T05:31:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 Processor -- start 2024-09-17T05:31:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- start start 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 0x7fefe009c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe009cc10 0x7fefe015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0168b90 con 0x7fefe006f110 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0168a10 con 0x7fefe0155d70 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fefe0168d10 con 0x7fefe009cc10 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53266/0 (socket says 172.21.15.43:53266) 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 -- 172.21.15.43:0/2514533580 learned_addr learned my addr 172.21.15.43:0/2514533580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefd7e00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 0x7fefe009c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe4c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe009cc10 0x7fefe015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 -- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe009cc10 msgr2=0x7fefe015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe009cc10 0x7fefe015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 -- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 msgr2=0x7fefe009c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 0x7fefe009c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 -- 172.21.15.43:0/2514533580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe01f8820 con 0x7fefe006f110 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefd7e00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 0x7fefe009c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdfe00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c430 tx=0x7fefd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011790 con 0x7fefe006f110 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd0017070 con 0x7fefe006f110 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011cc0 con 0x7fefe006f110 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe01f8ac0 con 0x7fefe006f110 2024-09-17T05:31:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.012+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01f9080 con 0x7fefe006f110 2024-09-17T05:31:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe00ad0b0 con 0x7fefe006f110 2024-09-17T05:31:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefd001c460 con 0x7fefe006f110 2024-09-17T05:31:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefdcc00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 0x7fefbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6741+0+0 (secure 0 0 0) 0x7fefd0094220 con 0x7fefe006f110 2024-09-17T05:31:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefd7e00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 0x7fefbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.016+0000 7fefd7e00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 0x7fefbc075710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fefe00c9ec0 tx=0x7fefc80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.020+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefd0021020 con 0x7fefe006f110 2024-09-17T05:31:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.236+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fefe00d1150 con 0x7fefe006f110 2024-09-17T05:31:25.236 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:25.236+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.236+0000 7fefdcc00640 1 -- 172.21.15.43:0/2514533580 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v164) ==== 161+0+0 (secure 0 0 0) 0x7fefe00d1150 con 0x7fefe006f110 2024-09-17T05:31:25.238 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 msgr2=0x7fefbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 0x7fefbc075710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fefe00c9ec0 tx=0x7fefc80091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 msgr2=0x7fefe00c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c430 tx=0x7fefd000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 shutdown_connections 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefbc073200 0x7fefbc075710 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefe0155d70 0x7fefe009c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefe009cc10 0x7fefe015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 --2- 172.21.15.43:0/2514533580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefe006f110 0x7fefe00c6270 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 >> 172.21.15.43:0/2514533580 conn(0x7fefe00b23d0 msgr2=0x7fefe00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 shutdown_connections 2024-09-17T05:31:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.240+0000 7fefe6c00640 1 -- 172.21.15.43:0/2514533580 wait complete. 2024-09-17T05:31:25.250 INFO:tasks.ceph.mds.d.smithi062.stderr: -535> 2024-09-17T05:31:23.533+0000 7f3f57600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:31:25.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:25.376 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:31:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- 172.21.15.43:0/3670409221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 msgr2=0x7fe88015d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 --2- 172.21.15.43:0/3670409221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe88015d760 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe8800c5900 tx=0x7fe87001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- 172.21.15.43:0/3670409221 shutdown_connections 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 --2- 172.21.15.43:0/3670409221 >> v2:172.21.15.62:3300/0 conn(0x7fe88006fa10 0x7fe88006fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 --2- 172.21.15.43:0/3670409221 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 0x7fe8800cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 --2- 172.21.15.43:0/3670409221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe88015d760 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- 172.21.15.43:0/3670409221 >> 172.21.15.43:0/3670409221 conn(0x7fe8800b24c0 msgr2=0x7fe8800b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- 172.21.15.43:0/3670409221 shutdown_connections 2024-09-17T05:31:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- 172.21.15.43:0/3670409221 wait complete. 2024-09-17T05:31:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 Processor -- start 2024-09-17T05:31:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.424+0000 7fe886c00640 1 -- start start 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88006fa10 0x7fe8801f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 0x7fe8800b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe880073e00 con 0x7fe880160d50 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe880073c80 con 0x7fe88006fa10 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe880073f80 con 0x7fe8800cf420 2024-09-17T05:31:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53284/0 (socket says 172.21.15.43:53284) 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 -- 172.21.15.43:0/951016164 learned_addr learned my addr 172.21.15.43:0/951016164 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe87fe00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88006fa10 0x7fe8801f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe877e00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 0x7fe8800b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 -- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 msgr2=0x7fe8800b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 0x7fe8800b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 -- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88006fa10 msgr2=0x7fe8801f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88006fa10 0x7fe8801f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 -- 172.21.15.43:0/951016164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8801fcf90 con 0x7fe880160d50 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe884c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe87800d9b0 tx=0x7fe87800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878016070 con 0x7fe880160d50 2024-09-17T05:31:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe878004e20 con 0x7fe880160d50 2024-09-17T05:31:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87800bcf0 con 0x7fe880160d50 2024-09-17T05:31:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8801fd270 con 0x7fe880160d50 2024-09-17T05:31:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8800b52b0 con 0x7fe880160d50 2024-09-17T05:31:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.428+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8800cf830 con 0x7fe880160d50 2024-09-17T05:31:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.432+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe878003980 con 0x7fe880160d50 2024-09-17T05:31:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.432+0000 7fe87cc00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 0x7fe85c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.432+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6827+0+0 (secure 0 0 0) 0x7fe878094f00 con 0x7fe880160d50 2024-09-17T05:31:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.432+0000 7fe87fe00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 0x7fe85c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.432+0000 7fe87fe00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 0x7fe85c075710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe8700af000 tx=0x7fe87001b0e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.436+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe87801f020 con 0x7fe880160d50 2024-09-17T05:31:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:25.672+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe8801fcd20 con 0x7fe880160d50 2024-09-17T05:31:26.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe87cc00640 1 -- 172.21.15.43:0/951016164 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v165) ==== 107+0+0 (secure 0 0 0) 0x7fe87805f460 con 0x7fe880160d50 2024-09-17T05:31:26.244 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 msgr2=0x7fe85c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 0x7fe85c075710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe8700af000 tx=0x7fe87001b0e0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 msgr2=0x7fe8801fc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe87800d9b0 tx=0x7fe87800de70 comp rx=0 tx=0).stop 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 shutdown_connections 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe85c073200 0x7fe85c075710 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe880160d50 0x7fe8801fc490 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8800cf420 0x7fe8800b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 --2- 172.21.15.43:0/951016164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe88006fa10 0x7fe8801f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 >> 172.21.15.43:0/951016164 conn(0x7fe8800b24c0 msgr2=0x7fe8800cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 shutdown_connections 2024-09-17T05:31:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.244+0000 7fe886c00640 1 -- 172.21.15.43:0/951016164 wait complete. 2024-09-17T05:31:26.271 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:26.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 Processor -- start 2024-09-17T05:31:26.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 -- start start 2024-09-17T05:31:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5635573c9890 0x5635574ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x563557413550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 0x563557413a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563557319a00 con 0x5635573facb0 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563557377f40 con 0x563557487da0 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe22f469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563557486b40 con 0x5635573c9890 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe227c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x563557413550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe227c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x563557413550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53296/0 (socket says 172.21.15.43:53296) 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe227c00640 1 -- 172.21.15.43:0/3885823187 learned_addr learned my addr 172.21.15.43:0/3885823187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 0x563557413a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe228600640 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5635573c9890 0x5635574ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 -- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5635573c9890 msgr2=0x5635574ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5635573c9890 0x5635574ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 -- 172.21.15.43:0/3885823187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 msgr2=0x563557413550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x563557413550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 -- 172.21.15.43:0/3885823187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635572f2c10 con 0x563557487da0 2024-09-17T05:31:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe229000640 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 0x563557413a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe220001510 tx=0x7fe2200019d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b336da77dfa953b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe226800640 1 -- 172.21.15.43:0/3885823187 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220017070 con 0x563557487da0 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe226800640 1 -- 172.21.15.43:0/3885823187 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2200027b0 con 0x563557487da0 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.332+0000 7fe226800640 1 -- 172.21.15.43:0/3885823187 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220002c40 con 0x563557487da0 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 msgr2=0x563557413a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 0x563557413a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe220001510 tx=0x7fe2200019d0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- 172.21.15.43:0/3885823187 shutdown_connections 2024-09-17T05:31:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563557487da0 0x563557413a90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x563557413550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- 172.21.15.43:0/3885823187 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5635573c9890 0x5635574ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- 172.21.15.43:0/3885823187 >> 172.21.15.43:0/3885823187 conn(0x5635573b7d80 msgr2=0x5635573cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- 172.21.15.43:0/3885823187 shutdown_connections 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- 172.21.15.43:0/3885823187 wait complete. 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 Processor -- start 2024-09-17T05:31:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- start start 2024-09-17T05:31:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635573c9890 0x56355749ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x56355749fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563557487da0 0x563557410ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563557483d40 con 0x5635573facb0 2024-09-17T05:31:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563557481600 con 0x5635573c9890 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe22f469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563557483560 con 0x563557487da0 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe227c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x56355749fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe227c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x56355749fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53308/0 (socket says 172.21.15.43:53308) 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 learned_addr learned my addr 172.21.15.43:0/298074993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563557487da0 0x563557410ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.336+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635573c9890 0x56355749ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563557487da0 msgr2=0x563557410ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563557487da0 0x563557410ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635573c9890 msgr2=0x56355749ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635573c9890 0x56355749ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635574a2040 con 0x5635573facb0 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563557487da0 0x563557410ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635573c9890 0x56355749ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:26.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x56355749fe40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe21400a7c0 tx=0x7fe21400ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe225400640 1 -- 172.21.15.43:0/298074993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21401b070 con 0x5635573facb0 2024-09-17T05:31:26.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe225400640 1 -- 172.21.15.43:0/298074993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe214009dc0 con 0x5635573facb0 2024-09-17T05:31:26.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe225400640 1 -- 172.21.15.43:0/298074993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140166c0 con 0x5635573facb0 2024-09-17T05:31:26.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56355749f460 con 0x5635573facb0 2024-09-17T05:31:26.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355749f9a0 con 0x5635573facb0 2024-09-17T05:31:26.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.340+0000 7fe225400640 1 -- 172.21.15.43:0/298074993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe21401e070 con 0x5635573facb0 2024-09-17T05:31:26.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe225400640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe208071be0 0x7fe2080740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe225400640 1 -- 172.21.15.43:0/298074993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6827+0+0 (secure 0 0 0) 0x7fe214093e50 con 0x5635573facb0 2024-09-17T05:31:26.344 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:26.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe208071be0 0x7fe2080740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5635575349e0 0x563557534df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635574a0380 con 0x5635575349e0 2024-09-17T05:31:26.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5635575349e0 0x563557534df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe208071be0 0x7fe2080740f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe2180ab000 tx=0x7fe218002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5635575349e0 0x563557534df0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v102'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.344+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557480b50 con 0x5635575349e0 2024-09-17T05:31:26.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v102'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548100 con 0x5635575349e0 2024-09-17T05:31:26.349 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:26.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v102'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575494b0 con 0x5635575349e0 2024-09-17T05:31:26.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.348+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557549850 con 0x5635575349e0 2024-09-17T05:31:26.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575484a0 con 0x5635575349e0 2024-09-17T05:31:26.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548890 con 0x5635575349e0 2024-09-17T05:31:26.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v102'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.352+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v102'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.357 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:26.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v102'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.356+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.360+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.364+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.364+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.364+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.366 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.364+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575490b0 con 0x5635575349e0 2024-09-17T05:31:26.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575490b0 con 0x5635575349e0 2024-09-17T05:31:26.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.368+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v102'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v102'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.372+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v102'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.376 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:26.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.376+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.380+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.380+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.380+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v102'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.380+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v102'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v102'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.385 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:26.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.384+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x5635575349e0 2024-09-17T05:31:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56355754a100 0x56355754c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557548c30 con 0x56355754a100 2024-09-17T05:31:26.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56355754a100 0x56355754c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.388+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56355754a100 0x56355754c520 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'286 uv265 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355754ca60 con 0x56355754a100 2024-09-17T05:31:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'287 uv287 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754ce00 con 0x56355754a100 2024-09-17T05:31:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.395 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:26.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.392+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754d650 con 0x56355754a100 2024-09-17T05:31:26.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v102'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754d9f0 con 0x56355754a100 2024-09-17T05:31:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754dd90 con 0x56355754a100 2024-09-17T05:31:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v102'291 uv284 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.396+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355754e130 con 0x56355754a100 2024-09-17T05:31:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v102'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v102'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.402 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:26.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.400+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.404+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.404+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.404+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v102'296 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.404+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v102'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v102'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.409 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:26.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575526a0 con 0x56355754a100 2024-09-17T05:31:26.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v102'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.408+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575526a0 con 0x56355754a100 2024-09-17T05:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v102'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v102'301 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v102'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.412+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v102'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.416 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:26.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754f4e0 con 0x56355754a100 2024-09-17T05:31:26.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v102'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754f4e0 con 0x56355754a100 2024-09-17T05:31:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v102'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.416+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355754e4d0 con 0x56355754a100 2024-09-17T05:31:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v102'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x56355754a100 2024-09-17T05:31:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56355754e9e0 0x563557550e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557551340 con 0x56355754e9e0 2024-09-17T05:31:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56355754e9e0 0x563557550e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56355754e9e0 0x563557550e00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v102'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.420+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557551810 con 0x56355754e9e0 2024-09-17T05:31:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557551cf0 con 0x56355754e9e0 2024-09-17T05:31:26.425 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575535a0 con 0x56355754e9e0 2024-09-17T05:31:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.424+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557553940 con 0x56355754e9e0 2024-09-17T05:31:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552590 con 0x56355754e9e0 2024-09-17T05:31:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552980 con 0x5635575349e0 2024-09-17T05:31:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.428+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557552980 con 0x5635575349e0 2024-09-17T05:31:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v102'58 uv58 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552980 con 0x5635575349e0 2024-09-17T05:31:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v102'59 uv59 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.434 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552980 con 0x5635575349e0 2024-09-17T05:31:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v102'60 uv60 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563557553ce0 0x563557556100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.432+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557556640 con 0x563557553ce0 2024-09-17T05:31:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563557553ce0 0x563557556100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe229000640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563557553ce0 0x563557556100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v102'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557556640 con 0x563557553ce0 2024-09-17T05:31:26.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.436+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557556640 con 0x563557553ce0 2024-09-17T05:31:26.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.440 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:26.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557557650 con 0x563557553ce0 2024-09-17T05:31:26.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557557650 con 0x563557553ce0 2024-09-17T05:31:26.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.440+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557556640 con 0x563557553ce0 2024-09-17T05:31:26.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563557556b50 0x563557558f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557559600 con 0x563557556b50 2024-09-17T05:31:26.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563557556b50 0x563557558f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe228600640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563557556b50 0x563557558f70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v102'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.444+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v102'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.449 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:26.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v102'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755a9e0 con 0x563557556b50 2024-09-17T05:31:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v102'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755a9e0 con 0x563557556b50 2024-09-17T05:31:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v102'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.448+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v102'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v102'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v102'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.452+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v102'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.456 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v102'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x5635575599d0 con 0x563557556b50 2024-09-17T05:31:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe228600640 1 -- 172.21.15.43:0/298074993 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v102'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2180abc00 con 0x563557556b50 2024-09-17T05:31:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355755a050 0x56355755c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755cb70 con 0x56355755a050 2024-09-17T05:31:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.456+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355755a050 0x56355755c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.460+0000 7fe227c00640 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355755a050 0x56355755c470 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.460+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v102'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.460+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755cf40 con 0x56355755a050 2024-09-17T05:31:26.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.460+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v102'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.460+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x56355755a050 2024-09-17T05:31:26.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v102'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.464 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:26.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x56355755a050 2024-09-17T05:31:26.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v102'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x56355755a050 2024-09-17T05:31:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v102'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.464+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x5635575349e0 2024-09-17T05:31:26.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v102'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d360 con 0x5635575349e0 2024-09-17T05:31:26.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x5635575349e0 2024-09-17T05:31:26.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.471 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:26.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.468+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755e370 con 0x5635575349e0 2024-09-17T05:31:26.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.472+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.472+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755e370 con 0x5635575349e0 2024-09-17T05:31:26.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.472+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.472+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x5635575349e0 2024-09-17T05:31:26.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x5635575349e0 2024-09-17T05:31:26.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x56355754e9e0 2024-09-17T05:31:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v102'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d360 con 0x56355754e9e0 2024-09-17T05:31:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v102'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.476+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d360 con 0x56355754e9e0 2024-09-17T05:31:26.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v102'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v102'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v102'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.480+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v102'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v102'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.487 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v102'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.484+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557552130 con 0x56355754e9e0 2024-09-17T05:31:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.488+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v102'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.488+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x563557560170 con 0x56355754e9e0 2024-09-17T05:31:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.488+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v102'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355754e9e0 2024-09-17T05:31:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.488+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v102'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v102'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.495 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:26.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v102'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.492+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.496+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v102'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.496+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.496+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v102'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.496+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.501 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:26.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755e820 con 0x563557553ce0 2024-09-17T05:31:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.500+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755e820 con 0x563557553ce0 2024-09-17T05:31:26.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x563557553ce0 2024-09-17T05:31:26.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe22000f070 con 0x563557553ce0 2024-09-17T05:31:26.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v102'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.504+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v102'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v102'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.510 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:26.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v102'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.508+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.512+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v102'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.512+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.512+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v102'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.512+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v102'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v102'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v102'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.516+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v102'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v102'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v102'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.520+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.523 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v102'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v102'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755e820 con 0x56355755a050 2024-09-17T05:31:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v102'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.524+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755d810 con 0x56355755a050 2024-09-17T05:31:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v102'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x56355755dbd0 con 0x56355755a050 2024-09-17T05:31:26.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe227c00640 1 -- 172.21.15.43:0/298074993 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v102'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe21401d030 con 0x56355755a050 2024-09-17T05:31:26.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56355754a100 msgr2=0x56355754c520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56355754a100 0x56355754c520 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563557556b50 msgr2=0x563557558f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563557556b50 0x563557558f70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355755a050 msgr2=0x56355755c470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355755a050 0x56355755c470 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56355754e9e0 msgr2=0x563557550e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56355754e9e0 0x563557550e00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 reap_dead start 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5635575349e0 msgr2=0x563557534df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5635575349e0 0x563557534df0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563557553ce0 msgr2=0x563557556100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563557553ce0 0x563557556100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe208071be0 msgr2=0x7fe2080740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe208071be0 0x7fe2080740f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe2180ab000 tx=0x7fe218002040 comp rx=0 tx=0).stop 2024-09-17T05:31:26.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 msgr2=0x56355749fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 --2- 172.21.15.43:0/298074993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635573facb0 0x56355749fe40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe21400a7c0 tx=0x7fe21400ac80 comp rx=0 tx=0).stop 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe229000640 1 -- 172.21.15.43:0/298074993 reap_dead start 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 shutdown_connections 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 >> 172.21.15.43:0/298074993 conn(0x5635573b7d80 msgr2=0x5635573cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 shutdown_connections 2024-09-17T05:31:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.528+0000 7fe22f469080 1 -- 172.21.15.43:0/298074993 wait complete. 2024-09-17T05:31:26.535 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-09-17T05:31:26.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 Processor -- start 2024-09-17T05:31:26.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 -- start start 2024-09-17T05:31:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822abe19c0 0x55822ac1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822ac935b0 0x55822abddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55822ab2aa00 con 0x55822ac939c0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55822ab88f40 con 0x55822abe19c0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6250dd9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55822acbd2d0 con 0x55822ac935b0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53320/0 (socket says 172.21.15.43:53320) 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 -- 172.21.15.43:0/1411635701 learned_addr learned my addr 172.21.15.43:0/1411635701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6249600640 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822ac935b0 0x55822abddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624a000640 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822abe19c0 0x55822ac1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 -- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822ac935b0 msgr2=0x55822abddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822ac935b0 0x55822abddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 -- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822abe19c0 msgr2=0x55822ac1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822abe19c0 0x55822ac1de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 -- 172.21.15.43:0/1411635701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822ab03c10 con 0x55822ac939c0 2024-09-17T05:31:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f624aa00640 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6244001620 tx=0x7f6244001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fc9b1df465d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.592+0000 7f6243e00640 1 -- 172.21.15.43:0/1411635701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244017070 con 0x55822ac939c0 2024-09-17T05:31:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6243e00640 1 -- 172.21.15.43:0/1411635701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6244002790 con 0x55822ac939c0 2024-09-17T05:31:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6243e00640 1 -- 172.21.15.43:0/1411635701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244002be0 con 0x55822ac939c0 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- 172.21.15.43:0/1411635701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 msgr2=0x55822ac20000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6244001620 tx=0x7f6244001a20 comp rx=0 tx=0).stop 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- 172.21.15.43:0/1411635701 shutdown_connections 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac939c0 0x55822ac20000 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822ac935b0 0x55822abddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- 172.21.15.43:0/1411635701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822abe19c0 0x55822ac1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- 172.21.15.43:0/1411635701 >> 172.21.15.43:0/1411635701 conn(0x55822abc8d80 msgr2=0x55822acb33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- 172.21.15.43:0/1411635701 shutdown_connections 2024-09-17T05:31:26.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- 172.21.15.43:0/1411635701 wait complete. 2024-09-17T05:31:26.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 Processor -- start 2024-09-17T05:31:26.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- start start 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822abe19c0 0x55822ad43bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 0x55822acb9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822ac939c0 0x55822ac923d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55822abdd4e0 con 0x55822ac935b0 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55822abdd240 con 0x55822ac939c0 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6250dd9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55822abdd780 con 0x55822abe19c0 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6249600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 0x55822acb9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6249600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 0x55822acb9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53334/0 (socket says 172.21.15.43:53334) 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f6249600640 1 -- 172.21.15.43:0/2978300695 learned_addr learned my addr 172.21.15.43:0/2978300695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:26.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f624a000640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822abe19c0 0x55822ad43bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.596+0000 7f624aa00640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822ac939c0 0x55822ac923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 -- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822abe19c0 msgr2=0x55822ad43bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55822abe19c0 0x55822ad43bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 -- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822ac939c0 msgr2=0x55822ac923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822ac939c0 0x55822ac923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 -- 172.21.15.43:0/2978300695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822acba240 con 0x55822ac935b0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f624aa00640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55822ac939c0 0x55822ac923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6249600640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 0x55822acb9d00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6238018da0 tx=0x7f62380ab980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 -- 172.21.15.43:0/2978300695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380b9070 con 0x55822ac935b0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55822ad441f0 con 0x55822ac935b0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 -- 172.21.15.43:0/2978300695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62380abc30 con 0x55822ac935b0 2024-09-17T05:31:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 -- 172.21.15.43:0/2978300695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380b5440 con 0x55822ac935b0 2024-09-17T05:31:26.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822ad42ae0 con 0x55822ac935b0 2024-09-17T05:31:26.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 -- 172.21.15.43:0/2978300695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f62380bc070 con 0x55822ac935b0 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f622c071be0 0x7f622c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.600+0000 7f6242a00640 1 -- 172.21.15.43:0/2978300695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6827+0+0 (secure 0 0 0) 0x7f623812e230 con 0x55822ac935b0 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624a000640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f622c071be0 0x7f622c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55822ad45c80 0x55822ac1dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x55822ad42f80 con 0x55822ad45c80 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624aa00640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55822ad45c80 0x55822ac1dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624a000640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f622c071be0 0x7f622c0740f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c8d0 tx=0x7f623c00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624aa00640 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55822ad45c80 0x55822ac1dea0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624aa00640 1 -- 172.21.15.43:0/2978300695 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [delete] v102'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f624400f070 con 0x55822ad45c80 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55822ad45c80 msgr2=0x55822ac1dea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55822ad45c80 0x55822ac1dea0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f622c071be0 msgr2=0x7f622c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f622c071be0 0x7f622c0740f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f623c00c8d0 tx=0x7f623c00f040 comp rx=0 tx=0).stop 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 msgr2=0x55822acb9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 --2- 172.21.15.43:0/2978300695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55822ac935b0 0x55822acb9d00 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6238018da0 tx=0x7f62380ab980 comp rx=0 tx=0).stop 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f624aa00640 1 -- 172.21.15.43:0/2978300695 reap_dead start 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 shutdown_connections 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 >> 172.21.15.43:0/2978300695 conn(0x55822abc8d80 msgr2=0x55822ac26590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 shutdown_connections 2024-09-17T05:31:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.604+0000 7f6250dd9080 1 -- 172.21.15.43:0/2978300695 wait complete. 2024-09-17T05:31:26.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 -- 172.21.15.43:0/2590341485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 msgr2=0x7f77280cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 --2- 172.21.15.43:0/2590341485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280cbc70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7720009bc0 tx=0x7f772001cab0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 -- 172.21.15.43:0/2590341485 shutdown_connections 2024-09-17T05:31:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 --2- 172.21.15.43:0/2590341485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280cbc70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 --2- 172.21.15.43:0/2590341485 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 0x7f772806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 -- 172.21.15.43:0/2590341485 >> 172.21.15.43:0/2590341485 conn(0x7f77280b23d0 msgr2=0x7f77280d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 -- 172.21.15.43:0/2590341485 shutdown_connections 2024-09-17T05:31:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.772+0000 7f772ca00640 1 -- 172.21.15.43:0/2590341485 wait complete. 2024-09-17T05:31:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 Processor -- start 2024-09-17T05:31:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 -- start start 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 0x7f77280c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77280ce690 con 0x7f7728070620 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77280ce510 con 0x7f77280cb860 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f772ca00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77280ce810 con 0x7f772806ce60 2024-09-17T05:31:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f771fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f771fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53338/0 (socket says 172.21.15.43:53338) 2024-09-17T05:31:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f771fe00640 1 -- 172.21.15.43:0/3892384150 learned_addr learned my addr 172.21.15.43:0/3892384150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f7726800640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.776+0000 7f7725e00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 0x7f77280c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 -- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 msgr2=0x7f77280c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 0x7f77280c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 -- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 msgr2=0x7f77280c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 -- 172.21.15.43:0/3892384150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77280c9740 con 0x7f7728070620 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771fe00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f771000eac0 tx=0x7f771000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771000cda0 con 0x7f7728070620 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7710004110 con 0x7f7728070620 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710003600 con 0x7f7728070620 2024-09-17T05:31:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77280c9a20 con 0x7f7728070620 2024-09-17T05:31:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77280c9fe0 con 0x7f7728070620 2024-09-17T05:31:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77280ceda0 con 0x7f7728070620 2024-09-17T05:31:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.780+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77100037a0 con 0x7f7728070620 2024-09-17T05:31:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.784+0000 7f771e000640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 0x7f7704075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.784+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6827+0+0 (secure 0 0 0) 0x7f7710094b60 con 0x7f7728070620 2024-09-17T05:31:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.784+0000 7f7725e00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 0x7f7704075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.784+0000 7f7725e00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 0x7f7704075710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7718007d70 tx=0x7f7718007d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:26.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.788+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f771005f030 con 0x7f7728070620 2024-09-17T05:31:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:26.996+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f77280c94d0 con 0x7f7728070620 2024-09-17T05:31:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.244+0000 7f771e000640 1 -- 172.21.15.43:0/3892384150 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v166) ==== 162+0+0 (secure 0 0 0) 0x7f7710063a60 con 0x7f7728070620 2024-09-17T05:31:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 msgr2=0x7f7704075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 0x7f7704075710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7718007d70 tx=0x7f7718007d00 comp rx=0 tx=0).stop 2024-09-17T05:31:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 msgr2=0x7f7728152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f771000eac0 tx=0x7f771000ef80 comp rx=0 tx=0).stop 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 shutdown_connections 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7704073200 0x7f7704075710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77280cb860 0x7f77280c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7728070620 0x7f7728152750 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 --2- 172.21.15.43:0/3892384150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772806ce60 0x7f77280c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 >> 172.21.15.43:0/3892384150 conn(0x7f77280b23d0 msgr2=0x7f77280b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 shutdown_connections 2024-09-17T05:31:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.248+0000 7f772ca00640 1 -- 172.21.15.43:0/3892384150 wait complete. 2024-09-17T05:31:27.259 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:27.260+0000 7f7545800640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-17T05:31:27.259 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:27.260+0000 7f7545800640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:31:27.260 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:27.260+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:27.260 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:27.260+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:27.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- 172.21.15.43:0/4002416376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa90015fe10 msgr2=0x7fa9000b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 --2- 172.21.15.43:0/4002416376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa90015fe10 0x7fa9000b4b80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009900 tx=0x7fa8f401c780 comp rx=0 tx=0).stop 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- 172.21.15.43:0/4002416376 shutdown_connections 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 --2- 172.21.15.43:0/4002416376 >> v2:172.21.15.62:3300/0 conn(0x7fa9000737c0 0x7fa9000700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 --2- 172.21.15.43:0/4002416376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa90015fe10 0x7fa9000b4b80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- 172.21.15.43:0/4002416376 >> 172.21.15.43:0/4002416376 conn(0x7fa9000b23d0 msgr2=0x7fa9000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- 172.21.15.43:0/4002416376 shutdown_connections 2024-09-17T05:31:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- 172.21.15.43:0/4002416376 wait complete. 2024-09-17T05:31:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 Processor -- start 2024-09-17T05:31:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.436+0000 7fa908800640 1 -- start start 2024-09-17T05:31:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa90006fc90 0x7fa90015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa90015fe10 0x7fa9000992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa900169f80 con 0x7fa9001fc720 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa900169e00 con 0x7fa90006fc90 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa90016a100 con 0x7fa90015fe10 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53354/0 (socket says 172.21.15.43:53354) 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 -- 172.21.15.43:0/2317867767 learned_addr learned my addr 172.21.15.43:0/2317867767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 -- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa90015fe10 msgr2=0x7fa9000992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa90015fe10 0x7fa9000992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 -- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa90006fc90 msgr2=0x7fa90015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa90006fc90 0x7fa90015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 -- 172.21.15.43:0/2317867767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90015c100 con 0x7fa9001fc720 2024-09-17T05:31:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa906800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e970 tx=0x7fa8fc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004d30 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8fc00cb20 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc005020 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa900099800 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90015c850 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.440+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90015d250 con 0x7fa9001fc720 2024-09-17T05:31:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.444+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8fc0051c0 con 0x7fa9001fc720 2024-09-17T05:31:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.444+0000 7fa8fa000640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.444+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7fa8fc095070 con 0x7fa9001fc720 2024-09-17T05:31:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.444+0000 7fa905e00640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.444+0000 7fa905e00640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 0x7fa8e4075710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40af000 tx=0x7fa8f4019520 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.448+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8fc016030 con 0x7fa9001fc720 2024-09-17T05:31:27.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.656+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa90015c2d0 con 0x7fa9001fc720 2024-09-17T05:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.656+0000 7fa8fa000640 1 -- 172.21.15.43:0/2317867767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) ==== 95+0+4383 (secure 0 0 0) 0x7fa8fc05fbc0 con 0x7fa9001fc720 2024-09-17T05:31:27.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:27.659 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":168,"btime":"2024-09-17T05:31:27:261213+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7205,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1077468406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1077468406},{"type":"v1","addr":"172.21.15.62:6835","nonce":1077468406}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":165}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:27.261121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 168 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 msgr2=0x7fa8e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 0x7fa8e4075710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40af000 tx=0x7fa8f4019520 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 msgr2=0x7fa9001fcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e970 tx=0x7fa8fc00ee30 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 shutdown_connections 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa8e4073200 0x7fa8e4075710 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9001fc720 0x7fa9001fcb30 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa90015fe10 0x7fa9000992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 --2- 172.21.15.43:0/2317867767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa90006fc90 0x7fa90015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 >> 172.21.15.43:0/2317867767 conn(0x7fa9000b23d0 msgr2=0x7fa9000702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 shutdown_connections 2024-09-17T05:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.660+0000 7fa908800640 1 -- 172.21.15.43:0/2317867767 wait complete. 2024-09-17T05:31:27.682 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:27.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 -- 172.21.15.43:0/913374882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 msgr2=0x7f7e800b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 --2- 172.21.15.43:0/913374882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e800b8b10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7e8008d510 tx=0x7f7e7801cd80 comp rx=0 tx=0).stop 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 -- 172.21.15.43:0/913374882 shutdown_connections 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 --2- 172.21.15.43:0/913374882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e800b8b10 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 --2- 172.21.15.43:0/913374882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 0x7f7e800d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 --2- 172.21.15.43:0/913374882 >> v2:172.21.15.62:3300/0 conn(0x7f7e800b9de0 0x7f7e8015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 -- 172.21.15.43:0/913374882 >> 172.21.15.43:0/913374882 conn(0x7f7e800d4600 msgr2=0x7f7e800d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 -- 172.21.15.43:0/913374882 shutdown_connections 2024-09-17T05:31:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.924+0000 7f7e87e00640 1 -- 172.21.15.43:0/913374882 wait complete. 2024-09-17T05:31:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 Processor -- start 2024-09-17T05:31:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 -- start start 2024-09-17T05:31:27.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 0x7f7e800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 0x7f7e800cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e800cf7b0 con 0x7f7e800b8700 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e800cf630 con 0x7f7e800b9de0 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e87e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7e800cf930 con 0x7f7e800d2bc0 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.928+0000 7f7e85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53386/0 (socket says 172.21.15.43:53386) 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 -- 172.21.15.43:0/950062313 learned_addr learned my addr 172.21.15.43:0/950062313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e84a00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 0x7f7e800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 0x7f7e800cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 -- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 msgr2=0x7f7e800cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 0x7f7e800cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 -- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 msgr2=0x7f7e800cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 0x7f7e800cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 -- 172.21.15.43:0/950062313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e80156d80 con 0x7f7e800b8700 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e84a00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 0x7f7e800cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e85400640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e9f0 tx=0x7f7e7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400ccc0 con 0x7f7e800b8700 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e74010430 con 0x7f7e800b8700 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e800cb640 con 0x7f7e800b8700 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74010860 con 0x7f7e800b8700 2024-09-17T05:31:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e800a2160 con 0x7f7e800b8700 2024-09-17T05:31:27.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8015ec30 con 0x7f7e800b8700 2024-09-17T05:31:27.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.932+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7e74010a00 con 0x7f7e800b8700 2024-09-17T05:31:27.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.936+0000 7f7e7d600640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 0x7f7e60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:27.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.936+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7f7e74095a70 con 0x7f7e800b8700 2024-09-17T05:31:27.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.936+0000 7f7e84a00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 0x7f7e60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:27.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.936+0000 7f7e84a00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 0x7f7e60075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7e80158230 tx=0x7f7e6c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:27.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:27.936+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7e80156d80 con 0x7f7e800b8700 2024-09-17T05:31:28.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.148+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e8006cf10 con 0x7f7e800b8700 2024-09-17T05:31:28.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.148+0000 7f7e7d600640 1 -- 172.21.15.43:0/950062313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) ==== 95+0+4383 (secure 0 0 0) 0x7f7e74067e20 con 0x7f7e800b8700 2024-09-17T05:31:28.151 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:28.152 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":168,"btime":"2024-09-17T05:31:27:261213+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7205,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1077468406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1077468406},{"type":"v1","addr":"172.21.15.62:6835","nonce":1077468406}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":165}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:27.261121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:28.152 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 168 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 msgr2=0x7f7e60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 0x7f7e60075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7e80158230 tx=0x7f7e6c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 msgr2=0x7f7e80156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e9f0 tx=0x7f7e7400eeb0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 shutdown_connections 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7e60073200 0x7f7e60075710 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e800b9de0 0x7f7e800cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7e800d2bc0 0x7f7e800cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 --2- 172.21.15.43:0/950062313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800b8700 0x7f7e80156840 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 >> 172.21.15.43:0/950062313 conn(0x7f7e800d4600 msgr2=0x7f7e800d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:28.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 shutdown_connections 2024-09-17T05:31:28.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.152+0000 7f7e87e00640 1 -- 172.21.15.43:0/950062313 wait complete. 2024-09-17T05:31:28.175 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:28.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 -- 172.21.15.43:0/4069048945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 msgr2=0x7f39e406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 --2- 172.21.15.43:0/4069048945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e406ce60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009900 tx=0x7f39d801c810 comp rx=0 tx=0).stop 2024-09-17T05:31:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 -- 172.21.15.43:0/4069048945 shutdown_connections 2024-09-17T05:31:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 --2- 172.21.15.43:0/4069048945 >> v2:172.21.15.62:3300/0 conn(0x7f39e40b2330 0x7f39e40b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 --2- 172.21.15.43:0/4069048945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e406ce60 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 -- 172.21.15.43:0/4069048945 >> 172.21.15.43:0/4069048945 conn(0x7f39e40b2e80 msgr2=0x7f39e40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 -- 172.21.15.43:0/4069048945 shutdown_connections 2024-09-17T05:31:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.332+0000 7f39eba00640 1 -- 172.21.15.43:0/4069048945 wait complete. 2024-09-17T05:31:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 Processor -- start 2024-09-17T05:31:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 -- start start 2024-09-17T05:31:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e40b2330 0x7f39e409ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39e416a350 con 0x7f39e40b6b10 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39e416a1d0 con 0x7f39e40b0cd0 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f39e416a4d0 con 0x7f39e40b2330 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e40b2330 0x7f39e409ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53408/0 (socket says 172.21.15.43:53408) 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:60150/0 (socket says 172.21.15.43:60150) 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 -- 172.21.15.43:0/4143236526 learned_addr learned my addr 172.21.15.43:0/4143236526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 -- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e40b2330 msgr2=0x7f39e409ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e40b2330 0x7f39e409ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 -- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 msgr2=0x7f39e40ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 -- 172.21.15.43:0/4143236526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e409c720 con 0x7f39e40b6b10 2024-09-17T05:31:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9000640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e9a00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c740 tx=0x7f39dc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc003b80 con 0x7f39e40b6b10 2024-09-17T05:31:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39dc00ce50 con 0x7f39e40b6b10 2024-09-17T05:31:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e409e100 con 0x7f39e40b6b10 2024-09-17T05:31:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.336+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00a850 con 0x7f39e40b6b10 2024-09-17T05:31:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.340+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e409e6c0 con 0x7f39e40b6b10 2024-09-17T05:31:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.340+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e40b2740 con 0x7f39e40b6b10 2024-09-17T05:31:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.340+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f39dc00a9f0 con 0x7f39e40b6b10 2024-09-17T05:31:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.340+0000 7f39e0c00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 0x7f39c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.340+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7f39dc094360 con 0x7f39e40b6b10 2024-09-17T05:31:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.344+0000 7f39e9000640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 0x7f39c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.344+0000 7f39e9000640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 0x7f39c4075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39d80af000 tx=0x7f39d801b050 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.348+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39e40b2c80 con 0x7f39e40b6b10 2024-09-17T05:31:28.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.556+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e409c4b0 con 0x7f39e40b6b10 2024-09-17T05:31:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.556+0000 7f39e0c00640 1 -- 172.21.15.43:0/4143236526 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) ==== 95+0+4383 (secure 0 0 0) 0x7f39dc05e8f0 con 0x7f39e40b6b10 2024-09-17T05:31:28.559 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:28.559 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":168,"btime":"2024-09-17T05:31:27:261213+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7205,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1077468406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1077468406},{"type":"v1","addr":"172.21.15.62:6835","nonce":1077468406}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":165}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:27.261121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 168 2024-09-17T05:31:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 msgr2=0x7f39c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 0x7f39c4075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39d80af000 tx=0x7f39d801b050 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 msgr2=0x7f39e41f8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c740 tx=0x7f39dc00cc00 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 shutdown_connections 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f39c4073200 0x7f39c4075710 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39e40b6b10 0x7f39e41f8b80 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39e40b2330 0x7f39e409ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 --2- 172.21.15.43:0/4143236526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39e40b0cd0 0x7f39e40ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 >> 172.21.15.43:0/4143236526 conn(0x7f39e40b2e80 msgr2=0x7f39e40b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 shutdown_connections 2024-09-17T05:31:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.560+0000 7f39eba00640 1 -- 172.21.15.43:0/4143236526 wait complete. 2024-09-17T05:31:28.581 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-09-17T05:31:28.582 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-09-17T05:31:28.582 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:28.582 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:28.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:28.591 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:28.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:28.591 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:28.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:28.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 -- 172.21.15.43:0/969105186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070620 msgr2=0x7fa3600b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 --2- 172.21.15.43:0/969105186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070620 0x7fa3600b30d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa3600cfc10 tx=0x7fa35001c830 comp rx=0 tx=0).stop 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 -- 172.21.15.43:0/969105186 shutdown_connections 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 --2- 172.21.15.43:0/969105186 >> v2:172.21.15.62:3300/0 conn(0x7fa36006ce60 0x7fa36006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 --2- 172.21.15.43:0/969105186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070620 0x7fa3600b30d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 -- 172.21.15.43:0/969105186 >> 172.21.15.43:0/969105186 conn(0x7fa3600b23d0 msgr2=0x7fa3600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 -- 172.21.15.43:0/969105186 shutdown_connections 2024-09-17T05:31:28.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.852+0000 7fa365400640 1 -- 172.21.15.43:0/969105186 wait complete. 2024-09-17T05:31:28.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 Processor -- start 2024-09-17T05:31:28.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 -- start start 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa360070620 0x7fa3601f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cb860 0x7fa3601fa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600ce540 con 0x7fa36006ce60 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600ce3c0 con 0x7fa360070620 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa365400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3600ce6c0 con 0x7fa3600cb860 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53426/0 (socket says 172.21.15.43:53426) 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 -- 172.21.15.43:0/4239836614 learned_addr learned my addr 172.21.15.43:0/4239836614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa355e00640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa360070620 0x7fa3601f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35f200640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cb860 0x7fa3601fa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 -- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cb860 msgr2=0x7fa3601fa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cb860 0x7fa3601fa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 -- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa360070620 msgr2=0x7fa3601f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa360070620 0x7fa3601f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.856+0000 7fa35e800640 1 -- 172.21.15.43:0/4239836614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3600c7d60 con 0x7fa36006ce60 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa35e800640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa350009850 tx=0x7fa3500a6b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500ac070 con 0x7fa36006ce60 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa350004030 con 0x7fa36006ce60 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3600c8040 con 0x7fa36006ce60 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350004480 con 0x7fa36006ce60 2024-09-17T05:31:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3600c8600 con 0x7fa36006ce60 2024-09-17T05:31:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3600cecd0 con 0x7fa36006ce60 2024-09-17T05:31:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3500af070 con 0x7fa36006ce60 2024-09-17T05:31:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 0x7fa33c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7fa35012e2e0 con 0x7fa36006ce60 2024-09-17T05:31:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.860+0000 7fa355e00640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 0x7fa33c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.864+0000 7fa355e00640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 0x7fa33c075710 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3600745f0 tx=0x7fa34800b3e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:28.864+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa36006d7b0 con 0x7fa36006ce60 2024-09-17T05:31:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.072+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa36006d310 con 0x7fa36006ce60 2024-09-17T05:31:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.264+0000 7fa357400640 1 -- 172.21.15.43:0/4239836614 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v169) ==== 161+0+0 (secure 0 0 0) 0x7fa3500164b0 con 0x7fa36006ce60 2024-09-17T05:31:29.267 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 msgr2=0x7fa33c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 0x7fa33c075710 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa3600745f0 tx=0x7fa34800b3e0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 msgr2=0x7fa36014d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa350009850 tx=0x7fa3500a6b50 comp rx=0 tx=0).stop 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 shutdown_connections 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa33c073200 0x7fa33c075710 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa360070620 0x7fa3601f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3600cb860 0x7fa3601fa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 --2- 172.21.15.43:0/4239836614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36006ce60 0x7fa36014d1a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 >> 172.21.15.43:0/4239836614 conn(0x7fa3600b23d0 msgr2=0x7fa3600b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 shutdown_connections 2024-09-17T05:31:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.268+0000 7fa365400640 1 -- 172.21.15.43:0/4239836614 wait complete. 2024-09-17T05:31:29.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 -- 172.21.15.43:0/1954239300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e406f110 msgr2=0x7f70e40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 --2- 172.21.15.43:0/1954239300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e406f110 0x7f70e40ad0b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009bc0 tx=0x7f70dc01cae0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 -- 172.21.15.43:0/1954239300 shutdown_connections 2024-09-17T05:31:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 --2- 172.21.15.43:0/1954239300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e406f110 0x7f70e40ad0b0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 --2- 172.21.15.43:0/1954239300 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e40b3af0 0x7f70e415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 -- 172.21.15.43:0/1954239300 >> 172.21.15.43:0/1954239300 conn(0x7f70e40b23d0 msgr2=0x7f70e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 -- 172.21.15.43:0/1954239300 shutdown_connections 2024-09-17T05:31:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.448+0000 7f70e9600640 1 -- 172.21.15.43:0/1954239300 wait complete. 2024-09-17T05:31:29.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 Processor -- start 2024-09-17T05:31:29.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 -- start start 2024-09-17T05:31:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e406f110 0x7f70e409ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e409dd10 0x7f70e40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4168be0 con 0x7f70e4155d70 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4168a60 con 0x7f70e409dd10 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f70e4168d60 con 0x7f70e406f110 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53442/0 (socket says 172.21.15.43:53442) 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 -- 172.21.15.43:0/2886694100 learned_addr learned my addr 172.21.15.43:0/2886694100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e3400640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e409dd10 0x7f70e40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2a00640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e406f110 0x7f70e409ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 -- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e406f110 msgr2=0x7f70e409ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e406f110 0x7f70e409ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 -- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e409dd10 msgr2=0x7f70e40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e409dd10 0x7f70e40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 -- 172.21.15.43:0/2886694100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e415acf0 con 0x7f70e4155d70 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e2000640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e7d0 tx=0x7f70d000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000ccc0 con 0x7f70e4155d70 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70d0010430 con 0x7f70e4155d70 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0010860 con 0x7f70e4155d70 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.452+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e40aefb0 con 0x7f70e4155d70 2024-09-17T05:31:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e415b270 con 0x7f70e4155d70 2024-09-17T05:31:29.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e40ad0b0 con 0x7f70e4155d70 2024-09-17T05:31:29.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70d0010a00 con 0x7f70e4155d70 2024-09-17T05:31:29.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70daa00640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 0x7f70c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:29.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7f70d0095100 con 0x7f70e4155d70 2024-09-17T05:31:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.456+0000 7f70e2a00640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 0x7f70c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.460+0000 7f70e2a00640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 0x7f70c4075740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007fd0 tx=0x7f70d4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.460+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f70d005f570 con 0x7f70e4155d70 2024-09-17T05:31:29.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:29.700+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f70e406dbe0 con 0x7f70e4155d70 2024-09-17T05:31:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.280+0000 7f70daa00640 1 -- 172.21.15.43:0/2886694100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v170) ==== 101+0+0 (secure 0 0 0) 0x7f70d0063fa0 con 0x7f70e4155d70 2024-09-17T05:31:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 msgr2=0x7f70c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 0x7f70c4075740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007fd0 tx=0x7f70d4009040 comp rx=0 tx=0).stop 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 msgr2=0x7f70e40ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e7d0 tx=0x7f70d000ec90 comp rx=0 tx=0).stop 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 shutdown_connections 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f70c4073230 0x7f70c4075740 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e409dd10 0x7f70e40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70e4155d70 0x7f70e40ccab0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 --2- 172.21.15.43:0/2886694100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70e406f110 0x7f70e409ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 >> 172.21.15.43:0/2886694100 conn(0x7f70e40b23d0 msgr2=0x7f70e406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 shutdown_connections 2024-09-17T05:31:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.284+0000 7f70e9600640 1 -- 172.21.15.43:0/2886694100 wait complete. 2024-09-17T05:31:30.310 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 Processor -- start 2024-09-17T05:31:30.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 -- start start 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dce52360 0x55c6dcf08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dce9bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c6dcda0a00 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c6dcdfef40 con 0x55c6dce9c9e0 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe486ec1080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c6dcf0dd10 con 0x55c6dce52360 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53470/0 (socket says 172.21.15.43:53470) 2024-09-17T05:31:30.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 -- 172.21.15.43:0/1508199289 learned_addr learned my addr 172.21.15.43:0/1508199289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480000640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dce52360 0x55c6dcf08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe47f600640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dce9bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 -- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dce52360 msgr2=0x55c6dcf08cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dce52360 0x55c6dcf08cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 -- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 msgr2=0x55c6dce9bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dce9bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe480a00640 1 -- 172.21.15.43:0/1508199289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6dcd79c10 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.372+0000 7fe47f600640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dce9bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe480a00640 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe4780015c0 tx=0x7fe4780019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57189b58c4fcbc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe47e200640 1 -- 172.21.15.43:0/1508199289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478017070 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe47e200640 1 -- 172.21.15.43:0/1508199289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4780027b0 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe47e200640 1 -- 172.21.15.43:0/1508199289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478002c40 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- 172.21.15.43:0/1508199289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 msgr2=0x55c6dcf29f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe4780015c0 tx=0x7fe4780019c0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- 172.21.15.43:0/1508199289 shutdown_connections 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dce9bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dcf0b4d0 0x55c6dcf29f60 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1508199289 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dce52360 0x55c6dcf08cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- 172.21.15.43:0/1508199289 >> 172.21.15.43:0/1508199289 conn(0x55c6dce3ed80 msgr2=0x55c6dcf0a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- 172.21.15.43:0/1508199289 shutdown_connections 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- 172.21.15.43:0/1508199289 wait complete. 2024-09-17T05:31:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 Processor -- start 2024-09-17T05:31:30.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.376+0000 7fe486ec1080 1 -- start start 2024-09-17T05:31:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dcf07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dcf0b4d0 0x55c6dcf084f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c6dcf09520 con 0x55c6dce52360 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c6dce9b290 con 0x55c6dce9c9e0 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c6dcf2cf10 con 0x55c6dcf0b4d0 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53482/0 (socket says 172.21.15.43:53482) 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 learned_addr learned my addr 172.21.15.43:0/1929980356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dcf07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dcf0b4d0 0x55c6dcf084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dcf0b4d0 msgr2=0x55c6dcf084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dcf0b4d0 0x55c6dcf084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 msgr2=0x55c6dcf07fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c6dce9c9e0 0x55c6dcf07fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6dcf2a110 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c6dcf0b4d0 0x55c6dcf084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe470096be0 tx=0x7fe4700ab750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe47ce00640 1 -- 172.21.15.43:0/1929980356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4700abb00 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6dcf2a3b0 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe47ce00640 1 -- 172.21.15.43:0/1929980356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4700bb070 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe47ce00640 1 -- 172.21.15.43:0/1929980356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4700b5440 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.380+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6dcf2a8f0 con 0x55c6dce52360 2024-09-17T05:31:30.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe47ce00640 1 -- 172.21.15.43:0/1929980356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4700b3040 con 0x55c6dce52360 2024-09-17T05:31:30.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe47ce00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 0x7fe460074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe47ce00640 1 -- 172.21.15.43:0/1929980356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7fe47012df50 con 0x55c6dce52360 2024-09-17T05:31:30.385 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 0x7fe460074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 0x55c6dcf2d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcf2dcc0 con 0x55c6dcf2d370 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 0x55c6dcf2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 0x7fe460074120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c650 tx=0x7fe46c004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.384+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 0x55c6dcf2d780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.388+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v103'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.388+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcf2e090 con 0x55c6dcf2d370 2024-09-17T05:31:30.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.388+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v103'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.388+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcf2e460 con 0x55c6dcf2d370 2024-09-17T05:31:30.392 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v103'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcf5b0 con 0x55c6dcf2d370 2024-09-17T05:31:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcf950 con 0x55c6dcf2d370 2024-09-17T05:31:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.392+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfce5a0 con 0x55c6dcf2d370 2024-09-17T05:31:30.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.396+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.396+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfce960 con 0x55c6dcf2d370 2024-09-17T05:31:30.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.396+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v103'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.396+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v103'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.401 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:30.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v103'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.400+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.404+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.410 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.408+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.412+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.412+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfced00 con 0x55c6dcf2d370 2024-09-17T05:31:30.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.412+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.412+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v103'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v103'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.416+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.420 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.420+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v103'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.420+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd0d00 con 0x55c6dcf2d370 2024-09-17T05:31:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.420+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.420+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd0d00 con 0x55c6dcf2d370 2024-09-17T05:31:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v103'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.424+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v103'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.429 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v103'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd0d00 con 0x55c6dcf2d370 2024-09-17T05:31:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.428+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd0d00 con 0x55c6dcf2d370 2024-09-17T05:31:30.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfcfcf0 con 0x55c6dcf2d370 2024-09-17T05:31:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c6dcfd0200 0x55c6dcfd2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd2b60 con 0x55c6dcfd0200 2024-09-17T05:31:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c6dcfd0200 0x55c6dcfd2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.432+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c6dcfd0200 0x55c6dcfd2620 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'307 uv290 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd2f00 con 0x55c6dcfd0200 2024-09-17T05:31:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'308 uv308 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd32a0 con 0x55c6dcfd0200 2024-09-17T05:31:30.439 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.436+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3b40 con 0x55c6dcfd0200 2024-09-17T05:31:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.440+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v103'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.440+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3ee0 con 0x55c6dcfd0200 2024-09-17T05:31:30.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.440+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.440+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd4280 con 0x55c6dcfd0200 2024-09-17T05:31:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v103'312 uv295 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v103'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd4a10 con 0x55c6dcfd0200 2024-09-17T05:31:30.446 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v103'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.444+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd4a10 con 0x55c6dcfd0200 2024-09-17T05:31:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v103'317 uv300 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.448+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.452+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v103'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.452+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.453 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.452+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v103'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.452+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v103'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v103'322 uv306 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.456+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v103'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.461 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v103'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd49c0 con 0x55c6dcfd0200 2024-09-17T05:31:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.460+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd5e60 con 0x55c6dcfd0200 2024-09-17T05:31:30.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd4d80 con 0x55c6dcfd0200 2024-09-17T05:31:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfd0200 2024-09-17T05:31:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c6dcfd5650 0x55c6dcfd7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd4d80 con 0x55c6dcfd5650 2024-09-17T05:31:30.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c6dcfd5650 0x55c6dcfd7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.464+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c6dcfd5650 0x55c6dcfd7e20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v103'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd8360 con 0x55c6dcfd5650 2024-09-17T05:31:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v103'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd8700 con 0x55c6dcfd5650 2024-09-17T05:31:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v103'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.471 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.468+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd9b10 con 0x55c6dcfd5650 2024-09-17T05:31:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd9f90 con 0x55c6dcfd5650 2024-09-17T05:31:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd8aa0 con 0x55c6dcfd5650 2024-09-17T05:31:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.472+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd8ec0 con 0x55c6dcf2d370 2024-09-17T05:31:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.476+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'61 uv60 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.476+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd8ec0 con 0x55c6dcf2d370 2024-09-17T05:31:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.476+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'62 uv62 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.476+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd8ec0 con 0x55c6dcf2d370 2024-09-17T05:31:30.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'63 uv63 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd8ec0 con 0x55c6dcf2d370 2024-09-17T05:31:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'64 uv64 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 0x55c6dcfdc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfdcce0 con 0x55c6dcfda380 2024-09-17T05:31:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 0x55c6dcfdc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.480+0000 7fe480a00640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 0x55c6dcfdc7a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v103'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfdcce0 con 0x55c6dcfda380 2024-09-17T05:31:30.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v103'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfdcce0 con 0x55c6dcfda380 2024-09-17T05:31:30.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v103'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.486 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:30.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.484+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfddcf0 con 0x55c6dcfda380 2024-09-17T05:31:30.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfddcf0 con 0x55c6dcfda380 2024-09-17T05:31:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfdcce0 con 0x55c6dcfda380 2024-09-17T05:31:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c6dcfdd360 0x55c6dcfdf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfdfe70 con 0x55c6dcfdd360 2024-09-17T05:31:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c6dcfdd360 0x55c6dcfdf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.488+0000 7fe480000640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c6dcfdd360 0x55c6dcfdf7a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v103'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v103'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v103'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.495 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:30.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.492+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe1250 con 0x55c6dcfdd360 2024-09-17T05:31:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe1250 con 0x55c6dcfdd360 2024-09-17T05:31:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.496+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v103'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v103'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.503 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:30.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v103'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.500+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe0240 con 0x55c6dcfdd360 2024-09-17T05:31:30.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe480000640 1 -- 172.21.15.43:0/1929980356 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4700f4620 con 0x55c6dcfdd360 2024-09-17T05:31:30.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c6dcfe08c0 0x55c6dcfe2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe33d0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c6dcfe08c0 0x55c6dcfe2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.504+0000 7fe47f600640 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c6dcfe08c0 0x55c6dcfe2d00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v103'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe37a0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v103'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v103'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.511 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:30.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.508+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcf2d370 2024-09-17T05:31:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v103'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.512+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcf2d370 2024-09-17T05:31:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v103'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcf2d370 2024-09-17T05:31:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v103'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.518 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe4bd0 con 0x55c6dcf2d370 2024-09-17T05:31:30.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.516+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe4bd0 con 0x55c6dcf2d370 2024-09-17T05:31:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.520+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.520+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcf2d370 2024-09-17T05:31:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.520+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcf2d370 2024-09-17T05:31:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.520+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v103'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v103'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v103'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.526 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.524+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v103'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.528+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.532+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v103'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.532+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.532+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v103'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.534 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.532+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe3bc0 con 0x55c6dcfd5650 2024-09-17T05:31:30.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.536+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v103'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.536+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3780 con 0x55c6dcfd5650 2024-09-17T05:31:30.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.536+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v103'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfd5650 2024-09-17T05:31:30.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.536+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.536+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v103'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.540+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.540+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v103'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.540+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.542 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.540+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v103'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.540+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v103'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v103'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v103'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.543+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v103'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.548 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3740 con 0x55c6dcfda380 2024-09-17T05:31:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v103'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3720 con 0x55c6dcfda380 2024-09-17T05:31:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v103'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.547+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfda380 2024-09-17T05:31:30.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v103'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe47800f070 con 0x55c6dcfda380 2024-09-17T05:31:30.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v103'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v103'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.551+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.556 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:30.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v103'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v103'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v103'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.555+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.559+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v103'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.561 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.559+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.559+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v103'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.559+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v103'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 1.00000000 [create] v103'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 1.00000000 [delete] v103'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.563+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v103'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.567 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd3700 con 0x55c6dcfe08c0 2024-09-17T05:31:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v103'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd36e0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v103'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfd36e0 con 0x55c6dcfe08c0 2024-09-17T05:31:30.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.567+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v103'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55c6dcfe4410 con 0x55c6dcfe08c0 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe47f600640 1 -- 172.21.15.43:0/1929980356 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v103'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe46c00a740 con 0x55c6dcfe08c0 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c6dcfd0200 msgr2=0x55c6dcfd2620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c6dcfd0200 0x55c6dcfd2620 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c6dcfdd360 msgr2=0x55c6dcfdf7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c6dcfdd360 0x55c6dcfdf7a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c6dcfe08c0 msgr2=0x55c6dcfe2d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c6dcfe08c0 0x55c6dcfe2d00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c6dcfd5650 msgr2=0x55c6dcfd7e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c6dcfd5650 0x55c6dcfd7e20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe480a00640 1 -- 172.21.15.43:0/1929980356 reap_dead start 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 msgr2=0x55c6dcf2d780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 0x55c6dcf2d780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 msgr2=0x55c6dcfdc7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 0x55c6dcfdc7a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 msgr2=0x7fe460074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 0x7fe460074120 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c650 tx=0x7fe46c004890 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 msgr2=0x55c6dcf27bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe470096be0 tx=0x7fe4700ab750 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 shutdown_connections 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c6dcfda380 0x55c6dcfdc7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe460071c10 0x7fe460074120 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c6dcf2d370 0x55c6dcf2d780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 --2- 172.21.15.43:0/1929980356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6dce52360 0x55c6dcf27bc0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 >> 172.21.15.43:0/1929980356 conn(0x55c6dce3ed80 msgr2=0x55c6dcf32b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 shutdown_connections 2024-09-17T05:31:30.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.571+0000 7fe486ec1080 1 -- 172.21.15.43:0/1929980356 wait complete. 2024-09-17T05:31:30.578 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-09-17T05:31:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 Processor -- start 2024-09-17T05:31:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- start start 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f4ae590 0x55e24f4f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f4f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e24f459f40 con 0x55e24f569da0 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e24f466d00 con 0x55e24f58e9c0 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e24f3fba00 con 0x55e24f4ae590 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53508/0 (socket says 172.21.15.43:53508) 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 -- 172.21.15.43:0/4199107684 learned_addr learned my addr 172.21.15.43:0/4199107684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a44a00640 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f4f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 -- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f4ae590 msgr2=0x55e24f4f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f4ae590 0x55e24f4f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 -- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 msgr2=0x55e24f4f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f4f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 -- 172.21.15.43:0/4199107684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24f4f62a0 con 0x55e24f569da0 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a44a00640 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f4f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a43600640 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7a30018e30 tx=0x7f7a30017fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0418aab7ceba0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a42200640 1 -- 172.21.15.43:0/4199107684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300ab980 con 0x55e24f569da0 2024-09-17T05:31:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a42200640 1 -- 172.21.15.43:0/4199107684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a300b9070 con 0x55e24f569da0 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a42200640 1 -- 172.21.15.43:0/4199107684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300b4400 con 0x55e24f569da0 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/4199107684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 msgr2=0x55e24f4f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f7a30018e30 tx=0x7f7a30017fa0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/4199107684 shutdown_connections 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f4f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f569da0 0x55e24f4f5730 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/4199107684 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f4ae590 0x55e24f4f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/4199107684 >> 172.21.15.43:0/4199107684 conn(0x55e24f499d80 msgr2=0x55e24f4dcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/4199107684 shutdown_connections 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.679+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/4199107684 wait complete. 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 Processor -- start 2024-09-17T05:31:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 -- start start 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 0x55e24f4f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f569da0 0x55e24f5805f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f580b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e24f581990 con 0x55e24f4ae590 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e24f4f6540 con 0x55e24f58e9c0 2024-09-17T05:31:30.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e24f581c30 con 0x55e24f569da0 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 0x55e24f4f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 0x55e24f4f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53518/0 (socket says 172.21.15.43:53518) 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 -- 172.21.15.43:0/882131771 learned_addr learned my addr 172.21.15.43:0/882131771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44a00640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f580b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a43600640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f569da0 0x55e24f5805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 -- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f569da0 msgr2=0x55e24f5805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e24f569da0 0x55e24f5805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 -- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 msgr2=0x55e24f580b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e24f58e9c0 0x55e24f580b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 -- 172.21.15.43:0/882131771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24f57dc50 con 0x55e24f4ae590 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a44000640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 0x55e24f4f5770 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400bcf0 tx=0x7f7a34009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a40e00640 1 -- 172.21.15.43:0/882131771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400e630 con 0x55e24f4ae590 2024-09-17T05:31:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e24f57def0 con 0x55e24f4ae590 2024-09-17T05:31:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a40e00640 1 -- 172.21.15.43:0/882131771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a3400e8e0 con 0x55e24f4ae590 2024-09-17T05:31:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.683+0000 7f7a40e00640 1 -- 172.21.15.43:0/882131771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ed30 con 0x55e24f4ae590 2024-09-17T05:31:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24f57e430 con 0x55e24f4ae590 2024-09-17T05:31:30.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a40e00640 1 -- 172.21.15.43:0/882131771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a3400d390 con 0x55e24f4ae590 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a40e00640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a24071be0 0x7f7a240740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a40e00640 1 -- 172.21.15.43:0/882131771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7f7a34093120 con 0x55e24f4ae590 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e24f57cad0 0x55e24f564bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55e24f57d4a0 con 0x55e24f57cad0 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a43600640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a24071be0 0x7f7a240740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.687+0000 7f7a44a00640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e24f57cad0 0x55e24f564bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a44a00640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e24f57cad0 0x55e24f564bb0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a43600640 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a24071be0 0x7f7a240740f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7a30004f50 tx=0x7f7a300bf040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a44a00640 1 -- 172.21.15.43:0/882131771 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [delete] v103'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7a3c002680 con 0x55e24f57cad0 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e24f57cad0 msgr2=0x55e24f564bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e24f57cad0 0x55e24f564bb0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a24071be0 msgr2=0x7f7a240740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7a24071be0 0x7f7a240740f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7a30004f50 tx=0x7f7a300bf040 comp rx=0 tx=0).stop 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 msgr2=0x55e24f4f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 --2- 172.21.15.43:0/882131771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24f4ae590 0x55e24f4f5770 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400bcf0 tx=0x7f7a34009600 comp rx=0 tx=0).stop 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a44a00640 1 -- 172.21.15.43:0/882131771 reap_dead start 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 shutdown_connections 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 >> 172.21.15.43:0/882131771 conn(0x55e24f499d80 msgr2=0x55e24f4f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 shutdown_connections 2024-09-17T05:31:30.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.691+0000 7f7a4ae1a080 1 -- 172.21.15.43:0/882131771 wait complete. 2024-09-17T05:31:30.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 -- 172.21.15.43:0/3095996679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80d0390 msgr2=0x7fbbc80b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 --2- 172.21.15.43:0/3095996679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80d0390 0x7fbbc80b4490 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009840 tx=0x7fbbb001c790 comp rx=0 tx=0).stop 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 -- 172.21.15.43:0/3095996679 shutdown_connections 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 --2- 172.21.15.43:0/3095996679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80d0390 0x7fbbc80b4490 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 --2- 172.21.15.43:0/3095996679 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc80b7c70 0x7fbbc80cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 -- 172.21.15.43:0/3095996679 >> 172.21.15.43:0/3095996679 conn(0x7fbbc80691b0 msgr2=0x7fbbc80695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 -- 172.21.15.43:0/3095996679 shutdown_connections 2024-09-17T05:31:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.859+0000 7fbbccc00640 1 -- 172.21.15.43:0/3095996679 wait complete. 2024-09-17T05:31:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 Processor -- start 2024-09-17T05:31:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- start start 2024-09-17T05:31:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 0x7fbbc8140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc815f6e0 0x7fbbc80d2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc80cd210 con 0x7fbbc80b7c70 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc80cd090 con 0x7fbbc80d0390 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc80cd390 con 0x7fbbc815f6e0 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbbfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 0x7fbbc8140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53524/0 (socket says 172.21.15.43:53524) 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 -- 172.21.15.43:0/1372966225 learned_addr learned my addr 172.21.15.43:0/1372966225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7e00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc815f6e0 0x7fbbc80d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 -- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc815f6e0 msgr2=0x7fbbc80d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc815f6e0 0x7fbbc80d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 -- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 msgr2=0x7fbbc8140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 0x7fbbc8140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 -- 172.21.15.43:0/1372966225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81f8e40 con 0x7fbbc80b7c70 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbbfe00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 0x7fbbc8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbc7400640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800e970 tx=0x7fbbb800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800cd20 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbb8004020 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81f9120 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004450 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.863+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc814f090 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.867+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc814f2c0 con 0x7fbbc80b7c70 2024-09-17T05:31:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.867+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbb8003600 con 0x7fbbc80b7c70 2024-09-17T05:31:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.867+0000 7fbbbf400640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 0x7fbba4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.867+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7fbbb8094ea0 con 0x7fbbc80b7c70 2024-09-17T05:31:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.867+0000 7fbbbfe00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 0x7fbba4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.871+0000 7fbbbfe00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 0x7fbba4075740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00af000 tx=0x7fbbb001afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:30.871+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbbc814f2c0 con 0x7fbbc80b7c70 2024-09-17T05:31:31.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.083+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbbc806a1e0 con 0x7fbbc80b7c70 2024-09-17T05:31:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.287+0000 7fbbbf400640 1 -- 172.21.15.43:0/1372966225 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v171) ==== 162+0+0 (secure 0 0 0) 0x7fbbb805f3a0 con 0x7fbbc80b7c70 2024-09-17T05:31:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 msgr2=0x7fbba4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 0x7fbba4075740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00af000 tx=0x7fbbb001afd0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 msgr2=0x7fbbc81403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800e970 tx=0x7fbbb800ee30 comp rx=0 tx=0).stop 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 shutdown_connections 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbba4073230 0x7fbba4075740 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc80d0390 0x7fbbc8140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbbc815f6e0 0x7fbbc80d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 --2- 172.21.15.43:0/1372966225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc80b7c70 0x7fbbc81403c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 >> 172.21.15.43:0/1372966225 conn(0x7fbbc80691b0 msgr2=0x7fbbc806e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 shutdown_connections 2024-09-17T05:31:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.291+0000 7fbbccc00640 1 -- 172.21.15.43:0/1372966225 wait complete. 2024-09-17T05:31:31.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 -- 172.21.15.43:0/1150743900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40b43c0 msgr2=0x7fcbe40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 --2- 172.21.15.43:0/1150743900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40b43c0 0x7fcbe40b47d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009c80 tx=0x7fcbdc01cd70 comp rx=0 tx=0).stop 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 -- 172.21.15.43:0/1150743900 shutdown_connections 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 --2- 172.21.15.43:0/1150743900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40b43c0 0x7fcbe40b47d0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 --2- 172.21.15.43:0/1150743900 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe415fc40 0x7fcbe4160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 -- 172.21.15.43:0/1150743900 >> 172.21.15.43:0/1150743900 conn(0x7fcbe40b2e80 msgr2=0x7fcbe40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 -- 172.21.15.43:0/1150743900 shutdown_connections 2024-09-17T05:31:31.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.471+0000 7fcbeb800640 1 -- 172.21.15.43:0/1150743900 wait complete. 2024-09-17T05:31:31.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 Processor -- start 2024-09-17T05:31:31.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 -- start start 2024-09-17T05:31:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe40b43c0 0x7fcbe4157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe40b5fe0 0x7fcbe4156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe41690c0 con 0x7fcbe415fc40 2024-09-17T05:31:31.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4168f40 con 0x7fcbe40b43c0 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbeb800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe4169240 con 0x7fcbe40b5fe0 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53552/0 (socket says 172.21.15.43:53552) 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 -- 172.21.15.43:0/3843646053 learned_addr learned my addr 172.21.15.43:0/3843646053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe8e00640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe40b43c0 0x7fcbe4157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbdbe00640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe40b5fe0 0x7fcbe4156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 -- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe40b5fe0 msgr2=0x7fcbe4156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe40b5fe0 0x7fcbe4156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 -- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe40b43c0 msgr2=0x7fcbe4157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe40b43c0 0x7fcbe4157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.475+0000 7fcbe9800640 1 -- 172.21.15.43:0/3843646053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe41fa650 con 0x7fcbe415fc40 2024-09-17T05:31:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe9800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc01c7e0 tx=0x7fcbdc003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:31.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0b2040 con 0x7fcbe415fc40 2024-09-17T05:31:31.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbdc004040 con 0x7fcbe415fc40 2024-09-17T05:31:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0043f0 con 0x7fcbe415fc40 2024-09-17T05:31:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe41fa930 con 0x7fcbe415fc40 2024-09-17T05:31:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe4145790 con 0x7fcbe415fc40 2024-09-17T05:31:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe41fa3c0 con 0x7fcbe415fc40 2024-09-17T05:31:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcbdc0bd030 con 0x7fcbe415fc40 2024-09-17T05:31:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 0x7fcbc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.479+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6913+0+0 (secure 0 0 0) 0x7fcbdc12f2a0 con 0x7fcbe415fc40 2024-09-17T05:31:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.483+0000 7fcbe8e00640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 0x7fcbc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.483+0000 7fcbe8e00640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 0x7fcbc0075710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4007fd0 tx=0x7fcbd4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:31.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.483+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcbdc0c4020 con 0x7fcbe415fc40 2024-09-17T05:31:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.691+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbe40b47d0 con 0x7fcbe415fc40 2024-09-17T05:31:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.691+0000 7fcbe1600640 1 -- 172.21.15.43:0/3843646053 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+4392 (secure 0 0 0) 0x7fcbdc016990 con 0x7fcbe415fc40 2024-09-17T05:31:31.694 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:31.694 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":172,"btime":"2024-09-17T05:31:31:292672+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:31.292652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7205":{"gid":7205,"name":"d","rank":0,"incarnation":172,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/1077468406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1077468406},{"type":"v1","addr":"172.21.15.62:6835","nonce":1077468406}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 172 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 msgr2=0x7fcbc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 0x7fcbc0075710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4007fd0 tx=0x7fcbd4009040 comp rx=0 tx=0).stop 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 msgr2=0x7fcbe4145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc01c7e0 tx=0x7fcbdc003f40 comp rx=0 tx=0).stop 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 shutdown_connections 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcbc0073200 0x7fcbc0075710 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe415fc40 0x7fcbe4145cb0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcbe40b5fe0 0x7fcbe4156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 --2- 172.21.15.43:0/3843646053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe40b43c0 0x7fcbe4157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 >> 172.21.15.43:0/3843646053 conn(0x7fcbe40b2e80 msgr2=0x7fcbe4070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 shutdown_connections 2024-09-17T05:31:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:31.695+0000 7fcbeb800640 1 -- 172.21.15.43:0/3843646053 wait complete. 2024-09-17T05:31:31.716 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:31:32.014 INFO:tasks.ceph.mds.c.smithi043.stderr: -44> 2024-09-17T05:31:27.260+0000 7f7545800640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-17T05:31:32.015 INFO:tasks.ceph.mds.c.smithi043.stderr: -34> 2024-09-17T05:31:27.260+0000 7f7545800640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:31:32.134 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:31:34.312 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:34.313+0000 7fb4d5a00640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-17T05:31:34.312 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:34.313+0000 7fb4d5a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-17T05:31:34.313 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:34.311+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:34.313 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:34.311+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:36.254 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:36.253+0000 7fb4daa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:31:36.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 -- 172.21.15.43:0/2991077173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e74155d70 msgr2=0x7f2e7415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/2991077173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e74155d70 0x7f2e7415c7e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009840 tx=0x7f2e6401c760 comp rx=0 tx=0).stop 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 -- 172.21.15.43:0/2991077173 shutdown_connections 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/2991077173 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e740b3af0 0x7f2e740ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/2991077173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e74155d70 0x7f2e7415c7e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 -- 172.21.15.43:0/2991077173 >> 172.21.15.43:0/2991077173 conn(0x7f2e740b23d0 msgr2=0x7f2e740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 -- 172.21.15.43:0/2991077173 shutdown_connections 2024-09-17T05:31:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.887+0000 7f2e7ae00640 1 -- 172.21.15.43:0/2991077173 wait complete. 2024-09-17T05:31:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 Processor -- start 2024-09-17T05:31:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 -- start start 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 0x7f2e740c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e74155d70 0x7f2e740c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74168c60 con 0x7f2e740c6120 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74168ae0 con 0x7f2e7406f110 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e7ae00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2e74168de0 con 0x7f2e74155d70 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49106/0 (socket says 172.21.15.43:49106) 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 -- 172.21.15.43:0/3132794075 learned_addr learned my addr 172.21.15.43:0/3132794075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e73e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 0x7f2e740c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e6be00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e74155d70 0x7f2e740c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 -- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e74155d70 msgr2=0x7f2e740c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e74155d70 0x7f2e740c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 -- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 msgr2=0x7f2e740c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 0x7f2e740c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 -- 172.21.15.43:0/3132794075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e741fa080 con 0x7f2e740c6120 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e73e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 0x7f2e740c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e78e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00eab0 tx=0x7f2e6c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.891+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00cea0 con 0x7f2e740c6120 2024-09-17T05:31:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e6c004b30 con 0x7f2e740c6120 2024-09-17T05:31:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c005020 con 0x7f2e740c6120 2024-09-17T05:31:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e741fa360 con 0x7f2e740c6120 2024-09-17T05:31:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e740c8220 con 0x7f2e740c6120 2024-09-17T05:31:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e740c6680 con 0x7f2e740c6120 2024-09-17T05:31:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e6c0051c0 con 0x7f2e740c6120 2024-09-17T05:31:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e70c00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 0x7f2e50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.895+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7f2e6c095670 con 0x7f2e740c6120 2024-09-17T05:31:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.899+0000 7f2e73e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 0x7f2e50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:36.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.899+0000 7f2e73e00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 0x7f2e50075710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af000 tx=0x7f2e6401afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:36.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:36.899+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2e6c05fc30 con 0x7f2e740c6120 2024-09-17T05:31:37.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e740c8480 con 0x7f2e740c6120 2024-09-17T05:31:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e70c00640 1 -- 172.21.15.43:0/3132794075 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) ==== 95+0+4383 (secure 0 0 0) 0x7f2e6c064660 con 0x7f2e740c6120 2024-09-17T05:31:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:37.112 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":176,"btime":"2024-09-17T05:31:34:313887+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2023604015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2023604015},{"type":"v1","addr":"172.21.15.43:6835","nonce":2023604015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":173}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:34.313881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:37.113 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 176 2024-09-17T05:31:37.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 msgr2=0x7f2e50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 0x7f2e50075710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af000 tx=0x7f2e6401afa0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 msgr2=0x7f2e741657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00eab0 tx=0x7f2e6c00ef70 comp rx=0 tx=0).stop 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 shutdown_connections 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2e50073200 0x7f2e50075710 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740c6120 0x7f2e741657c0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2e74155d70 0x7f2e740c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 --2- 172.21.15.43:0/3132794075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7406f110 0x7f2e740c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 >> 172.21.15.43:0/3132794075 conn(0x7f2e740b23d0 msgr2=0x7f2e7415d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 shutdown_connections 2024-09-17T05:31:37.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.111+0000 7f2e7ae00640 1 -- 172.21.15.43:0/3132794075 wait complete. 2024-09-17T05:31:37.135 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:31:37.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:37.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 -- 172.21.15.43:0/2045613904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 msgr2=0x7fafb815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 --2- 172.21.15.43:0/2045613904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815c7e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009840 tx=0x7fafa801c790 comp rx=0 tx=0).stop 2024-09-17T05:31:37.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 -- 172.21.15.43:0/2045613904 shutdown_connections 2024-09-17T05:31:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 --2- 172.21.15.43:0/2045613904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815c7e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 -- 172.21.15.43:0/2045613904 >> 172.21.15.43:0/2045613904 conn(0x7fafb80b23d0 msgr2=0x7fafb80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 -- 172.21.15.43:0/2045613904 shutdown_connections 2024-09-17T05:31:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.291+0000 7fafbcc00640 1 -- 172.21.15.43:0/2045613904 wait complete. 2024-09-17T05:31:37.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 Processor -- start 2024-09-17T05:31:37.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- start start 2024-09-17T05:31:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb815ac50 0x7fafb80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8169d70 con 0x7fafb806f110 2024-09-17T05:31:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8169bf0 con 0x7fafb8155d70 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fafb8169ef0 con 0x7fafb815ac50 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49114/0 (socket says 172.21.15.43:49114) 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 -- 172.21.15.43:0/1089523958 learned_addr learned my addr 172.21.15.43:0/1089523958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafafe00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb6800640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb815ac50 0x7fafb80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 -- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb815ac50 msgr2=0x7fafb80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb815ac50 0x7fafb80ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 -- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 msgr2=0x7fafb815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 -- 172.21.15.43:0/1089523958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb815b060 con 0x7fafb806f110 2024-09-17T05:31:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafb5e00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fafa801cc70 tx=0x7fafa8005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:37.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80bb070 con 0x7fafb806f110 2024-09-17T05:31:37.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa8004330 con 0x7fafb806f110 2024-09-17T05:31:37.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb80c6110 con 0x7fafb806f110 2024-09-17T05:31:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80b2040 con 0x7fafb806f110 2024-09-17T05:31:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.295+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb80c66d0 con 0x7fafb806f110 2024-09-17T05:31:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb815c7e0 con 0x7fafb806f110 2024-09-17T05:31:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafa80be070 con 0x7fafb806f110 2024-09-17T05:31:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafae000640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 0x7faf94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7fafa812e1e0 con 0x7fafb806f110 2024-09-17T05:31:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafafe00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 0x7faf94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.299+0000 7fafafe00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 0x7faf94075710 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fafb815b2d0 tx=0x7fafa0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.303+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafb815b060 con 0x7fafb806f110 2024-09-17T05:31:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafb815fe90 con 0x7fafb806f110 2024-09-17T05:31:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafae000640 1 -- 172.21.15.43:0/1089523958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) ==== 95+0+4383 (secure 0 0 0) 0x7fafa8016410 con 0x7fafb806f110 2024-09-17T05:31:37.516 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:37.516 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":176,"btime":"2024-09-17T05:31:34:313887+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2023604015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2023604015},{"type":"v1","addr":"172.21.15.43:6835","nonce":2023604015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":173}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:34.313881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 176 2024-09-17T05:31:37.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 msgr2=0x7faf94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 0x7faf94075710 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fafb815b2d0 tx=0x7fafa0009040 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 msgr2=0x7fafb809de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fafa801cc70 tx=0x7fafa8005ca0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 shutdown_connections 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf94073200 0x7faf94075710 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafb8155d70 0x7fafb815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafb815ac50 0x7fafb80ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 --2- 172.21.15.43:0/1089523958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb806f110 0x7fafb809de40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 >> 172.21.15.43:0/1089523958 conn(0x7fafb80b23d0 msgr2=0x7fafb815f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 shutdown_connections 2024-09-17T05:31:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.515+0000 7fafbcc00640 1 -- 172.21.15.43:0/1089523958 wait complete. 2024-09-17T05:31:37.538 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:37.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- 172.21.15.43:0/1503486897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f772006fc90 msgr2=0x7f77200700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 --2- 172.21.15.43:0/1503486897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f772006fc90 0x7f77200700a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7708009840 tx=0x7f770801c700 comp rx=0 tx=0).stop 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- 172.21.15.43:0/1503486897 shutdown_connections 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 --2- 172.21.15.43:0/1503486897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f772006fc90 0x7f77200700a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 --2- 172.21.15.43:0/1503486897 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 0x7f77200b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- 172.21.15.43:0/1503486897 >> 172.21.15.43:0/1503486897 conn(0x7f77200b23d0 msgr2=0x7f77200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- 172.21.15.43:0/1503486897 shutdown_connections 2024-09-17T05:31:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- 172.21.15.43:0/1503486897 wait complete. 2024-09-17T05:31:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 Processor -- start 2024-09-17T05:31:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.695+0000 7f7725e00640 1 -- start start 2024-09-17T05:31:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f772006fc90 0x7f77200c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 0x7f7720158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7720169fd0 con 0x7f77200730b0 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7720169e50 con 0x7f772006fc90 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f772016a150 con 0x7f772015fe10 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49140/0 (socket says 172.21.15.43:49140) 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 -- 172.21.15.43:0/3581235038 learned_addr learned my addr 172.21.15.43:0/3581235038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f771f400640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 0x7f7720158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f771ea00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f772006fc90 0x7f77200c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 -- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 msgr2=0x7f7720158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 0x7f7720158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 -- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f772006fc90 msgr2=0x7f77200c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f772006fc90 0x7f77200c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 -- 172.21.15.43:0/3581235038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772015a690 con 0x7f77200730b0 2024-09-17T05:31:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f770801cbe0 tx=0x7f77080a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080bd020 con 0x7f77200730b0 2024-09-17T05:31:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77080a66d0 con 0x7f77200730b0 2024-09-17T05:31:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708003600 con 0x7f77200730b0 2024-09-17T05:31:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772015cac0 con 0x7f77200730b0 2024-09-17T05:31:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772015c7b0 con 0x7f77200730b0 2024-09-17T05:31:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.699+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77200700a0 con 0x7f77200730b0 2024-09-17T05:31:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.703+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77080c0020 con 0x7f77200730b0 2024-09-17T05:31:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.703+0000 7f7717400640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 0x7f76fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.703+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7f770812f700 con 0x7f77200730b0 2024-09-17T05:31:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.703+0000 7f771ea00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 0x7f76fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.703+0000 7f771ea00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 0x7f76fc075740 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f771000bb90 tx=0x7f7710005ec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:37.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.707+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77080c7070 con 0x7f77200730b0 2024-09-17T05:31:37.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f772015d120 con 0x7f77200730b0 2024-09-17T05:31:37.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7717400640 1 -- 172.21.15.43:0/3581235038 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) ==== 95+0+4383 (secure 0 0 0) 0x7f7708016410 con 0x7f77200730b0 2024-09-17T05:31:37.921 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:37.921 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":176,"btime":"2024-09-17T05:31:34:313887+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2023604015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2023604015},{"type":"v1","addr":"172.21.15.43:6835","nonce":2023604015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":173}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:34.313881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:37.921 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 176 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 msgr2=0x7f76fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 0x7f76fc075740 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f771000bb90 tx=0x7f7710005ec0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 msgr2=0x7f7720158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f770801cbe0 tx=0x7f77080a64e0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 shutdown_connections 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f76fc073230 0x7f76fc075740 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f772015fe10 0x7f7720158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77200730b0 0x7f7720158210 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 --2- 172.21.15.43:0/3581235038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f772006fc90 0x7f77200c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:37.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 >> 172.21.15.43:0/3581235038 conn(0x7f77200b23d0 msgr2=0x7f77200ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:37.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 shutdown_connections 2024-09-17T05:31:37.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:37.919+0000 7f7725e00640 1 -- 172.21.15.43:0/3581235038 wait complete. 2024-09-17T05:31:37.944 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-09-17T05:31:37.944 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-09-17T05:31:37.944 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:37.944 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:37.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:37.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:37.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:37.953 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:38.003 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:38.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:38.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 -- 172.21.15.43:0/1720543421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc278155d70 msgr2=0x7fc27815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 --2- 172.21.15.43:0/1720543421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc278155d70 0x7fc27815c7e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc268009900 tx=0x7fc26801c820 comp rx=0 tx=0).stop 2024-09-17T05:31:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 -- 172.21.15.43:0/1720543421 shutdown_connections 2024-09-17T05:31:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 --2- 172.21.15.43:0/1720543421 >> v2:172.21.15.62:3300/0 conn(0x7fc27806f110 0x7fc2780b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 --2- 172.21.15.43:0/1720543421 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc278155d70 0x7fc27815c7e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 -- 172.21.15.43:0/1720543421 >> 172.21.15.43:0/1720543421 conn(0x7fc2780b23d0 msgr2=0x7fc2780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 -- 172.21.15.43:0/1720543421 shutdown_connections 2024-09-17T05:31:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 -- 172.21.15.43:0/1720543421 wait complete. 2024-09-17T05:31:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.203+0000 7fc27e200640 1 Processor -- start 2024-09-17T05:31:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- start start 2024-09-17T05:31:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc27806f110 0x7fc2781fc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2780b3920 0x7fc2781fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc278168ab0 con 0x7fc278155d70 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc278168930 con 0x7fc27806f110 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc278168c30 con 0x7fc2780b3920 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49148/0 (socket says 172.21.15.43:49148) 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 -- 172.21.15.43:0/996516281 learned_addr learned my addr 172.21.15.43:0/996516281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc26ea00640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2780b3920 0x7fc2781fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277400640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc27806f110 0x7fc2781fc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 -- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2780b3920 msgr2=0x7fc2781fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2780b3920 0x7fc2781fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 -- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc27806f110 msgr2=0x7fc2781fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc27806f110 0x7fc2781fc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 -- 172.21.15.43:0/996516281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780c4a70 con 0x7fc278155d70 2024-09-17T05:31:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc277e00640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c760 tx=0x7fc27000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270003b80 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc270003d20 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2780c4d50 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27000aab0 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.207+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2780c5310 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2780b33e0 con 0x7fc278155d70 2024-09-17T05:31:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc27000ac50 con 0x7fc278155d70 2024-09-17T05:31:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc26fe00640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 0x7fc254075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7fc270094380 con 0x7fc278155d70 2024-09-17T05:31:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc277400640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 0x7fc254075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.211+0000 7fc277400640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 0x7fc254075740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc2680af000 tx=0x7fc26801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:38.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.215+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc2780b3d30 con 0x7fc278155d70 2024-09-17T05:31:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:38.427+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc2780c47e0 con 0x7fc278155d70 2024-09-17T05:31:39.069 INFO:tasks.ceph.mds.d.smithi062.stderr: -55> 2024-09-17T05:31:34.313+0000 7fb4d5a00640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-17T05:31:39.069 INFO:tasks.ceph.mds.d.smithi062.stderr: -53> 2024-09-17T05:31:34.313+0000 7fb4d5a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-17T05:31:39.069 INFO:tasks.ceph.mds.d.smithi062.stderr: -30> 2024-09-17T05:31:36.253+0000 7fb4daa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:31:39.193 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:31:39.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.343+0000 7fc26fe00640 1 -- 172.21.15.43:0/996516281 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v177) ==== 161+0+0 (secure 0 0 0) 0x7fc27005e830 con 0x7fc278155d70 2024-09-17T05:31:39.348 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 msgr2=0x7fc254075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 0x7fc254075740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc2680af000 tx=0x7fc26801b060 comp rx=0 tx=0).stop 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 msgr2=0x7fc2780c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c760 tx=0x7fc27000cc20 comp rx=0 tx=0).stop 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 shutdown_connections 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc254073230 0x7fc254075740 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278155d70 0x7fc2780c92d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2780b3920 0x7fc2781fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 --2- 172.21.15.43:0/996516281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc27806f110 0x7fc2781fc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 >> 172.21.15.43:0/996516281 conn(0x7fc2780b23d0 msgr2=0x7fc278074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 shutdown_connections 2024-09-17T05:31:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.351+0000 7fc27e200640 1 -- 172.21.15.43:0/996516281 wait complete. 2024-09-17T05:31:39.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:39.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 -- 172.21.15.43:0/739408302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b406f110 msgr2=0x7ff9b40b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/739408302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b406f110 0x7ff9b40b33e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009900 tx=0x7ff9a001c820 comp rx=0 tx=0).stop 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 -- 172.21.15.43:0/739408302 shutdown_connections 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/739408302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b406f110 0x7ff9b40b33e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/739408302 >> v2:172.21.15.62:3300/0 conn(0x7ff9b4155d70 0x7ff9b415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 -- 172.21.15.43:0/739408302 >> 172.21.15.43:0/739408302 conn(0x7ff9b40b23d0 msgr2=0x7ff9b40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 -- 172.21.15.43:0/739408302 shutdown_connections 2024-09-17T05:31:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 -- 172.21.15.43:0/739408302 wait complete. 2024-09-17T05:31:39.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.539+0000 7ff9bbe00640 1 Processor -- start 2024-09-17T05:31:39.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- start start 2024-09-17T05:31:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b40b3920 0x7ff9b409cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b4155d70 0x7ff9b40c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4168b10 con 0x7ff9b406f110 2024-09-17T05:31:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4168990 con 0x7ff9b4155d70 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b4168c90 con 0x7ff9b40b3920 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49152/0 (socket says 172.21.15.43:49152) 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 -- 172.21.15.43:0/3244390627 learned_addr learned my addr 172.21.15.43:0/3244390627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9e00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b4155d70 0x7ff9b40c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b8a00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b40b3920 0x7ff9b409cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 -- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b40b3920 msgr2=0x7ff9b409cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b40b3920 0x7ff9b409cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 -- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b4155d70 msgr2=0x7ff9b40c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b4155d70 0x7ff9b40c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 -- 172.21.15.43:0/3244390627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b40c9df0 con 0x7ff9b406f110 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b9400640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800e9c0 tx=0x7ff9a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800cd80 con 0x7ff9b406f110 2024-09-17T05:31:39.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a8010520 con 0x7ff9b406f110 2024-09-17T05:31:39.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b40ca0d0 con 0x7ff9b406f110 2024-09-17T05:31:39.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8010970 con 0x7ff9b406f110 2024-09-17T05:31:39.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41fc920 con 0x7ff9b406f110 2024-09-17T05:31:39.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.543+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b40b33e0 con 0x7ff9b406f110 2024-09-17T05:31:39.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.547+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9a8010b10 con 0x7ff9b406f110 2024-09-17T05:31:39.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.547+0000 7ff9b1600640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:39.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.547+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7ff9a8095240 con 0x7ff9b406f110 2024-09-17T05:31:39.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.547+0000 7ff9b8a00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:39.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.547+0000 7ff9b8a00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 0x7ff994075710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00af000 tx=0x7ff9a001b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:39.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.551+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff9a805fde0 con 0x7ff9b406f110 2024-09-17T05:31:39.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:39.787+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff9b415be40 con 0x7ff9b406f110 2024-09-17T05:31:40.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9b1600640 1 -- 172.21.15.43:0/3244390627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v178) ==== 101+0+0 (secure 0 0 0) 0x7ff9a805f6e0 con 0x7ff9b406f110 2024-09-17T05:31:40.356 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 msgr2=0x7ff994075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 0x7ff994075710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00af000 tx=0x7ff9a001b060 comp rx=0 tx=0).stop 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 msgr2=0x7ff9b41fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800e9c0 tx=0x7ff9a800ee80 comp rx=0 tx=0).stop 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 shutdown_connections 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9b4155d70 0x7ff9b40c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff9b40b3920 0x7ff9b409cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 --2- 172.21.15.43:0/3244390627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b406f110 0x7ff9b41fd120 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 >> 172.21.15.43:0/3244390627 conn(0x7ff9b40b23d0 msgr2=0x7ff9b415a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 shutdown_connections 2024-09-17T05:31:40.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.355+0000 7ff9bbe00640 1 -- 172.21.15.43:0/3244390627 wait complete. 2024-09-17T05:31:40.381 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:40.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 Processor -- start 2024-09-17T05:31:40.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 -- start start 2024-09-17T05:31:40.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d2accb0 0x55ed9d2c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 0x55ed9d3d98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ed9d1cba00 con 0x55ed9d27dd70 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ed9d229f40 con 0x55ed9d2accb0 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f35a3f28080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ed9d334890 con 0x55ed9d333e40 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49176/0 (socket says 172.21.15.43:49176) 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 -- 172.21.15.43:0/1860380243 learned_addr learned my addr 172.21.15.43:0/1860380243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359da00640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 0x55ed9d3d98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f3597e00640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d2accb0 0x55ed9d2c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 -- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 msgr2=0x55ed9d3d98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:40.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 0x55ed9d3d98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 -- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d2accb0 msgr2=0x55ed9d2c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d2accb0 0x55ed9d2c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359d000640 1 -- 172.21.15.43:0/1860380243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9d1a4c10 con 0x55ed9d27dd70 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.439+0000 7f359da00640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 0x55ed9d3d98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f359d000640 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f359000b5f0 tx=0x7f359000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb269c375b305c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f3596a00640 1 -- 172.21.15.43:0/1860380243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359000a8f0 con 0x55ed9d27dd70 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f3596a00640 1 -- 172.21.15.43:0/1860380243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f359000aa90 con 0x55ed9d27dd70 2024-09-17T05:31:40.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f3596a00640 1 -- 172.21.15.43:0/1860380243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590015470 con 0x55ed9d27dd70 2024-09-17T05:31:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- 172.21.15.43:0/1860380243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 msgr2=0x55ed9d352e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f359000b5f0 tx=0x7f359000bab0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- 172.21.15.43:0/1860380243 shutdown_connections 2024-09-17T05:31:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d2accb0 0x55ed9d2c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d333e40 0x55ed9d3d98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- 172.21.15.43:0/1860380243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d27dd70 0x55ed9d352e70 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- 172.21.15.43:0/1860380243 >> 172.21.15.43:0/1860380243 conn(0x55ed9d269d80 msgr2=0x55ed9d2c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- 172.21.15.43:0/1860380243 shutdown_connections 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- 172.21.15.43:0/1860380243 wait complete. 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 Processor -- start 2024-09-17T05:31:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- start start 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d27dd70 0x55ed9d2a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d333e40 0x55ed9d2be560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ed9d2bf280 con 0x55ed9d2accb0 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ed9d2ac6a0 con 0x55ed9d333e40 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.443+0000 7f35a3f28080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55ed9d2ac940 con 0x55ed9d27dd70 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49188/0 (socket says 172.21.15.43:49188) 2024-09-17T05:31:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 learned_addr learned my addr 172.21.15.43:0/2875082673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d27dd70 0x55ed9d2a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d333e40 0x55ed9d2be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d27dd70 msgr2=0x55ed9d2a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55ed9d27dd70 0x55ed9d2a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d333e40 msgr2=0x55ed9d2be560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d333e40 0x55ed9d2be560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9d3dd350 con 0x55ed9d2accb0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed9d333e40 0x55ed9d2be560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f358800deb0 tx=0x7f358800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3595600640 1 -- 172.21.15.43:0/2875082673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588014020 con 0x55ed9d2accb0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3595600640 1 -- 172.21.15.43:0/2875082673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3588001400 con 0x55ed9d2accb0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3595600640 1 -- 172.21.15.43:0/2875082673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358800a040 con 0x55ed9d2accb0 2024-09-17T05:31:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed9d3dd5f0 con 0x55ed9d2accb0 2024-09-17T05:31:40.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9d3e9610 con 0x55ed9d2accb0 2024-09-17T05:31:40.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.447+0000 7f3595600640 1 -- 172.21.15.43:0/2875082673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3588017020 con 0x55ed9d2accb0 2024-09-17T05:31:40.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f3595600640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 0x7f357c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f3595600640 1 -- 172.21.15.43:0/2875082673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7f3588093290 con 0x55ed9d2accb0 2024-09-17T05:31:40.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 0x7f357c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:40.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55ed9d3ead60 0x55ed9d3f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f3bf0 con 0x55ed9d3ead60 2024-09-17T05:31:40.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55ed9d3ead60 0x55ed9d3f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 0x7f357c0740f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f359000b5f0 tx=0x7f3590002fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.451+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55ed9d3ead60 0x55ed9d3f36b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v104'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f3f90 con 0x55ed9d3ead60 2024-09-17T05:31:40.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v104'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4330 con 0x55ed9d3ead60 2024-09-17T05:31:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:40.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v104'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.455+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f56e0 con 0x55ed9d3ead60 2024-09-17T05:31:40.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5a80 con 0x55ed9d3ead60 2024-09-17T05:31:40.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f46d0 con 0x55ed9d3ead60 2024-09-17T05:31:40.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.459+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4ac0 con 0x55ed9d3ead60 2024-09-17T05:31:40.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v104'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v104'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v104'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.463+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.467+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.467+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.467+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.467+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.467+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.471+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.471+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.471+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.471+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.471+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.475+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f4e60 con 0x55ed9d3ead60 2024-09-17T05:31:40.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v104'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.479+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v104'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:40.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v104'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f6e30 con 0x55ed9d3ead60 2024-09-17T05:31:40.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.483+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f6e30 con 0x55ed9d3ead60 2024-09-17T05:31:40.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.487+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.487+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.487+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.487+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v104'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v104'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:40.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v104'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.491+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f6e30 con 0x55ed9d3ead60 2024-09-17T05:31:40.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f6e30 con 0x55ed9d3ead60 2024-09-17T05:31:40.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f5e20 con 0x55ed9d3ead60 2024-09-17T05:31:40.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55ed9d3f6330 0x55ed9d3f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f8c90 con 0x55ed9d3f6330 2024-09-17T05:31:40.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55ed9d3f6330 0x55ed9d3f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.495+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55ed9d3f6330 0x55ed9d3f8750 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.499+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'329 uv311 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.499+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3f9030 con 0x55ed9d3f6330 2024-09-17T05:31:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.499+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'330 uv330 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.499+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f93d0 con 0x55ed9d3f6330 2024-09-17T05:31:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.503+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.504 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.503+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f9c20 con 0x55ed9d3f6330 2024-09-17T05:31:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.503+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.503+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3f9fc0 con 0x55ed9d3f6330 2024-09-17T05:31:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.503+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.507+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa360 con 0x55ed9d3f6330 2024-09-17T05:31:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.507+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v104'334 uv316 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.507+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.507+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v104'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.507+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v104'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.511+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.515+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v104'339 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.515+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.515+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v104'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.515+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.515+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v104'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.519 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.519+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fe8d0 con 0x55ed9d3f6330 2024-09-17T05:31:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.519+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v104'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.519+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fe8d0 con 0x55ed9d3f6330 2024-09-17T05:31:40.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.519+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.519+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v104'344 uv327 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v104'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v104'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.523+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fb710 con 0x55ed9d3f6330 2024-09-17T05:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.527+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.527+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fb710 con 0x55ed9d3f6330 2024-09-17T05:31:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.527+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.527+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fa700 con 0x55ed9d3f6330 2024-09-17T05:31:40.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d3f6330 2024-09-17T05:31:40.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ed9d3fb000 0x55ed9d3fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fd960 con 0x55ed9d3fb000 2024-09-17T05:31:40.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ed9d3fb000 0x55ed9d3fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ed9d3fb000 0x55ed9d3fd420 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v104'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.531+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fde20 con 0x55ed9d3fb000 2024-09-17T05:31:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fe300 con 0x55ed9d3fb000 2024-09-17T05:31:40.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v104'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.537 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:40.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3ffbb0 con 0x55ed9d3fb000 2024-09-17T05:31:40.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.535+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fff50 con 0x55ed9d3fb000 2024-09-17T05:31:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3feba0 con 0x55ed9d3fb000 2024-09-17T05:31:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d3ead60 2024-09-17T05:31:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'65 uv64 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.539+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d3ead60 2024-09-17T05:31:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.543+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'66 uv66 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.543+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d3ead60 2024-09-17T05:31:40.546 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.543+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.543+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d3ead60 2024-09-17T05:31:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55ed9d4002f0 0x55ed9d402710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d4002f0 2024-09-17T05:31:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55ed9d4002f0 0x55ed9d402710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f359da00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55ed9d4002f0 0x55ed9d402710 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v104'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.547+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d4002f0 2024-09-17T05:31:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d3fef90 con 0x55ed9d4002f0 2024-09-17T05:31:40.552 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:40.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v104'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d403cb0 con 0x55ed9d4002f0 2024-09-17T05:31:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.551+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d403cb0 con 0x55ed9d4002f0 2024-09-17T05:31:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d402ca0 con 0x55ed9d4002f0 2024-09-17T05:31:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55ed9d4031b0 0x55ed9d4055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405ba0 con 0x55ed9d4031b0 2024-09-17T05:31:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55ed9d4031b0 0x55ed9d4055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f359d000640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55ed9d4031b0 0x55ed9d4055d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v104'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.555+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v104'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v104'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.562 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:40.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d406f80 con 0x55ed9d4031b0 2024-09-17T05:31:40.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.559+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d406f80 con 0x55ed9d4031b0 2024-09-17T05:31:40.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v104'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.563+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v104'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v104'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.569 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:40.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.567+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d405f70 con 0x55ed9d4031b0 2024-09-17T05:31:40.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f359d000640 1 -- 172.21.15.43:0/2875082673 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f359000acd0 con 0x55ed9d4031b0 2024-09-17T05:31:40.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55ed9d406890 0x55ed9d406ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4093a0 con 0x55ed9d406890 2024-09-17T05:31:40.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55ed9d406890 0x55ed9d406ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f3597e00640 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55ed9d406890 0x55ed9d406ca0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v104'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.571+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d409740 con 0x55ed9d406890 2024-09-17T05:31:40.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v104'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409740 con 0x55ed9d406890 2024-09-17T05:31:40.577 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:40.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v104'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409b30 con 0x55ed9d406890 2024-09-17T05:31:40.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.575+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409b30 con 0x55ed9d406890 2024-09-17T05:31:40.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3ead60 2024-09-17T05:31:40.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v104'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3ead60 2024-09-17T05:31:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.579+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3ead60 2024-09-17T05:31:40.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v104'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.584 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:40.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d40a8f0 con 0x55ed9d3ead60 2024-09-17T05:31:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d40a8b0 con 0x55ed9d3ead60 2024-09-17T05:31:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.583+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3ead60 2024-09-17T05:31:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d3ead60 2024-09-17T05:31:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3fb000 2024-09-17T05:31:40.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v104'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d4098a0 con 0x55ed9d3fb000 2024-09-17T05:31:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v104'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.587+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409850 con 0x55ed9d3fb000 2024-09-17T05:31:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v104'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.592 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409c40 con 0x55ed9d3fb000 2024-09-17T05:31:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v104'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409c40 con 0x55ed9d3fb000 2024-09-17T05:31:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v104'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.591+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098f0 con 0x55ed9d3fb000 2024-09-17T05:31:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v104'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d4098f0 con 0x55ed9d3fb000 2024-09-17T05:31:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v104'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098f0 con 0x55ed9d3fb000 2024-09-17T05:31:40.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v104'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.599 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:40.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.595+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d4098f0 con 0x55ed9d3fb000 2024-09-17T05:31:40.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.599+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v104'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.599+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409de0 con 0x55ed9d3fb000 2024-09-17T05:31:40.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.599+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v104'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d3fb000 2024-09-17T05:31:40.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.599+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v104'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v104'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.607 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:40.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v104'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.603+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.607+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v104'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.607+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.607+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v104'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.607+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.607+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.612 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:40.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v104'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409da0 con 0x55ed9d4002f0 2024-09-17T05:31:40.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409d80 con 0x55ed9d4002f0 2024-09-17T05:31:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.611+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409d80 con 0x55ed9d4002f0 2024-09-17T05:31:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.615+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3598002050 con 0x55ed9d4002f0 2024-09-17T05:31:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.615+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409d80 con 0x55ed9d406890 2024-09-17T05:31:40.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.615+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v104'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.615+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d409d30 con 0x55ed9d406890 2024-09-17T05:31:40.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v104'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.620 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:40.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v104'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v104'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.619+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v104'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v104'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v104'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.623+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.627 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v104'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v104'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v104'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.627+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.632 INFO:teuthology.orchestra.run.smithi043.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v104'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.631+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d409cb0 con 0x55ed9d406890 2024-09-17T05:31:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55ed9d40a0d0 con 0x55ed9d406890 2024-09-17T05:31:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f3597e00640 1 -- 172.21.15.43:0/2875082673 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3588020070 con 0x55ed9d406890 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55ed9d3f6330 msgr2=0x55ed9d3f8750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55ed9d3f6330 0x55ed9d3f8750 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55ed9d4031b0 msgr2=0x55ed9d4055d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55ed9d4031b0 0x55ed9d4055d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55ed9d406890 msgr2=0x55ed9d406ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55ed9d406890 0x55ed9d406ca0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ed9d3fb000 msgr2=0x55ed9d3fd420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55ed9d3fb000 0x55ed9d3fd420 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55ed9d3ead60 msgr2=0x55ed9d3f36b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55ed9d3ead60 0x55ed9d3f36b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55ed9d4002f0 msgr2=0x55ed9d402710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55ed9d4002f0 0x55ed9d402710 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 reap_dead start 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 reap_dead start 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f359da00640 1 -- 172.21.15.43:0/2875082673 reap_dead start 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 msgr2=0x7f357c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 0x7f357c0740f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f359000b5f0 tx=0x7f3590002fc0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 msgr2=0x55ed9d3e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f358800deb0 tx=0x7f358800b5c0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 shutdown_connections 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f357c071be0 0x7f357c0740f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 --2- 172.21.15.43:0/2875082673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ed9d2accb0 0x55ed9d3e8b00 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 >> 172.21.15.43:0/2875082673 conn(0x55ed9d269d80 msgr2=0x55ed9d2bf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 shutdown_connections 2024-09-17T05:31:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.635+0000 7f35a3f28080 1 -- 172.21.15.43:0/2875082673 wait complete. 2024-09-17T05:31:40.643 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-09-17T05:31:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.743+0000 7ff957a4e080 1 Processor -- start 2024-09-17T05:31:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.743+0000 7ff957a4e080 1 -- start start 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 0x55f79ec60010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 0x55f79ed148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f79eb54f40 con 0x55f79ebd7cb0 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f79eb61d00 con 0x55f79ec64da0 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f79eaf6a00 con 0x55f79eba9590 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49204/0 (socket says 172.21.15.43:49204) 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 -- 172.21.15.43:0/170302648 learned_addr learned my addr 172.21.15.43:0/170302648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:40.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff951600640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 0x55f79ed148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff950c00640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 0x55f79ec60010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 -- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 msgr2=0x55f79ec60010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 0x55f79ec60010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 -- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 msgr2=0x55f79ed148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 0x55f79ed148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 -- 172.21.15.43:0/170302648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79ec7db90 con 0x55f79ebd7cb0 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff950c00640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 0x55f79ec60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:40.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff951600640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 0x55f79ed148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:40.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94be00640 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff93c001e40 tx=0x7ff93c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5ef50e80dbac9b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94aa00640 1 -- 172.21.15.43:0/170302648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c012070 con 0x55f79ebd7cb0 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94aa00640 1 -- 172.21.15.43:0/170302648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff93c00e930 con 0x55f79ebd7cb0 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff94aa00640 1 -- 172.21.15.43:0/170302648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00edc0 con 0x55f79ebd7cb0 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- 172.21.15.43:0/170302648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 msgr2=0x55f79ebf0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff93c001e40 tx=0x7ff93c00b740 comp rx=0 tx=0).stop 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- 172.21.15.43:0/170302648 shutdown_connections 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79ec64da0 0x55f79ed148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ebd7cb0 0x55f79ebf0590 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 --2- 172.21.15.43:0/170302648 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79eba9590 0x55f79ec60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- 172.21.15.43:0/170302648 >> 172.21.15.43:0/170302648 conn(0x55f79eb94d80 msgr2=0x55f79ebaaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- 172.21.15.43:0/170302648 shutdown_connections 2024-09-17T05:31:40.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.747+0000 7ff957a4e080 1 -- 172.21.15.43:0/170302648 wait complete. 2024-09-17T05:31:40.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 Processor -- start 2024-09-17T05:31:40.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- start start 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79eba9590 0x55f79ec69260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79ebd7cb0 0x55f79ec848d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 0x55f79ec7c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f79ec7de30 con 0x55f79ec64da0 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f79ebee6c0 con 0x55f79eba9590 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f79ec7e0d0 con 0x55f79ebd7cb0 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 0x55f79ec7c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 0x55f79ec7c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49216/0 (socket says 172.21.15.43:49216) 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 -- 172.21.15.43:0/885320397 learned_addr learned my addr 172.21.15.43:0/885320397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff94be00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79ebd7cb0 0x55f79ec848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff950c00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79eba9590 0x55f79ec69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 -- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79ebd7cb0 msgr2=0x55f79ec848d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f79ebd7cb0 0x55f79ec848d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 -- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79eba9590 msgr2=0x55f79ec69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79eba9590 0x55f79ec69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 -- 172.21.15.43:0/885320397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79ec7bfe0 con 0x55f79ec64da0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff950c00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f79eba9590 0x55f79ec69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff951600640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 0x55f79ec7c530 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7ff94c004290 tx=0x7ff94c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff949600640 1 -- 172.21.15.43:0/885320397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c018070 con 0x55f79ec64da0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff949600640 1 -- 172.21.15.43:0/885320397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff94c0030d0 con 0x55f79ec64da0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff949600640 1 -- 172.21.15.43:0/885320397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0015a0 con 0x55f79ec64da0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f79ed19a10 con 0x55f79ec64da0 2024-09-17T05:31:40.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.751+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79ed19230 con 0x55f79ec64da0 2024-09-17T05:31:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff949600640 1 -- 172.21.15.43:0/885320397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff94c01b070 con 0x55f79ec64da0 2024-09-17T05:31:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff949600640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff930071be0 0x7ff9300740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff949600640 1 -- 172.21.15.43:0/885320397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7ff94c092d30 con 0x55f79ec64da0 2024-09-17T05:31:40.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff957a4e080 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f79ec7a740 0x55f79ed1c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55f79ed18a80 con 0x55f79ec7a740 2024-09-17T05:31:40.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff950c00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff930071be0 0x7ff9300740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff94be00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f79ec7a740 0x55f79ed1c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff94be00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f79ec7a740 0x55f79ed1c560 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.755+0000 7ff950c00640 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff930071be0 0x7ff9300740f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9440ab000 tx=0x7ff9440ad040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff94be00640 1 -- 172.21.15.43:0/885320397 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v104'350 uv350 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff93c014030 con 0x55f79ec7a740 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f79ec7a740 msgr2=0x55f79ed1c560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f79ec7a740 0x55f79ed1c560 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff930071be0 msgr2=0x7ff9300740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff930071be0 0x7ff9300740f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff9440ab000 tx=0x7ff9440ad040 comp rx=0 tx=0).stop 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 msgr2=0x55f79ec7c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 --2- 172.21.15.43:0/885320397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f79ec64da0 0x55f79ec7c530 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7ff94c004290 tx=0x7ff94c00d5f0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff951600640 1 -- 172.21.15.43:0/885320397 reap_dead start 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 shutdown_connections 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 >> 172.21.15.43:0/885320397 conn(0x55f79eb94d80 msgr2=0x55f79ec64160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 shutdown_connections 2024-09-17T05:31:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.759+0000 7ff957a4e080 1 -- 172.21.15.43:0/885320397 wait complete. 2024-09-17T05:31:40.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- 172.21.15.43:0/1079020610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc68006f110 msgr2=0x7fc6800ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 --2- 172.21.15.43:0/1079020610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc68006f110 0x7fc6800ad0b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc678009c80 tx=0x7fc67801cce0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- 172.21.15.43:0/1079020610 shutdown_connections 2024-09-17T05:31:40.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 --2- 172.21.15.43:0/1079020610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc68006f110 0x7fc6800ad0b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 --2- 172.21.15.43:0/1079020610 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6800b3af0 0x7fc68015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- 172.21.15.43:0/1079020610 >> 172.21.15.43:0/1079020610 conn(0x7fc6800b23d0 msgr2=0x7fc6800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:40.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- 172.21.15.43:0/1079020610 shutdown_connections 2024-09-17T05:31:40.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- 172.21.15.43:0/1079020610 wait complete. 2024-09-17T05:31:40.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 Processor -- start 2024-09-17T05:31:40.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.967+0000 7fc684c00640 1 -- start start 2024-09-17T05:31:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc68006f110 0x7fc680156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6800ccab0 0x7fc6800ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc680168be0 con 0x7fc680155d70 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc680168a60 con 0x7fc6800ccab0 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc680168d60 con 0x7fc68006f110 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49230/0 (socket says 172.21.15.43:49230) 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 -- 172.21.15.43:0/205421375 learned_addr learned my addr 172.21.15.43:0/205421375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc67e800640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6800ccab0 0x7fc6800ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc67de00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc68006f110 0x7fc680156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 -- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc68006f110 msgr2=0x7fc680156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc68006f110 0x7fc680156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 -- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6800ccab0 msgr2=0x7fc6800ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6800ccab0 0x7fc6800ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:40.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 -- 172.21.15.43:0/205421375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6800cd400 con 0x7fc680155d70 2024-09-17T05:31:40.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc677e00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d750 tx=0x7fc66800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668017070 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc668009d20 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc680160360 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668005280 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc680156f50 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.971+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6800ad0b0 con 0x7fc680155d70 2024-09-17T05:31:40.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.975+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc668005420 con 0x7fc680155d70 2024-09-17T05:31:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.975+0000 7fc676000640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 0x7fc65c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:40.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.975+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6999+0+0 (secure 0 0 0) 0x7fc668094800 con 0x7fc680155d70 2024-09-17T05:31:40.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.975+0000 7fc67de00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 0x7fc65c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:40.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.975+0000 7fc67de00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 0x7fc65c075740 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc670007fd0 tx=0x7fc670009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:40.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:40.979+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6800cd400 con 0x7fc680155d70 2024-09-17T05:31:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.187+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc6800b04a0 con 0x7fc680155d70 2024-09-17T05:31:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.359+0000 7fc676000640 1 -- 172.21.15.43:0/205421375 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v179) ==== 162+0+0 (secure 0 0 0) 0x7fc66800de60 con 0x7fc680155d70 2024-09-17T05:31:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:41.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 msgr2=0x7fc65c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 0x7fc65c075740 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc670007fd0 tx=0x7fc670009040 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 msgr2=0x7fc68009cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d750 tx=0x7fc66800dc10 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 shutdown_connections 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc65c073230 0x7fc65c075740 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6800ccab0 0x7fc6800ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc680155d70 0x7fc68009cd50 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 --2- 172.21.15.43:0/205421375 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc68006f110 0x7fc680156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 >> 172.21.15.43:0/205421375 conn(0x7fc6800b23d0 msgr2=0x7fc68015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 shutdown_connections 2024-09-17T05:31:41.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.363+0000 7fc684c00640 1 -- 172.21.15.43:0/205421375 wait complete. 2024-09-17T05:31:41.374 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:41.371+0000 7f9e6ea00640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-17T05:31:41.374 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:41.371+0000 7f9e6ea00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:31:41.375 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:41.371+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:41.375 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:41.375+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:41.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 -- 172.21.15.43:0/4140342136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d700b1db0 msgr2=0x7f3d701632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 --2- 172.21.15.43:0/4140342136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d700b1db0 0x7f3d701632b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009bc0 tx=0x7f3d6801cb10 comp rx=0 tx=0).stop 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 -- 172.21.15.43:0/4140342136 shutdown_connections 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 --2- 172.21.15.43:0/4140342136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d700b1db0 0x7f3d701632b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 --2- 172.21.15.43:0/4140342136 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d7006e390 0x7f3d7006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 -- 172.21.15.43:0/4140342136 >> 172.21.15.43:0/4140342136 conn(0x7f3d700cee80 msgr2=0x7f3d700cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 -- 172.21.15.43:0/4140342136 shutdown_connections 2024-09-17T05:31:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.547+0000 7f3d78600640 1 -- 172.21.15.43:0/4140342136 wait complete. 2024-09-17T05:31:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 Processor -- start 2024-09-17T05:31:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- start start 2024-09-17T05:31:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d700b19a0 0x7f3d701fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 0x7f3d700c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700b7680 con 0x7f3d7006e390 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700b7500 con 0x7f3d700b1db0 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3d701637f0 con 0x7f3d700b19a0 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49248/0 (socket says 172.21.15.43:49248) 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 -- 172.21.15.43:0/1303202341 learned_addr learned my addr 172.21.15.43:0/1303202341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75200640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d700b19a0 0x7f3d701fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d76600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 0x7f3d700c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 -- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d700b19a0 msgr2=0x7f3d701fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d700b19a0 0x7f3d701fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 -- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 msgr2=0x7f3d700c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 0x7f3d700c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 -- 172.21.15.43:0/1303202341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d701fc7a0 con 0x7f3d7006e390 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d76600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 0x7f3d700c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d75c00640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400d770 tx=0x7f3d6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64009c30 con 0x7f3d7006e390 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d6400de80 con 0x7f3d7006e390 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d701fca80 con 0x7f3d7006e390 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64004280 con 0x7f3d7006e390 2024-09-17T05:31:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.551+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701588e0 con 0x7f3d7006e390 2024-09-17T05:31:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7006ea70 con 0x7f3d7006e390 2024-09-17T05:31:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d64003600 con 0x7f3d7006e390 2024-09-17T05:31:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d6d600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 0x7f3d50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f3d64094d10 con 0x7f3d7006e390 2024-09-17T05:31:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d75200640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 0x7f3d50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.555+0000 7f3d75200640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 0x7f3d50075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d700b7060 tx=0x7f3d5c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:41.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.559+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d6409a020 con 0x7f3d7006e390 2024-09-17T05:31:41.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.771+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d700b5dc0 con 0x7f3d7006e390 2024-09-17T05:31:41.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.771+0000 7f3d6d600640 1 -- 172.21.15.43:0/1303202341 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+4383 (secure 0 0 0) 0x7f3d6405f7c0 con 0x7f3d7006e390 2024-09-17T05:31:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:41.774 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":181,"btime":"2024-09-17T05:31:41:376078+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2584621842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2584621842},{"type":"v1","addr":"172.21.15.62:6835","nonce":2584621842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:41.375998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 181 2024-09-17T05:31:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 msgr2=0x7f3d50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 0x7f3d50075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d700b7060 tx=0x7f3d5c009040 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 msgr2=0x7f3d700c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400d770 tx=0x7f3d6400dc30 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 shutdown_connections 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3d50073200 0x7f3d50075710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d700b1db0 0x7f3d700c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3d700b19a0 0x7f3d701fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 --2- 172.21.15.43:0/1303202341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7006e390 0x7f3d700c7d60 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 >> 172.21.15.43:0/1303202341 conn(0x7f3d700cee80 msgr2=0x7f3d700b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 shutdown_connections 2024-09-17T05:31:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.775+0000 7f3d78600640 1 -- 172.21.15.43:0/1303202341 wait complete. 2024-09-17T05:31:41.797 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:41.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- 172.21.15.43:0/4127525164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b80b3af0 msgr2=0x7f77b815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 --2- 172.21.15.43:0/4127525164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b80b3af0 0x7f77b815c7e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a001c7d0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- 172.21.15.43:0/4127525164 shutdown_connections 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 --2- 172.21.15.43:0/4127525164 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b80b3af0 0x7f77b815c7e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- 172.21.15.43:0/4127525164 >> 172.21.15.43:0/4127525164 conn(0x7f77b80b23d0 msgr2=0x7f77b80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- 172.21.15.43:0/4127525164 shutdown_connections 2024-09-17T05:31:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- 172.21.15.43:0/4127525164 wait complete. 2024-09-17T05:31:41.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 Processor -- start 2024-09-17T05:31:41.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.955+0000 7f77be800640 1 -- start start 2024-09-17T05:31:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 0x7f77b80cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b81f72c0 0x7f77b80b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8169e50 con 0x7f77b8155d70 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8169cd0 con 0x7f77b806f110 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77b8169fd0 con 0x7f77b81f72c0 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49254/0 (socket says 172.21.15.43:49254) 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 -- 172.21.15.43:0/789167566 learned_addr learned my addr 172.21.15.43:0/789167566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77b7400640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 0x7f77b80cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77b7e00640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b81f72c0 0x7f77b80b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 -- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b81f72c0 msgr2=0x7f77b80b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b81f72c0 0x7f77b80b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 -- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 msgr2=0x7f77b80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 0x7f77b80cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 -- 172.21.15.43:0/789167566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b80cdbc0 con 0x7f77b8155d70 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77b7400640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 0x7f77b80cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77afe00640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f77a001ae80 tx=0x7f77a00a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a00b2040 con 0x7f77b8155d70 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77a0004320 con 0x7f77b8155d70 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a00044f0 con 0x7f77b8155d70 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b81f6540 con 0x7f77b8155d70 2024-09-17T05:31:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b81f6ac0 con 0x7f77b8155d70 2024-09-17T05:31:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.959+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b81f6f00 con 0x7f77b8155d70 2024-09-17T05:31:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.963+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77a00052d0 con 0x7f77b8155d70 2024-09-17T05:31:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.963+0000 7f77af400640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 0x7f7794075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.963+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f77a012e260 con 0x7f77b8155d70 2024-09-17T05:31:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.963+0000 7f77b7400640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 0x7f7794075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.963+0000 7f77b7400640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 0x7f7794075710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f77b80715c0 tx=0x7f77a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:41.967+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77a0016450 con 0x7f77b8155d70 2024-09-17T05:31:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.175+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b80cd950 con 0x7f77b8155d70 2024-09-17T05:31:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.175+0000 7f77af400640 1 -- 172.21.15.43:0/789167566 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+4383 (secure 0 0 0) 0x7f77a00f8d10 con 0x7f77b8155d70 2024-09-17T05:31:42.178 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:42.179 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":181,"btime":"2024-09-17T05:31:41:376078+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2584621842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2584621842},{"type":"v1","addr":"172.21.15.62:6835","nonce":2584621842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:41.375998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 181 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 msgr2=0x7f7794075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 0x7f7794075710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f77b80715c0 tx=0x7f77a8009040 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 msgr2=0x7f77b8155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f77a001ae80 tx=0x7f77a00a5f90 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 shutdown_connections 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7794073200 0x7f7794075710 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77b81f72c0 0x7f77b80b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8155d70 0x7f77b8155270 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 --2- 172.21.15.43:0/789167566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b806f110 0x7f77b80cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 >> 172.21.15.43:0/789167566 conn(0x7f77b80b23d0 msgr2=0x7f77b80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:42.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 shutdown_connections 2024-09-17T05:31:42.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.179+0000 7f77be800640 1 -- 172.21.15.43:0/789167566 wait complete. 2024-09-17T05:31:42.200 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:42.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 -- 172.21.15.43:0/3948078761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc155d70 msgr2=0x7f63dc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 --2- 172.21.15.43:0/3948078761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc155d70 0x7f63dc15c7e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0af0a0 tx=0x7f63cc01c800 comp rx=0 tx=0).stop 2024-09-17T05:31:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 -- 172.21.15.43:0/3948078761 shutdown_connections 2024-09-17T05:31:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 --2- 172.21.15.43:0/3948078761 >> v2:172.21.15.62:3300/0 conn(0x7f63dc06f110 0x7f63dc0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 --2- 172.21.15.43:0/3948078761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc155d70 0x7f63dc15c7e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 -- 172.21.15.43:0/3948078761 >> 172.21.15.43:0/3948078761 conn(0x7f63dc0b23d0 msgr2=0x7f63dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:42.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 -- 172.21.15.43:0/3948078761 shutdown_connections 2024-09-17T05:31:42.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 -- 172.21.15.43:0/3948078761 wait complete. 2024-09-17T05:31:42.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.359+0000 7f63e2e00640 1 Processor -- start 2024-09-17T05:31:42.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 -- start start 2024-09-17T05:31:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc06f110 0x7f63dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 0x7f63dc156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc168a30 con 0x7f63dc0b3920 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1688b0 con 0x7f63dc06f110 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e2e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc168bb0 con 0x7f63dc155d70 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc06f110 0x7f63dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49276/0 (socket says 172.21.15.43:49276) 2024-09-17T05:31:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 -- 172.21.15.43:0/2644382451 learned_addr learned my addr 172.21.15.43:0/2644382451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e0e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 0x7f63dc156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 -- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 msgr2=0x7f63dc156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 0x7f63dc156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 -- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc06f110 msgr2=0x7f63dc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc06f110 0x7f63dc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 -- 172.21.15.43:0/2644382451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc1f86b0 con 0x7f63dc0b3920 2024-09-17T05:31:42.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63e0e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 0x7f63dc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.363+0000 7f63d3e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f63c400e7c0 tx=0x7f63c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c400cc60 con 0x7f63dc0b3920 2024-09-17T05:31:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63c400ce00 con 0x7f63dc0b3920 2024-09-17T05:31:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4005280 con 0x7f63dc0b3920 2024-09-17T05:31:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc0cd020 con 0x7f63dc0b3920 2024-09-17T05:31:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc0cd5e0 con 0x7f63dc0b3920 2024-09-17T05:31:42.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc0b33e0 con 0x7f63dc0b3920 2024-09-17T05:31:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.367+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f63c4005420 con 0x7f63dc0b3920 2024-09-17T05:31:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.371+0000 7f63d8c00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 0x7f63b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.371+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f63c4095100 con 0x7f63dc0b3920 2024-09-17T05:31:42.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.371+0000 7f63dbe00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 0x7f63b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.371+0000 7f63dbe00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 0x7f63b8075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0af000 tx=0x7f63cc0a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.375+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f63c405fba0 con 0x7f63dc0b3920 2024-09-17T05:31:42.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.587+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63dc0ccdb0 con 0x7f63dc0b3920 2024-09-17T05:31:42.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.587+0000 7f63d8c00640 1 -- 172.21.15.43:0/2644382451 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+4383 (secure 0 0 0) 0x7f63c405f4a0 con 0x7f63dc0b3920 2024-09-17T05:31:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:42.591 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":181,"btime":"2024-09-17T05:31:41:376078+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8050,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2584621842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2584621842},{"type":"v1","addr":"172.21.15.62:6835","nonce":2584621842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:41.375998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:42.591 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 181 2024-09-17T05:31:42.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 msgr2=0x7f63b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 0x7f63b8075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0af000 tx=0x7f63cc0a7040 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 msgr2=0x7f63dc09d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f63c400e7c0 tx=0x7f63c400ec80 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 shutdown_connections 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63b8073200 0x7f63b8075710 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f63dc155d70 0x7f63dc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63dc0b3920 0x7f63dc09d840 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 --2- 172.21.15.43:0/2644382451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63dc06f110 0x7f63dc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 >> 172.21.15.43:0/2644382451 conn(0x7f63dc0b23d0 msgr2=0x7f63dc15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 shutdown_connections 2024-09-17T05:31:42.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.591+0000 7f63e2e00640 1 -- 172.21.15.43:0/2644382451 wait complete. 2024-09-17T05:31:42.614 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-09-17T05:31:42.614 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-09-17T05:31:42.615 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:42.615 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:42.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:42.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:42.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:42.624 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:42.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:42.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 -- 172.21.15.43:0/4197143386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c15fe10 msgr2=0x7f067c0b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 --2- 172.21.15.43:0/4197143386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c15fe10 0x7f067c0b4b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f066c009900 tx=0x7f066c01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 -- 172.21.15.43:0/4197143386 shutdown_connections 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 --2- 172.21.15.43:0/4197143386 >> v2:172.21.15.62:3300/0 conn(0x7f067c0737c0 0x7f067c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 --2- 172.21.15.43:0/4197143386 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c15fe10 0x7f067c0b4b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 -- 172.21.15.43:0/4197143386 >> 172.21.15.43:0/4197143386 conn(0x7f067c0b23d0 msgr2=0x7f067c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:42.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 -- 172.21.15.43:0/4197143386 shutdown_connections 2024-09-17T05:31:42.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.875+0000 7f0682c00640 1 -- 172.21.15.43:0/4197143386 wait complete. 2024-09-17T05:31:42.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 Processor -- start 2024-09-17T05:31:42.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- start start 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c06fc90 0x7f067c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 0x7f067c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f067c169fa0 con 0x7f067c15fe10 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f067c169e20 con 0x7f067c15b0e0 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f067c16a120 con 0x7f067c06fc90 2024-09-17T05:31:42.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49284/0 (socket says 172.21.15.43:49284) 2024-09-17T05:31:42.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 -- 172.21.15.43:0/891596290 learned_addr learned my addr 172.21.15.43:0/891596290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:42.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067be00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c06fc90 0x7f067c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0680c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 0x7f067c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 -- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c06fc90 msgr2=0x7f067c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c06fc90 0x7f067c155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 -- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 msgr2=0x7f067c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 0x7f067c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 -- 172.21.15.43:0/891596290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c0b1430 con 0x7f067c15fe10 2024-09-17T05:31:42.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0680c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 0x7f067c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f067b400640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f066800c3c0 tx=0x7f066800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668017070 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06680053c0 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c0b1710 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668003860 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.879+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c15d650 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.883+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c0b11a0 con 0x7f067c15fe10 2024-09-17T05:31:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.883+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f066801a070 con 0x7f067c15fe10 2024-09-17T05:31:42.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.883+0000 7f0673e00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 0x7f065c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:42.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.883+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f0668093a40 con 0x7f067c15fe10 2024-09-17T05:31:42.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.883+0000 7f067be00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 0x7f065c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:42.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.887+0000 7f067be00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 0x7f065c075710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c01b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:42.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:42.887+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f066805dea0 con 0x7f067c15fe10 2024-09-17T05:31:43.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.095+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f067c0700a0 con 0x7f067c15fe10 2024-09-17T05:31:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.379+0000 7f0673e00640 1 -- 172.21.15.43:0/891596290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v182) ==== 161+0+0 (secure 0 0 0) 0x7f06680628d0 con 0x7f067c15fe10 2024-09-17T05:31:43.381 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 msgr2=0x7f065c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 0x7f065c075710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c01b000 comp rx=0 tx=0).stop 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 msgr2=0x7f067c153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f066800c3c0 tx=0x7f066800c880 comp rx=0 tx=0).stop 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 shutdown_connections 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f065c073200 0x7f065c075710 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f067c15b0e0 0x7f067c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f067c15fe10 0x7f067c153650 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 --2- 172.21.15.43:0/891596290 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f067c06fc90 0x7f067c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 >> 172.21.15.43:0/891596290 conn(0x7f067c0b23d0 msgr2=0x7f067c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 shutdown_connections 2024-09-17T05:31:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.383+0000 7f0682c00640 1 -- 172.21.15.43:0/891596290 wait complete. 2024-09-17T05:31:43.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 -- 172.21.15.43:0/1295449097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d8155d70 msgr2=0x7fb3d815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 --2- 172.21.15.43:0/1295449097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d8155d70 0x7fb3d815c7e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80af0a0 tx=0x7fb3c801c800 comp rx=0 tx=0).stop 2024-09-17T05:31:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 -- 172.21.15.43:0/1295449097 shutdown_connections 2024-09-17T05:31:43.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 --2- 172.21.15.43:0/1295449097 >> v2:172.21.15.62:3300/0 conn(0x7fb3d806f110 0x7fb3d80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 --2- 172.21.15.43:0/1295449097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d8155d70 0x7fb3d815c7e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 -- 172.21.15.43:0/1295449097 >> 172.21.15.43:0/1295449097 conn(0x7fb3d80b23d0 msgr2=0x7fb3d80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 -- 172.21.15.43:0/1295449097 shutdown_connections 2024-09-17T05:31:43.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.563+0000 7fb3df200640 1 -- 172.21.15.43:0/1295449097 wait complete. 2024-09-17T05:31:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 Processor -- start 2024-09-17T05:31:43.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- start start 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 0x7fb3d81f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8155d70 0x7fb3d80cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8168a30 con 0x7fb3d806f110 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81688b0 con 0x7fb3d80b3920 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d8168bb0 con 0x7fb3d8155d70 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:43.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49302/0 (socket says 172.21.15.43:49302) 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 -- 172.21.15.43:0/3372289209 learned_addr learned my addr 172.21.15.43:0/3372289209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7400640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 0x7fb3d81f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3dd200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8155d70 0x7fb3d80cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 -- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8155d70 msgr2=0x7fb3d80cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8155d70 0x7fb3d80cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 -- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 msgr2=0x7fb3d81f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 0x7fb3d81f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 -- 172.21.15.43:0/3372289209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81608b0 con 0x7fb3d806f110 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7400640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 0x7fb3d81f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3d7e00640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b0040 tx=0x7fb3c80a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80b2040 con 0x7fb3d806f110 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3c80043b0 con 0x7fb3d806f110 2024-09-17T05:31:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8005290 con 0x7fb3d806f110 2024-09-17T05:31:43.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d8160b90 con 0x7fb3d806f110 2024-09-17T05:31:43.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.567+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d8161150 con 0x7fb3d806f110 2024-09-17T05:31:43.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.571+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d80b33e0 con 0x7fb3d806f110 2024-09-17T05:31:43.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.571+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb3c8005430 con 0x7fb3d806f110 2024-09-17T05:31:43.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.571+0000 7fb3cfe00640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 0x7fb3b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:43.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.571+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7fb3c812dcf0 con 0x7fb3d806f110 2024-09-17T05:31:43.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.575+0000 7fb3d7400640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 0x7fb3b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:43.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.575+0000 7fb3d7400640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 0x7fb3b8075740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8166380 tx=0x7fb3c40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:43.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.579+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb3c80164b0 con 0x7fb3d806f110 2024-09-17T05:31:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:43.811+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb3d80b0db0 con 0x7fb3d806f110 2024-09-17T05:31:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.391+0000 7fb3cfe00640 1 -- 172.21.15.43:0/3372289209 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v183) ==== 101+0+0 (secure 0 0 0) 0x7fb3c80f8720 con 0x7fb3d806f110 2024-09-17T05:31:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 msgr2=0x7fb3b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 0x7fb3b8075740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8166380 tx=0x7fb3c40091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 msgr2=0x7fb3d81fa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80b0040 tx=0x7fb3c80a5740 comp rx=0 tx=0).stop 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 shutdown_connections 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb3b8073230 0x7fb3b8075740 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3d80b3920 0x7fb3d81f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb3d8155d70 0x7fb3d80cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 --2- 172.21.15.43:0/3372289209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3d806f110 0x7fb3d81fa9f0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 >> 172.21.15.43:0/3372289209 conn(0x7fb3d80b23d0 msgr2=0x7fb3d80b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 shutdown_connections 2024-09-17T05:31:44.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.395+0000 7fb3df200640 1 -- 172.21.15.43:0/3372289209 wait complete. 2024-09-17T05:31:44.420 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 Processor -- start 2024-09-17T05:31:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 -- start start 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d255d70 0x56124d311630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d284cb0 0x56124d297ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56124d1a3a00 con 0x56124d30be40 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56124d201f40 con 0x56124d284cb0 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56124d30cb30 con 0x56124d255d70 2024-09-17T05:31:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f3e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f3e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54482/0 (socket says 172.21.15.43:54482) 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f3e600640 1 -- 172.21.15.43:0/3912043735 learned_addr learned my addr 172.21.15.43:0/3912043735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d255d70 0x56124d311630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.479+0000 7f5f3d200640 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d284cb0 0x56124d297ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 -- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d255d70 msgr2=0x56124d311630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d255d70 0x56124d311630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 -- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d284cb0 msgr2=0x56124d297ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d284cb0 0x56124d297ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 -- 172.21.15.43:0/3912043735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124d17cc10 con 0x56124d30be40 2024-09-17T05:31:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f3e600640 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800d510 tx=0x7f5f3800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6a1372b49dd604 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f37400640 1 -- 172.21.15.43:0/3912043735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800f8a0 con 0x56124d30be40 2024-09-17T05:31:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f37400640 1 -- 172.21.15.43:0/3912043735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f38004120 con 0x56124d30be40 2024-09-17T05:31:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f37400640 1 -- 172.21.15.43:0/3912043735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800fdb0 con 0x56124d30be40 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- 172.21.15.43:0/3912043735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 msgr2=0x56124d298010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800d510 tx=0x7f5f3800d9d0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- 172.21.15.43:0/3912043735 shutdown_connections 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d284cb0 0x56124d297ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d30be40 0x56124d298010 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/3912043735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d255d70 0x56124d311630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- 172.21.15.43:0/3912043735 >> 172.21.15.43:0/3912043735 conn(0x56124d241d80 msgr2=0x56124d29c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- 172.21.15.43:0/3912043735 shutdown_connections 2024-09-17T05:31:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- 172.21.15.43:0/3912043735 wait complete. 2024-09-17T05:31:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 Processor -- start 2024-09-17T05:31:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.483+0000 7f5f44b2f080 1 -- start start 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d255d70 0x56124d32ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d30be40 0x56124d30e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56124d32b550 con 0x56124d284cb0 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56124d2970f0 con 0x56124d255d70 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56124d297390 con 0x56124d30be40 2024-09-17T05:31:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54486/0 (socket says 172.21.15.43:54486) 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 learned_addr learned my addr 172.21.15.43:0/1676956257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d255d70 0x56124d32ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d30be40 0x56124d30e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d30be40 msgr2=0x56124d30e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d30be40 0x56124d30e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d255d70 msgr2=0x56124d32ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d255d70 0x56124d32ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124d29b1e0 con 0x56124d284cb0 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56124d255d70 0x56124d32ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56124d30be40 0x56124d30e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a550 tx=0x7f5f2c0a76f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f36000640 1 -- 172.21.15.43:0/1676956257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0b0880 con 0x56124d284cb0 2024-09-17T05:31:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f36000640 1 -- 172.21.15.43:0/1676956257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f2c0c1070 con 0x56124d284cb0 2024-09-17T05:31:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56124d29af40 con 0x56124d284cb0 2024-09-17T05:31:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f36000640 1 -- 172.21.15.43:0/1676956257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0b0cc0 con 0x56124d284cb0 2024-09-17T05:31:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124d298a80 con 0x56124d284cb0 2024-09-17T05:31:44.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.487+0000 7f5f36000640 1 -- 172.21.15.43:0/1676956257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f2c0bb070 con 0x56124d284cb0 2024-09-17T05:31:44.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f36000640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 0x7f5f200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f36000640 1 -- 172.21.15.43:0/1676956257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f5f2c1316c0 con 0x56124d284cb0 2024-09-17T05:31:44.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 0x7f5f200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.493 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56124d329fe0 0x56124d298f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d299460 con 0x56124d329fe0 2024-09-17T05:31:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56124d329fe0 0x56124d298f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 0x7f5f200740f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b5c0 tx=0x7f5f3000e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.491+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56124d329fe0 0x56124d298f20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.495+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v105'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.495+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3ca660 con 0x56124d329fe0 2024-09-17T05:31:44.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.495+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v105'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.495+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3caa00 con 0x56124d329fe0 2024-09-17T05:31:44.499 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:44.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v105'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cbdb0 con 0x56124d329fe0 2024-09-17T05:31:44.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cc150 con 0x56124d329fe0 2024-09-17T05:31:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.499+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cada0 con 0x56124d329fe0 2024-09-17T05:31:44.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb190 con 0x56124d329fe0 2024-09-17T05:31:44.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v105'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v105'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.503+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.507+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v105'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:44.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.507+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.507+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.507+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.511+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.515+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.515+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.515+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.518 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.515+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.519+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.523+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.523+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.523+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v105'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.523+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v105'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.529 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:44.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v105'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.527+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v105'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.531+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.535+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v105'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.535+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.538 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.535+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v105'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.535+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cb530 con 0x56124d329fe0 2024-09-17T05:31:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56124d3cc4f0 0x56124d3ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.539+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cee50 con 0x56124d3cc4f0 2024-09-17T05:31:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.543+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56124d3cc4f0 0x56124d3ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.543+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56124d3cc4f0 0x56124d3ce910 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.543+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'351 uv350 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.543+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3cf1f0 con 0x56124d3cc4f0 2024-09-17T05:31:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'352 uv352 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cf590 con 0x56124d3cc4f0 2024-09-17T05:31:44.549 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3cfde0 con 0x56124d3cc4f0 2024-09-17T05:31:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.547+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0180 con 0x56124d3cc4f0 2024-09-17T05:31:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0520 con 0x56124d3cc4f0 2024-09-17T05:31:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v105'356 uv338 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d08c0 con 0x56124d3cc4f0 2024-09-17T05:31:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v105'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.551+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v105'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.556 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:44.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.555+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v105'361 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v105'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v105'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.563 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.559+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d4e30 con 0x56124d3cc4f0 2024-09-17T05:31:44.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.563+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v105'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.563+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d4e30 con 0x56124d3cc4f0 2024-09-17T05:31:44.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.563+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.563+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v105'366 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v105'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v105'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.571 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:44.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.567+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d1c70 con 0x56124d3cc4f0 2024-09-17T05:31:44.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d1c70 con 0x56124d3cc4f0 2024-09-17T05:31:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d0c60 con 0x56124d3cc4f0 2024-09-17T05:31:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3cc4f0 2024-09-17T05:31:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56124d3d1400 0x56124d3d3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d3e40 con 0x56124d3d1400 2024-09-17T05:31:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.571+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56124d3d1400 0x56124d3d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.575+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56124d3d1400 0x56124d3d3820 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.575+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v105'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.575+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d4330 con 0x56124d3d1400 2024-09-17T05:31:44.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.575+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v105'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.575+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d4810 con 0x56124d3d1400 2024-09-17T05:31:44.580 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d60c0 con 0x56124d3d1400 2024-09-17T05:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d6460 con 0x56124d3d1400 2024-09-17T05:31:44.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.579+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d50b0 con 0x56124d3d1400 2024-09-17T05:31:44.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d329fe0 2024-09-17T05:31:44.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'69 uv68 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d329fe0 2024-09-17T05:31:44.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.583+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d329fe0 2024-09-17T05:31:44.588 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d329fe0 2024-09-17T05:31:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'72 uv72 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56124d3d6800 0x56124d3d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d3d6800 2024-09-17T05:31:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56124d3d6800 0x56124d3d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.587+0000 7f5f3e600640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56124d3d6800 0x56124d3d8c20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v105'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d3d6800 2024-09-17T05:31:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v105'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d54a0 con 0x56124d3d6800 2024-09-17T05:31:44.595 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.591+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3da170 con 0x56124d3d6800 2024-09-17T05:31:44.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3da170 con 0x56124d3d6800 2024-09-17T05:31:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3d9160 con 0x56124d3d6800 2024-09-17T05:31:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56124d3d9670 0x56124d3dba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.595+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc100 con 0x56124d3d9670 2024-09-17T05:31:44.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.599+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56124d3d9670 0x56124d3dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.599+0000 7f5f3dc00640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56124d3d9670 0x56124d3dba90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.599+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v105'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.599+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.599+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v105'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.603+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.605 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:44.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.603+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v105'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.603+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dd500 con 0x56124d3d9670 2024-09-17T05:31:44.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.603+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.603+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dd500 con 0x56124d3d9670 2024-09-17T05:31:44.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v105'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.607+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v105'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.613 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:44.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v105'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.611+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dc4f0 con 0x56124d3d9670 2024-09-17T05:31:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f3dc00640 1 -- 172.21.15.43:0/1676956257 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f30008ae0 con 0x56124d3d9670 2024-09-17T05:31:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56124d3dcb70 0x56124d3def90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3df6e0 con 0x56124d3dcb70 2024-09-17T05:31:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56124d3dcb70 0x56124d3def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f3d200640 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56124d3dcb70 0x56124d3def90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v105'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3df6e0 con 0x56124d3dcb70 2024-09-17T05:31:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v105'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.615+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3dcb70 2024-09-17T05:31:44.620 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:44.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v105'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3dcb70 2024-09-17T05:31:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v105'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3dcb70 2024-09-17T05:31:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v105'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.619+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d329fe0 2024-09-17T05:31:44.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v105'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d329fe0 2024-09-17T05:31:44.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v105'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d329fe0 2024-09-17T05:31:44.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.627 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:44.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.623+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3e0b90 con 0x56124d329fe0 2024-09-17T05:31:44.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3e0b90 con 0x56124d329fe0 2024-09-17T05:31:44.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d329fe0 2024-09-17T05:31:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d329fe0 2024-09-17T05:31:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.627+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v105'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v105'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v105'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.635 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.631+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v105'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v105'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v105'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.635+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.639+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v105'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.639+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.639+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v105'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.642 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.639+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dfb80 con 0x56124d3d1400 2024-09-17T05:31:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v105'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dffa0 con 0x56124d3d1400 2024-09-17T05:31:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v105'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3d1400 2024-09-17T05:31:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v105'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.643+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.647+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v105'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.647+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.650 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:44.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.647+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v105'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.647+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v105'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v105'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v105'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.651+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.656 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:44.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.655+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3d6800 2024-09-17T05:31:44.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.659+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f3800f6a0 con 0x56124d3d6800 2024-09-17T05:31:44.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.659+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.659+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v105'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.659+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v105'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.664 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:44.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v105'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v105'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.663+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v105'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v105'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v105'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.667+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.672 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v105'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v105'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v105'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.671+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v105'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v105'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.678 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v105'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.675+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v105'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v105'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3dff40 con 0x56124d3dcb70 2024-09-17T05:31:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v105'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.679+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56124d3e0330 con 0x56124d3dcb70 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f3d200640 1 -- 172.21.15.43:0/1676956257 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v105'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5f2c0f7d70 con 0x56124d3dcb70 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56124d3cc4f0 msgr2=0x56124d3ce910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56124d3cc4f0 0x56124d3ce910 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56124d3d9670 msgr2=0x56124d3dba90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56124d3d9670 0x56124d3dba90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56124d3dcb70 msgr2=0x56124d3def90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56124d3dcb70 0x56124d3def90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56124d3d1400 msgr2=0x56124d3d3820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56124d3d1400 0x56124d3d3820 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56124d329fe0 msgr2=0x56124d298f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56124d329fe0 0x56124d298f20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56124d3d6800 msgr2=0x56124d3d8c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56124d3d6800 0x56124d3d8c20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 reap_dead start 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f3e600640 1 -- 172.21.15.43:0/1676956257 reap_dead start 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 msgr2=0x7f5f200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 0x7f5f200740f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b5c0 tx=0x7f5f3000e040 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 msgr2=0x56124d3279d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a550 tx=0x7f5f2c0a76f0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 shutdown_connections 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f20071be0 0x7f5f200740f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 --2- 172.21.15.43:0/1676956257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56124d284cb0 0x56124d3279d0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 >> 172.21.15.43:0/1676956257 conn(0x56124d241d80 msgr2=0x56124d30ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 shutdown_connections 2024-09-17T05:31:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.683+0000 7f5f44b2f080 1 -- 172.21.15.43:0/1676956257 wait complete. 2024-09-17T05:31:44.690 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-09-17T05:31:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.787+0000 7fdae2c89080 1 Processor -- start 2024-09-17T05:31:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.787+0000 7fdae2c89080 1 -- start start 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac0002300 0x564f2e076740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e097290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f2df6cf40 con 0x564f2e009480 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564f2df79d00 con 0x7fdac0002300 2024-09-17T05:31:44.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564f2df0ea00 con 0x564f2e096e80 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54512/0 (socket says 172.21.15.43:54512) 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 -- 172.21.15.43:0/201109718 learned_addr learned my addr 172.21.15.43:0/201109718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadbe00640 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac0002300 0x564f2e076740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadb400640 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e097290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 -- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 msgr2=0x564f2e097290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e097290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 -- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac0002300 msgr2=0x564f2e076740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac0002300 0x564f2e076740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 -- 172.21.15.43:0/201109718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2e001220 con 0x564f2e009480 2024-09-17T05:31:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdadc800640 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fdad4001340 tx=0x7fdad4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd6701f475dec68 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdada000640 1 -- 172.21.15.43:0/201109718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4004040 con 0x564f2e009480 2024-09-17T05:31:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdada000640 1 -- 172.21.15.43:0/201109718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdad4009070 con 0x564f2e009480 2024-09-17T05:31:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdada000640 1 -- 172.21.15.43:0/201109718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007590 con 0x564f2e009480 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- 172.21.15.43:0/201109718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 msgr2=0x564f2e009890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fdad4001340 tx=0x7fdad4009660 comp rx=0 tx=0).stop 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- 172.21.15.43:0/201109718 shutdown_connections 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f2e009480 0x564f2e009890 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e097290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 --2- 172.21.15.43:0/201109718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac0002300 0x564f2e076740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- 172.21.15.43:0/201109718 >> 172.21.15.43:0/201109718 conn(0x564f2dfacd80 msgr2=0x564f2e0067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- 172.21.15.43:0/201109718 shutdown_connections 2024-09-17T05:31:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.791+0000 7fdae2c89080 1 -- 172.21.15.43:0/201109718 wait complete. 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 Processor -- start 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- start start 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2e009480 0x564f2e0942d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e08fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 0x564f2e006580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564f2e001760 con 0x7fdac0002300 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564f2e0014c0 con 0x564f2e009480 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564f2dfc0be0 con 0x564f2e096e80 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 0x564f2e006580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2e009480 0x564f2e0942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 0x564f2e006580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54514/0 (socket says 172.21.15.43:54514) 2024-09-17T05:31:44.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 -- 172.21.15.43:0/3768257481 learned_addr learned my addr 172.21.15.43:0/3768257481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadb400640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e08fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 -- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 msgr2=0x564f2e08fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564f2e096e80 0x564f2e08fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 -- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2e009480 msgr2=0x564f2e0942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2e009480 0x564f2e0942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 -- 172.21.15.43:0/3768257481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2e093e90 con 0x7fdac0002300 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadbe00640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564f2e009480 0x564f2e0942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdadc800640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 0x564f2e006580 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x564f2dfc39f0 tx=0x7fdad4016ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdad8c00640 1 -- 172.21.15.43:0/3768257481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007590 con 0x7fdac0002300 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdad8c00640 1 -- 172.21.15.43:0/3768257481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdad401e070 con 0x7fdac0002300 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdad8c00640 1 -- 172.21.15.43:0/3768257481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40079f0 con 0x7fdac0002300 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564f2e08ee90 con 0x7fdac0002300 2024-09-17T05:31:44.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.795+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2e08f3d0 con 0x7fdac0002300 2024-09-17T05:31:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdad8c00640 1 -- 172.21.15.43:0/3768257481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdad4007b90 con 0x7fdac0002300 2024-09-17T05:31:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdad8c00640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdabc071be0 0x7fdabc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdad8c00640 1 -- 172.21.15.43:0/3768257481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7fdad4093c30 con 0x7fdac0002300 2024-09-17T05:31:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdadbe00640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdabc071be0 0x7fdabc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdae2c89080 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564f2e092770 0x564f2e092b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x564f2e006ac0 con 0x564f2e092770 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdadb400640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564f2e092770 0x564f2e092b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdadbe00640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdabc071be0 0x7fdabc0740f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0ab000 tx=0x7fdacc017fc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.799+0000 7fdadb400640 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564f2e092770 0x564f2e092b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdadb400640 1 -- 172.21.15.43:0/3768257481 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v105'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdac800cb40 con 0x564f2e092770 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564f2e092770 msgr2=0x564f2e092b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564f2e092770 0x564f2e092b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdabc071be0 msgr2=0x7fdabc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdabc071be0 0x7fdabc0740f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0ab000 tx=0x7fdacc017fc0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 msgr2=0x564f2e006580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 --2- 172.21.15.43:0/3768257481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdac0002300 0x564f2e006580 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x564f2dfc39f0 tx=0x7fdad4016ed0 comp rx=0 tx=0).stop 2024-09-17T05:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdadc800640 1 -- 172.21.15.43:0/3768257481 reap_dead start 2024-09-17T05:31:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 shutdown_connections 2024-09-17T05:31:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 >> 172.21.15.43:0/3768257481 conn(0x564f2dfacd80 msgr2=0x564f2e009890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 shutdown_connections 2024-09-17T05:31:44.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:44.803+0000 7fdae2c89080 1 -- 172.21.15.43:0/3768257481 wait complete. 2024-09-17T05:31:44.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:45.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- 172.21.15.43:0/118228471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 msgr2=0x7f407815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 --2- 172.21.15.43:0/118228471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f407815c7e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f406c009840 tx=0x7f406c01c790 comp rx=0 tx=0).stop 2024-09-17T05:31:45.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- 172.21.15.43:0/118228471 shutdown_connections 2024-09-17T05:31:45.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 --2- 172.21.15.43:0/118228471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f407815c7e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- 172.21.15.43:0/118228471 >> 172.21.15.43:0/118228471 conn(0x7f40780b23d0 msgr2=0x7f40780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:45.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- 172.21.15.43:0/118228471 shutdown_connections 2024-09-17T05:31:45.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- 172.21.15.43:0/118228471 wait complete. 2024-09-17T05:31:45.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 Processor -- start 2024-09-17T05:31:45.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.011+0000 7f407fa00640 1 -- start start 2024-09-17T05:31:45.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f407806f110 0x7f407809bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781f9de0 0x7f40780cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4078169d70 con 0x7f4078155d70 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4078169bf0 con 0x7f40781f9de0 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4078169ef0 con 0x7f407806f110 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54534/0 (socket says 172.21.15.43:54534) 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 -- 172.21.15.43:0/835262076 learned_addr learned my addr 172.21.15.43:0/835262076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407da00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781f9de0 0x7f40780cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407d000640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f407806f110 0x7f407809bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 -- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f407806f110 msgr2=0x7f407809bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f407806f110 0x7f407809bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 -- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781f9de0 msgr2=0x7f40780cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781f9de0 0x7f40780cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 -- 172.21.15.43:0/835262076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40780c46b0 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4077e00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f406400c400 tx=0x7f406400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064011850 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4064017070 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064011da0 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40780cd0f0 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40780cda10 con 0x7f4078155d70 2024-09-17T05:31:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.015+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407815c7e0 con 0x7f4078155d70 2024-09-17T05:31:45.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.019+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f406401c460 con 0x7f4078155d70 2024-09-17T05:31:45.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.019+0000 7f4074c00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 0x7f4058075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.019+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7085+0+0 (secure 0 0 0) 0x7f40640945d0 con 0x7f4078155d70 2024-09-17T05:31:45.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.019+0000 7f407d000640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 0x7f4058075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.019+0000 7f407d000640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 0x7f4058075710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f406c0af000 tx=0x7f406c01afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:45.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.023+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f406405f100 con 0x7f4078155d70 2024-09-17T05:31:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.231+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f40780ad150 con 0x7f4078155d70 2024-09-17T05:31:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f4074c00640 1 -- 172.21.15.43:0/835262076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v184) ==== 162+0+0 (secure 0 0 0) 0x7f406405ea00 con 0x7f4078155d70 2024-09-17T05:31:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 msgr2=0x7f4058075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 0x7f4058075710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f406c0af000 tx=0x7f406c01afd0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 msgr2=0x7f40781f98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f406400c400 tx=0x7f406400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 shutdown_connections 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4058073200 0x7f4058075710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40781f9de0 0x7f40780cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4078155d70 0x7f40781f98a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 --2- 172.21.15.43:0/835262076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f407806f110 0x7f407809bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.399+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 >> 172.21.15.43:0/835262076 conn(0x7f40780b23d0 msgr2=0x7f407815d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.403+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 shutdown_connections 2024-09-17T05:31:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.403+0000 7f407fa00640 1 -- 172.21.15.43:0/835262076 wait complete. 2024-09-17T05:31:45.413 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:45.412+0000 7fdebc200640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-17T05:31:45.414 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:45.411+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:45.414 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:45.411+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:45.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 -- 172.21.15.43:0/159597450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80b4660 msgr2=0x7f8bd80736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 --2- 172.21.15.43:0/159597450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80b4660 0x7f8bd80736a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80b3980 tx=0x7f8bc401c700 comp rx=0 tx=0).stop 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 -- 172.21.15.43:0/159597450 shutdown_connections 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 --2- 172.21.15.43:0/159597450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80b4660 0x7f8bd80736a0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 --2- 172.21.15.43:0/159597450 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd80b4250 0x7f8bd815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 -- 172.21.15.43:0/159597450 >> 172.21.15.43:0/159597450 conn(0x7f8bd80d0870 msgr2=0x7f8bd80d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 -- 172.21.15.43:0/159597450 shutdown_connections 2024-09-17T05:31:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.583+0000 7f8bdce00640 1 -- 172.21.15.43:0/159597450 wait complete. 2024-09-17T05:31:45.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 Processor -- start 2024-09-17T05:31:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 -- start start 2024-09-17T05:31:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd80b4250 0x7f8bd8153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd815b540 0x7f8bd81473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8166050 con 0x7f8bd8159870 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8165ed0 con 0x7f8bd80b4250 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bdce00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd81661d0 con 0x7f8bd815b540 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54560/0 (socket says 172.21.15.43:54560) 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 -- 172.21.15.43:0/252426111 learned_addr learned my addr 172.21.15.43:0/252426111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7400640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd80b4250 0x7f8bd8153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd6a00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd815b540 0x7f8bd81473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 -- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd815b540 msgr2=0x7f8bd81473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd815b540 0x7f8bd81473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 -- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd80b4250 msgr2=0x7f8bd8153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd80b4250 0x7f8bd8153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.587+0000 7f8bd7e00640 1 -- 172.21.15.43:0/252426111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd815a1c0 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bd7e00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000cb10 tx=0x7f8bd000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0016020 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd000ce50 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000ae00 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd8147930 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81fc4c0 con 0x7f8bd8159870 2024-09-17T05:31:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd80b4660 con 0x7f8bd8159870 2024-09-17T05:31:45.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8bd0019020 con 0x7f8bd8159870 2024-09-17T05:31:45.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.591+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7f8bd0094a00 con 0x7f8bd8159870 2024-09-17T05:31:45.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.595+0000 7f8bd7400640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.595+0000 7f8bd7400640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 0x7f8bb8075710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bb50 tx=0x7f8bc800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:45.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.595+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8bd0020020 con 0x7f8bd8159870 2024-09-17T05:31:45.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd80739c0 con 0x7f8bd8159870 2024-09-17T05:31:45.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bcf400640 1 -- 172.21.15.43:0/252426111 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) ==== 95+0+3557 (secure 0 0 0) 0x7f8bd815a1c0 con 0x7f8bd8159870 2024-09-17T05:31:45.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:45.809 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":186,"btime":"2024-09-17T05:31:45:414649+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:45.414587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:45.809 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 186 2024-09-17T05:31:45.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 msgr2=0x7f8bb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 0x7f8bb8075710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bb50 tx=0x7f8bc800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 msgr2=0x7f8bd8159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000cb10 tx=0x7f8bd000cc00 comp rx=0 tx=0).stop 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 shutdown_connections 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd8159870 0x7f8bd8159c80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8bd815b540 0x7f8bd81473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 --2- 172.21.15.43:0/252426111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd80b4250 0x7f8bd8153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 >> 172.21.15.43:0/252426111 conn(0x7f8bd80d0870 msgr2=0x7f8bd815e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.807+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 shutdown_connections 2024-09-17T05:31:45.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.811+0000 7f8bdce00640 1 -- 172.21.15.43:0/252426111 wait complete. 2024-09-17T05:31:45.831 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:45.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 -- 172.21.15.43:0/3650520583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3b80040f0 msgr2=0x7fa3b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 --2- 172.21.15.43:0/3650520583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3b80040f0 0x7fa3b8004810 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4074250 tx=0x7fa3c001c750 comp rx=0 tx=0).stop 2024-09-17T05:31:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3d8c00640 1 -- 172.21.15.43:0/3650520583 reap_dead start 2024-09-17T05:31:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 -- 172.21.15.43:0/3650520583 shutdown_connections 2024-09-17T05:31:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 -- 172.21.15.43:0/3650520583 >> 172.21.15.43:0/3650520583 conn(0x7fa3d40cee80 msgr2=0x7fa3d40cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 -- 172.21.15.43:0/3650520583 shutdown_connections 2024-09-17T05:31:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.987+0000 7fa3dac00640 1 -- 172.21.15.43:0/3650520583 wait complete. 2024-09-17T05:31:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 Processor -- start 2024-09-17T05:31:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 -- start start 2024-09-17T05:31:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b80040f0 0x7fa3d4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3d41566d0 0x7fa3d415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4165df0 con 0x7fa3d41581a0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d4165c70 con 0x7fa3b80040f0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d4165f70 con 0x7fa3d41566d0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54578/0 (socket says 172.21.15.43:54578) 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 -- 172.21.15.43:0/2425932463 learned_addr learned my addr 172.21.15.43:0/2425932463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d3400640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3d41566d0 0x7fa3d415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d3e00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b80040f0 0x7fa3d4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 -- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3d41566d0 msgr2=0x7fa3d415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3d41566d0 0x7fa3d415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 -- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b80040f0 msgr2=0x7fa3d4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b80040f0 0x7fa3d4155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 -- 172.21.15.43:0/2425932463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d40b7240 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3d8c00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4073a20 tx=0x7fa3cc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3cc015070 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004d90 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.991+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d4165aa0 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d4159bd0 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d41564f0 con 0x7fa3d41581a0 2024-09-17T05:31:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3cc003980 con 0x7fa3d41581a0 2024-09-17T05:31:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3cbe00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 0x7fa3b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7fa3cc094d20 con 0x7fa3d41581a0 2024-09-17T05:31:45.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.995+0000 7fa3d3e00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 0x7fa3b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:45.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.999+0000 7fa3d3e00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 0x7fa3b4075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4001380 tx=0x7fa3c4001090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:45.999+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa3d41564f0 con 0x7fa3d41581a0 2024-09-17T05:31:46.127 INFO:tasks.ceph.mds.c.smithi043.stderr: -40> 2024-09-17T05:31:41.371+0000 7f9e6ea00640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-17T05:31:46.127 INFO:tasks.ceph.mds.c.smithi043.stderr: -38> 2024-09-17T05:31:41.371+0000 7f9e6ea00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:31:46.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.211+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d40d0840 con 0x7fa3d41581a0 2024-09-17T05:31:46.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.211+0000 7fa3cbe00640 1 -- 172.21.15.43:0/2425932463 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) ==== 95+0+3557 (secure 0 0 0) 0x7fa3cc05f1b0 con 0x7fa3d41581a0 2024-09-17T05:31:46.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:46.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":186,"btime":"2024-09-17T05:31:45:414649+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:45.414587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:46.215 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 186 2024-09-17T05:31:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 msgr2=0x7fa3b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 0x7fa3b4075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4001380 tx=0x7fa3c4001090 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 msgr2=0x7fa3d415dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4073a20 tx=0x7fa3cc010f40 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 shutdown_connections 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa3b4073200 0x7fa3b4075710 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d41581a0 0x7fa3d415dcf0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa3d41566d0 0x7fa3d415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 --2- 172.21.15.43:0/2425932463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b80040f0 0x7fa3d4155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 >> 172.21.15.43:0/2425932463 conn(0x7fa3d40cee80 msgr2=0x7fa3d415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 shutdown_connections 2024-09-17T05:31:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.215+0000 7fa3dac00640 1 -- 172.21.15.43:0/2425932463 wait complete. 2024-09-17T05:31:46.238 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:46.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:46.250 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 -- 172.21.15.43:0/4144684156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200ce3c0 msgr2=0x7fc2200cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 --2- 172.21.15.43:0/4144684156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200ce3c0 0x7fc2200cd8b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc208009900 tx=0x7fc20801c850 comp rx=0 tx=0).stop 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 -- 172.21.15.43:0/4144684156 shutdown_connections 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 --2- 172.21.15.43:0/4144684156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200ce3c0 0x7fc2200cd8b0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 --2- 172.21.15.43:0/4144684156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200cdfb0 0x7fc2200cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 -- 172.21.15.43:0/4144684156 >> 172.21.15.43:0/4144684156 conn(0x7fc2200b2810 msgr2=0x7fc2200b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 -- 172.21.15.43:0/4144684156 shutdown_connections 2024-09-17T05:31:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.395+0000 7fc227c00640 1 -- 172.21.15.43:0/4144684156 wait complete. 2024-09-17T05:31:46.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 Processor -- start 2024-09-17T05:31:46.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 -- start start 2024-09-17T05:31:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 0x7fc2201487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2201603e0 0x7fc2201fa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc22016a150 con 0x7fc2200cdfb0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc220169fd0 con 0x7fc2201603e0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc227c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc22016a2d0 con 0x7fc2200ce3c0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54596/0 (socket says 172.21.15.43:54596) 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 -- 172.21.15.43:0/2150370317 learned_addr learned my addr 172.21.15.43:0/2150370317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2201603e0 0x7fc2201fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc217e00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 0x7fc2201487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 -- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 msgr2=0x7fc2201487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 0x7fc2201487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 -- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2201603e0 msgr2=0x7fc2201fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2201603e0 0x7fc2201fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 -- 172.21.15.43:0/2150370317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22015a330 con 0x7fc2200cdfb0 2024-09-17T05:31:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc217e00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 0x7fc2201487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.399+0000 7fc225200640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d750 tx=0x7fc21000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:46.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210017070 con 0x7fc2200cdfb0 2024-09-17T05:31:46.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc210009cc0 con 0x7fc2200cdfb0 2024-09-17T05:31:46.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc220150a60 con 0x7fc2200cdfb0 2024-09-17T05:31:46.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210005280 con 0x7fc2200cdfb0 2024-09-17T05:31:46.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc220151020 con 0x7fc2200cdfb0 2024-09-17T05:31:46.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.403+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2201607f0 con 0x7fc2200cdfb0 2024-09-17T05:31:46.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.407+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc210005420 con 0x7fc2200cdfb0 2024-09-17T05:31:46.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.407+0000 7fc21d600640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 0x7fc1fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.407+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7fc210094860 con 0x7fc2200cdfb0 2024-09-17T05:31:46.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.407+0000 7fc217e00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 0x7fc1fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.407+0000 7fc217e00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 0x7fc1fc075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc2080af000 tx=0x7fc20801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.411+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc21000de60 con 0x7fc2200cdfb0 2024-09-17T05:31:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.623+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2200b0d30 con 0x7fc2200cdfb0 2024-09-17T05:31:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.623+0000 7fc21d600640 1 -- 172.21.15.43:0/2150370317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) ==== 95+0+4383 (secure 0 0 0) 0x7fc21005ec60 con 0x7fc2200cdfb0 2024-09-17T05:31:46.627 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:46.627 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":187,"btime":"2024-09-17T05:31:46:416999+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8586,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1587862715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1587862715},{"type":"v1","addr":"172.21.15.43:6835","nonce":1587862715}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:45.414587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 187 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 msgr2=0x7fc1fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 0x7fc1fc075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc2080af000 tx=0x7fc20801b090 comp rx=0 tx=0).stop 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 msgr2=0x7fc220159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d750 tx=0x7fc21000dc10 comp rx=0 tx=0).stop 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 shutdown_connections 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc1fc073200 0x7fc1fc075710 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2201603e0 0x7fc2201fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc2200ce3c0 0x7fc2201487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 --2- 172.21.15.43:0/2150370317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2200cdfb0 0x7fc220159df0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 >> 172.21.15.43:0/2150370317 conn(0x7fc2200b2810 msgr2=0x7fc22015f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 shutdown_connections 2024-09-17T05:31:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.627+0000 7fc227c00640 1 -- 172.21.15.43:0/2150370317 wait complete. 2024-09-17T05:31:46.648 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-09-17T05:31:46.649 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-09-17T05:31:46.649 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:46.649 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:46.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:46.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:46.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:46.700 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:46.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:46.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 -- 172.21.15.43:0/4189531632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56806f110 msgr2=0x7ff5680b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 --2- 172.21.15.43:0/4189531632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56806f110 0x7ff5680b33e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff554009900 tx=0x7ff55401c820 comp rx=0 tx=0).stop 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 -- 172.21.15.43:0/4189531632 shutdown_connections 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 --2- 172.21.15.43:0/4189531632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56806f110 0x7ff5680b33e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 --2- 172.21.15.43:0/4189531632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff568155d70 0x7ff56815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 -- 172.21.15.43:0/4189531632 >> 172.21.15.43:0/4189531632 conn(0x7ff5680b23d0 msgr2=0x7ff5680d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 -- 172.21.15.43:0/4189531632 shutdown_connections 2024-09-17T05:31:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.951+0000 7ff56dc00640 1 -- 172.21.15.43:0/4189531632 wait complete. 2024-09-17T05:31:46.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 Processor -- start 2024-09-17T05:31:46.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- start start 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff56806f110 0x7ff5680c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff568155d70 0x7ff56809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff568168b10 con 0x7ff5680b3920 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff568168990 con 0x7ff568155d70 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff568168c90 con 0x7ff56806f110 2024-09-17T05:31:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54614/0 (socket says 172.21.15.43:54614) 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 -- 172.21.15.43:0/2908565016 learned_addr learned my addr 172.21.15.43:0/2908565016 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566a00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff56806f110 0x7ff5680c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff567400640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff568155d70 0x7ff56809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 -- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff56806f110 msgr2=0x7ff5680c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff56806f110 0x7ff5680c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 -- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff568155d70 msgr2=0x7ff56809dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff568155d70 0x7ff56809dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 -- 172.21.15.43:0/2908565016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5680c7350 con 0x7ff5680b3920 2024-09-17T05:31:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff566000640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff55401aed0 tx=0x7ff5540a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540b2040 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5540043b0 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554005270 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5680cd5b0 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.955+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5680d0460 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5680b33e0 con 0x7ff5680b3920 2024-09-17T05:31:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff554005410 con 0x7ff5680b3920 2024-09-17T05:31:46.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff55ea00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 0x7ff548075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:46.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7ff55412dae0 con 0x7ff5680b3920 2024-09-17T05:31:46.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff566a00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 0x7ff548075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:46.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.959+0000 7ff566a00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 0x7ff548075710 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff5680715c0 tx=0x7ff558009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:46.963+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff5540bc060 con 0x7ff5680b3920 2024-09-17T05:31:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.171+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff5680b0270 con 0x7ff5680b3920 2024-09-17T05:31:47.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.427+0000 7ff55ea00640 1 -- 172.21.15.43:0/2908565016 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v188) ==== 161+0+0 (secure 0 0 0) 0x7ff5540164d0 con 0x7ff5680b3920 2024-09-17T05:31:47.430 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:47.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 msgr2=0x7ff548075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:47.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 0x7ff548075710 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff5680715c0 tx=0x7ff558009500 comp rx=0 tx=0).stop 2024-09-17T05:31:47.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 msgr2=0x7ff5680c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff55401aed0 tx=0x7ff5540a5720 comp rx=0 tx=0).stop 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 shutdown_connections 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff548073200 0x7ff548075710 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff568155d70 0x7ff56809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5680b3920 0x7ff5680c85b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 --2- 172.21.15.43:0/2908565016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff56806f110 0x7ff5680c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 >> 172.21.15.43:0/2908565016 conn(0x7ff5680b23d0 msgr2=0x7ff56806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 shutdown_connections 2024-09-17T05:31:47.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.431+0000 7ff56dc00640 1 -- 172.21.15.43:0/2908565016 wait complete. 2024-09-17T05:31:47.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:47.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 -- 172.21.15.43:0/2935968599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c8004810 msgr2=0x7fb0c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:47.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/2935968599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c8004810 0x7fb0c8004c20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009b40 tx=0x7fb0cc01d210 comp rx=0 tx=0).stop 2024-09-17T05:31:47.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e3400640 1 -- 172.21.15.43:0/2935968599 reap_dead start 2024-09-17T05:31:47.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 -- 172.21.15.43:0/2935968599 shutdown_connections 2024-09-17T05:31:47.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 -- 172.21.15.43:0/2935968599 >> 172.21.15.43:0/2935968599 conn(0x7fb0e40b23d0 msgr2=0x7fb0e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:47.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 -- 172.21.15.43:0/2935968599 shutdown_connections 2024-09-17T05:31:47.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.611+0000 7fb0e9e00640 1 -- 172.21.15.43:0/2935968599 wait complete. 2024-09-17T05:31:47.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 Processor -- start 2024-09-17T05:31:47.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- start start 2024-09-17T05:31:47.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0c8004810 0x7fb0e415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0e409dc90 0x7fb0e409e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e4168e80 con 0x7fb0e40ae230 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e4168d00 con 0x7fb0e409dc90 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e4169000 con 0x7fb0c8004810 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54624/0 (socket says 172.21.15.43:54624) 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 -- 172.21.15.43:0/1959918259 learned_addr learned my addr 172.21.15.43:0/1959918259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e2a00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0c8004810 0x7fb0e415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0dbe00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0e409dc90 0x7fb0e409e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 -- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0c8004810 msgr2=0x7fb0e415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0c8004810 0x7fb0e415bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 -- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0e409dc90 msgr2=0x7fb0e409e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0e409dc90 0x7fb0e409e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 -- 172.21.15.43:0/1959918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e415c310 con 0x7fb0e40ae230 2024-09-17T05:31:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e3400640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8004790 tx=0x7fb0dc006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00fa00 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0dc00fba0 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e40aeb80 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011430 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.615+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e41fa710 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.619+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e41fa1a0 con 0x7fb0e40ae230 2024-09-17T05:31:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.619+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0dc0115d0 con 0x7fb0e40ae230 2024-09-17T05:31:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.619+0000 7fb0db400640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.619+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7fb0dc094fe0 con 0x7fb0e40ae230 2024-09-17T05:31:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.619+0000 7fb0e2a00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.623+0000 7fb0e2a00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 0x7fb0c0075710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006860 tx=0x7fb0d40067f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.623+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb0e40aeb80 con 0x7fb0e40ae230 2024-09-17T05:31:47.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:47.863+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb0e40b4b80 con 0x7fb0e40ae230 2024-09-17T05:31:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0db400640 1 -- 172.21.15.43:0/1959918259 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v189) ==== 101+0+0 (secure 0 0 0) 0x7fb0dc05f440 con 0x7fb0e40ae230 2024-09-17T05:31:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 msgr2=0x7fb0c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 0x7fb0c0075710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006860 tx=0x7fb0d40067f0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 msgr2=0x7fb0e40ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8004790 tx=0x7fb0dc006720 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 shutdown_connections 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0e409dc90 0x7fb0e409e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e40ae230 0x7fb0e40ae640 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 --2- 172.21.15.43:0/1959918259 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb0c8004810 0x7fb0e415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 >> 172.21.15.43:0/1959918259 conn(0x7fb0e40b23d0 msgr2=0x7fb0e4074ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 shutdown_connections 2024-09-17T05:31:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.435+0000 7fb0e9e00640 1 -- 172.21.15.43:0/1959918259 wait complete. 2024-09-17T05:31:48.461 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 Processor -- start 2024-09-17T05:31:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 -- start start 2024-09-17T05:31:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 0x564e29834dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 0x564e297ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e2973aa00 con 0x564e298a2e40 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564e29798f40 con 0x564e297ecd70 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2527c53080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564e298a3890 con 0x564e2981bcb0 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2521800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2521800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54634/0 (socket says 172.21.15.43:54634) 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2521800640 1 -- 172.21.15.43:0/4179350166 learned_addr learned my addr 172.21.15.43:0/4179350166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f2520e00640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 0x564e29834dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.519+0000 7f251be00640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 0x564e297ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 -- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 msgr2=0x564e297ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 0x564e297ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 -- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 msgr2=0x564e29834dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 0x564e29834dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 -- 172.21.15.43:0/4179350166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e29713c10 con 0x564e298a2e40 2024-09-17T05:31:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f251be00640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 0x564e297ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2520e00640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 0x564e29834dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2521800640 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f251c01c3c0 tx=0x7f251c01aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8da3e701d0d6f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f251aa00640 1 -- 172.21.15.43:0/4179350166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c004020 con 0x564e298a2e40 2024-09-17T05:31:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f251aa00640 1 -- 172.21.15.43:0/4179350166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f251c0bb470 con 0x564e298a2e40 2024-09-17T05:31:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f251aa00640 1 -- 172.21.15.43:0/4179350166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0042d0 con 0x564e298a2e40 2024-09-17T05:31:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- 172.21.15.43:0/4179350166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 msgr2=0x564e297eadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f251c01c3c0 tx=0x7f251c01aab0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- 172.21.15.43:0/4179350166 shutdown_connections 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e297eadd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e2981bcb0 0x564e297ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 --2- 172.21.15.43:0/4179350166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e297ecd70 0x564e29834dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- 172.21.15.43:0/4179350166 >> 172.21.15.43:0/4179350166 conn(0x564e297d8d80 msgr2=0x564e298c1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- 172.21.15.43:0/4179350166 shutdown_connections 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- 172.21.15.43:0/4179350166 wait complete. 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 Processor -- start 2024-09-17T05:31:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.523+0000 7f2527c53080 1 -- start start 2024-09-17T05:31:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e297ecd70 0x564e29815870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e2981bcb0 0x564e298beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e2982f8f0 con 0x564e298a2e40 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564e298c1860 con 0x564e2981bcb0 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564e29818e80 con 0x564e297ecd70 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54640/0 (socket says 172.21.15.43:54640) 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 learned_addr learned my addr 172.21.15.43:0/3032710283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e297ecd70 0x564e29815870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e2981bcb0 0x564e298beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e297ecd70 msgr2=0x564e29815870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x564e297ecd70 0x564e29815870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e2981bcb0 msgr2=0x564e298beaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e2981bcb0 0x564e298beaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e29833680 con 0x564e298a2e40 2024-09-17T05:31:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2521800640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f251c004fd0 tx=0x7f251c0bbae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2519600640 1 -- 172.21.15.43:0/3032710283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c004020 con 0x564e298a2e40 2024-09-17T05:31:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2519600640 1 -- 172.21.15.43:0/3032710283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f251c0ca070 con 0x564e298a2e40 2024-09-17T05:31:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e2982f1a0 con 0x564e298a2e40 2024-09-17T05:31:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2519600640 1 -- 172.21.15.43:0/3032710283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0c56d0 con 0x564e298a2e40 2024-09-17T05:31:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e29812640 con 0x564e298a2e40 2024-09-17T05:31:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.527+0000 7f2519600640 1 -- 172.21.15.43:0/3032710283 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f251c0c3040 con 0x564e298a2e40 2024-09-17T05:31:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2519600640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 0x7f25000740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2519600640 1 -- 172.21.15.43:0/3032710283 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7f251c133620 con 0x564e298a2e40 2024-09-17T05:31:48.532 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 0x7f25000740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564e297eada0 0x564e29812d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e298132c0 con 0x564e297eada0 2024-09-17T05:31:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564e297eada0 0x564e29812d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 0x7f25000740f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2514004fd0 tx=0x7f25140049b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.531+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564e297eada0 0x564e29812d80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v106'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e29813690 con 0x564e297eada0 2024-09-17T05:31:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v106'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299657c0 con 0x564e297eada0 2024-09-17T05:31:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v106'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.539 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.535+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29966b70 con 0x564e297eada0 2024-09-17T05:31:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.539+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.539+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29966f10 con 0x564e297eada0 2024-09-17T05:31:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.539+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.539+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29965b60 con 0x564e297eada0 2024-09-17T05:31:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29965f00 con 0x564e297eada0 2024-09-17T05:31:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v106'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e299662f0 con 0x564e297eada0 2024-09-17T05:31:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v106'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.543+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662f0 con 0x564e297eada0 2024-09-17T05:31:48.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v106'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.548 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:48.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.547+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.551+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.556 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.555+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.559+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.559+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.559+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.559+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v106'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v106'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v106'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.566 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.563+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.567+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v106'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v106'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v106'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.575 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.571+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.575+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.575+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.575+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.575+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e297eada0 2024-09-17T05:31:48.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.575+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564e299672b0 0x564e299696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299662a0 con 0x564e299672b0 2024-09-17T05:31:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564e299672b0 0x564e299696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564e299672b0 0x564e299696d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'373 uv355 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.579+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e29969c10 con 0x564e299672b0 2024-09-17T05:31:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'374 uv374 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996a030 con 0x564e299672b0 2024-09-17T05:31:48.585 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:48.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996a920 con 0x564e299672b0 2024-09-17T05:31:48.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v106'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.583+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996a4f0 con 0x564e299672b0 2024-09-17T05:31:48.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996b170 con 0x564e299672b0 2024-09-17T05:31:48.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v106'378 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2996b510 con 0x564e299672b0 2024-09-17T05:31:48.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v106'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.587+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996b8b0 con 0x564e299672b0 2024-09-17T05:31:48.592 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:48.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.591+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v106'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.591+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.591+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.591+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v106'383 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v106'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.595+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v106'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.600 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:48.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v106'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.599+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v106'388 uv371 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v106'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ada0 con 0x564e299672b0 2024-09-17T05:31:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v106'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.607 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.603+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996cd70 con 0x564e299672b0 2024-09-17T05:31:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996cd70 con 0x564e299672b0 2024-09-17T05:31:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996bd60 con 0x564e299672b0 2024-09-17T05:31:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e299672b0 2024-09-17T05:31:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564e2996c270 0x564e2996e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.607+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996ebd0 con 0x564e2996c270 2024-09-17T05:31:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564e2996c270 0x564e2996e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564e2996c270 0x564e2996e690 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v106'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2996ec50 con 0x564e2996c270 2024-09-17T05:31:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.611+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e2996c270 2024-09-17T05:31:48.616 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29970080 con 0x564e2996c270 2024-09-17T05:31:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29970080 con 0x564e2996c270 2024-09-17T05:31:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.615+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e2996c270 2024-09-17T05:31:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e2996c270 2024-09-17T05:31:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e297eada0 2024-09-17T05:31:48.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'73 uv72 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e297eada0 2024-09-17T05:31:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'74 uv74 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.619+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e297eada0 2024-09-17T05:31:48.624 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'75 uv75 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2996f070 con 0x564e297eada0 2024-09-17T05:31:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'76 uv76 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f250c00c220 con 0x564e297eada0 2024-09-17T05:31:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 0x564e29971b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299720e0 con 0x564e2996f6f0 2024-09-17T05:31:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f2521800640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 0x564e29971b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.623+0000 7f2521800640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 0x564e29971b10 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v106'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e29972580 con 0x564e2996f6f0 2024-09-17T05:31:48.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29972a60 con 0x564e2996f6f0 2024-09-17T05:31:48.631 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:48.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.627+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29974310 con 0x564e2996f6f0 2024-09-17T05:31:48.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299746b0 con 0x564e2996f6f0 2024-09-17T05:31:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29973300 con 0x564e2996f6f0 2024-09-17T05:31:48.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564e29973c00 0x564e29976a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.631+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29976fa0 con 0x564e29973c00 2024-09-17T05:31:48.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564e29973c00 0x564e29976a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2520e00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564e29973c00 0x564e29976a60 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v106'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e29976fa0 con 0x564e29973c00 2024-09-17T05:31:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v106'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.635+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v106'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.641 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299783a0 con 0x564e29973c00 2024-09-17T05:31:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e299783a0 con 0x564e29973c00 2024-09-17T05:31:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.639+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v106'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v106'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.643+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v106'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.648 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e29977390 con 0x564e29973c00 2024-09-17T05:31:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2520e00640 1 -- 172.21.15.43:0/3032710283 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2514008980 con 0x564e29973c00 2024-09-17T05:31:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564e299778a0 0x564e29979cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a3f0 con 0x564e299778a0 2024-09-17T05:31:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.647+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564e299778a0 0x564e29979cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.651+0000 7f251be00640 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564e299778a0 0x564e29979cc0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.651+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v106'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.651+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997a3f0 con 0x564e299778a0 2024-09-17T05:31:48.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.651+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v106'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.651+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e299778a0 2024-09-17T05:31:48.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v106'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.655 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:48.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e299778a0 2024-09-17T05:31:48.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e299778a0 2024-09-17T05:31:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.655+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e297eada0 2024-09-17T05:31:48.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v106'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e297eada0 2024-09-17T05:31:48.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e297eada0 2024-09-17T05:31:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.663 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.659+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b850 con 0x564e297eada0 2024-09-17T05:31:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.663+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.663+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b850 con 0x564e297eada0 2024-09-17T05:31:48.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.663+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.663+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e297eada0 2024-09-17T05:31:48.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.663+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e297eada0 2024-09-17T05:31:48.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.667+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.667+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v106'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.667+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.667+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v106'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.667+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v106'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.671 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:48.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v106'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v106'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.671+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v106'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v106'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.678 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v106'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.675+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997a840 con 0x564e2996c270 2024-09-17T05:31:48.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v106'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997d650 con 0x564e2996c270 2024-09-17T05:31:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v106'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e2996c270 2024-09-17T05:31:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997acf0 con 0x564e2996f6f0 2024-09-17T05:31:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v106'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.679+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.683+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v106'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.683+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.683+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v106'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.686 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:48.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.683+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.683+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v106'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.687+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.687+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v106'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.687+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.687+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.687+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.692 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997c0a0 con 0x564e2996f6f0 2024-09-17T05:31:48.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997c0a0 con 0x564e2996f6f0 2024-09-17T05:31:48.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.691+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e2996f6f0 2024-09-17T05:31:48.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.695+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v106'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f251c0a7d70 con 0x564e2996f6f0 2024-09-17T05:31:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.695+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.695+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v106'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.695+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v106'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v106'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.701 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:48.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v106'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.699+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v106'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v106'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v106'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.703+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.707+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v106'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.709 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:48.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.707+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.707+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v106'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.707+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v106'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v106'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v106'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.711+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v106'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.715 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:48.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997c0a0 con 0x564e299778a0 2024-09-17T05:31:48.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v106'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997c0a0 con 0x564e299778a0 2024-09-17T05:31:48.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v106'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.715+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b090 con 0x564e299778a0 2024-09-17T05:31:48.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v106'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x564e2997b540 con 0x564e299778a0 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f251be00640 1 -- 172.21.15.43:0/3032710283 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v106'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f250c024040 con 0x564e299778a0 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564e299672b0 msgr2=0x564e299696d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x564e299672b0 0x564e299696d0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564e29973c00 msgr2=0x564e29976a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x564e29973c00 0x564e29976a60 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564e299778a0 msgr2=0x564e29979cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x564e299778a0 0x564e29979cc0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564e2996c270 msgr2=0x564e2996e690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564e2996c270 0x564e2996e690 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564e297eada0 msgr2=0x564e29812d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x564e297eada0 0x564e29812d80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 reap_dead start 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 msgr2=0x564e29971b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 0x564e29971b10 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2521800640 1 -- 172.21.15.43:0/3032710283 reap_dead start 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 msgr2=0x7f25000740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 0x7f25000740f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2514004fd0 tx=0x7f25140049b0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 msgr2=0x564e2982e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f251c004fd0 tx=0x7f251c0bbae0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 shutdown_connections 2024-09-17T05:31:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x564e2996f6f0 0x564e29971b10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2500071be0 0x7f25000740f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 --2- 172.21.15.43:0/3032710283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e298a2e40 0x564e2982e690 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 >> 172.21.15.43:0/3032710283 conn(0x564e297d8d80 msgr2=0x564e298321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 shutdown_connections 2024-09-17T05:31:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.719+0000 7f2527c53080 1 -- 172.21.15.43:0/3032710283 wait complete. 2024-09-17T05:31:48.727 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-09-17T05:31:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 Processor -- start 2024-09-17T05:31:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 -- start start 2024-09-17T05:31:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d37ad70 0x55886d4e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d3bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55886d2c8a00 con 0x55886d3a9cb0 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55886d326f40 con 0x55886d37ad70 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c904b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55886d431b80 con 0x55886d430e40 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54654/0 (socket says 172.21.15.43:54654) 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c0c00640 1 -- 172.21.15.43:0/2517249372 learned_addr learned my addr 172.21.15.43:0/2517249372 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c1600640 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d37ad70 0x55886d4e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.827+0000 7f81c2000640 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d3bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 -- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 msgr2=0x55886d3bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d3bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 -- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d37ad70 msgr2=0x55886d4e1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d37ad70 0x55886d4e1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 -- 172.21.15.43:0/2517249372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886d2a1c10 con 0x55886d3a9cb0 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c0c00640 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b610 tx=0x7f81b0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fdf234a25bec3e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81bb400640 1 -- 172.21.15.43:0/2517249372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000a890 con 0x55886d3a9cb0 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81bb400640 1 -- 172.21.15.43:0/2517249372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b000aa30 con 0x55886d3a9cb0 2024-09-17T05:31:48.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81bb400640 1 -- 172.21.15.43:0/2517249372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000ae60 con 0x55886d3a9cb0 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- 172.21.15.43:0/2517249372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 msgr2=0x55886d4e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b610 tx=0x7f81b0004f90 comp rx=0 tx=0).stop 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- 172.21.15.43:0/2517249372 shutdown_connections 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d3bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d3a9cb0 0x55886d4e1750 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 --2- 172.21.15.43:0/2517249372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d37ad70 0x55886d4e1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- 172.21.15.43:0/2517249372 >> 172.21.15.43:0/2517249372 conn(0x55886d366d80 msgr2=0x55886d378890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- 172.21.15.43:0/2517249372 shutdown_connections 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- 172.21.15.43:0/2517249372 wait complete. 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 Processor -- start 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.831+0000 7f81c904b080 1 -- start start 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d37ad70 0x55886d42fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 0x55886d3bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d446730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55886d3aa240 con 0x55886d37ad70 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55886d44d790 con 0x55886d3a9cb0 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55886d430910 con 0x55886d430e40 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 0x55886d3bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 0x55886d3bcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:50498/0 (socket says 172.21.15.43:50498) 2024-09-17T05:31:48.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 -- 172.21.15.43:0/518989085 learned_addr learned my addr 172.21.15.43:0/518989085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c2000640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d446730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c1600640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d37ad70 0x55886d42fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 -- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 msgr2=0x55886d446730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55886d430e40 0x55886d446730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 -- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d37ad70 msgr2=0x55886d42fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55886d37ad70 0x55886d42fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 -- 172.21.15.43:0/518989085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886d4de9e0 con 0x55886d3a9cb0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c0c00640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 0x55886d3bcac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f81b00166d0 tx=0x7f81b0016700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81ba000640 1 -- 172.21.15.43:0/518989085 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0016b30 con 0x55886d3a9cb0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81ba000640 1 -- 172.21.15.43:0/518989085 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b0016cd0 con 0x55886d3a9cb0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55886d4dec80 con 0x55886d3a9cb0 2024-09-17T05:31:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81ba000640 1 -- 172.21.15.43:0/518989085 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0020420 con 0x55886d3a9cb0 2024-09-17T05:31:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886d4df1c0 con 0x55886d3a9cb0 2024-09-17T05:31:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.835+0000 7f81ba000640 1 -- 172.21.15.43:0/518989085 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f81b0029430 con 0x55886d3a9cb0 2024-09-17T05:31:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81ba000640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f81a4071be0 0x7f81a40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81ba000640 1 -- 172.21.15.43:0/518989085 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7f81b0095130 con 0x55886d3a9cb0 2024-09-17T05:31:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c1600640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f81a4071be0 0x7f81a40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c904b080 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55886d4de4d0 0x55886d4e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55886d4e8eb0 con 0x55886d4de4d0 2024-09-17T05:31:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c2000640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55886d4de4d0 0x55886d4e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c1600640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f81a4071be0 0x7f81a40740f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81b40af000 tx=0x7f81b40b6040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.839+0000 7f81c2000640 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55886d4de4d0 0x55886d4e8970 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c2000640 1 -- 172.21.15.43:0/518989085 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v106'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f81bc002810 con 0x55886d4de4d0 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55886d4de4d0 msgr2=0x55886d4e8970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55886d4de4d0 0x55886d4e8970 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f81a4071be0 msgr2=0x7f81a40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f81a4071be0 0x7f81a40740f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f81b40af000 tx=0x7f81b40b6040 comp rx=0 tx=0).stop 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 msgr2=0x55886d3bcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 --2- 172.21.15.43:0/518989085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55886d3a9cb0 0x55886d3bcac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f81b00166d0 tx=0x7f81b0016700 comp rx=0 tx=0).stop 2024-09-17T05:31:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c2000640 1 -- 172.21.15.43:0/518989085 reap_dead start 2024-09-17T05:31:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 shutdown_connections 2024-09-17T05:31:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 >> 172.21.15.43:0/518989085 conn(0x55886d366d80 msgr2=0x55886d44fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 shutdown_connections 2024-09-17T05:31:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:48.843+0000 7f81c904b080 1 -- 172.21.15.43:0/518989085 wait complete. 2024-09-17T05:31:48.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 -- 172.21.15.43:0/1850720038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc15f470 msgr2=0x7ff6dc0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 --2- 172.21.15.43:0/1850720038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc15f470 0x7ff6dc0d0390 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009840 tx=0x7ff6d001c700 comp rx=0 tx=0).stop 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 -- 172.21.15.43:0/1850720038 shutdown_connections 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 --2- 172.21.15.43:0/1850720038 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc06fce0 0x7ff6dc0700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 --2- 172.21.15.43:0/1850720038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc15f470 0x7ff6dc0d0390 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 -- 172.21.15.43:0/1850720038 >> 172.21.15.43:0/1850720038 conn(0x7ff6dc0b2810 msgr2=0x7ff6dc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 -- 172.21.15.43:0/1850720038 shutdown_connections 2024-09-17T05:31:49.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.055+0000 7ff6e4800640 1 -- 172.21.15.43:0/1850720038 wait complete. 2024-09-17T05:31:49.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 Processor -- start 2024-09-17T05:31:49.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- start start 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc0b5610 0x7ff6dc1fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15f470 0x7ff6dc154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc169600 con 0x7ff6dc06fce0 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc169480 con 0x7ff6dc15f470 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc169780 con 0x7ff6dc0b5610 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54682/0 (socket says 172.21.15.43:54682) 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 -- 172.21.15.43:0/203116515 learned_addr learned my addr 172.21.15.43:0/203116515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e2800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15f470 0x7ff6dc154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1400640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc0b5610 0x7ff6dc1fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 -- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc0b5610 msgr2=0x7ff6dc1fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc0b5610 0x7ff6dc1fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 -- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15f470 msgr2=0x7ff6dc154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15f470 0x7ff6dc154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 -- 172.21.15.43:0/203116515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc1fda10 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e1e00640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009790 tx=0x7ff6d00a8600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00a8c80 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6d00a8e20 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc153320 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0005300 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.059+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc0b0720 con 0x7ff6dc06fce0 2024-09-17T05:31:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc0700f0 con 0x7ff6dc06fce0 2024-09-17T05:31:49.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6d00054a0 con 0x7ff6dc06fce0 2024-09-17T05:31:49.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6d6000640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 0x7ff6c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7171+0+0 (secure 0 0 0) 0x7ff6d012f290 con 0x7ff6dc06fce0 2024-09-17T05:31:49.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6e1400640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 0x7ff6c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.063+0000 7ff6e1400640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 0x7ff6c0075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b1fa0 tx=0x7ff6cc007ec0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:49.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.067+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6d00c9080 con 0x7ff6dc06fce0 2024-09-17T05:31:49.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.283+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff6dc1653f0 con 0x7ff6dc06fce0 2024-09-17T05:31:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6d6000640 1 -- 172.21.15.43:0/203116515 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v190) ==== 162+0+0 (secure 0 0 0) 0x7ff6dc1fda10 con 0x7ff6dc06fce0 2024-09-17T05:31:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 msgr2=0x7ff6c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 0x7ff6c0075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0b1fa0 tx=0x7ff6cc007ec0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 msgr2=0x7ff6dc0b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009790 tx=0x7ff6d00a8600 comp rx=0 tx=0).stop 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 shutdown_connections 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6c0073200 0x7ff6c0075710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6dc15f470 0x7ff6dc154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6dc0b5610 0x7ff6dc1fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 --2- 172.21.15.43:0/203116515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc06fce0 0x7ff6dc0b1200 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 >> 172.21.15.43:0/203116515 conn(0x7ff6dc0b2810 msgr2=0x7ff6dc164a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 shutdown_connections 2024-09-17T05:31:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.443+0000 7ff6e4800640 1 -- 172.21.15.43:0/203116515 wait complete. 2024-09-17T05:31:49.458 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:31:49.455+0000 7f22c9e00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:31:49.458 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:49.455+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:31:49.458 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:31:49.455+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:31:49.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 -- 172.21.15.43:0/1092063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 msgr2=0x7f7690005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 --2- 172.21.15.43:0/1092063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f7690005160 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009b60 tx=0x7f76a401d2b0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76aa800640 1 -- 172.21.15.43:0/1092063 reap_dead start 2024-09-17T05:31:49.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 -- 172.21.15.43:0/1092063 shutdown_connections 2024-09-17T05:31:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 -- 172.21.15.43:0/1092063 >> 172.21.15.43:0/1092063 conn(0x7f76ac0b2e80 msgr2=0x7f76ac0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 -- 172.21.15.43:0/1092063 shutdown_connections 2024-09-17T05:31:49.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.627+0000 7f76b0e00640 1 -- 172.21.15.43:0/1092063 wait complete. 2024-09-17T05:31:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 Processor -- start 2024-09-17T05:31:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 -- start start 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 0x7f76ac0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 0x7f76ac0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0b2ad0 con 0x7f7690004d50 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0b2950 con 0x7f76ac09bcb0 2024-09-17T05:31:49.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76b0e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0ce2f0 con 0x7f76ac158b30 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54688/0 (socket says 172.21.15.43:54688) 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 -- 172.21.15.43:0/222826406 learned_addr learned my addr 172.21.15.43:0/222826406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a1400640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 0x7f76ac0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 -- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 msgr2=0x7f76ac0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76aa800640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 0x7f76ac0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 0x7f76ac0ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 -- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 msgr2=0x7f76ac0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 0x7f76ac0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a9e00640 1 -- 172.21.15.43:0/222826406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac1f95f0 con 0x7f7690004d50 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.631+0000 7f76a1400640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 0x7f76ac0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:49.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76aa800640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 0x7f76ac0ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a9e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f769c0013e0 tx=0x7f769c006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c003b90 con 0x7f7690004d50 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f769c018020 con 0x7f7690004d50 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac1f9b70 con 0x7f7690004d50 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c005110 con 0x7f7690004d50 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac151ba0 con 0x7f7690004d50 2024-09-17T05:31:49.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac0ce850 con 0x7f7690004d50 2024-09-17T05:31:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f769c0052b0 con 0x7f7690004d50 2024-09-17T05:31:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.635+0000 7f76a2a00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 0x7f7688075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:49.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.639+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f769c097fe0 con 0x7f7690004d50 2024-09-17T05:31:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.639+0000 7f76a1400640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 0x7f7688075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.639+0000 7f76a1400640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 0x7f7688075710 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0cb900 tx=0x7f76940091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.643+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f769c022030 con 0x7f7690004d50 2024-09-17T05:31:49.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76ac06dd50 con 0x7f7690004d50 2024-09-17T05:31:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76a2a00640 1 -- 172.21.15.43:0/222826406 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) ==== 95+0+3557 (secure 0 0 0) 0x7f76ac1f95f0 con 0x7f7690004d50 2024-09-17T05:31:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:49.853 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":192,"btime":"2024-09-17T05:31:49:459390+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:49.459323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 192 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 msgr2=0x7f7688075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 0x7f7688075710 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0cb900 tx=0x7f76940091e0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 msgr2=0x7f76ac151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f769c0013e0 tx=0x7f769c006410 comp rx=0 tx=0).stop 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 shutdown_connections 2024-09-17T05:31:49.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7688073200 0x7f7688075710 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ac09bcb0 0x7f76ac0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f76ac158b30 0x7f76ac0ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 --2- 172.21.15.43:0/222826406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7690004d50 0x7f76ac151660 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 >> 172.21.15.43:0/222826406 conn(0x7f76ac0b2e80 msgr2=0x7f76ac0cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 shutdown_connections 2024-09-17T05:31:49.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:49.851+0000 7f76b0e00640 1 -- 172.21.15.43:0/222826406 wait complete. 2024-09-17T05:31:49.875 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:49.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 -- 172.21.15.43:0/547202853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 msgr2=0x7fbf80162ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 --2- 172.21.15.43:0/547202853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf80162ba0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbf80073fa0 tx=0x7fbf6801c830 comp rx=0 tx=0).stop 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 -- 172.21.15.43:0/547202853 shutdown_connections 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 --2- 172.21.15.43:0/547202853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf80162ba0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 --2- 172.21.15.43:0/547202853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf800b1db0 0x7fbf801630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 --2- 172.21.15.43:0/547202853 >> v2:172.21.15.62:3300/0 conn(0x7fbf8006e390 0x7fbf8006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 -- 172.21.15.43:0/547202853 >> 172.21.15.43:0/547202853 conn(0x7fbf800cee80 msgr2=0x7fbf800cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 -- 172.21.15.43:0/547202853 shutdown_connections 2024-09-17T05:31:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.031+0000 7fbf85200640 1 -- 172.21.15.43:0/547202853 wait complete. 2024-09-17T05:31:50.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 Processor -- start 2024-09-17T05:31:50.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 -- start start 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8006e390 0x7fbf800cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf800b1db0 0x7fbf801fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800b7680 con 0x7fbf800b19a0 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800b7500 con 0x7fbf800b1db0 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf85200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf800d0da0 con 0x7fbf8006e390 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54692/0 (socket says 172.21.15.43:54692) 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 -- 172.21.15.43:0/2629029408 learned_addr learned my addr 172.21.15.43:0/2629029408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf7de00640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8006e390 0x7fbf800cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf7e800640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf800b1db0 0x7fbf801fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 -- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8006e390 msgr2=0x7fbf800cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8006e390 0x7fbf800cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 -- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf800b1db0 msgr2=0x7fbf801fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf800b1db0 0x7fbf801fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 -- 172.21.15.43:0/2629029408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf801fd600 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.035+0000 7fbf75400640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbf6801aee0 tx=0x7fbf680a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf680b2040 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf680042c0 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf680052d0 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf800cc850 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf800cce10 con 0x7fbf800b19a0 2024-09-17T05:31:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf800d12c0 con 0x7fbf800b19a0 2024-09-17T05:31:50.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf68005470 con 0x7fbf800b19a0 2024-09-17T05:31:50.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.039+0000 7fbf76a00640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.043+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7fbf6812dce0 con 0x7fbf800b19a0 2024-09-17T05:31:50.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.043+0000 7fbf7de00640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.043+0000 7fbf7de00640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 0x7fbf5c075710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005fd0 tx=0x7fbf7000b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.047+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf680164b0 con 0x7fbf800b19a0 2024-09-17T05:31:50.168 INFO:tasks.ceph.mds.d.smithi062.stderr: -38> 2024-09-17T05:31:45.412+0000 7fdebc200640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-17T05:31:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf800b5e10 con 0x7fbf800b19a0 2024-09-17T05:31:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf76a00640 1 -- 172.21.15.43:0/2629029408 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) ==== 95+0+3557 (secure 0 0 0) 0x7fbf680044b0 con 0x7fbf800b19a0 2024-09-17T05:31:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":192,"btime":"2024-09-17T05:31:49:459390+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:49.459323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 192 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 msgr2=0x7fbf5c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 0x7fbf5c075710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbf70005fd0 tx=0x7fbf7000b260 comp rx=0 tx=0).stop 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 msgr2=0x7fbf801563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbf6801aee0 tx=0x7fbf680a5650 comp rx=0 tx=0).stop 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 shutdown_connections 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf800b1db0 0x7fbf801fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf800b19a0 0x7fbf801563f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 --2- 172.21.15.43:0/2629029408 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf8006e390 0x7fbf800cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 >> 172.21.15.43:0/2629029408 conn(0x7fbf800cee80 msgr2=0x7fbf800b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:50.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 shutdown_connections 2024-09-17T05:31:50.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.255+0000 7fbf85200640 1 -- 172.21.15.43:0/2629029408 wait complete. 2024-09-17T05:31:50.279 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:31:50.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:50.292 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:31:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- 172.21.15.43:0/251813903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2015fe10 msgr2=0x7f4f200b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 --2- 172.21.15.43:0/251813903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2015fe10 0x7f4f200b4b80 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009870 tx=0x7f4f1001c700 comp rx=0 tx=0).stop 2024-09-17T05:31:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- 172.21.15.43:0/251813903 shutdown_connections 2024-09-17T05:31:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 --2- 172.21.15.43:0/251813903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2015fe10 0x7f4f200b4b80 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- 172.21.15.43:0/251813903 >> 172.21.15.43:0/251813903 conn(0x7f4f200b23d0 msgr2=0x7f4f200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:50.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- 172.21.15.43:0/251813903 shutdown_connections 2024-09-17T05:31:50.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- 172.21.15.43:0/251813903 wait complete. 2024-09-17T05:31:50.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 Processor -- start 2024-09-17T05:31:50.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.439+0000 7f4f25400640 1 -- start start 2024-09-17T05:31:50.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f2015fe10 0x7f4f20074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20160c20 0x7f4f20161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2016aff0 con 0x7f4f2006fc90 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2016ae70 con 0x7f4f20160c20 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f25400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2016b170 con 0x7f4f2015fe10 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54714/0 (socket says 172.21.15.43:54714) 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 -- 172.21.15.43:0/1141338681 learned_addr learned my addr 172.21.15.43:0/1141338681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f15e00640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f2015fe10 0x7f4f20074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1f200640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20160c20 0x7f4f20161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 -- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f2015fe10 msgr2=0x7f4f20074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f2015fe10 0x7f4f20074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 -- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20160c20 msgr2=0x7f4f20161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20160c20 0x7f4f20161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 -- 172.21.15.43:0/1141338681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f20158eb0 con 0x7f4f2006fc90 2024-09-17T05:31:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.443+0000 7f4f1e800640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4f100b0040 tx=0x7f4f100a6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100bd020 con 0x7f4f2006fc90 2024-09-17T05:31:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f100041f0 con 0x7f4f2006fc90 2024-09-17T05:31:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10003600 con 0x7f4f2006fc90 2024-09-17T05:31:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f20160420 con 0x7f4f2006fc90 2024-09-17T05:31:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f201595b0 con 0x7f4f2006fc90 2024-09-17T05:31:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f200b4b80 con 0x7f4f2006fc90 2024-09-17T05:31:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.447+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f100c0020 con 0x7f4f2006fc90 2024-09-17T05:31:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.451+0000 7f4f17400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 0x7f4efc075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.451+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f4f1012f720 con 0x7f4f2006fc90 2024-09-17T05:31:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.451+0000 7f4f15e00640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 0x7f4efc075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.451+0000 7f4f15e00640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 0x7f4efc075770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4f08007fd0 tx=0x7f4f08009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.455+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f10016410 con 0x7f4f2006fc90 2024-09-17T05:31:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.663+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f2015da50 con 0x7f4f2006fc90 2024-09-17T05:31:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.663+0000 7f4f17400640 1 -- 172.21.15.43:0/1141338681 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) ==== 95+0+4383 (secure 0 0 0) 0x7f4f100f9b60 con 0x7f4f2006fc90 2024-09-17T05:31:50.666 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:50.666 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":193,"btime":"2024-09-17T05:31:50:461358+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8215,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:49.459323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 193 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 msgr2=0x7f4efc075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 0x7f4efc075770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4f08007fd0 tx=0x7f4f08009040 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 msgr2=0x7f4f20073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4f100b0040 tx=0x7f4f100a6640 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 shutdown_connections 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4efc073260 0x7f4efc075770 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20160c20 0x7f4f20161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4f2015fe10 0x7f4f20074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 --2- 172.21.15.43:0/1141338681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f2006fc90 0x7f4f20073f50 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 >> 172.21.15.43:0/1141338681 conn(0x7f4f200b23d0 msgr2=0x7f4f200ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 shutdown_connections 2024-09-17T05:31:50.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.667+0000 7f4f25400640 1 -- 172.21.15.43:0/1141338681 wait complete. 2024-09-17T05:31:50.689 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-09-17T05:31:50.689 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-09-17T05:31:50.689 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:31:50.689 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:50.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:31:50.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:31:50.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:50.698 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:31:50.747 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:31:50.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:31:50.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:31:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 -- 172.21.15.43:0/1128140863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40b3af0 msgr2=0x7f43c40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 --2- 172.21.15.43:0/1128140863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40b3af0 0x7f43c40ad0b0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f43c4072b60 tx=0x7f43bc01cac0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 -- 172.21.15.43:0/1128140863 shutdown_connections 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 --2- 172.21.15.43:0/1128140863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40b3af0 0x7f43c40ad0b0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 --2- 172.21.15.43:0/1128140863 >> v2:172.21.15.62:3300/0 conn(0x7f43c4155d70 0x7f43c415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 -- 172.21.15.43:0/1128140863 >> 172.21.15.43:0/1128140863 conn(0x7f43c40b23d0 msgr2=0x7f43c40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 -- 172.21.15.43:0/1128140863 shutdown_connections 2024-09-17T05:31:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.947+0000 7f43cb000640 1 -- 172.21.15.43:0/1128140863 wait complete. 2024-09-17T05:31:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 Processor -- start 2024-09-17T05:31:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- start start 2024-09-17T05:31:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c4155d70 0x7f43c40c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c40ca8e0 0x7f43c409db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4168c60 con 0x7f43c406f110 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4168ae0 con 0x7f43c40ca8e0 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43c4168de0 con 0x7f43c4155d70 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54732/0 (socket says 172.21.15.43:54732) 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 -- 172.21.15.43:0/4055886987 learned_addr learned my addr 172.21.15.43:0/4055886987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c9000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c40ca8e0 0x7f43c409db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43bbe00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c4155d70 0x7f43c40c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 -- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c4155d70 msgr2=0x7f43c40c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c4155d70 0x7f43c40c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 -- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c40ca8e0 msgr2=0x7f43c409db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c40ca8e0 0x7f43c409db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 -- 172.21.15.43:0/4055886987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c40cb390 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c3e00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d6d0 tx=0x7f43b400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4017070 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43b4009b10 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4010460 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c40c62e0 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.951+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c40cb910 con 0x7f43c406f110 2024-09-17T05:31:50.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.955+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c40ad0b0 con 0x7f43c406f110 2024-09-17T05:31:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.955+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43b401a070 con 0x7f43c406f110 2024-09-17T05:31:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.955+0000 7f43c0c00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 0x7f43a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.955+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f43b4094970 con 0x7f43c406f110 2024-09-17T05:31:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.955+0000 7f43bbe00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 0x7f43a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.959+0000 7f43bbe00640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 0x7f43a0075740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f43c40cdd80 tx=0x7f43ac009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:50.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:50.959+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43b405ed20 con 0x7f43c406f110 2024-09-17T05:31:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.171+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43c406dac0 con 0x7f43c406f110 2024-09-17T05:31:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43c0c00640 1 -- 172.21.15.43:0/4055886987 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v194) ==== 161+0+0 (secure 0 0 0) 0x7f43b4063750 con 0x7f43c406f110 2024-09-17T05:31:51.473 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 msgr2=0x7f43a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 0x7f43a0075740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f43c40cdd80 tx=0x7f43ac009040 comp rx=0 tx=0).stop 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 msgr2=0x7f43c40ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d6d0 tx=0x7f43b400db90 comp rx=0 tx=0).stop 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 shutdown_connections 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43a0073230 0x7f43a0075740 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c40ca8e0 0x7f43c409db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43c4155d70 0x7f43c40c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 --2- 172.21.15.43:0/4055886987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c406f110 0x7f43c40ca1e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 >> 172.21.15.43:0/4055886987 conn(0x7f43c40b23d0 msgr2=0x7f43c406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:51.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 shutdown_connections 2024-09-17T05:31:51.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.471+0000 7f43cb000640 1 -- 172.21.15.43:0/4055886987 wait complete. 2024-09-17T05:31:51.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:31:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 -- 172.21.15.43:0/3376182871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e24003b40 msgr2=0x7f0e24003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 --2- 172.21.15.43:0/3376182871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e24003b40 0x7f0e24003fb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c2e0 tx=0x7f0e3001cf20 comp rx=0 tx=0).stop 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 -- 172.21.15.43:0/3376182871 shutdown_connections 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 --2- 172.21.15.43:0/3376182871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e24003b40 0x7f0e24003fb0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 -- 172.21.15.43:0/3376182871 >> 172.21.15.43:0/3376182871 conn(0x7f0e3c0b2810 msgr2=0x7f0e3c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 -- 172.21.15.43:0/3376182871 shutdown_connections 2024-09-17T05:31:51.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.651+0000 7f0e43a00640 1 -- 172.21.15.43:0/3376182871 wait complete. 2024-09-17T05:31:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 Processor -- start 2024-09-17T05:31:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 -- start start 2024-09-17T05:31:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e24003b40 0x7f0e3c157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e3c150b80 0x7f0e3c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c0b41b0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c0b4030 con 0x7f0e24003b40 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e43a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c0b4330 con 0x7f0e3c150b80 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54738/0 (socket says 172.21.15.43:54738) 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 -- 172.21.15.43:0/2554319131 learned_addr learned my addr 172.21.15.43:0/2554319131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41000640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e24003b40 0x7f0e3c157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e3be00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e3c150b80 0x7f0e3c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 -- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e3c150b80 msgr2=0x7f0e3c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e3c150b80 0x7f0e3c150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 -- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e24003b40 msgr2=0x7f0e3c157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e24003b40 0x7f0e3c157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 -- 172.21.15.43:0/2554319131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c1514d0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.655+0000 7f0e41a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072810 tx=0x7f0e340019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011660 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e34017070 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c1f87d0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011b90 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c1f7cf0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c0b4800 con 0x7f0e3c158ca0 2024-09-17T05:31:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0e3401c460 con 0x7f0e3c158ca0 2024-09-17T05:31:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 0x7f0e1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.659+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f0e34094fe0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.663+0000 7f0e41000640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 0x7f0e1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.663+0000 7f0e41000640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 0x7f0e1c075710 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0e300ac000 tx=0x7f0e30019d60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.663+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0e3c1514d0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:51.899+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f0e3c1595f0 con 0x7f0e3c158ca0 2024-09-17T05:31:51.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:51.945 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.479+0000 7f0e38c00640 1 -- 172.21.15.43:0/2554319131 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v195) ==== 101+0+0 (secure 0 0 0) 0x7f0e3405f930 con 0x7f0e3c158ca0 2024-09-17T05:31:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 msgr2=0x7f0e1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 0x7f0e1c075710 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0e300ac000 tx=0x7f0e30019d60 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 msgr2=0x7f0e3c1590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c072810 tx=0x7f0e340019c0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 shutdown_connections 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0e1c073200 0x7f0e1c075710 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e3c158ca0 0x7f0e3c1590b0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0e3c150b80 0x7f0e3c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 --2- 172.21.15.43:0/2554319131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e24003b40 0x7f0e3c157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 >> 172.21.15.43:0/2554319131 conn(0x7f0e3c0b2810 msgr2=0x7f0e3c0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 shutdown_connections 2024-09-17T05:31:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.483+0000 7f0e43a00640 1 -- 172.21.15.43:0/2554319131 wait complete. 2024-09-17T05:31:52.505 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:31:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 Processor -- start 2024-09-17T05:31:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 -- start start 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652eac20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652eac90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652eac191a0 con 0x5652eac928b0 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5652eab27a00 con 0x5652eac08cb0 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5652eac19320 con 0x5652eac91010 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54756/0 (socket says 172.21.15.43:54756) 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 -- 172.21.15.43:0/118361565 learned_addr learned my addr 172.21.15.43:0/118361565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077200640 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652eac20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1076800640 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652eac90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 -- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 msgr2=0x5652eac90770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652eac90770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 -- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 msgr2=0x5652eac20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652eac20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 -- 172.21.15.43:0/118361565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652eab26a20 con 0x5652eac928b0 2024-09-17T05:31:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1077c00640 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f107000c6f0 tx=0x7f107000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc71f154c771100f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1075400640 1 -- 172.21.15.43:0/118361565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070004020 con 0x5652eac928b0 2024-09-17T05:31:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1075400640 1 -- 172.21.15.43:0/118361565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10700042b0 con 0x5652eac928b0 2024-09-17T05:31:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f1075400640 1 -- 172.21.15.43:0/118361565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107000aa80 con 0x5652eac928b0 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 -- 172.21.15.43:0/118361565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 msgr2=0x5652eacaf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.563+0000 7f107e09b080 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f107000c6f0 tx=0x7f107000cbb0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- 172.21.15.43:0/118361565 shutdown_connections 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652eacaf3b0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652eac90770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- 172.21.15.43:0/118361565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652eac20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- 172.21.15.43:0/118361565 >> 172.21.15.43:0/118361565 conn(0x5652eabc5d80 msgr2=0x5652eabdc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- 172.21.15.43:0/118361565 shutdown_connections 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- 172.21.15.43:0/118361565 wait complete. 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 Processor -- start 2024-09-17T05:31:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- start start 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652ead496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652ead4c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5652eacacfe0 con 0x5652eac928b0 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5652eacad280 con 0x5652eac08cb0 2024-09-17T05:31:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f107e09b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5652eacad520 con 0x5652eac91010 2024-09-17T05:31:52.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54768/0 (socket says 172.21.15.43:54768) 2024-09-17T05:31:52.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 learned_addr learned my addr 172.21.15.43:0/1887335030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652ead496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652ead4c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 msgr2=0x5652ead4c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5652eac91010 0x5652ead4c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 msgr2=0x5652ead496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652eac08cb0 0x5652ead496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652eab00c10 con 0x5652eac928b0 2024-09-17T05:31:52.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.567+0000 7f1077c00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1070004fd0 tx=0x7f1070004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 -- 172.21.15.43:0/1887335030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070025070 con 0x5652eac928b0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5652ead4db40 con 0x5652eac928b0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 -- 172.21.15.43:0/1887335030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f107000f670 con 0x5652eac928b0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 -- 172.21.15.43:0/1887335030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107000faa0 con 0x5652eac928b0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ead4d8a0 con 0x5652eac928b0 2024-09-17T05:31:52.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 -- 172.21.15.43:0/1887335030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1070028070 con 0x5652eac928b0 2024-09-17T05:31:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 0x7f10580740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f106fe00640 1 -- 172.21.15.43:0/1887335030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f1070094df0 con 0x5652eac928b0 2024-09-17T05:31:52.575 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:31:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5652eacacb90 0x5652ead43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead4d1c0 con 0x5652eacacb90 2024-09-17T05:31:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.571+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 0x7f10580740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.575+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5652eacacb90 0x5652ead43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.575+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 0x7f10580740f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1068004240 tx=0x7f106800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.575+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5652eacacb90 0x5652ead43ff0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.575+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v107'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.575+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead44530 con 0x5652eacacb90 2024-09-17T05:31:52.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v107'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead44900 con 0x5652eacacb90 2024-09-17T05:31:52.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v107'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.581 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:31:52.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56de0 con 0x5652eacacb90 2024-09-17T05:31:52.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.579+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead57180 con 0x5652eacacb90 2024-09-17T05:31:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead55dd0 con 0x5652eacacb90 2024-09-17T05:31:52.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead561c0 con 0x5652eacacb90 2024-09-17T05:31:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v107'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.583+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead565b0 con 0x5652eacacb90 2024-09-17T05:31:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.587+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v107'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.587+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead565b0 con 0x5652eacacb90 2024-09-17T05:31:52.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.587+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v107'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.590 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:31:52.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.587+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.591+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.598 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:31:52.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.595+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56560 con 0x5652eacacb90 2024-09-17T05:31:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.599+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead56940 con 0x5652eacacb90 2024-09-17T05:31:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v107'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v107'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.603+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v107'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.609 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:31:52.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.607+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v107'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v107'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.611+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v107'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.616 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:31:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.615+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652eacacb90 2024-09-17T05:31:52.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5652ead57a30 0x5652ead59e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead567e0 con 0x5652ead57a30 2024-09-17T05:31:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5652ead57a30 0x5652ead59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5652ead57a30 0x5652ead59e50 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'395 uv377 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.619+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5a390 con 0x5652ead57a30 2024-09-17T05:31:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.623+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'396 uv396 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.623+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5a730 con 0x5652ead57a30 2024-09-17T05:31:52.626 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:31:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.623+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.623+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5af80 con 0x5652ead57a30 2024-09-17T05:31:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.627+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.627+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5b320 con 0x5652ead57a30 2024-09-17T05:31:52.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.627+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.627+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5b6c0 con 0x5652ead57a30 2024-09-17T05:31:52.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.627+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v107'400 uv382 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.631+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.631+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v107'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.631+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.631+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v107'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.634 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:31:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.631+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v107'405 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.635+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.639+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v107'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.639+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.639+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v107'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.642 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:31:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.639+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5fc30 con 0x5652ead57a30 2024-09-17T05:31:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v107'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5fc30 con 0x5652ead57a30 2024-09-17T05:31:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v107'410 uv393 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.643+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v107'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v107'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.649 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:31:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ca70 con 0x5652ead57a30 2024-09-17T05:31:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.647+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ca70 con 0x5652ead57a30 2024-09-17T05:31:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5ba60 con 0x5652ead57a30 2024-09-17T05:31:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead57a30 2024-09-17T05:31:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5652ead5c0e0 0x5652ead5e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eb10 con 0x5652ead5c0e0 2024-09-17T05:31:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5652ead5c0e0 0x5652ead5e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.651+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5652ead5c0e0 0x5652ead5e500 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v107'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5ebd0 con 0x5652ead5c0e0 2024-09-17T05:31:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v107'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652ead5c0e0 2024-09-17T05:31:52.659 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:31:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v107'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.655+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead60000 con 0x5652ead5c0e0 2024-09-17T05:31:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead60000 con 0x5652ead5c0e0 2024-09-17T05:31:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652ead5c0e0 2024-09-17T05:31:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.659+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652eacacb90 2024-09-17T05:31:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.663+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'77 uv76 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.663+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652eacacb90 2024-09-17T05:31:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.663+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'78 uv78 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.663+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652eacacb90 2024-09-17T05:31:52.667 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:31:52.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.663+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'79 uv79 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead5eff0 con 0x5652eacacb90 2024-09-17T05:31:52.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'80 uv80 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5652ead5f670 0x5652ead61a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead620a0 con 0x5652ead5f670 2024-09-17T05:31:52.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f1077c00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5652ead5f670 0x5652ead61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.667+0000 7f1077c00640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5652ead5f670 0x5652ead61a90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v107'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead62520 con 0x5652ead5f670 2024-09-17T05:31:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v107'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead629d0 con 0x5652ead5f670 2024-09-17T05:31:52.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v107'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.675 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:31:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.671+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead64280 con 0x5652ead5f670 2024-09-17T05:31:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead64620 con 0x5652ead5f670 2024-09-17T05:31:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead63270 con 0x5652ead5f670 2024-09-17T05:31:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5652ead63b70 0x5652ead669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead66f10 con 0x5652ead63b70 2024-09-17T05:31:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.675+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5652ead63b70 0x5652ead669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.679+0000 7f1077200640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5652ead63b70 0x5652ead669d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.679+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v107'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.679+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead66f10 con 0x5652ead63b70 2024-09-17T05:31:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.679+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v107'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.679+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead66f90 con 0x5652ead63b70 2024-09-17T05:31:52.684 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:31:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v107'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead68390 con 0x5652ead63b70 2024-09-17T05:31:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead68390 con 0x5652ead63b70 2024-09-17T05:31:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.683+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v107'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v107'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.687+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.692 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:31:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v107'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead67380 con 0x5652ead63b70 2024-09-17T05:31:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f1077200640 1 -- 172.21.15.43:0/1887335030 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1068009970 con 0x5652ead63b70 2024-09-17T05:31:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5652ead67890 0x5652ead69cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a360 con 0x5652ead67890 2024-09-17T05:31:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.691+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5652ead67890 0x5652ead69cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.695+0000 7f1076800640 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5652ead67890 0x5652ead69cb0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.695+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v107'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.695+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6a360 con 0x5652ead67890 2024-09-17T05:31:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.695+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v107'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.695+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead67890 2024-09-17T05:31:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v107'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.700 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:31:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead67890 2024-09-17T05:31:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead67890 2024-09-17T05:31:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.699+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652eacacb90 2024-09-17T05:31:52.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v107'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652eacacb90 2024-09-17T05:31:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v107'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652eacacb90 2024-09-17T05:31:52.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v107'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.707 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:31:52.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.703+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b7c0 con 0x5652eacacb90 2024-09-17T05:31:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b7c0 con 0x5652eacacb90 2024-09-17T05:31:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652eacacb90 2024-09-17T05:31:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652eacacb90 2024-09-17T05:31:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.707+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead5c0e0 2024-09-17T05:31:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.711+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v107'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.711+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead5c0e0 2024-09-17T05:31:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.711+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v107'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.711+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6a7b0 con 0x5652ead5c0e0 2024-09-17T05:31:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v107'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.715 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:31:52.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead62e80 con 0x5652ead5c0e0 2024-09-17T05:31:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead62e80 con 0x5652ead5c0e0 2024-09-17T05:31:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.715+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead62e80 con 0x5652ead5c0e0 2024-09-17T05:31:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v107'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead62e80 con 0x5652ead5c0e0 2024-09-17T05:31:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v107'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead62e60 con 0x5652ead5c0e0 2024-09-17T05:31:52.722 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:31:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v107'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.719+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead62e60 con 0x5652ead5c0e0 2024-09-17T05:31:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.723+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.723+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6d5c0 con 0x5652ead5c0e0 2024-09-17T05:31:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.723+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead5c0e0 2024-09-17T05:31:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.723+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6ac60 con 0x5652ead5f670 2024-09-17T05:31:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.727+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v107'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.727+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6b050 con 0x5652ead5f670 2024-09-17T05:31:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.727+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v107'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.727+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b050 con 0x5652ead5f670 2024-09-17T05:31:52.731 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:31:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.727+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v107'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.731+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead5f670 2024-09-17T05:31:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.731+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v107'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.731+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead5f670 2024-09-17T05:31:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.731+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v107'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.731+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead5f670 2024-09-17T05:31:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v107'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead5f670 2024-09-17T05:31:52.737 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:31:52.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v107'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6c010 con 0x5652ead5f670 2024-09-17T05:31:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v107'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.735+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6c010 con 0x5652ead5f670 2024-09-17T05:31:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead5f670 2024-09-17T05:31:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1070027030 con 0x5652ead5f670 2024-09-17T05:31:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v107'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.739+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v107'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v107'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.745 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:31:52.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v107'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.743+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v107'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v107'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v107'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.747+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.752 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:31:52.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v107'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v107'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v107'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.751+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v107'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v107'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.758 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:31:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v107'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.755+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v107'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v107'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b000 con 0x5652ead67890 2024-09-17T05:31:52.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v107'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.759+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5652ead6b440 con 0x5652ead67890 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f1076800640 1 -- 172.21.15.43:0/1887335030 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v107'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f10640ac990 con 0x5652ead67890 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5652ead57a30 msgr2=0x5652ead59e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5652ead57a30 0x5652ead59e50 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5652ead63b70 msgr2=0x5652ead669d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5652ead63b70 0x5652ead669d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5652ead67890 msgr2=0x5652ead69cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5652ead67890 0x5652ead69cb0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5652ead5c0e0 msgr2=0x5652ead5e500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5652ead5c0e0 0x5652ead5e500 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5652eacacb90 msgr2=0x5652ead43ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5652eacacb90 0x5652ead43ff0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 reap_dead start 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5652ead5f670 msgr2=0x5652ead61a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5652ead5f670 0x5652ead61a90 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f1077c00640 1 -- 172.21.15.43:0/1887335030 reap_dead start 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 msgr2=0x7f10580740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 0x7f10580740f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1068004240 tx=0x7f106800d040 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 msgr2=0x5652ead4cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1070004fd0 tx=0x7f1070004090 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 shutdown_connections 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1058071be0 0x7f10580740f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 --2- 172.21.15.43:0/1887335030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5652eac928b0 0x5652ead4cc80 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 >> 172.21.15.43:0/1887335030 conn(0x5652eabc5d80 msgr2=0x5652eabdc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 shutdown_connections 2024-09-17T05:31:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.763+0000 7f107e09b080 1 -- 172.21.15.43:0/1887335030 wait complete. 2024-09-17T05:31:52.769 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-09-17T05:31:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 Processor -- start 2024-09-17T05:31:52.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 -- start start 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496007bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 0x56249600a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562495f10a00 con 0x562495fc54d0 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562495f6ef40 con 0x562496078e40 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc15b985080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562496079b80 con 0x562495ff1cb0 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc154c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc154c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54800/0 (socket says 172.21.15.43:54800) 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.871+0000 7fc154c00640 1 -- 172.21.15.43:0/1814922907 learned_addr learned my addr 172.21.15.43:0/1814922907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc155600640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 0x56249600a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc14fe00640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496007bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 -- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 msgr2=0x562496007bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496007bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 -- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 msgr2=0x56249600a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 0x56249600a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 -- 172.21.15.43:0/1814922907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562495ee9c10 con 0x562495fc54d0 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc155600640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 0x56249600a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc154c00640 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b230 tx=0x7fc14800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b78f9b098be8078 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc14ea00640 1 -- 172.21.15.43:0/1814922907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148016070 con 0x562495fc54d0 2024-09-17T05:31:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc14ea00640 1 -- 172.21.15.43:0/1814922907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc14800d4c0 con 0x562495fc54d0 2024-09-17T05:31:52.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc14ea00640 1 -- 172.21.15.43:0/1814922907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14800d710 con 0x562495fc54d0 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- 172.21.15.43:0/1814922907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 msgr2=0x56249611f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b230 tx=0x7fc14800b6f0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- 172.21.15.43:0/1814922907 shutdown_connections 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562496078e40 0x56249600a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496007bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 --2- 172.21.15.43:0/1814922907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495fc54d0 0x56249611f920 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- 172.21.15.43:0/1814922907 >> 172.21.15.43:0/1814922907 conn(0x562495faed80 msgr2=0x562495fc0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- 172.21.15.43:0/1814922907 shutdown_connections 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- 172.21.15.43:0/1814922907 wait complete. 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 Processor -- start 2024-09-17T05:31:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.875+0000 7fc15b985080 1 -- start start 2024-09-17T05:31:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 0x562496094970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496077980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562496078e40 0x562495fe4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56249600bf40 con 0x562496078e40 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562495fc10b0 con 0x562495fc54d0 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562495fc1350 con 0x562495ff1cb0 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc155600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562496078e40 0x562495fe4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 0x562496094970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc14fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496077980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 0x562496094970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:50632/0 (socket says 172.21.15.43:50632) 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 -- 172.21.15.43:0/2988590036 learned_addr learned my addr 172.21.15.43:0/2988590036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 -- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 msgr2=0x562496077980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562495ff1cb0 0x562496077980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 -- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562496078e40 msgr2=0x562495fe4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562496078e40 0x562495fe4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 -- 172.21.15.43:0/2988590036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56249612fc90 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc155600640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562496078e40 0x562495fe4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc154c00640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 0x562496094970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c10 tx=0x7fc148010f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc14d600640 1 -- 172.21.15.43:0/2988590036 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148016040 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc14d600640 1 -- 172.21.15.43:0/2988590036 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc148001a60 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc14d600640 1 -- 172.21.15.43:0/2988590036 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14800d4c0 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56249612ff00 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.879+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562496130440 con 0x562495fc54d0 2024-09-17T05:31:52.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc14d600640 1 -- 172.21.15.43:0/2988590036 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc148022020 con 0x562495fc54d0 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc14d600640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc134071be0 0x7fc1340740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc14d600640 1 -- 172.21.15.43:0/2988590036 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7fc148093a40 con 0x562495fc54d0 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc14fe00640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc134071be0 0x7fc1340740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc15b985080 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562496130b80 0x562496130f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5624961314d0 con 0x562496130b80 2024-09-17T05:31:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc155600640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562496130b80 0x562496130f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:52.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc14fe00640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc134071be0 0x7fc1340740f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c670 tx=0x7fc140004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc155600640 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562496130b80 0x562496130f90 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.883+0000 7fc155600640 1 -- 172.21.15.43:0/2988590036 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v107'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc150002050 con 0x562496130b80 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562496130b80 msgr2=0x562496130f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562496130b80 0x562496130f90 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc134071be0 msgr2=0x7fc1340740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc134071be0 0x7fc1340740f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c670 tx=0x7fc140004890 comp rx=0 tx=0).stop 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 msgr2=0x562496094970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 --2- 172.21.15.43:0/2988590036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562495fc54d0 0x562496094970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c10 tx=0x7fc148010f80 comp rx=0 tx=0).stop 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc155600640 1 -- 172.21.15.43:0/2988590036 reap_dead start 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 shutdown_connections 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 >> 172.21.15.43:0/2988590036 conn(0x562495faed80 msgr2=0x562495fc0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 shutdown_connections 2024-09-17T05:31:52.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:52.887+0000 7fc15b985080 1 -- 172.21.15.43:0/2988590036 wait complete. 2024-09-17T05:31:52.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:31:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 -- 172.21.15.43:0/925034866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 msgr2=0x7fe620160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 --2- 172.21.15.43:0/925034866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe620160620 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fe620162580 tx=0x7fe61801caf0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 -- 172.21.15.43:0/925034866 shutdown_connections 2024-09-17T05:31:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 --2- 172.21.15.43:0/925034866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe620160620 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 --2- 172.21.15.43:0/925034866 >> v2:172.21.15.62:3300/0 conn(0x7fe6200b4570 0x7fe6200b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 -- 172.21.15.43:0/925034866 >> 172.21.15.43:0/925034866 conn(0x7fe6200b2810 msgr2=0x7fe6200b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 -- 172.21.15.43:0/925034866 shutdown_connections 2024-09-17T05:31:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 -- 172.21.15.43:0/925034866 wait complete. 2024-09-17T05:31:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.095+0000 7fe626800640 1 Processor -- start 2024-09-17T05:31:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- start start 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe62006f940 0x7fe62015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6200b4570 0x7fe620156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe62016bea0 con 0x7fe620160210 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe62016c020 con 0x7fe62006f940 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe62016c1a0 con 0x7fe6200b4570 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54836/0 (socket says 172.21.15.43:54836) 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 -- 172.21.15.43:0/1500561558 learned_addr learned my addr 172.21.15.43:0/1500561558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61f400640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe62006f940 0x7fe62015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe616a00640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6200b4570 0x7fe620156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 -- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6200b4570 msgr2=0x7fe620156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6200b4570 0x7fe620156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 -- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe62006f940 msgr2=0x7fe62015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe62006f940 0x7fe62015e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 -- 172.21.15.43:0/1500561558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62014c320 con 0x7fe620160210 2024-09-17T05:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe61fe00640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe61801cfd0 tx=0x7fe618003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6180b2040 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6180bb070 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618004330 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62014bb60 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.099+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201576a0 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe620157900 con 0x7fe620160210 2024-09-17T05:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6180044d0 con 0x7fe620160210 2024-09-17T05:31:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe617e00640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 0x7fe5fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7fe61812ef50 con 0x7fe620160210 2024-09-17T05:31:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe61f400640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 0x7fe5fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.103+0000 7fe61f400640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 0x7fe5fc075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe610007d70 tx=0x7fe610007d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.107+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe618016770 con 0x7fe620160210 2024-09-17T05:31:53.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.319+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe6200b4980 con 0x7fe620160210 2024-09-17T05:31:53.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.487+0000 7fe617e00640 1 -- 172.21.15.43:0/1500561558 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v196) ==== 162+0+0 (secure 0 0 0) 0x7fe6180f9430 con 0x7fe620160210 2024-09-17T05:31:53.491 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:31:53.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 msgr2=0x7fe5fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 0x7fe5fc075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe610007d70 tx=0x7fe610007d00 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 msgr2=0x7fe6201570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe61801cfd0 tx=0x7fe618003f40 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 shutdown_connections 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe5fc073200 0x7fe5fc075710 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe620160210 0x7fe6201570a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6200b4570 0x7fe620156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 --2- 172.21.15.43:0/1500561558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe62006f940 0x7fe62015e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 >> 172.21.15.43:0/1500561558 conn(0x7fe6200b2810 msgr2=0x7fe6200b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 shutdown_connections 2024-09-17T05:31:53.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.491+0000 7fe626800640 1 -- 172.21.15.43:0/1500561558 wait complete. 2024-09-17T05:31:53.506 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:53.504+0000 7f7c55800640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-17T05:31:53.506 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:31:53.504+0000 7f7c55800640 -1 mds.0.197 unhandled write error (22) Invalid argument, force readonly... 2024-09-17T05:31:53.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 -- 172.21.15.43:0/230288608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c15adc0 msgr2=0x7f9e6c06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 --2- 172.21.15.43:0/230288608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c15adc0 0x7f9e6c06ce60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009900 tx=0x7f9e5c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 -- 172.21.15.43:0/230288608 shutdown_connections 2024-09-17T05:31:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 --2- 172.21.15.43:0/230288608 >> v2:172.21.15.62:3300/0 conn(0x7f9e6c0cc160 0x7f9e6c06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 --2- 172.21.15.43:0/230288608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c15adc0 0x7f9e6c06ce60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 -- 172.21.15.43:0/230288608 >> 172.21.15.43:0/230288608 conn(0x7f9e6c0b23d0 msgr2=0x7f9e6c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 -- 172.21.15.43:0/230288608 shutdown_connections 2024-09-17T05:31:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.671+0000 7f9e72a00640 1 -- 172.21.15.43:0/230288608 wait complete. 2024-09-17T05:31:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 Processor -- start 2024-09-17T05:31:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 -- start start 2024-09-17T05:31:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0cc6d0 0x7f9e6c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c15adc0 0x7f9e6c1fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1662e0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c166160 con 0x7f9e6c15adc0 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e72a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c166460 con 0x7f9e6c0cc6d0 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54852/0 (socket says 172.21.15.43:54852) 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.675+0000 7f9e6be00640 1 -- 172.21.15.43:0/2196481749 learned_addr learned my addr 172.21.15.43:0/2196481749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e70a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c15adc0 0x7f9e6c1fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e63e00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0cc6d0 0x7f9e6c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 -- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0cc6d0 msgr2=0x7f9e6c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0cc6d0 0x7f9e6c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 -- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c15adc0 msgr2=0x7f9e6c1fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c15adc0 0x7f9e6c1fd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 -- 172.21.15.43:0/2196481749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c164c60 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e6be00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009850 tx=0x7f9e5c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0b2040 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c0043b0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c005270 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c1fcdb0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c0736f0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c0ce100 con 0x7f9e6c0cc160 2024-09-17T05:31:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.679+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9e5c0bd060 con 0x7f9e6c0cc160 2024-09-17T05:31:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.683+0000 7f9e68c00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 0x7f9e48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.683+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f9e5c12e120 con 0x7f9e6c0cc160 2024-09-17T05:31:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.683+0000 7f9e63e00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 0x7f9e48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.683+0000 7f9e63e00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 0x7f9e48075710 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0b47a0 tx=0x7f9e54009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.687+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9e5c0164d0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e6c0cdae0 con 0x7f9e6c0cc160 2024-09-17T05:31:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e68c00640 1 -- 172.21.15.43:0/2196481749 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) ==== 95+0+4392 (secure 0 0 0) 0x7f9e5c0f8b20 con 0x7f9e6c0cc160 2024-09-17T05:31:53.897 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:53.897 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":197,"btime":"2024-09-17T05:31:53:494578+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":197,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:53.494560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8215":{"gid":8215,"name":"d","rank":0,"incarnation":197,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 197 2024-09-17T05:31:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 msgr2=0x7f9e48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 0x7f9e48075710 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0b47a0 tx=0x7f9e54009040 comp rx=0 tx=0).stop 2024-09-17T05:31:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 msgr2=0x7f9e6c072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009850 tx=0x7f9e5c0a56b0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 shutdown_connections 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073200 0x7f9e48075710 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c15adc0 0x7f9e6c1fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0cc6d0 0x7f9e6c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 --2- 172.21.15.43:0/2196481749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c0cc160 0x7f9e6c072f00 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 >> 172.21.15.43:0/2196481749 conn(0x7f9e6c0b23d0 msgr2=0x7f9e6c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 shutdown_connections 2024-09-17T05:31:53.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:53.895+0000 7f9e72a00640 1 -- 172.21.15.43:0/2196481749 wait complete. 2024-09-17T05:31:54.211 INFO:tasks.ceph.mds.c.smithi043.stderr: -38> 2024-09-17T05:31:49.455+0000 7f22c9e00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:31:54.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:54.329 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:31:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- 172.21.15.43:0/2441165681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9204004da0 msgr2=0x7f9204005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 --2- 172.21.15.43:0/2441165681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9204004da0 0x7f9204005250 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9218009b40 tx=0x7f921801d270 comp rx=0 tx=0).stop 2024-09-17T05:31:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f921fe00640 1 -- 172.21.15.43:0/2441165681 reap_dead start 2024-09-17T05:31:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- 172.21.15.43:0/2441165681 shutdown_connections 2024-09-17T05:31:54.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- 172.21.15.43:0/2441165681 >> 172.21.15.43:0/2441165681 conn(0x7f92200b23d0 msgr2=0x7f92200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:54.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- 172.21.15.43:0/2441165681 shutdown_connections 2024-09-17T05:31:54.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- 172.21.15.43:0/2441165681 wait complete. 2024-09-17T05:31:54.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 Processor -- start 2024-09-17T05:31:54.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.479+0000 7f9226400640 1 -- start start 2024-09-17T05:31:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9204004da0 0x7f92200cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200cbf70 0x7f92200c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92200ad7c0 con 0x7f92201f5b60 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f922006f5f0 con 0x7f92200cbf70 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f92200ad940 con 0x7f9204004da0 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921f400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9204004da0 0x7f92200cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54406/0 (socket says 172.21.15.43:54406) 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 -- 172.21.15.43:0/2888340151 learned_addr learned my addr 172.21.15.43:0/2888340151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9216a00640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200cbf70 0x7f92200c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 -- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9204004da0 msgr2=0x7f92200cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9204004da0 0x7f92200cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 -- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200cbf70 msgr2=0x7f92200c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200cbf70 0x7f92200c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 -- 172.21.15.43:0/2888340151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92201f9950 con 0x7f92201f5b60 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f921fe00640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9218009b40 tx=0x7f9218005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218009710 con 0x7f92201f5b60 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92180be070 con 0x7f92201f5b60 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201f90b0 con 0x7f92201f5b60 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218004c20 con 0x7f92201f5b60 2024-09-17T05:31:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.483+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201f9670 con 0x7f92201f5b60 2024-09-17T05:31:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92200ad0b0 con 0x7f92201f5b60 2024-09-17T05:31:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9218004dc0 con 0x7f92201f5b60 2024-09-17T05:31:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f9217e00640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 0x7f91fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f92181300d0 con 0x7f92201f5b60 2024-09-17T05:31:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f921f400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 0x7f91fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.487+0000 7f921f400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 0x7f91fc075710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9210002240 tx=0x7f9210006b30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.491+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92201f9950 con 0x7f92201f5b60 2024-09-17T05:31:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.703+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92201f8e40 con 0x7f92201f5b60 2024-09-17T05:31:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9217e00640 1 -- 172.21.15.43:0/2888340151 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5221 (secure 0 0 0) 0x7f9218016f80 con 0x7f92201f5b60 2024-09-17T05:31:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:31:54.708 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":198,"btime":"2024-09-17T05:31:54:499761+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3742224456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3742224456},{"type":"v1","addr":"172.21.15.43:6835","nonce":3742224456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":198}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:53.511612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8215":{"gid":8215,"name":"d","rank":0,"incarnation":197,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:31:54.708 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 198 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 msgr2=0x7f91fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 0x7f91fc075710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9210002240 tx=0x7f9210006b30 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 msgr2=0x7f92201f5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9218009b40 tx=0x7f9218005f20 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 shutdown_connections 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f91fc073200 0x7f91fc075710 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92200cbf70 0x7f92200c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92201f5b60 0x7f92201f5f70 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 --2- 172.21.15.43:0/2888340151 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9204004da0 0x7f92200cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 >> 172.21.15.43:0/2888340151 conn(0x7f92200b23d0 msgr2=0x7f92200b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 shutdown_connections 2024-09-17T05:31:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:54.707+0000 7f9226400640 1 -- 172.21.15.43:0/2888340151 wait complete. 2024-09-17T05:31:54.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 198, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:31:53.511612+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 107, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8215}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8215': {'gid': 8215, 'name': 'd', 'rank': 0, 'incarnation': 197, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.62:6835/2196052200', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2196052200}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2196052200}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:31:54.730 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8215:up:reconnect 2024-09-17T05:31:54.730 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:31:59.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:31:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 -- 172.21.15.43:0/3700730046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 msgr2=0x7f1438004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 --2- 172.21.15.43:0/3700730046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f1438004c20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f144400c2e0 tx=0x7f144401d980 comp rx=0 tx=0).stop 2024-09-17T05:31:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 -- 172.21.15.43:0/3700730046 shutdown_connections 2024-09-17T05:31:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 --2- 172.21.15.43:0/3700730046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f1438004c20 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 -- 172.21.15.43:0/3700730046 >> 172.21.15.43:0/3700730046 conn(0x7f14540b23d0 msgr2=0x7f14540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:31:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 -- 172.21.15.43:0/3700730046 shutdown_connections 2024-09-17T05:31:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.903+0000 7f1459600640 1 -- 172.21.15.43:0/3700730046 wait complete. 2024-09-17T05:31:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 Processor -- start 2024-09-17T05:31:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- start start 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14541fc7c0 0x7f14541fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14540cfd60 0x7f14540d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14540ad6f0 con 0x7f1438004810 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f14540ad570 con 0x7f14541fc7c0 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f14540ad870 con 0x7f14540cfd60 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54432/0 (socket says 172.21.15.43:54432) 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 -- 172.21.15.43:0/3741450916 learned_addr learned my addr 172.21.15.43:0/3741450916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 -- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14540cfd60 msgr2=0x7f14540d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14540cfd60 0x7f14540d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 -- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14541fc7c0 msgr2=0x7f14541fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:31:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14541fc7c0 0x7f14541fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:31:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 -- 172.21.15.43:0/3741450916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14540c9f00 con 0x7f1438004810 2024-09-17T05:31:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1452a00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f143c011a30 tx=0x7f143c011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c01e070 con 0x7f1438004810 2024-09-17T05:31:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f143c00fc10 con 0x7f1438004810 2024-09-17T05:31:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14540ca1a0 con 0x7f1438004810 2024-09-17T05:31:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c019410 con 0x7f1438004810 2024-09-17T05:31:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.907+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1454073130 con 0x7f1438004810 2024-09-17T05:31:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14540addd0 con 0x7f1438004810 2024-09-17T05:31:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f143c021070 con 0x7f1438004810 2024-09-17T05:31:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f144aa00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 0x7f1430075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:31:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f143c0951e0 con 0x7f1438004810 2024-09-17T05:31:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f144be00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 0x7f1430075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:31:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.911+0000 7f144be00640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 0x7f1430075710 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14440ab000 tx=0x7f1438005840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:31:59.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:31:59.915+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f143c05f480 con 0x7f1438004810 2024-09-17T05:32:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.127+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145415bee0 con 0x7f1438004810 2024-09-17T05:32:00.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.127+0000 7f144aa00640 1 -- 172.21.15.43:0/3741450916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5225 (secure 0 0 0) 0x7f143c063eb0 con 0x7f1438004810 2024-09-17T05:32:00.130 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:00.130 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":200,"btime":"2024-09-17T05:31:56:508616+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3742224456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3742224456},{"type":"v1","addr":"172.21.15.43:6835","nonce":3742224456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":198}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:56.508607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8215":{"gid":8215,"name":"d","rank":0,"incarnation":197,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8215,"qdb_cluster":[8215]},"id":6}]} 2024-09-17T05:32:00.131 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 200 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 msgr2=0x7f1430075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 0x7f1430075710 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14440ab000 tx=0x7f1438005840 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 msgr2=0x7f14540c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f143c011a30 tx=0x7f143c011ef0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 shutdown_connections 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1430073200 0x7f1430075710 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14541fc7c0 0x7f14541fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14540cfd60 0x7f14540d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 --2- 172.21.15.43:0/3741450916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1438004810 0x7f14540c6b50 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 >> 172.21.15.43:0/3741450916 conn(0x7f14540b23d0 msgr2=0x7f145406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 shutdown_connections 2024-09-17T05:32:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.131+0000 7f1459600640 1 -- 172.21.15.43:0/3741450916 wait complete. 2024-09-17T05:32:00.476 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:00.476+0000 7f7c59400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:32:00.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 -- 172.21.15.43:0/3780870512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec0b3af0 msgr2=0x7f88ec0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 --2- 172.21.15.43:0/3780870512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec0b3af0 0x7f88ec0ad0b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009c80 tx=0x7f88e401cce0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 -- 172.21.15.43:0/3780870512 shutdown_connections 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 --2- 172.21.15.43:0/3780870512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec0b3af0 0x7f88ec0ad0b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 --2- 172.21.15.43:0/3780870512 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec155d70 0x7f88ec15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 -- 172.21.15.43:0/3780870512 >> 172.21.15.43:0/3780870512 conn(0x7f88ec0b23d0 msgr2=0x7f88ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 -- 172.21.15.43:0/3780870512 shutdown_connections 2024-09-17T05:32:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.715+0000 7f88f3a00640 1 -- 172.21.15.43:0/3780870512 wait complete. 2024-09-17T05:32:00.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 Processor -- start 2024-09-17T05:32:00.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- start start 2024-09-17T05:32:00.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec06f110 0x7f88ec09d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec1fc9a0 0x7f88ec0cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec168cb0 con 0x7f88ec155d70 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec168b30 con 0x7f88ec1fc9a0 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec168e30 con 0x7f88ec06f110 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec1fc9a0 0x7f88ec0cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f1000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec06f110 0x7f88ec09d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54452/0 (socket says 172.21.15.43:54452) 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 -- 172.21.15.43:0/2856574977 learned_addr learned my addr 172.21.15.43:0/2856574977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 -- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec06f110 msgr2=0x7f88ec09d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec06f110 0x7f88ec09d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 -- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec1fc9a0 msgr2=0x7f88ec0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec1fc9a0 0x7f88ec0cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 -- 172.21.15.43:0/2856574977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec09ddf0 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b440 tx=0x7f88d800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8015020 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88d8011980 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec14fbc0 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011dd0 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.719+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec073510 con 0x7f88ec155d70 2024-09-17T05:32:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec0ad0b0 con 0x7f88ec155d70 2024-09-17T05:32:00.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f88d8018020 con 0x7f88ec155d70 2024-09-17T05:32:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88e8c00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 0x7f88cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f88d80944e0 con 0x7f88ec155d70 2024-09-17T05:32:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88f1000640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 0x7f88cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:00.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.723+0000 7f88f1000640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 0x7f88cc075740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0715c0 tx=0x7f88e000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:00.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.727+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f88d801f030 con 0x7f88ec155d70 2024-09-17T05:32:00.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.939+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88ec14f980 con 0x7f88ec155d70 2024-09-17T05:32:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.939+0000 7f88e8c00640 1 -- 172.21.15.43:0/2856574977 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5225 (secure 0 0 0) 0x7f88d800d040 con 0x7f88ec155d70 2024-09-17T05:32:00.942 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:00.942 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":200,"btime":"2024-09-17T05:31:56:508616+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3742224456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3742224456},{"type":"v1","addr":"172.21.15.43:6835","nonce":3742224456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":198}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:56.508607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8215":{"gid":8215,"name":"d","rank":0,"incarnation":197,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8215,"qdb_cluster":[8215]},"id":6}]} 2024-09-17T05:32:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 200 2024-09-17T05:32:00.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 msgr2=0x7f88cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 0x7f88cc075740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0715c0 tx=0x7f88e000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 msgr2=0x7f88ec0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b440 tx=0x7f88d800b900 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 shutdown_connections 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88cc073230 0x7f88cc075740 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec1fc9a0 0x7f88ec0cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88ec155d70 0x7f88ec0cf4d0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 --2- 172.21.15.43:0/2856574977 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec06f110 0x7f88ec09d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 >> 172.21.15.43:0/2856574977 conn(0x7f88ec0b23d0 msgr2=0x7f88ec152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 shutdown_connections 2024-09-17T05:32:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:00.943+0000 7f88f3a00640 1 -- 172.21.15.43:0/2856574977 wait complete. 2024-09-17T05:32:00.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 200, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:31:56.508607+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 107, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8215}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8215': {'gid': 8215, 'name': 'd', 'rank': 0, 'incarnation': 197, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6835/2196052200', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2196052200}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2196052200}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8215, 'qdb_cluster': [8215]} 2024-09-17T05:32:00.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:32:00.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- 172.21.15.43:0/13855849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55380737c0 msgr2=0x7f55380700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 --2- 172.21.15.43:0/13855849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55380737c0 0x7f55380700a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5534009bf0 tx=0x7f553401ca80 comp rx=0 tx=0).stop 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- 172.21.15.43:0/13855849 shutdown_connections 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 --2- 172.21.15.43:0/13855849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55380737c0 0x7f55380700a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- 172.21.15.43:0/13855849 >> 172.21.15.43:0/13855849 conn(0x7f55380b23d0 msgr2=0x7f55380d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- 172.21.15.43:0/13855849 shutdown_connections 2024-09-17T05:32:01.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- 172.21.15.43:0/13855849 wait complete. 2024-09-17T05:32:01.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 Processor -- start 2024-09-17T05:32:01.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.123+0000 7f5540800640 1 -- start start 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 0x7f55380c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5538099280 0x7f553814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f553816b6e0 con 0x7f553806fc90 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f553816b560 con 0x7f5538099280 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f553816b860 con 0x7f553815fe10 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54478/0 (socket says 172.21.15.43:54478) 2024-09-17T05:32:01.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 -- 172.21.15.43:0/3496285305 learned_addr learned my addr 172.21.15.43:0/3496285305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553d400640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 0x7f55380c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553e800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5538099280 0x7f553814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 -- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 msgr2=0x7f55380c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 0x7f55380c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 -- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5538099280 msgr2=0x7f553814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5538099280 0x7f553814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 -- 172.21.15.43:0/3496285305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55380c8970 con 0x7f553806fc90 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553d400640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 0x7f55380c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f553de00640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c420 tx=0x7f552c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5532000640 1 -- 172.21.15.43:0/3496285305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c011790 con 0x7f553806fc90 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5532000640 1 -- 172.21.15.43:0/3496285305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f552c017070 con 0x7f553806fc90 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5532000640 1 -- 172.21.15.43:0/3496285305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c011ce0 con 0x7f553806fc90 2024-09-17T05:32:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55380c8c50 con 0x7f553806fc90 2024-09-17T05:32:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.127+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55380c9210 con 0x7f553806fc90 2024-09-17T05:32:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5532000640 1 -- 172.21.15.43:0/3496285305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f552c01c460 con 0x7f553806fc90 2024-09-17T05:32:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5532000640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 0x7f551c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5532000640 1 -- 172.21.15.43:0/3496285305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f552c0946d0 con 0x7f553806fc90 2024-09-17T05:32:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 Processor -- start 2024-09-17T05:32:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 -- start start 2024-09-17T05:32:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f553d400640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 0x7f551c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f553d400640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 0x7f551c075710 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f55380af8c0 tx=0x7f5528009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x55625250b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625250c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5562523e0410 con 0x5562523ba950 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55625231db40 con 0x556251fca460 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55625230cf70 con 0x5562523a3ca0 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f553d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f553d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54492/0 (socket says 172.21.15.43:54492) 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.131+0000 7f553d400640 1 -- 172.21.15.43:0/2492892329 learned_addr learned my addr 172.21.15.43:0/2492892329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553e800640 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625250c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553de00640 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x55625250b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 -- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 msgr2=0x55625250c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625250c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 -- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 msgr2=0x55625250b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x55625250b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 -- 172.21.15.43:0/2492892329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562522d45b0 con 0x5562523ba950 2024-09-17T05:32:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f553d400640 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5528001e30 tx=0x7f5528011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6c2eedb34cddcc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5526600640 1 -- 172.21.15.43:0/2492892329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528003600 con 0x5562523ba950 2024-09-17T05:32:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5526600640 1 -- 172.21.15.43:0/2492892329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5528003890 con 0x5562523ba950 2024-09-17T05:32:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5526600640 1 -- 172.21.15.43:0/2492892329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528003cc0 con 0x5562523ba950 2024-09-17T05:32:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- 172.21.15.43:0/2492892329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 msgr2=0x55625250bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5528001e30 tx=0x7f5528011000 comp rx=0 tx=0).stop 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- 172.21.15.43:0/2492892329 shutdown_connections 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625250c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x55625250bbe0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 --2- 172.21.15.43:0/2492892329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x55625250b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- 172.21.15.43:0/2492892329 >> 172.21.15.43:0/2492892329 conn(0x556252490d90 msgr2=0x5562524c0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- 172.21.15.43:0/2492892329 shutdown_connections 2024-09-17T05:32:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- 172.21.15.43:0/2492892329 wait complete. 2024-09-17T05:32:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 -- start start 2024-09-17T05:32:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.135+0000 7f5547d66000 1 Processor -- start 2024-09-17T05:32:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x556252512920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 0x5562525133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5562522a75f0 con 0x5562523a3ca0 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556252310d80 con 0x556251fca460 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5562522b3520 con 0x5562523ba950 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 10 client.0 _ll_register_callbacks cb 0x5562522b7830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54506/0 (socket says 172.21.15.43:54506) 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 -- 172.21.15.43:0/4118744447 learned_addr learned my addr 172.21.15.43:0/4118744447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553de00640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x556252512920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553e800640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 0x5562525133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 -- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 msgr2=0x5562525133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 0x5562525133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 -- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 msgr2=0x556252512920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x556252512920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 -- 172.21.15.43:0/4118744447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562522ab190 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553e800640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 0x5562525133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553d400640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x5562523177e0 tx=0x7f5528003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f5ef41f289bd82 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528003c30 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f5528003dd0 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562522a5110 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) ==== 24+0+0 (secure 0 0 0) 0x7f5528007030 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 10 client.8745 fetch_fsmap learned FSMap version 200 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) -- 0x556252391f10 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5562522a5110 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 200) ==== 3502+0+0 (secure 0 0 0) 0x7f55280071d0 con 0x5562523a3ca0 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 10 client.8745 fetch_fsmap finished waiting for FSMap version 200 2024-09-17T05:32:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 10 client.8745 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:32:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 20 client.8745 populate_metadata read hostname 'smithi043' 2024-09-17T05:32:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251faf7d0 0x55625251cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 4 client.8745 mds_command: new command op to 8215 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:32:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 --> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55625231fa90 con 0x556251faf7d0 2024-09-17T05:32:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553de00640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251faf7d0 0x55625251cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f553de00640 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251faf7d0 0x55625251cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31d82276025a16e server_cookie=c04fb6b5e18d06b7 in_seq=0 out_seq=0 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.139+0000 7f551aa00640 10 client.8745 ms_handle_connect on v2:172.21.15.62:6834/2196052200 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f551aa00640 1 -- 172.21.15.43:0/4118744447 <== mds.0 v2:172.21.15.62:6834/2196052200 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55625231fa90 con 0x556251faf7d0 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f551aa00640 10 client.8745 handle_command_reply: tid=0 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 client.8745 shutdown 2024-09-17T05:32:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 msgr2=0x556252512e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x5562523177e0 tx=0x7f5528003830 comp rx=0 tx=0).stop 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 shutdown_connections 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251faf7d0 0x55625251cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523ba950 0x5562525133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523a3ca0 0x556252512e60 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 --2- 172.21.15.43:0/4118744447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556251fca460 0x556252512920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f551aa00640 0 client.8745 ms_handle_reset on v2:172.21.15.62:6834/2196052200 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 >> 172.21.15.43:0/4118744447 conn(0x556252490d90 msgr2=0x5562524c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 shutdown_connections 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 1 -- 172.21.15.43:0/4118744447 wait complete. 2024-09-17T05:32:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.143+0000 7f5547d66000 20 client.8745 trim_cache size 0 max 16384 2024-09-17T05:32:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 Processor -- start 2024-09-17T05:32:01.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- start start 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625257dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 0x55625257e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5538206af0 con 0x556251fca460 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5538205b30 con 0x5562523ba950 2024-09-17T05:32:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5538204ec0 con 0x5562523a3ca0 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54520/0 (socket says 172.21.15.43:54520) 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 -- 172.21.15.43:0/1390759780 learned_addr learned my addr 172.21.15.43:0/1390759780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553d400640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625257dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553e800640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 0x55625257e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 -- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 msgr2=0x55625257dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625257dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 -- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 msgr2=0x55625257e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 0x55625257e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 -- 172.21.15.43:0/1390759780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556252368db0 con 0x556251fca460 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553d400640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625257dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553e800640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 0x55625257e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f553de00640 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f552c00cde0 tx=0x7f552c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe317e86ce117ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5525c00640 1 -- 172.21.15.43:0/1390759780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c068360 con 0x556251fca460 2024-09-17T05:32:01.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5525c00640 1 -- 172.21.15.43:0/1390759780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f552c011e80 con 0x556251fca460 2024-09-17T05:32:01.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5525c00640 1 -- 172.21.15.43:0/1390759780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0800b0 con 0x556251fca460 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- 172.21.15.43:0/1390759780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 msgr2=0x55625257d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f552c00cde0 tx=0x7f552c00a000 comp rx=0 tx=0).stop 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- 172.21.15.43:0/1390759780 shutdown_connections 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523ba950 0x55625257e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5562523a3ca0 0x55625257dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 --2- 172.21.15.43:0/1390759780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556251fca460 0x55625257d840 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- 172.21.15.43:0/1390759780 >> 172.21.15.43:0/1390759780 conn(0x556252490d90 msgr2=0x5562524bfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.163+0000 7f5547d66000 1 -- 172.21.15.43:0/1390759780 shutdown_connections 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- 172.21.15.43:0/1390759780 wait complete. 2024-09-17T05:32:01.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- start start 2024-09-17T05:32:01.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 Processor -- start 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556251fca460 0x5562525813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523a3ca0 0x556252581910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556252310d80 con 0x5562523ba950 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5562523153b0 con 0x5562523a3ca0 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5562522d8ee0 con 0x556251fca460 2024-09-17T05:32:01.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 10 client.0 _ll_register_callbacks cb 0x7f552c05b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553de00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556251fca460 0x5562525813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54530/0 (socket says 172.21.15.43:54530) 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 -- 172.21.15.43:0/325521778 learned_addr learned my addr 172.21.15.43:0/325521778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553d400640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523a3ca0 0x556252581910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 -- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556251fca460 msgr2=0x5562525813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556251fca460 0x5562525813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 -- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523a3ca0 msgr2=0x556252581910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523a3ca0 0x556252581910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 -- 172.21.15.43:0/325521778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562522d3ab0 con 0x5562523ba950 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f553e800640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f55340b0040 tx=0x7f5534004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4921e9ad7e9a4c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f551aa00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340bd070 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f5534003c40 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.167+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552c011e80 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) ==== 24+0+0 (secure 0 0 0) 0x7f55340b2040 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 10 client.8751 fetch_fsmap learned FSMap version 200 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) -- 0x5562522b2070 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f552c011e80 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 200) ==== 3502+0+0 (secure 0 0 0) 0x7f553806d670 con 0x5562523ba950 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 10 client.8751 fetch_fsmap finished waiting for FSMap version 200 2024-09-17T05:32:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 10 client.8751 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:32:01.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 20 client.8751 populate_metadata read hostname 'smithi043' 2024-09-17T05:32:01.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251f9d830 0x55625253d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 4 client.8751 mds_command: new command op to 8215 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:32:01.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 --> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] -- command(tid 0: {"prefix": "status"}) -- 0x55625231b540 con 0x556251f9d830 2024-09-17T05:32:01.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f553de00640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251f9d830 0x55625253d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f553de00640 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251f9d830 0x55625253d2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f6fddb454ecb5d server_cookie=b2f870c65d88f1a6 in_seq=0 out_seq=0 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 10 client.8751 ms_handle_connect on v2:172.21.15.62:6834/2196052200 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 1 -- 172.21.15.43:0/325521778 <== mds.0 v2:172.21.15.62:6834/2196052200 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55625231b540 con 0x556251f9d830 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 10 client.8751 handle_command_reply: tid=0 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 client.8751 shutdown 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 msgr2=0x556252581e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f55340b0040 tx=0x7f5534004f20 comp rx=0 tx=0).stop 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 shutdown_connections 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:6834/2196052200,v1:172.21.15.62:6835/2196052200] conn(0x556251f9d830 0x55625253d2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562523a3ca0 0x556252581910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5562523ba950 0x556252581e50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 --2- 172.21.15.43:0/325521778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556251fca460 0x5562525813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f551aa00640 0 client.8751 ms_handle_reset on v2:172.21.15.62:6834/2196052200 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 >> 172.21.15.43:0/325521778 conn(0x556252490d90 msgr2=0x5562524c0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 shutdown_connections 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 1 -- 172.21.15.43:0/325521778 wait complete. 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5547d66000 20 client.8751 trim_cache size 0 max 16384 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8215, 2024-09-17T05:32:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 7.6713402689999999, 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 200, 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 107, 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 107, 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.877866805 2024-09-17T05:32:01.179 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 msgr2=0x7f551c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 0x7f551c075710 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f55380af8c0 tx=0x7f5528009040 comp rx=0 tx=0).stop 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 msgr2=0x7f55380c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c420 tx=0x7f552c00c8e0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 shutdown_connections 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f551c073200 0x7f551c075710 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5538099280 0x7f553814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f553815fe10 0x7f55380c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 --2- 172.21.15.43:0/3496285305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f553806fc90 0x7f55380c8140 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 >> 172.21.15.43:0/3496285305 conn(0x7f55380b23d0 msgr2=0x7f55380b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 shutdown_connections 2024-09-17T05:32:01.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.171+0000 7f5540800640 1 -- 172.21.15.43:0/3496285305 wait complete. 2024-09-17T05:32:01.193 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:32:01.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- 172.21.15.43:0/442286947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c0b3af0 msgr2=0x7fc12c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 --2- 172.21.15.43:0/442286947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c0b3af0 0x7fc12c15c7e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0d0c40 tx=0x7fc11c01c740 comp rx=0 tx=0).stop 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- 172.21.15.43:0/442286947 shutdown_connections 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 --2- 172.21.15.43:0/442286947 >> v2:172.21.15.62:3300/0 conn(0x7fc12c06f110 0x7fc12c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 --2- 172.21.15.43:0/442286947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c0b3af0 0x7fc12c15c7e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- 172.21.15.43:0/442286947 >> 172.21.15.43:0/442286947 conn(0x7fc12c0b23d0 msgr2=0x7fc12c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- 172.21.15.43:0/442286947 shutdown_connections 2024-09-17T05:32:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- 172.21.15.43:0/442286947 wait complete. 2024-09-17T05:32:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 Processor -- start 2024-09-17T05:32:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.351+0000 7fc134800640 1 -- start start 2024-09-17T05:32:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 0x7fc12c09a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c0b1870 0x7fc12c164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c168b90 con 0x7fc12c155d70 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c168a10 con 0x7fc12c06f110 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c168d10 con 0x7fc12c0b1870 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54538/0 (socket says 172.21.15.43:54538) 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 -- 172.21.15.43:0/2722415269 learned_addr learned my addr 172.21.15.43:0/2722415269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131e00640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 0x7fc12c09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 -- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c0b1870 msgr2=0x7fc12c164940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c0b1870 0x7fc12c164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 -- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 msgr2=0x7fc12c09a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 0x7fc12c09a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 -- 172.21.15.43:0/2722415269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c0cb830 con 0x7fc12c155d70 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131e00640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 0x7fc12c09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc131400640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc11c01cc20 tx=0x7fc11c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0b2040 con 0x7fc12c155d70 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc11c004130 con 0x7fc12c155d70 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c0751a0 con 0x7fc12c155d70 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003600 con 0x7fc12c155d70 2024-09-17T05:32:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c0cc090 con 0x7fc12c155d70 2024-09-17T05:32:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.355+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0ad0b0 con 0x7fc12c155d70 2024-09-17T05:32:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.359+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc11c0bd030 con 0x7fc12c155d70 2024-09-17T05:32:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.359+0000 7fc126000640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 0x7fc110075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.359+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7fc11c12e370 con 0x7fc12c155d70 2024-09-17T05:32:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.359+0000 7fc131e00640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 0x7fc110075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.359+0000 7fc131e00640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 0x7fc110075740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc120001230 tx=0x7fc120009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.363+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc11c0c4060 con 0x7fc12c155d70 2024-09-17T05:32:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.571+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc12c0cbb10 con 0x7fc12c155d70 2024-09-17T05:32:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.571+0000 7fc126000640 1 -- 172.21.15.43:0/2722415269 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+5225 (secure 0 0 0) 0x7fc11c0163f0 con 0x7fc12c155d70 2024-09-17T05:32:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:01.574 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":200,"btime":"2024-09-17T05:31:56:508616+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3742224456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3742224456},{"type":"v1","addr":"172.21.15.43:6835","nonce":3742224456}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":198}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:31:56.508607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8215":{"gid":8215,"name":"d","rank":0,"incarnation":197,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2196052200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2196052200},{"type":"v1","addr":"172.21.15.62:6835","nonce":2196052200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8215,"qdb_cluster":[8215]},"id":6}]} 2024-09-17T05:32:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 200 2024-09-17T05:32:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 msgr2=0x7fc110075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 0x7fc110075740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc120001230 tx=0x7fc120009040 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 msgr2=0x7fc12c0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc11c01cc20 tx=0x7fc11c005db0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 shutdown_connections 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc110073230 0x7fc110075740 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc12c0b1870 0x7fc12c164940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc12c155d70 0x7fc12c0cb2f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 --2- 172.21.15.43:0/2722415269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc12c06f110 0x7fc12c09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 >> 172.21.15.43:0/2722415269 conn(0x7fc12c0b23d0 msgr2=0x7fc12c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 shutdown_connections 2024-09-17T05:32:01.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.575+0000 7fc134800640 1 -- 172.21.15.43:0/2722415269 wait complete. 2024-09-17T05:32:01.596 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-09-17T05:32:01.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:32:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- 172.21.15.43:0/2932303736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f303406fc90 msgr2=0x7f30340700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 --2- 172.21.15.43:0/2932303736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f303406fc90 0x7f30340700a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f30340ccb60 tx=0x7f301c01c6e0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- 172.21.15.43:0/2932303736 shutdown_connections 2024-09-17T05:32:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 --2- 172.21.15.43:0/2932303736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f303406fc90 0x7f30340700a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 --2- 172.21.15.43:0/2932303736 >> v2:172.21.15.62:3300/0 conn(0x7f303415fe10 0x7f30340b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- 172.21.15.43:0/2932303736 >> 172.21.15.43:0/2932303736 conn(0x7f30340b23d0 msgr2=0x7f30340d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- 172.21.15.43:0/2932303736 shutdown_connections 2024-09-17T05:32:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- 172.21.15.43:0/2932303736 wait complete. 2024-09-17T05:32:01.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 Processor -- start 2024-09-17T05:32:01.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.755+0000 7f3038c00640 1 -- start start 2024-09-17T05:32:01.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 0x7f3034156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 0x7f303415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f303416a000 con 0x7f30340730b0 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3034169e80 con 0x7f303415fe10 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f303416a180 con 0x7f303406fc90 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54552/0 (socket says 172.21.15.43:54552) 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 -- 172.21.15.43:0/1461035613 learned_addr learned my addr 172.21.15.43:0/1461035613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3031e00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 0x7f3034156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 -- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 msgr2=0x7f3034156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 0x7f3034156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 -- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 msgr2=0x7f303415b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3032800640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 0x7f303415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 0x7f303415b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 -- 172.21.15.43:0/1461035613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3034150080 con 0x7f30340730b0 2024-09-17T05:32:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3031e00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 0x7f3034156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3032800640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 0x7f303415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3029400640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f301c0b0040 tx=0x7f301c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0bd030 con 0x7f30340730b0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f301c004040 con 0x7f30340730b0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3034153f10 con 0x7f30340730b0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c004230 con 0x7f30340730b0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30340b1530 con 0x7f30340730b0 2024-09-17T05:32:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.759+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30340700a0 con 0x7f30340730b0 2024-09-17T05:32:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.763+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f301c0c0070 con 0x7f30340730b0 2024-09-17T05:32:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.763+0000 7f302aa00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 0x7f3010075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.763+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f301c12ef40 con 0x7f30340730b0 2024-09-17T05:32:01.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.763+0000 7f3031e00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 0x7f3010075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:01.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.763+0000 7f3031e00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 0x7f3010075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ac0 tx=0x7f30240091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:01.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.767+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f301c0c7020 con 0x7f30340730b0 2024-09-17T05:32:01.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.979+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3034074d20 con 0x7f30340730b0 2024-09-17T05:32:01.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f302aa00640 1 -- 172.21.15.43:0/1461035613 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v200) ==== 53+0+73 (secure 0 0 0) 0x7f301c0163f0 con 0x7f30340730b0 2024-09-17T05:32:01.984 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 msgr2=0x7f3010075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 0x7f3010075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ac0 tx=0x7f30240091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 msgr2=0x7f3034152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f301c0b0040 tx=0x7f301c005db0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 shutdown_connections 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3010073200 0x7f3010075710 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303415fe10 0x7f303415b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340730b0 0x7f3034152b60 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 --2- 172.21.15.43:0/1461035613 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f303406fc90 0x7f3034156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:01.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 >> 172.21.15.43:0/1461035613 conn(0x7f30340b23d0 msgr2=0x7f303415c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:01.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 shutdown_connections 2024-09-17T05:32:01.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:01.983+0000 7f3038c00640 1 -- 172.21.15.43:0/1461035613 wait complete. 2024-09-17T05:32:02.006 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:32:02.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:32:02.007 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:02.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.007 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:32:02.017 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:32:02.017 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:32:02.017 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:32:02.017 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41922sec preferred_lft 41922sec 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:32:02.018 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:02.019 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:32:02.020 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:02.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.021 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:32:02.077 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:32:02.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.079 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:32:02.132 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:32:02.133 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:32:02.133 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:32:02.133 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:32:02.133 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:32:02.133 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:32:02.133 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:32:02.133 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:32:02.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:32:02.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:32:02.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:32:02.134 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:32:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:32:02.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:02.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.270 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:32:02.280 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:32:02.280 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:32:02.281 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:02.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:32:02.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:02.344 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:02.395 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:02.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:02.396 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:02.451 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:32:02.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:02.556 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:02.556 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:02.556 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:02.574 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:02.574 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:02.575 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:02.575 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:32:02.575 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:32:02.588 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:32:02.588 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:32:02.591 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:32:02.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:02.611 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:32:02.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:32:02.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:02.713 INFO:teuthology.orchestra.run.smithi043.stdout:client7514 2024-09-17T05:32:02.713 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:02.713 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:02.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.764 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:32:02.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:02.810 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-09-17T05:32:02.818 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:32:02.818 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:32:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:02.818 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:32:02.819 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:32:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:mkdir: cannot create directory 'foo': Read-only file system 2024-09-17T05:32:02.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:02.827 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory 'foo': Read-only file system 2024-09-17T05:32:02.827 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-09-17T05:32:02.827 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:02.827 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:02.879 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:02.879 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:02.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- 172.21.15.43:0/1746198201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 msgr2=0x7f0c840ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 --2- 172.21.15.43:0/1746198201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840ccc40 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009840 tx=0x7f0c7401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- 172.21.15.43:0/1746198201 shutdown_connections 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 --2- 172.21.15.43:0/1746198201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840ccc40 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- 172.21.15.43:0/1746198201 >> 172.21.15.43:0/1746198201 conn(0x7f0c84068fe0 msgr2=0x7f0c84069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- 172.21.15.43:0/1746198201 shutdown_connections 2024-09-17T05:32:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- 172.21.15.43:0/1746198201 wait complete. 2024-09-17T05:32:03.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 Processor -- start 2024-09-17T05:32:03.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- start start 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c8415ca80 0x7f0c840c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c84160cf0 0x7f0c8413ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.135+0000 7f0c88e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8415ce90 con 0x7f0c840d1cb0 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c88e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84153960 con 0x7f0c8415ca80 2024-09-17T05:32:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c88e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8415d010 con 0x7f0c84160cf0 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54580/0 (socket says 172.21.15.43:54580) 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 -- 172.21.15.43:0/4281556564 learned_addr learned my addr 172.21.15.43:0/4281556564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c82a00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c8415ca80 0x7f0c840c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c84160cf0 0x7f0c8413ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 -- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c84160cf0 msgr2=0x7f0c8413ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c84160cf0 0x7f0c8413ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 -- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c8415ca80 msgr2=0x7f0c840c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c8415ca80 0x7f0c840c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 -- 172.21.15.43:0/4281556564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8414e7f0 con 0x7f0c840d1cb0 2024-09-17T05:32:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c83400640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0c7401ccd0 tx=0x7f0c74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740ba020 con 0x7f0c840d1cb0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c74003600 con 0x7f0c840d1cb0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8414ead0 con 0x7f0c840d1cb0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740b2040 con 0x7f0c840d1cb0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8414f090 con 0x7f0c840d1cb0 2024-09-17T05:32:03.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c840ccc40 con 0x7f0c840d1cb0 2024-09-17T05:32:03.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.139+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c740bd020 con 0x7f0c840d1cb0 2024-09-17T05:32:03.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.143+0000 7f0c7b400640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 0x7f0c64075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.143+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f0c7412ec70 con 0x7f0c840d1cb0 2024-09-17T05:32:03.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.143+0000 7f0c82a00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 0x7f0c64075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.143+0000 7f0c82a00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 0x7f0c64075740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0c840c51d0 tx=0x7f0c70009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.147+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c740c4030 con 0x7f0c840d1cb0 2024-09-17T05:32:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.331+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0c8406a1e0 con 0x7f0c840d1cb0 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.331+0000 7f0c7b400640 1 -- 172.21.15.43:0/4281556564 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v107) ==== 81+0+2188 (secure 0 0 0) 0x7f0c74016410 con 0x7f0c840d1cb0 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:32:03.335 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:32:03.336 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:32:03.337 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:32:03.338 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:listed 36 entries 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 msgr2=0x7f0c64075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 0x7f0c64075740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0c840c51d0 tx=0x7f0c70009040 comp rx=0 tx=0).stop 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 msgr2=0x7f0c840c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0c7401ccd0 tx=0x7f0c74004040 comp rx=0 tx=0).stop 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 shutdown_connections 2024-09-17T05:32:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c64073230 0x7f0c64075740 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c8415ca80 0x7f0c840c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c84160cf0 0x7f0c8413ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 --2- 172.21.15.43:0/4281556564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c840d1cb0 0x7f0c840c79d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 >> 172.21.15.43:0/4281556564 conn(0x7f0c84068fe0 msgr2=0x7f0c8415e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 shutdown_connections 2024-09-17T05:32:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.335+0000 7f0c88e00640 1 -- 172.21.15.43:0/4281556564 wait complete. 2024-09-17T05:32:03.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:32:03.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:03.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:32:03.381 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:03.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:03.381 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:03.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 -- 172.21.15.43:0/1363764465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c06f110 msgr2=0x7f212c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 --2- 172.21.15.43:0/1363764465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c06f110 0x7f212c0b33e0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2114009900 tx=0x7f211401c820 comp rx=0 tx=0).stop 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 -- 172.21.15.43:0/1363764465 shutdown_connections 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 --2- 172.21.15.43:0/1363764465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c06f110 0x7f212c0b33e0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 --2- 172.21.15.43:0/1363764465 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c155d70 0x7f212c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 -- 172.21.15.43:0/1363764465 >> 172.21.15.43:0/1363764465 conn(0x7f212c0b23d0 msgr2=0x7f212c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 -- 172.21.15.43:0/1363764465 shutdown_connections 2024-09-17T05:32:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.635+0000 7f2132800640 1 -- 172.21.15.43:0/1363764465 wait complete. 2024-09-17T05:32:03.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 Processor -- start 2024-09-17T05:32:03.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- start start 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c06f110 0x7f212c09dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c0b3920 0x7f212c09cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f212c168b10 con 0x7f212c155d70 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f212c168990 con 0x7f212c06f110 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f212c168c90 con 0x7f212c0b3920 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c06f110 0x7f212c09dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2122a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c0b3920 0x7f212c09cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54598/0 (socket says 172.21.15.43:54598) 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 -- 172.21.15.43:0/1471509195 learned_addr learned my addr 172.21.15.43:0/1471509195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 -- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c0b3920 msgr2=0x7f212c09cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c0b3920 0x7f212c09cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 -- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c06f110 msgr2=0x7f212c09dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c06f110 0x7f212c09dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 -- 172.21.15.43:0/1471509195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c0caa60 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f212be00640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f212400c760 tx=0x7f212400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124003b80 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2124003d20 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c0cc240 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212400aab0 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.639+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c09d950 con 0x7f212c155d70 2024-09-17T05:32:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c0b33e0 con 0x7f212c155d70 2024-09-17T05:32:03.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f212400ac50 con 0x7f212c155d70 2024-09-17T05:32:03.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f2123e00640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 0x7f2108075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:03.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7257+0+0 (secure 0 0 0) 0x7f2124094480 con 0x7f212c155d70 2024-09-17T05:32:03.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f212b400640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 0x7f2108075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:03.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.643+0000 7f212b400640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 0x7f2108075710 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f212c0715c0 tx=0x7f211c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:03.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.647+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f212c0b3d30 con 0x7f212c155d70 2024-09-17T05:32:03.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.859+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f212c1fa190 con 0x7f212c155d70 2024-09-17T05:32:03.861 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:03.859+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.859+0000 7f2123e00640 1 -- 172.21.15.43:0/1471509195 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v201) ==== 161+0+0 (secure 0 0 0) 0x7f212405e830 con 0x7f212c155d70 2024-09-17T05:32:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 msgr2=0x7f2108075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 0x7f2108075710 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f212c0715c0 tx=0x7f211c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 msgr2=0x7f212c0cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f212400c760 tx=0x7f212400cc20 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 shutdown_connections 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2108073200 0x7f2108075710 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f212c155d70 0x7f212c0cf150 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f212c0b3920 0x7f212c09cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 --2- 172.21.15.43:0/1471509195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f212c06f110 0x7f212c09dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 >> 172.21.15.43:0/1471509195 conn(0x7f212c0b23d0 msgr2=0x7f212c165b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 shutdown_connections 2024-09-17T05:32:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:03.863+0000 7f2132800640 1 -- 172.21.15.43:0/1471509195 wait complete. 2024-09-17T05:32:03.870 INFO:tasks.ceph.mds.d.smithi062.stderr: -1883> 2024-09-17T05:31:53.504+0000 7f7c55800640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-17T05:32:03.870 INFO:tasks.ceph.mds.d.smithi062.stderr: -1880> 2024-09-17T05:31:53.504+0000 7f7c55800640 -1 mds.0.197 unhandled write error (22) Invalid argument, force readonly... 2024-09-17T05:32:03.873 INFO:tasks.ceph.mds.d.smithi062.stderr: -640> 2024-09-17T05:32:00.476+0000 7f7c59400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:32:03.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:04.003 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3897166464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f00b4570 msgr2=0x7fa2f00b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3897166464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f00b4570 0x7fa2f00b4980 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009840 tx=0x7fa2d801c790 comp rx=0 tx=0).stop 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3897166464 shutdown_connections 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3897166464 >> v2:172.21.15.62:3300/0 conn(0x7fa2f0160210 0x7fa2f0160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3897166464 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f00b4570 0x7fa2f00b4980 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3897166464 >> 172.21.15.43:0/3897166464 conn(0x7fa2f00b2810 msgr2=0x7fa2f00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3897166464 shutdown_connections 2024-09-17T05:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3897166464 wait complete. 2024-09-17T05:32:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 Processor -- start 2024-09-17T05:32:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.047+0000 7fa2f6e00640 1 -- start start 2024-09-17T05:32:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f006f940 0x7fa2f0145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f0160210 0x7fa2f014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f016c020 con 0x7fa2f00b4570 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f016bea0 con 0x7fa2f006f940 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f016c1a0 con 0x7fa2f0160210 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45496/0 (socket says 172.21.15.43:45496) 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 -- 172.21.15.43:0/3459884982 learned_addr learned my addr 172.21.15.43:0/3459884982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2efe00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f006f940 0x7fa2f0145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f4e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f0160210 0x7fa2f014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 -- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f0160210 msgr2=0x7fa2f014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f0160210 0x7fa2f014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 -- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f006f940 msgr2=0x7fa2f0145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f006f940 0x7fa2f0145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 -- 172.21.15.43:0/3459884982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f014a380 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2e7400640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009790 tx=0x7fa2d80a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80ac030 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d80053b0 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f014a5f0 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80046d0 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f0153b00 con 0x7fa2f00b4570 2024-09-17T05:32:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.051+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f00b4980 con 0x7fa2f00b4570 2024-09-17T05:32:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.055+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2d80af070 con 0x7fa2f00b4570 2024-09-17T05:32:04.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.055+0000 7fa2ecc00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 0x7fa2cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.055+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7343+0+0 (secure 0 0 0) 0x7fa2d812e9a0 con 0x7fa2f00b4570 2024-09-17T05:32:04.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.055+0000 7fa2efe00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 0x7fa2cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.055+0000 7fa2efe00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 0x7fa2cc075740 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0007fd0 tx=0x7fa2e0007ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.059+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa2d80b6070 con 0x7fa2f00b4570 2024-09-17T05:32:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.295+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa2f015eb70 con 0x7fa2f00b4570 2024-09-17T05:32:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.867+0000 7fa2ecc00640 1 -- 172.21.15.43:0/3459884982 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v202) ==== 107+0+0 (secure 0 0 0) 0x7fa2d8016410 con 0x7fa2f00b4570 2024-09-17T05:32:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 msgr2=0x7fa2cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 0x7fa2cc075740 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0007fd0 tx=0x7fa2e0007ea0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 msgr2=0x7fa2f014e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009790 tx=0x7fa2d80a64e0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 shutdown_connections 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa2cc073230 0x7fa2cc075740 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa2f0160210 0x7fa2f014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2f00b4570 0x7fa2f014e5d0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 --2- 172.21.15.43:0/3459884982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f006f940 0x7fa2f0145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 >> 172.21.15.43:0/3459884982 conn(0x7fa2f00b2810 msgr2=0x7fa2f00af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 shutdown_connections 2024-09-17T05:32:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.871+0000 7fa2f6e00640 1 -- 172.21.15.43:0/3459884982 wait complete. 2024-09-17T05:32:04.895 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 Processor -- start 2024-09-17T05:32:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 -- start start 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d414ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 0x561f2d42f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f2d49f8e0 con 0x561f2d49ee40 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561f2d336a00 con 0x561f2d417cb0 2024-09-17T05:32:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75dd672080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561f2d49fb30 con 0x561f2d3e8d70 2024-09-17T05:32:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45512/0 (socket says 172.21.15.43:45512) 2024-09-17T05:32:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.951+0000 7f75d7200640 1 -- 172.21.15.43:0/2779832919 learned_addr learned my addr 172.21.15.43:0/2779832919 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d6800640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d414ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d5e00640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 0x561f2d42f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 -- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 msgr2=0x561f2d414ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d414ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 -- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 msgr2=0x561f2d42f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 0x561f2d42f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 -- 172.21.15.43:0/2779832919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2d335a20 con 0x561f2d49ee40 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d5e00640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 0x561f2d42f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d7200640 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f75d001e5b0 tx=0x7f75d001aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9109ce3c7160f34e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d4a00640 1 -- 172.21.15.43:0/2779832919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0004020 con 0x561f2d49ee40 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d4a00640 1 -- 172.21.15.43:0/2779832919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75d00042b0 con 0x561f2d49ee40 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75d4a00640 1 -- 172.21.15.43:0/2779832919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d00bb790 con 0x561f2d49ee40 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- 172.21.15.43:0/2779832919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 msgr2=0x561f2d430b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f75d001e5b0 tx=0x7f75d001aab0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- 172.21.15.43:0/2779832919 shutdown_connections 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d417cb0 0x561f2d42f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d49ee40 0x561f2d430b40 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 --2- 172.21.15.43:0/2779832919 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d414ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- 172.21.15.43:0/2779832919 >> 172.21.15.43:0/2779832919 conn(0x561f2d3d4d80 msgr2=0x561f2d4bd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- 172.21.15.43:0/2779832919 shutdown_connections 2024-09-17T05:32:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- 172.21.15.43:0/2779832919 wait complete. 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 Processor -- start 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.955+0000 7f75dd672080 1 -- start start 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d40e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 0x561f2d4bac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d49ee40 0x561f2d3e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f2d4bcba0 con 0x561f2d417cb0 2024-09-17T05:32:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561f2d3d6f60 con 0x561f2d49ee40 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561f2d3d7200 con 0x561f2d3e8d70 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 0x561f2d4bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 0x561f2d4bac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45514/0 (socket says 172.21.15.43:45514) 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 learned_addr learned my addr 172.21.15.43:0/1570944014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d7200640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d49ee40 0x561f2d3e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d40e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 msgr2=0x561f2d40e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f2d3e8d70 0x561f2d40e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d49ee40 msgr2=0x561f2d3e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f2d49ee40 0x561f2d3e6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2d30fc10 con 0x561f2d417cb0 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 0x561f2d4bac50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f75c400a6d0 tx=0x7f75c400a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75cf400640 1 -- 172.21.15.43:0/1570944014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4008800 con 0x561f2d417cb0 2024-09-17T05:32:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75cf400640 1 -- 172.21.15.43:0/1570944014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75c40089a0 con 0x561f2d417cb0 2024-09-17T05:32:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75cf400640 1 -- 172.21.15.43:0/1570944014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4008dd0 con 0x561f2d417cb0 2024-09-17T05:32:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f2d40ecb0 con 0x561f2d417cb0 2024-09-17T05:32:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.959+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2d40f1f0 con 0x561f2d417cb0 2024-09-17T05:32:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75cf400640 1 -- 172.21.15.43:0/1570944014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75c401e440 con 0x561f2d417cb0 2024-09-17T05:32:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75cf400640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75b8071be0 0x7f75b80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75cf400640 1 -- 172.21.15.43:0/1570944014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7343+0+0 (secure 0 0 0) 0x7f75c40937b0 con 0x561f2d417cb0 2024-09-17T05:32:04.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75b8071be0 0x7f75b80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.965 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561f2d411e70 0x561f2d412280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d4127c0 con 0x561f2d411e70 2024-09-17T05:32:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75d7200640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561f2d411e70 0x561f2d412280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75b8071be0 0x7f75b80740f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b5e0 tx=0x7f75c800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.963+0000 7f75d7200640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561f2d411e70 0x561f2d412280 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v108'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d412b90 con 0x561f2d411e70 2024-09-17T05:32:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v108'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d561be0 con 0x561f2d411e70 2024-09-17T05:32:04.970 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v108'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.967+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562f90 con 0x561f2d411e70 2024-09-17T05:32:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d563330 con 0x561f2d411e70 2024-09-17T05:32:04.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d561f80 con 0x561f2d411e70 2024-09-17T05:32:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.971+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562370 con 0x561f2d411e70 2024-09-17T05:32:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.975+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v108'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.975+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.975+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v108'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.975+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.979 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v108'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.979+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.983+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.988 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:04.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.987+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v108'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.991+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v108'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.997 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v108'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.995+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v108'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:04.999+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v108'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.006 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:05.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v108'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.003+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d562710 con 0x561f2d411e70 2024-09-17T05:32:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561f2d5636d0 0x561f2d565af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d566030 con 0x561f2d5636d0 2024-09-17T05:32:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561f2d5636d0 0x561f2d565af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.007+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561f2d5636d0 0x561f2d565af0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.011+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'420 uv399 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.011+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d5663d0 con 0x561f2d5636d0 2024-09-17T05:32:05.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.011+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'421 uv421 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.011+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d566770 con 0x561f2d5636d0 2024-09-17T05:32:05.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.015 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:05.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d566b10 con 0x561f2d5636d0 2024-09-17T05:32:05.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d566f00 con 0x561f2d5636d0 2024-09-17T05:32:05.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.015+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5672a0 con 0x561f2d5636d0 2024-09-17T05:32:05.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.019+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v108'425 uv418 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.019+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d567690 con 0x561f2d5636d0 2024-09-17T05:32:05.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.019+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v108'426 uv426 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.019+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567a80 con 0x561f2d5636d0 2024-09-17T05:32:05.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.019+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v108'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.023 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:05.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.023+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567e70 con 0x561f2d5636d0 2024-09-17T05:32:05.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.023+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.023+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567e70 con 0x561f2d5636d0 2024-09-17T05:32:05.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.023+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.023+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567bb0 con 0x561f2d5636d0 2024-09-17T05:32:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v108'430 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v108'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v108'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.031 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.027+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.031+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v108'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.031+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.031+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.031+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v108'435 uv415 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v108'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v108'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.039 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.035+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d567b60 con 0x561f2d5636d0 2024-09-17T05:32:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5636d0 2024-09-17T05:32:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x561f2d568480 0x561f2d568890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.039+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56b020 con 0x561f2d568480 2024-09-17T05:32:05.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.043+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x561f2d568480 0x561f2d568890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.043+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x561f2d568480 0x561f2d568890 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.043+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v108'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.043+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d56b3c0 con 0x561f2d568480 2024-09-17T05:32:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v108'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56b860 con 0x561f2d568480 2024-09-17T05:32:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v108'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.049 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56d110 con 0x561f2d568480 2024-09-17T05:32:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.047+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56d4b0 con 0x561f2d568480 2024-09-17T05:32:05.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56c100 con 0x561f2d568480 2024-09-17T05:32:05.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d411e70 2024-09-17T05:32:05.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'81 uv80 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.051+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d411e70 2024-09-17T05:32:05.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.055+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'82 uv82 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.055+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d411e70 2024-09-17T05:32:05.057 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.055+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'83 uv83 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.055+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d411e70 2024-09-17T05:32:05.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.055+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'84 uv84 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.059+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x561f2d56d850 0x561f2d56fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.059+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d56d850 2024-09-17T05:32:05.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.059+0000 7f75d7200640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x561f2d56d850 0x561f2d56fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.059+0000 7f75d7200640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x561f2d56d850 0x561f2d56fc70 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v108'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d56c4f0 con 0x561f2d56d850 2024-09-17T05:32:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v108'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5701b0 con 0x561f2d56d850 2024-09-17T05:32:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v108'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.067 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:05.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.063+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5711c0 con 0x561f2d56d850 2024-09-17T05:32:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5711c0 con 0x561f2d56d850 2024-09-17T05:32:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5701b0 con 0x561f2d56d850 2024-09-17T05:32:05.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x561f2d5706c0 0x561f2d572ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d5731a0 con 0x561f2d5706c0 2024-09-17T05:32:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.067+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x561f2d5706c0 0x561f2d572ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75d6800640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x561f2d5706c0 0x561f2d572ae0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v108'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d573260 con 0x561f2d5706c0 2024-09-17T05:32:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v108'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v108'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.075 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.071+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d574690 con 0x561f2d5706c0 2024-09-17T05:32:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d574690 con 0x561f2d5706c0 2024-09-17T05:32:05.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.075+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v108'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v108'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v108'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.083 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:05.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.079+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d573680 con 0x561f2d5706c0 2024-09-17T05:32:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75d6800640 1 -- 172.21.15.43:0/1570944014 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c8009040 con 0x561f2d5706c0 2024-09-17T05:32:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f2d573d00 0x561f2d576120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576730 con 0x561f2d573d00 2024-09-17T05:32:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f2d573d00 0x561f2d576120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.083+0000 7f75d5e00640 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f2d573d00 0x561f2d576120 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v108'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v108'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v108'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.091 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.087+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d411e70 2024-09-17T05:32:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v108'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.091+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d411e70 2024-09-17T05:32:05.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.095+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v108'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.095+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d411e70 2024-09-17T05:32:05.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.095+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v108'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.098 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:05.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.095+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577b50 con 0x561f2d411e70 2024-09-17T05:32:05.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577b10 con 0x561f2d411e70 2024-09-17T05:32:05.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d411e70 2024-09-17T05:32:05.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d411e70 2024-09-17T05:32:05.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.099+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d568480 2024-09-17T05:32:05.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v108'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d568480 2024-09-17T05:32:05.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v108'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d568480 2024-09-17T05:32:05.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v108'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.107 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:05.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.103+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.107+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.107+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.107+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.107+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v108'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v108'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v108'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.114 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:05.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.111+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d56bcf0 con 0x561f2d568480 2024-09-17T05:32:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577060 con 0x561f2d568480 2024-09-17T05:32:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d568480 2024-09-17T05:32:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577020 con 0x561f2d56d850 2024-09-17T05:32:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v108'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.115+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d577020 con 0x561f2d56d850 2024-09-17T05:32:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.119+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.119+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577020 con 0x561f2d56d850 2024-09-17T05:32:05.122 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.119+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v108'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.119+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577020 con 0x561f2d56d850 2024-09-17T05:32:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v108'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v108'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.123+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.128 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v108'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.127+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d56d850 2024-09-17T05:32:05.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.131+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75d00a7d70 con 0x561f2d56d850 2024-09-17T05:32:05.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.131+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.131+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v108'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.131+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576b00 con 0x561f2d573d00 2024-09-17T05:32:05.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v108'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v108'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.137 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:05.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.135+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v108'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v108'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.139+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.144 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v108'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v108'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v108'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.143+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v108'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v108'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.151 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:05.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v108'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.147+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.151+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v108'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.151+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.151+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v108'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.151+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d576fb0 con 0x561f2d573d00 2024-09-17T05:32:05.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v108'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x561f2d577370 con 0x561f2d573d00 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75d5e00640 1 -- 172.21.15.43:0/1570944014 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v108'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f75c401b020 con 0x561f2d573d00 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561f2d5636d0 msgr2=0x561f2d565af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x561f2d5636d0 0x561f2d565af0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x561f2d5706c0 msgr2=0x561f2d572ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x561f2d5706c0 0x561f2d572ae0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f2d573d00 msgr2=0x561f2d576120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f2d573d00 0x561f2d576120 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x561f2d568480 msgr2=0x561f2d568890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 reap_dead start 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x561f2d568480 0x561f2d568890 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561f2d411e70 msgr2=0x561f2d412280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x561f2d411e70 0x561f2d412280 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x561f2d56d850 msgr2=0x561f2d56fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x561f2d56d850 0x561f2d56fc70 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75b8071be0 msgr2=0x7f75b80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75b8071be0 0x7f75b80740f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b5e0 tx=0x7f75c800d040 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 msgr2=0x561f2d4bac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 --2- 172.21.15.43:0/1570944014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f2d417cb0 0x561f2d4bac50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f75c400a6d0 tx=0x7f75c400a7c0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75d7200640 1 -- 172.21.15.43:0/1570944014 reap_dead start 2024-09-17T05:32:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 shutdown_connections 2024-09-17T05:32:05.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 >> 172.21.15.43:0/1570944014 conn(0x561f2d3d4d80 msgr2=0x561f2d4be3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:05.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 shutdown_connections 2024-09-17T05:32:05.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.155+0000 7f75dd672080 1 -- 172.21.15.43:0/1570944014 wait complete. 2024-09-17T05:32:05.163 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-09-17T05:32:05.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.263+0000 7f381dde8080 1 Processor -- start 2024-09-17T05:32:05.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.263+0000 7f381dde8080 1 -- start start 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836cf34010 0x55836ce7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836ce7da00 0x55836cec35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55836cdcaa00 con 0x55836cf358b0 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55836ce28f40 con 0x55836ce7da00 2024-09-17T05:32:05.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55836ce7f760 con 0x55836cf34010 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45542/0 (socket says 172.21.15.43:45542) 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 -- 172.21.15.43:0/3502624111 learned_addr learned my addr 172.21.15.43:0/3502624111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3817a00640 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836ce7da00 0x55836cec35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3817000640 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836cf34010 0x55836ce7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 -- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836cf34010 msgr2=0x55836ce7d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836cf34010 0x55836ce7d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 -- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836ce7da00 msgr2=0x55836cec35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836ce7da00 0x55836cec35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 -- 172.21.15.43:0/3502624111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836cda3c10 con 0x55836cf358b0 2024-09-17T05:32:05.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3816600640 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3804019740 tx=0x7f3804017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb270d38f522ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3815200640 1 -- 172.21.15.43:0/3502624111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38040ab980 con 0x55836cf358b0 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3815200640 1 -- 172.21.15.43:0/3502624111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38040abc10 con 0x55836cf358b0 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f3815200640 1 -- 172.21.15.43:0/3502624111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38040abe00 con 0x55836cf358b0 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- 172.21.15.43:0/3502624111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 msgr2=0x55836cec30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3804019740 tx=0x7f3804017e30 comp rx=0 tx=0).stop 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- 172.21.15.43:0/3502624111 shutdown_connections 2024-09-17T05:32:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836ce7da00 0x55836cec35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf358b0 0x55836cec30a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 --2- 172.21.15.43:0/3502624111 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836cf34010 0x55836ce7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- 172.21.15.43:0/3502624111 >> 172.21.15.43:0/3502624111 conn(0x55836ce68d80 msgr2=0x55836cec50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- 172.21.15.43:0/3502624111 shutdown_connections 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- 172.21.15.43:0/3502624111 wait complete. 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 Processor -- start 2024-09-17T05:32:05.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.267+0000 7f381dde8080 1 -- start start 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 0x55836cec0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf34010 0x55836cec13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836cf358b0 0x55836cf51c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55836cf53a40 con 0x55836cf34010 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55836ce7d840 con 0x55836cf358b0 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55836cebf2c0 con 0x55836ce7da00 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 0x55836cec0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 0x55836cec0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:36842/0 (socket says 172.21.15.43:36842) 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 -- 172.21.15.43:0/782099811 learned_addr learned my addr 172.21.15.43:0/782099811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3816600640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf34010 0x55836cec13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817a00640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836cf358b0 0x55836cf51c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 -- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836cf358b0 msgr2=0x55836cf51c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55836cf358b0 0x55836cf51c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 -- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf34010 msgr2=0x55836cec13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf34010 0x55836cec13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 -- 172.21.15.43:0/782099811 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836cebf560 con 0x55836ce7da00 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3816600640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836cf34010 0x55836cec13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f3817000640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 0x55836cec0e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f380800ef80 tx=0x7f3808002f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f380f400640 1 -- 172.21.15.43:0/782099811 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808015020 con 0x55836ce7da00 2024-09-17T05:32:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f380f400640 1 -- 172.21.15.43:0/782099811 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808011650 con 0x55836ce7da00 2024-09-17T05:32:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55836cebf800 con 0x55836ce7da00 2024-09-17T05:32:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f380f400640 1 -- 172.21.15.43:0/782099811 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808011aa0 con 0x55836ce7da00 2024-09-17T05:32:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.271+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55836cf521a0 con 0x55836ce7da00 2024-09-17T05:32:05.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f380f400640 1 -- 172.21.15.43:0/782099811 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3808018020 con 0x55836ce7da00 2024-09-17T05:32:05.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f380f400640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f37f8071be0 0x7f37f80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f380f400640 1 -- 172.21.15.43:0/782099811 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 7343+0+0 (secure 0 0 0) 0x7f38080934d0 con 0x55836ce7da00 2024-09-17T05:32:05.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f3816600640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f37f8071be0 0x7f37f80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f381dde8080 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55836cfeb8d0 0x55836cfebce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x55836cf52430 con 0x55836cfeb8d0 2024-09-17T05:32:05.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f3817a00640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55836cfeb8d0 0x55836cfebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f3816600640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f37f8071be0 0x7f37f80740f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55836ce7de10 tx=0x7f3804002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.275+0000 7f3817a00640 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55836cfeb8d0 0x55836cfebce0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f3817a00640 1 -- 172.21.15.43:0/782099811 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v108'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f381000b040 con 0x55836cfeb8d0 2024-09-17T05:32:05.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55836cfeb8d0 msgr2=0x55836cfebce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55836cfeb8d0 0x55836cfebce0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f37f8071be0 msgr2=0x7f37f80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f37f8071be0 0x7f37f80740f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55836ce7de10 tx=0x7f3804002040 comp rx=0 tx=0).stop 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 msgr2=0x55836cec0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 --2- 172.21.15.43:0/782099811 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55836ce7da00 0x55836cec0e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f380800ef80 tx=0x7f3808002f90 comp rx=0 tx=0).stop 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f3817a00640 1 -- 172.21.15.43:0/782099811 reap_dead start 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 shutdown_connections 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 >> 172.21.15.43:0/782099811 conn(0x55836ce68d80 msgr2=0x55836cf4f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 shutdown_connections 2024-09-17T05:32:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.279+0000 7f381dde8080 1 -- 172.21.15.43:0/782099811 wait complete. 2024-09-17T05:32:05.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:05.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 -- 172.21.15.43:0/3258005615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 msgr2=0x7f7628003a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 --2- 172.21.15.43:0/3258005615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 0x7f7628003a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7634009b40 tx=0x7f763401d290 comp rx=0 tx=0).stop 2024-09-17T05:32:05.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643400640 1 -- 172.21.15.43:0/3258005615 reap_dead start 2024-09-17T05:32:05.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 -- 172.21.15.43:0/3258005615 shutdown_connections 2024-09-17T05:32:05.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 -- 172.21.15.43:0/3258005615 >> 172.21.15.43:0/3258005615 conn(0x7f764406ce60 msgr2=0x7f764406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:05.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 -- 172.21.15.43:0/3258005615 shutdown_connections 2024-09-17T05:32:05.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.487+0000 7f7643e00640 1 -- 172.21.15.43:0/3258005615 wait complete. 2024-09-17T05:32:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 Processor -- start 2024-09-17T05:32:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 -- start start 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 0x7f76441f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76440b00f0 0x7f76440b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f764415fc90 con 0x7f764415b0d0 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f764415fb10 con 0x7f76440b00f0 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f76440b4360 con 0x7f7644158f60 2024-09-17T05:32:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45558/0 (socket says 172.21.15.43:45558) 2024-09-17T05:32:05.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7643400640 1 -- 172.21.15.43:0/2889690991 learned_addr learned my addr 172.21.15.43:0/2889690991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:05.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7642a00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 0x7f76441f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.491+0000 7f7642000640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76440b00f0 0x7f76440b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 -- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 msgr2=0x7f76441f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 0x7f76441f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 -- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76440b00f0 msgr2=0x7f76440b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76440b00f0 0x7f76440b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 -- 172.21.15.43:0/2889690991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441f5e50 con 0x7f764415b0d0 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643400640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f76440cd670 tx=0x7f763c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000c260 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f763c018070 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c011bc0 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441f6130 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441595f0 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76440b48c0 con 0x7f764415b0d0 2024-09-17T05:32:05.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.495+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f763c01d460 con 0x7f764415b0d0 2024-09-17T05:32:05.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.499+0000 7f763aa00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 0x7f7624075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:05.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.499+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7343+0+0 (secure 0 0 0) 0x7f763c094e30 con 0x7f764415b0d0 2024-09-17T05:32:05.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.499+0000 7f7642a00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 0x7f7624075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:05.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.499+0000 7f7642a00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 0x7f7624075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f76340ad000 tx=0x7f763401a030 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:05.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.503+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f763c05f210 con 0x7f764415b0d0 2024-09-17T05:32:05.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:05.711+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f76441f5be0 con 0x7f764415b0d0 2024-09-17T05:32:06.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.051+0000 7f763aa00640 1 -- 172.21.15.43:0/2889690991 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v203) ==== 162+0+0 (secure 0 0 0) 0x7f763c063c40 con 0x7f764415b0d0 2024-09-17T05:32:06.054 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 msgr2=0x7f7624075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 0x7f7624075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f76340ad000 tx=0x7f763401a030 comp rx=0 tx=0).stop 2024-09-17T05:32:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 msgr2=0x7f764415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f76440cd670 tx=0x7f763c006430 comp rx=0 tx=0).stop 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 shutdown_connections 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7624073230 0x7f7624075740 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76440b00f0 0x7f76440b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f764415b0d0 0x7f764415dd40 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 --2- 172.21.15.43:0/2889690991 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7644158f60 0x7f76441f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 >> 172.21.15.43:0/2889690991 conn(0x7f764406ce60 msgr2=0x7f7644156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 shutdown_connections 2024-09-17T05:32:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.055+0000 7f7643e00640 1 -- 172.21.15.43:0/2889690991 wait complete. 2024-09-17T05:32:06.063 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:06.063+0000 7f4c81400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:32:06.064 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:06.063+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:06.064 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:06.063+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:06.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 -- 172.21.15.43:0/3772095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c0b3af0 msgr2=0x7f684c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 --2- 172.21.15.43:0/3772095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c0b3af0 0x7f684c0ad0b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f684c072b60 tx=0x7f684401cac0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 -- 172.21.15.43:0/3772095949 shutdown_connections 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 --2- 172.21.15.43:0/3772095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c0b3af0 0x7f684c0ad0b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 --2- 172.21.15.43:0/3772095949 >> v2:172.21.15.62:3300/0 conn(0x7f684c155d70 0x7f684c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 -- 172.21.15.43:0/3772095949 >> 172.21.15.43:0/3772095949 conn(0x7f684c0b23d0 msgr2=0x7f684c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 -- 172.21.15.43:0/3772095949 shutdown_connections 2024-09-17T05:32:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.239+0000 7f6852800640 1 -- 172.21.15.43:0/3772095949 wait complete. 2024-09-17T05:32:06.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 Processor -- start 2024-09-17T05:32:06.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- start start 2024-09-17T05:32:06.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f684c155d70 0x7f684c06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f684c09cca0 0x7f684c06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f684c168c60 con 0x7f684c06f110 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f684c168ae0 con 0x7f684c09cca0 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f684c168de0 con 0x7f684c155d70 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45586/0 (socket says 172.21.15.43:45586) 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 -- 172.21.15.43:0/1252124881 learned_addr learned my addr 172.21.15.43:0/1252124881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684be00640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f684c09cca0 0x7f684c06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6843e00640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f684c155d70 0x7f684c06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 -- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f684c155d70 msgr2=0x7f684c06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f684c155d70 0x7f684c06ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 -- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f684c09cca0 msgr2=0x7f684c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f684c09cca0 0x7f684c06d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 -- 172.21.15.43:0/1252124881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c1fcbc0 con 0x7f684c06f110 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f684b400640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f683c00d750 tx=0x7f683c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c017070 con 0x7f684c06f110 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f683c004310 con 0x7f684c06f110 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c00de60 con 0x7f684c06f110 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c1f95b0 con 0x7f684c06f110 2024-09-17T05:32:06.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.243+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c1f8ad0 con 0x7f684c06f110 2024-09-17T05:32:06.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c0ad0b0 con 0x7f684c06f110 2024-09-17T05:32:06.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f683c009c30 con 0x7f684c06f110 2024-09-17T05:32:06.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6843400640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 0x7f6828075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f683c094e60 con 0x7f684c06f110 2024-09-17T05:32:06.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6843e00640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 0x7f6828075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.247+0000 7f6843e00640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 0x7f6828075710 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f684c075540 tx=0x7f6834009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:06.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.251+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f683c05f150 con 0x7f684c06f110 2024-09-17T05:32:06.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.463+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f684c15edf0 con 0x7f684c06f110 2024-09-17T05:32:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.463+0000 7f6843400640 1 -- 172.21.15.43:0/1252124881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+4383 (secure 0 0 0) 0x7f683c063b80 con 0x7f684c06f110 2024-09-17T05:32:06.466 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:06.466 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":205,"btime":"2024-09-17T05:32:06:065099+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1306620592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1306620592},{"type":"v1","addr":"172.21.15.62:6835","nonce":1306620592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:06.065013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 205 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 msgr2=0x7f6828075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 0x7f6828075710 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f684c075540 tx=0x7f6834009040 comp rx=0 tx=0).stop 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 msgr2=0x7f684c09c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f683c00d750 tx=0x7f683c00dc10 comp rx=0 tx=0).stop 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 shutdown_connections 2024-09-17T05:32:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6828073200 0x7f6828075710 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f684c09cca0 0x7f684c06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f684c155d70 0x7f684c06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 --2- 172.21.15.43:0/1252124881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684c06f110 0x7f684c09c4e0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 >> 172.21.15.43:0/1252124881 conn(0x7f684c0b23d0 msgr2=0x7f684c165c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 shutdown_connections 2024-09-17T05:32:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.467+0000 7f6852800640 1 -- 172.21.15.43:0/1252124881 wait complete. 2024-09-17T05:32:06.488 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:06.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 -- 172.21.15.43:0/592437887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040d0d80 msgr2=0x7f2b040cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 --2- 172.21.15.43:0/592437887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040d0d80 0x7f2b040cce70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009840 tx=0x7f2aec01c820 comp rx=0 tx=0).stop 2024-09-17T05:32:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 -- 172.21.15.43:0/592437887 shutdown_connections 2024-09-17T05:32:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 --2- 172.21.15.43:0/592437887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040d0d80 0x7f2b040cce70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 --2- 172.21.15.43:0/592437887 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 0x7f2b040d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 -- 172.21.15.43:0/592437887 >> 172.21.15.43:0/592437887 conn(0x7f2b040cee80 msgr2=0x7f2b040cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 -- 172.21.15.43:0/592437887 shutdown_connections 2024-09-17T05:32:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.643+0000 7f2b0a000640 1 -- 172.21.15.43:0/592437887 wait complete. 2024-09-17T05:32:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 Processor -- start 2024-09-17T05:32:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 -- start start 2024-09-17T05:32:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 0x7f2b041fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040cd3b0 0x7f2b04153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04169e10 con 0x7f2b040d0d80 2024-09-17T05:32:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04169c90 con 0x7f2b040cd3b0 2024-09-17T05:32:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b0a000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2b04169f90 con 0x7f2b040b7ce0 2024-09-17T05:32:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45590/0 (socket says 172.21.15.43:45590) 2024-09-17T05:32:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 -- 172.21.15.43:0/376928787 learned_addr learned my addr 172.21.15.43:0/376928787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2afaa00640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040cd3b0 0x7f2b04153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03400640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 0x7f2b041fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 -- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 msgr2=0x7f2b041fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 0x7f2b041fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 -- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040cd3b0 msgr2=0x7f2b04153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040cd3b0 0x7f2b04153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 -- 172.21.15.43:0/376928787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0415aa40 con 0x7f2b040d0d80 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.647+0000 7f2b03e00640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00c6b0 tx=0x7f2afc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc003b80 con 0x7f2b040d0d80 2024-09-17T05:32:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2afc003d40 con 0x7f2b040d0d80 2024-09-17T05:32:06.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b041fc910 con 0x7f2b040d0d80 2024-09-17T05:32:06.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc005330 con 0x7f2b040d0d80 2024-09-17T05:32:06.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041fced0 con 0x7f2b040d0d80 2024-09-17T05:32:06.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b041fc660 con 0x7f2b040d0d80 2024-09-17T05:32:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2afc0054d0 con 0x7f2b040d0d80 2024-09-17T05:32:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 0x7f2ae0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f2afc0946a0 con 0x7f2b040d0d80 2024-09-17T05:32:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.651+0000 7f2b03400640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 0x7f2ae0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.655+0000 7f2b03400640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 0x7f2ae0075710 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009660 tx=0x7f2af40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.655+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2afc05e890 con 0x7f2b040d0d80 2024-09-17T05:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.867+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b040d0840 con 0x7f2b040d0d80 2024-09-17T05:32:06.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.867+0000 7f2afbe00640 1 -- 172.21.15.43:0/376928787 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+4383 (secure 0 0 0) 0x7f2afc0632c0 con 0x7f2b040d0d80 2024-09-17T05:32:06.870 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:06.870 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":205,"btime":"2024-09-17T05:32:06:065099+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1306620592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1306620592},{"type":"v1","addr":"172.21.15.62:6835","nonce":1306620592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:06.065013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:06.871 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 205 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 msgr2=0x7f2ae0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 0x7f2ae0075710 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009660 tx=0x7f2af40091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 msgr2=0x7f2b04157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00c6b0 tx=0x7f2afc00cb70 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 shutdown_connections 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2ae0073200 0x7f2ae0075710 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b040cd3b0 0x7f2b04153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b040d0d80 0x7f2b04157e10 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 --2- 172.21.15.43:0/376928787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2b040b7ce0 0x7f2b041fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 >> 172.21.15.43:0/376928787 conn(0x7f2b040cee80 msgr2=0x7f2b04072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:06.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 shutdown_connections 2024-09-17T05:32:06.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:06.871+0000 7f2b0a000640 1 -- 172.21.15.43:0/376928787 wait complete. 2024-09-17T05:32:06.893 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:06.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 -- 172.21.15.43:0/2526800227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc0d0650 msgr2=0x7f00dc0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 --2- 172.21.15.43:0/2526800227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc0d0650 0x7f00dc0cd750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009bc0 tx=0x7f00d401cac0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 -- 172.21.15.43:0/2526800227 shutdown_connections 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 --2- 172.21.15.43:0/2526800227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc0d0650 0x7f00dc0cd750 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 -- 172.21.15.43:0/2526800227 >> 172.21.15.43:0/2526800227 conn(0x7f00dc0cee60 msgr2=0x7f00dc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 -- 172.21.15.43:0/2526800227 shutdown_connections 2024-09-17T05:32:07.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.047+0000 7f00e3800640 1 -- 172.21.15.43:0/2526800227 wait complete. 2024-09-17T05:32:07.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 Processor -- start 2024-09-17T05:32:07.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 -- start start 2024-09-17T05:32:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc09ba60 0x7f00dc14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00dc0d0650 0x7f00dc0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc16ab30 con 0x7f00dc0d0240 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc16a9b0 con 0x7f00dc09ba60 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.051+0000 7f00e3800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc16acb0 con 0x7f00dc0d0650 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00e0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc09ba60 0x7f00dc14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00e1800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00dc0d0650 0x7f00dc0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45592/0 (socket says 172.21.15.43:45592) 2024-09-17T05:32:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 -- 172.21.15.43:0/2959237051 learned_addr learned my addr 172.21.15.43:0/2959237051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 -- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00dc0d0650 msgr2=0x7f00dc0c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00dc0d0650 0x7f00dc0c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 -- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc09ba60 msgr2=0x7f00dc14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc09ba60 0x7f00dc14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 -- 172.21.15.43:0/2959237051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc0c6b70 con 0x7f00dc0d0240 2024-09-17T05:32:07.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d3e00640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c700 tx=0x7f00c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4016020 con 0x7f00dc0d0240 2024-09-17T05:32:07.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00c400a930 con 0x7f00dc0d0240 2024-09-17T05:32:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc0c6e50 con 0x7f00dc0d0240 2024-09-17T05:32:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c400ad80 con 0x7f00dc0d0240 2024-09-17T05:32:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.055+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc167810 con 0x7f00dc0d0240 2024-09-17T05:32:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.059+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc167af0 con 0x7f00dc0d0240 2024-09-17T05:32:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.059+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00c4019020 con 0x7f00dc0d0240 2024-09-17T05:32:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.059+0000 7f00d9600640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 0x7f00b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.059+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f00c4094940 con 0x7f00dc0d0240 2024-09-17T05:32:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.059+0000 7f00e0e00640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 0x7f00b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.063+0000 7f00e0e00640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 0x7f00b8075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f00cc007fd0 tx=0x7f00cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.063+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00c4020020 con 0x7f00dc0d0240 2024-09-17T05:32:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.275+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00dc0b3130 con 0x7f00dc0d0240 2024-09-17T05:32:07.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00d9600640 1 -- 172.21.15.43:0/2959237051 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+4383 (secure 0 0 0) 0x7f00c405ec40 con 0x7f00dc0d0240 2024-09-17T05:32:07.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":205,"btime":"2024-09-17T05:32:06:065099+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1306620592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1306620592},{"type":"v1","addr":"172.21.15.62:6835","nonce":1306620592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:06.065013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 205 2024-09-17T05:32:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 msgr2=0x7f00b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 0x7f00b8075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f00cc007fd0 tx=0x7f00cc009040 comp rx=0 tx=0).stop 2024-09-17T05:32:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 msgr2=0x7f00dc0c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c700 tx=0x7f00c400cbc0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 shutdown_connections 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f00b8073200 0x7f00b8075710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f00dc0d0650 0x7f00dc0c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00dc0d0240 0x7f00dc0c7df0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 --2- 172.21.15.43:0/2959237051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00dc09ba60 0x7f00dc14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 >> 172.21.15.43:0/2959237051 conn(0x7f00dc0cee60 msgr2=0x7f00dc0b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 shutdown_connections 2024-09-17T05:32:07.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.279+0000 7f00e3800640 1 -- 172.21.15.43:0/2959237051 wait complete. 2024-09-17T05:32:07.302 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-09-17T05:32:07.302 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-09-17T05:32:07.302 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:07.302 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:07.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:07.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:07.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:07.312 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:07.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:07.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:07.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 -- 172.21.15.43:0/3390732610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b00070620 msgr2=0x7f4b000b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 --2- 172.21.15.43:0/3390732610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b00070620 0x7f4b000b30d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4b000cfc10 tx=0x7f4af001c830 comp rx=0 tx=0).stop 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 -- 172.21.15.43:0/3390732610 shutdown_connections 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 --2- 172.21.15.43:0/3390732610 >> v2:172.21.15.62:3300/0 conn(0x7f4b0006ce60 0x7f4b0006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 --2- 172.21.15.43:0/3390732610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b00070620 0x7f4b000b30d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 -- 172.21.15.43:0/3390732610 >> 172.21.15.43:0/3390732610 conn(0x7f4b000b23d0 msgr2=0x7f4b000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 -- 172.21.15.43:0/3390732610 shutdown_connections 2024-09-17T05:32:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.563+0000 7f4b05000640 1 -- 172.21.15.43:0/3390732610 wait complete. 2024-09-17T05:32:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 Processor -- start 2024-09-17T05:32:07.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- start start 2024-09-17T05:32:07.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b00070620 0x7f4b00159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b000cb860 0x7f4b00157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000ce540 con 0x7f4b0006ce60 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000ce3c0 con 0x7f4b000cb860 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4b000ce6c0 con 0x7f4b00070620 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45614/0 (socket says 172.21.15.43:45614) 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 -- 172.21.15.43:0/1503673896 learned_addr learned my addr 172.21.15.43:0/1503673896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afd400640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b00070620 0x7f4b00159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afe800640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b000cb860 0x7f4b00157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 -- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b00070620 msgr2=0x7f4b00159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b00070620 0x7f4b00159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 -- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b000cb860 msgr2=0x7f4b00157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b000cb860 0x7f4b00157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 -- 172.21.15.43:0/1503673896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b001f9fb0 con 0x7f4b0006ce60 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4afde00640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009850 tx=0x7f4af00a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00ac070 con 0x7f4b0006ce60 2024-09-17T05:32:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af0004120 con 0x7f4b0006ce60 2024-09-17T05:32:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0015a150 con 0x7f4b0006ce60 2024-09-17T05:32:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0005040 con 0x7f4b0006ce60 2024-09-17T05:32:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.567+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001f94d0 con 0x7f4b0006ce60 2024-09-17T05:32:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.571+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b000cecb0 con 0x7f4b0006ce60 2024-09-17T05:32:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.571+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4af00af070 con 0x7f4b0006ce60 2024-09-17T05:32:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.571+0000 7f4af6000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 0x7f4ae0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.571+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f4af012e6e0 con 0x7f4b0006ce60 2024-09-17T05:32:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.571+0000 7f4afd400640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 0x7f4ae0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.575+0000 7f4afd400640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 0x7f4ae0075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec005d10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.575+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b0006d7b0 con 0x7f4b0006ce60 2024-09-17T05:32:07.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:07.791+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b0009e190 con 0x7f4b0006ce60 2024-09-17T05:32:08.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.059+0000 7f4af6000640 1 -- 172.21.15.43:0/1503673896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v206) ==== 161+0+0 (secure 0 0 0) 0x7f4af00164b0 con 0x7f4b0006ce60 2024-09-17T05:32:08.062 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 msgr2=0x7f4ae0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 0x7f4ae0075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec005d10 comp rx=0 tx=0).stop 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 msgr2=0x7f4b0009d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009850 tx=0x7f4af00a6ba0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 shutdown_connections 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4ae0073200 0x7f4ae0075710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b000cb860 0x7f4b00157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b00070620 0x7f4b00159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 --2- 172.21.15.43:0/1503673896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0006ce60 0x7f4b0009d580 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 >> 172.21.15.43:0/1503673896 conn(0x7f4b000b23d0 msgr2=0x7f4b000b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 shutdown_connections 2024-09-17T05:32:08.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.063+0000 7f4b05000640 1 -- 172.21.15.43:0/1503673896 wait complete. 2024-09-17T05:32:08.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 -- 172.21.15.43:0/1349377520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 msgr2=0x7f37040700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 --2- 172.21.15.43:0/1349377520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040700a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009840 tx=0x7f36f001c620 comp rx=0 tx=0).stop 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 -- 172.21.15.43:0/1349377520 shutdown_connections 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 --2- 172.21.15.43:0/1349377520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040700a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 --2- 172.21.15.43:0/1349377520 >> v2:172.21.15.62:3300/0 conn(0x7f370415fe10 0x7f37040b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 -- 172.21.15.43:0/1349377520 >> 172.21.15.43:0/1349377520 conn(0x7f37040b23d0 msgr2=0x7f37040d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 -- 172.21.15.43:0/1349377520 shutdown_connections 2024-09-17T05:32:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.243+0000 7f370c200640 1 -- 172.21.15.43:0/1349377520 wait complete. 2024-09-17T05:32:08.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 Processor -- start 2024-09-17T05:32:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 -- start start 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370415fe10 0x7f3704156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f370416a000 con 0x7f37040730b0 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3704169e80 con 0x7f370415fe10 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370c200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f370416a180 con 0x7f370406fc90 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45616/0 (socket says 172.21.15.43:45616) 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 -- 172.21.15.43:0/539285225 learned_addr learned my addr 172.21.15.43:0/539285225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3709800640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f370a200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370415fe10 0x7f3704156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 -- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 msgr2=0x7f37040995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 -- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370415fe10 msgr2=0x7f3704156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370415fe10 0x7f3704156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 -- 172.21.15.43:0/539285225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3704155b60 con 0x7f37040730b0 2024-09-17T05:32:08.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.247+0000 7f3708e00640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009790 tx=0x7f36f00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00ad070 con 0x7f37040730b0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f00aaa60 con 0x7f37040730b0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00046d0 con 0x7f37040730b0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370415bfb0 con 0x7f37040730b0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3704155e00 con 0x7f37040730b0 2024-09-17T05:32:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37040700a0 con 0x7f37040730b0 2024-09-17T05:32:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36f00b0070 con 0x7f37040730b0 2024-09-17T05:32:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 0x7f36e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f36f012e910 con 0x7f37040730b0 2024-09-17T05:32:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.251+0000 7f3709800640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 0x7f36e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.255+0000 7f3709800640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 0x7f36e4075740 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ad0 tx=0x7f36f80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.259+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36f00b8070 con 0x7f37040730b0 2024-09-17T05:32:08.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:08.499+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3704075fe0 con 0x7f37040730b0 2024-09-17T05:32:09.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f3701600640 1 -- 172.21.15.43:0/539285225 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v207) ==== 101+0+0 (secure 0 0 0) 0x7f36f0016330 con 0x7f37040730b0 2024-09-17T05:32:09.072 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:09.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 msgr2=0x7f36e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 0x7f36e4075740 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ad0 tx=0x7f36f80091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 msgr2=0x7f3704099af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009790 tx=0x7f36f00056a0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 shutdown_connections 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f36e4073230 0x7f36e4075740 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370415fe10 0x7f3704156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040730b0 0x7f3704099af0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 --2- 172.21.15.43:0/539285225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f370406fc90 0x7f37040995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 >> 172.21.15.43:0/539285225 conn(0x7f37040b23d0 msgr2=0x7f37040ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 shutdown_connections 2024-09-17T05:32:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.071+0000 7f370c200640 1 -- 172.21.15.43:0/539285225 wait complete. 2024-09-17T05:32:09.098 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:09.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.155+0000 7f9b50e2b080 1 Processor -- start 2024-09-17T05:32:09.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.155+0000 7f9b50e2b080 1 -- start start 2024-09-17T05:32:09.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee34c24d0 0x55dee35040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3500560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dee340da00 con 0x55dee3575e40 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dee346bf40 con 0x55dee34c24d0 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55dee3576b80 con 0x55dee34eecb0 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b48e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee34c24d0 0x55dee35040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45634/0 (socket says 172.21.15.43:45634) 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 -- 172.21.15.43:0/454334800 learned_addr learned my addr 172.21.15.43:0/454334800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b43e00640 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3500560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 -- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 msgr2=0x55dee3500560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:09.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3500560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 -- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee34c24d0 msgr2=0x55dee35040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:09.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee34c24d0 0x55dee35040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 -- 172.21.15.43:0/454334800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee33e6c10 con 0x55dee3575e40 2024-09-17T05:32:09.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b49800640 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f9b4401a960 tx=0x7f9b4401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f300438d60c2e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b42a00640 1 -- 172.21.15.43:0/454334800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44004020 con 0x55dee3575e40 2024-09-17T05:32:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b42a00640 1 -- 172.21.15.43:0/454334800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b440bb470 con 0x55dee3575e40 2024-09-17T05:32:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b42a00640 1 -- 172.21.15.43:0/454334800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b440042d0 con 0x55dee3575e40 2024-09-17T05:32:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- 172.21.15.43:0/454334800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 msgr2=0x55dee3500aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f9b4401a960 tx=0x7f9b4401aaf0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- 172.21.15.43:0/454334800 shutdown_connections 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee3575e40 0x55dee3500aa0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3500560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/454334800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee34c24d0 0x55dee35040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- 172.21.15.43:0/454334800 >> 172.21.15.43:0/454334800 conn(0x55dee34abd80 msgr2=0x55dee34bd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- 172.21.15.43:0/454334800 shutdown_connections 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.159+0000 7f9b50e2b080 1 -- 172.21.15.43:0/454334800 wait complete. 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 Processor -- start 2024-09-17T05:32:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- start start 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3578570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee3575e40 0x55dee3590fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dee3505790 con 0x55dee34c24d0 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dee35054f0 con 0x55dee3575e40 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55dee3501610 con 0x55dee34eecb0 2024-09-17T05:32:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45644/0 (socket says 172.21.15.43:45644) 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 learned_addr learned my addr 172.21.15.43:0/869267135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee3575e40 0x55dee3590fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3578570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 msgr2=0x55dee3578570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3578570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee3575e40 msgr2=0x55dee3590fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee3575e40 0x55dee3590fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee3591500 con 0x55dee34c24d0 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dee3575e40 0x55dee3590fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55dee34eecb0 0x55dee3578570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00ef20 tx=0x7f9b3c00c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b41600640 1 -- 172.21.15.43:0/869267135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c016030 con 0x55dee34c24d0 2024-09-17T05:32:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b41600640 1 -- 172.21.15.43:0/869267135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b3c001980 con 0x55dee34c24d0 2024-09-17T05:32:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b41600640 1 -- 172.21.15.43:0/869267135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00a040 con 0x55dee34c24d0 2024-09-17T05:32:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dee3575a20 con 0x55dee34c24d0 2024-09-17T05:32:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.163+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee3506d70 con 0x55dee34c24d0 2024-09-17T05:32:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b41600640 1 -- 172.21.15.43:0/869267135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b3c019070 con 0x55dee34c24d0 2024-09-17T05:32:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b41600640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 0x7f9b280740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b41600640 1 -- 172.21.15.43:0/869267135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f9b3c0934f0 con 0x55dee34c24d0 2024-09-17T05:32:09.170 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 0x7f9b280740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 0x55dee362d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee362dc20 con 0x55dee362d2d0 2024-09-17T05:32:09.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 0x55dee362d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 0x7f9b280740f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c850 tx=0x7f9b34004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.167+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 0x55dee362d6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.171+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v109'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.171+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3638900 con 0x55dee362d2d0 2024-09-17T05:32:09.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.171+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v109'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.171+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3638ca0 con 0x55dee362d2d0 2024-09-17T05:32:09.176 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:09.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v109'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363a050 con 0x55dee362d2d0 2024-09-17T05:32:09.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363a3f0 con 0x55dee362d2d0 2024-09-17T05:32:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.175+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639040 con 0x55dee362d2d0 2024-09-17T05:32:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.179+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.179+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36393e0 con 0x55dee362d2d0 2024-09-17T05:32:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.179+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v109'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.179+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v109'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v109'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.185 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:09.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.183+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.187+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.193 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:09.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.191+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.195+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v109'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v109'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.203 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v109'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.199+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.203+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.207+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v109'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.207+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.207+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v109'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.207+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v109'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.211 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.211+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639780 con 0x55dee362d2d0 2024-09-17T05:32:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55dee363a790 0x55dee363cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3639c90 con 0x55dee363a790 2024-09-17T05:32:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55dee363a790 0x55dee363cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55dee363a790 0x55dee363cbb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'441 uv424 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.215+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee363d0f0 con 0x55dee363a790 2024-09-17T05:32:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'442 uv442 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d490 con 0x55dee363a790 2024-09-17T05:32:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.221 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:09.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363dce0 con 0x55dee363a790 2024-09-17T05:32:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.219+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363e080 con 0x55dee363a790 2024-09-17T05:32:09.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.223+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.223+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363e420 con 0x55dee363a790 2024-09-17T05:32:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.223+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v109'446 uv429 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.223+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee363e7c0 con 0x55dee363a790 2024-09-17T05:32:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v109'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363eb60 con 0x55dee363a790 2024-09-17T05:32:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v109'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.229 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.227+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v109'451 uv434 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v109'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.231+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.235+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v109'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.237 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:09.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.235+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.235+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v109'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.235+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.239+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.239+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.239+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v109'456 uv440 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.239+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v109'457 uv457 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.245 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:09.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v109'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.243+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363d910 con 0x55dee363a790 2024-09-17T05:32:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee363a790 2024-09-17T05:32:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55dee363f520 0x55dee3641940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3641e80 con 0x55dee363f520 2024-09-17T05:32:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55dee363f520 0x55dee3641940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.247+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55dee363f520 0x55dee3641940 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v109'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3400aac0 con 0x55dee363f520 2024-09-17T05:32:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3642220 con 0x55dee363f520 2024-09-17T05:32:09.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v109'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee363ec50 con 0x55dee363f520 2024-09-17T05:32:09.254 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:09.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v109'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3644000 con 0x55dee363f520 2024-09-17T05:32:09.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.251+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.255+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36443a0 con 0x55dee363f520 2024-09-17T05:32:09.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.255+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.255+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3642ff0 con 0x55dee363f520 2024-09-17T05:32:09.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.255+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.255+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3643410 con 0x55dee362d2d0 2024-09-17T05:32:09.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'85 uv84 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3643410 con 0x55dee362d2d0 2024-09-17T05:32:09.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'86 uv86 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3643410 con 0x55dee362d2d0 2024-09-17T05:32:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.263 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.259+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3643410 con 0x55dee362d2d0 2024-09-17T05:32:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 0x55dee3646b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36470a0 con 0x55dee3644740 2024-09-17T05:32:09.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 0x55dee3646b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b49800640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 0x55dee3646b60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v109'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.263+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee36470a0 con 0x55dee3644740 2024-09-17T05:32:09.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v109'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36470a0 con 0x55dee3644740 2024-09-17T05:32:09.269 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:09.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v109'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36480b0 con 0x55dee3644740 2024-09-17T05:32:09.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36480b0 con 0x55dee3644740 2024-09-17T05:32:09.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.267+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee36470a0 con 0x55dee3644740 2024-09-17T05:32:09.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55dee3647720 0x55dee3649b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a0c0 con 0x55dee3647720 2024-09-17T05:32:09.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55dee3647720 0x55dee3649b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b48e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55dee3647720 0x55dee3649b40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v109'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.271+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364a1a0 con 0x55dee3647720 2024-09-17T05:32:09.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v109'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.277 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:09.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v109'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364b5d0 con 0x55dee3647720 2024-09-17T05:32:09.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.275+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364b5d0 con 0x55dee3647720 2024-09-17T05:32:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v109'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.279+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v109'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.284 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:09.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v109'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364a5c0 con 0x55dee3647720 2024-09-17T05:32:09.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.283+0000 7f9b48e00640 1 -- 172.21.15.43:0/869267135 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b3c00d040 con 0x55dee3647720 2024-09-17T05:32:09.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dee364ac40 0x55dee364d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364d680 con 0x55dee364ac40 2024-09-17T05:32:09.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dee364ac40 0x55dee364d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b43e00640 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dee364ac40 0x55dee364d060 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v109'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364da50 con 0x55dee364ac40 2024-09-17T05:32:09.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v109'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.287+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee364ac40 2024-09-17T05:32:09.292 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:09.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v109'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee364ac40 2024-09-17T05:32:09.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee364ac40 2024-09-17T05:32:09.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.291+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee362d2d0 2024-09-17T05:32:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v109'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee362d2d0 2024-09-17T05:32:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v109'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee362d2d0 2024-09-17T05:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v109'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.299 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.295+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364ee80 con 0x55dee362d2d0 2024-09-17T05:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364ee80 con 0x55dee362d2d0 2024-09-17T05:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee362d2d0 2024-09-17T05:32:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee362d2d0 2024-09-17T05:32:09.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.299+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v109'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v109'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.306 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v109'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.303+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.307+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.307+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.307+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.307+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v109'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v109'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v109'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.315 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.311+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364de70 con 0x55dee363f520 2024-09-17T05:32:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.315+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.315+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650cc0 con 0x55dee363f520 2024-09-17T05:32:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.315+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee363f520 2024-09-17T05:32:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.315+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v109'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.322 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:09.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v109'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.319+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.323+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.323+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.323+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v109'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.323+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.323+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v109'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.327+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.327+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v109'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:09.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.327+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee3650c80 con 0x55dee3644740 2024-09-17T05:32:09.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.327+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.327+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364f290 con 0x55dee3644740 2024-09-17T05:32:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee3644740 2024-09-17T05:32:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b44002050 con 0x55dee3644740 2024-09-17T05:32:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee364ac40 2024-09-17T05:32:09.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.331+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee364ac40 2024-09-17T05:32:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v109'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee364ac40 2024-09-17T05:32:09.337 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee364ac40 2024-09-17T05:32:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.335+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e280 con 0x55dee364ac40 2024-09-17T05:32:09.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v109'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v109'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.339+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v109'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.344 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v109'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.343+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v109'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v109'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v109'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.351 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:09.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.347+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1f0 con 0x55dee364ac40 2024-09-17T05:32:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v109'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1d0 con 0x55dee364ac40 2024-09-17T05:32:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v109'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e1d0 con 0x55dee364ac40 2024-09-17T05:32:09.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v109'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.351+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55dee364e5c0 con 0x55dee364ac40 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b43e00640 1 -- 172.21.15.43:0/869267135 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v109'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9b34019490 con 0x55dee364ac40 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55dee363a790 msgr2=0x55dee363cbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55dee363a790 0x55dee363cbb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55dee3647720 msgr2=0x55dee3649b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55dee3647720 0x55dee3649b40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dee364ac40 msgr2=0x55dee364d060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55dee364ac40 0x55dee364d060 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55dee363f520 msgr2=0x55dee3641940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55dee363f520 0x55dee3641940 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 msgr2=0x55dee362d6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b49800640 1 -- 172.21.15.43:0/869267135 reap_dead start 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 0x55dee362d6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 msgr2=0x55dee3646b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 0x55dee3646b60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 msgr2=0x7f9b280740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 0x7f9b280740f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c850 tx=0x7f9b34004890 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 msgr2=0x55dee35074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00ef20 tx=0x7f9b3c00c6f0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 shutdown_connections 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55dee3644740 0x55dee3646b60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b28071be0 0x7f9b280740f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55dee362d2d0 0x55dee362d6e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 --2- 172.21.15.43:0/869267135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dee34c24d0 0x55dee35074b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 >> 172.21.15.43:0/869267135 conn(0x55dee34abd80 msgr2=0x55dee35921e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 shutdown_connections 2024-09-17T05:32:09.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.355+0000 7f9b50e2b080 1 -- 172.21.15.43:0/869267135 wait complete. 2024-09-17T05:32:09.361 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-17T05:32:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.463+0000 7f6d7685e080 1 Processor -- start 2024-09-17T05:32:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.463+0000 7f6d7685e080 1 -- start start 2024-09-17T05:32:09.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af717e40 0x5576af6a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576af5afa00 con 0x5576af690cb0 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5576af60df40 con 0x5576af717e40 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5576af718b80 con 0x5576af661d70 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45670/0 (socket says 172.21.15.43:45670) 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 -- 172.21.15.43:0/45867760 learned_addr learned my addr 172.21.15.43:0/45867760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6fa00640 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d70400640 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af717e40 0x5576af6a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 -- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 msgr2=0x5576af6a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:09.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 -- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af717e40 msgr2=0x5576af6a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af717e40 0x5576af6a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 -- 172.21.15.43:0/45867760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576af588c10 con 0x5576af690cb0 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6f000640 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00b5c0 tx=0x7f6d5c00ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5210c54ecdd1d09d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6dc00640 1 -- 172.21.15.43:0/45867760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00a950 con 0x5576af690cb0 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6dc00640 1 -- 172.21.15.43:0/45867760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d5c00aaf0 con 0x5576af690cb0 2024-09-17T05:32:09.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d6dc00640 1 -- 172.21.15.43:0/45867760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c008460 con 0x5576af690cb0 2024-09-17T05:32:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- 172.21.15.43:0/45867760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 msgr2=0x5576af6a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00b5c0 tx=0x7f6d5c00ba80 comp rx=0 tx=0).stop 2024-09-17T05:32:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- 172.21.15.43:0/45867760 shutdown_connections 2024-09-17T05:32:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af717e40 0x5576af6a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af690cb0 0x5576af6a7e90 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 --2- 172.21.15.43:0/45867760 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- 172.21.15.43:0/45867760 >> 172.21.15.43:0/45867760 conn(0x5576af64dd80 msgr2=0x5576af65f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- 172.21.15.43:0/45867760 shutdown_connections 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- 172.21.15.43:0/45867760 wait complete. 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 Processor -- start 2024-09-17T05:32:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.467+0000 7f6d7685e080 1 -- start start 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6865c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af690cb0 0x5576af7d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 0x5576af7d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576af7bcc00 con 0x5576af717e40 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5576af6910c0 con 0x5576af690cb0 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5576af691360 con 0x5576af661d70 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 0x5576af7d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 0x5576af7d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45678/0 (socket says 172.21.15.43:45678) 2024-09-17T05:32:09.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 -- 172.21.15.43:0/1707545524 learned_addr learned my addr 172.21.15.43:0/1707545524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d6f000640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af690cb0 0x5576af7d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d6fa00640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 -- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 msgr2=0x5576af6865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5576af661d70 0x5576af6865c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 -- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af690cb0 msgr2=0x5576af7d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5576af690cb0 0x5576af7d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 -- 172.21.15.43:0/1707545524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576af688230 con 0x5576af717e40 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d70400640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 0x5576af7d20a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d68006f90 tx=0x7f6d68003230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d67e00640 1 -- 172.21.15.43:0/1707545524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68013480 con 0x5576af717e40 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d67e00640 1 -- 172.21.15.43:0/1707545524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d68018070 con 0x5576af717e40 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d67e00640 1 -- 172.21.15.43:0/1707545524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d680139b0 con 0x5576af717e40 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576af7cf9a0 con 0x5576af717e40 2024-09-17T05:32:09.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.471+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576af686b00 con 0x5576af717e40 2024-09-17T05:32:09.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d67e00640 1 -- 172.21.15.43:0/1707545524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d68013b50 con 0x5576af717e40 2024-09-17T05:32:09.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d67e00640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6d50071be0 0x7f6d500740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d67e00640 1 -- 172.21.15.43:0/1707545524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f6d68093290 con 0x5576af717e40 2024-09-17T05:32:09.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d6fa00640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6d50071be0 0x7f6d500740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d7685e080 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5576af7ce570 0x5576af7ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x5576af7ceec0 con 0x5576af7ce570 2024-09-17T05:32:09.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d6f000640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5576af7ce570 0x5576af7ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d6fa00640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6d50071be0 0x7f6d500740f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6d600af000 tx=0x7f6d6001c250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.475+0000 7f6d6f000640 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5576af7ce570 0x5576af7ce980 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d6f000640 1 -- 172.21.15.43:0/1707545524 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v109'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6d5c004d40 con 0x5576af7ce570 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5576af7ce570 msgr2=0x5576af7ce980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5576af7ce570 0x5576af7ce980 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6d50071be0 msgr2=0x7f6d500740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6d50071be0 0x7f6d500740f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6d600af000 tx=0x7f6d6001c250 comp rx=0 tx=0).stop 2024-09-17T05:32:09.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 msgr2=0x5576af7d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 --2- 172.21.15.43:0/1707545524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576af717e40 0x5576af7d20a0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d68006f90 tx=0x7f6d68003230 comp rx=0 tx=0).stop 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d70400640 1 -- 172.21.15.43:0/1707545524 reap_dead start 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 shutdown_connections 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 >> 172.21.15.43:0/1707545524 conn(0x5576af64dd80 msgr2=0x5576af65f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 shutdown_connections 2024-09-17T05:32:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.479+0000 7f6d7685e080 1 -- 172.21.15.43:0/1707545524 wait complete. 2024-09-17T05:32:09.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- 172.21.15.43:0/941248887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 msgr2=0x7f9a0c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 --2- 172.21.15.43:0/941248887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a0c004c20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9a100016f0 tx=0x7f9a1001c7b0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a27e00640 1 -- 172.21.15.43:0/941248887 reap_dead start 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- 172.21.15.43:0/941248887 shutdown_connections 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- 172.21.15.43:0/941248887 >> 172.21.15.43:0/941248887 conn(0x7f9a280cee80 msgr2=0x7f9a280cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- 172.21.15.43:0/941248887 shutdown_connections 2024-09-17T05:32:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- 172.21.15.43:0/941248887 wait complete. 2024-09-17T05:32:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 Processor -- start 2024-09-17T05:32:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.691+0000 7f9a2e200640 1 -- start start 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a280b0b90 0x7f9a2806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a2815cac0 0x7f9a2815e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28070be0 con 0x7f9a0c004810 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a28070a60 con 0x7f9a280b0b90 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9a280d0840 con 0x7f9a2815cac0 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a2815cac0 0x7f9a2815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a1fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a280b0b90 0x7f9a2806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45690/0 (socket says 172.21.15.43:45690) 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 -- 172.21.15.43:0/1387204272 learned_addr learned my addr 172.21.15.43:0/1387204272 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 -- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a2815cac0 msgr2=0x7f9a2815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a2815cac0 0x7f9a2815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 -- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a280b0b90 msgr2=0x7f9a2806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a280b0b90 0x7f9a2806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 -- 172.21.15.43:0/1387204272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2815d6a0 con 0x7f9a0c004810 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a27400640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9a180119b0 tx=0x7f9a18011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18004460 con 0x7f9a0c004810 2024-09-17T05:32:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a18016070 con 0x7f9a0c004810 2024-09-17T05:32:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800fc30 con 0x7f9a0c004810 2024-09-17T05:32:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2815d980 con 0x7f9a0c004810 2024-09-17T05:32:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2815df40 con 0x7f9a0c004810 2024-09-17T05:32:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.695+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a280cce70 con 0x7f9a0c004810 2024-09-17T05:32:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.699+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a18018030 con 0x7f9a0c004810 2024-09-17T05:32:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.699+0000 7f9a1f400640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 0x7f9a04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.699+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7429+0+0 (secure 0 0 0) 0x7f9a180951a0 con 0x7f9a0c004810 2024-09-17T05:32:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.699+0000 7f9a1fe00640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 0x7f9a04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.699+0000 7f9a1fe00640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 0x7f9a04075710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a100016c0 tx=0x7f9a100b1040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.703+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9a1805f4a0 con 0x7f9a0c004810 2024-09-17T05:32:09.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:09.911+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9a280cc5e0 con 0x7f9a0c004810 2024-09-17T05:32:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a1f400640 1 -- 172.21.15.43:0/1387204272 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v208) ==== 162+0+0 (secure 0 0 0) 0x7f9a18063ed0 con 0x7f9a0c004810 2024-09-17T05:32:10.076 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 msgr2=0x7f9a04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 0x7f9a04075710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a100016c0 tx=0x7f9a100b1040 comp rx=0 tx=0).stop 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 msgr2=0x7f9a281fa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9a180119b0 tx=0x7f9a18011db0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 shutdown_connections 2024-09-17T05:32:10.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9a04073200 0x7f9a04075710 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a280b0b90 0x7f9a2806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9a2815cac0 0x7f9a2815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 --2- 172.21.15.43:0/1387204272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a0c004810 0x7f9a281fa690 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 >> 172.21.15.43:0/1387204272 conn(0x7f9a280cee80 msgr2=0x7f9a280b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 shutdown_connections 2024-09-17T05:32:10.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.075+0000 7f9a2e200640 1 -- 172.21.15.43:0/1387204272 wait complete. 2024-09-17T05:32:10.087 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:10.084+0000 7fc21e400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:32:10.088 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:10.087+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:10.088 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:10.087+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:10.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- 172.21.15.43:0/583157613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 msgr2=0x7f88d0004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 --2- 172.21.15.43:0/583157613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88d0004070 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009b60 tx=0x7f88dc01d290 comp rx=0 tx=0).stop 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88eb400640 1 -- 172.21.15.43:0/583157613 reap_dead start 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- 172.21.15.43:0/583157613 shutdown_connections 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- 172.21.15.43:0/583157613 >> 172.21.15.43:0/583157613 conn(0x7f88ec0d0870 msgr2=0x7f88ec0d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- 172.21.15.43:0/583157613 shutdown_connections 2024-09-17T05:32:10.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- 172.21.15.43:0/583157613 wait complete. 2024-09-17T05:32:10.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 Processor -- start 2024-09-17T05:32:10.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.263+0000 7f88ebe00640 1 -- start start 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec14eac0 0x7f88ec15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec14eed0 0x7f88ec15dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec167ca0 con 0x7f88d0003c20 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec167b20 con 0x7f88ec14eac0 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec167e20 con 0x7f88ec14eed0 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45702/0 (socket says 172.21.15.43:45702) 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 -- 172.21.15.43:0/2659620516 learned_addr learned my addr 172.21.15.43:0/2659620516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:10.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88e3e00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec14eac0 0x7f88ec15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eb400640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec14eed0 0x7f88ec15dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 -- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec14eed0 msgr2=0x7f88ec15dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec14eed0 0x7f88ec15dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 -- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec14eac0 msgr2=0x7f88ec15d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec14eac0 0x7f88ec15d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 -- 172.21.15.43:0/2659620516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1fe0f0 con 0x7f88d0003c20 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88eaa00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009b60 tx=0x7f88dc005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0b1040 con 0x7f88d0003c20 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88dc0bb070 con 0x7f88d0003c20 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1f9e50 con 0x7f88d0003c20 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc009730 con 0x7f88d0003c20 2024-09-17T05:32:10.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1fa410 con 0x7f88d0003c20 2024-09-17T05:32:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec14d2a0 con 0x7f88d0003c20 2024-09-17T05:32:10.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.267+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f88dc004270 con 0x7f88d0003c20 2024-09-17T05:32:10.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.271+0000 7f88e2a00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 0x7f88c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.271+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7f88dc12fa60 con 0x7f88d0003c20 2024-09-17T05:32:10.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.271+0000 7f88e3e00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 0x7f88c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.271+0000 7f88e3e00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 0x7f88c8075710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88ec14cbe0 tx=0x7f88d4006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:10.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.275+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f88dc016fa0 con 0x7f88d0003c20 2024-09-17T05:32:10.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.483+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88ec151ed0 con 0x7f88d0003c20 2024-09-17T05:32:10.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.483+0000 7f88e2a00640 1 -- 172.21.15.43:0/2659620516 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) ==== 95+0+3557 (secure 0 0 0) 0x7f88dc0fa2e0 con 0x7f88d0003c20 2024-09-17T05:32:10.487 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:10.487 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":210,"btime":"2024-09-17T05:32:10:088901+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:10.088842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:10.487 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 210 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 msgr2=0x7f88c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 0x7f88c8075710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88ec14cbe0 tx=0x7f88d4006b20 comp rx=0 tx=0).stop 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 msgr2=0x7f88ec149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009b60 tx=0x7f88dc005590 comp rx=0 tx=0).stop 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 shutdown_connections 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f88c8073200 0x7f88c8075710 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88ec14eac0 0x7f88ec15d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f88ec14eed0 0x7f88ec15dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 --2- 172.21.15.43:0/2659620516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d0003c20 0x7f88ec149050 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 >> 172.21.15.43:0/2659620516 conn(0x7f88ec0d0870 msgr2=0x7f88ec166610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 shutdown_connections 2024-09-17T05:32:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.487+0000 7f88ebe00640 1 -- 172.21.15.43:0/2659620516 wait complete. 2024-09-17T05:32:10.509 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:10.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 -- 172.21.15.43:0/1207066457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 msgr2=0x7efbf400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 --2- 172.21.15.43:0/1207066457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efbf400b9d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7efc100c7280 tx=0x7efc0801fa40 comp rx=0 tx=0).stop 2024-09-17T05:32:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc0fe00640 1 -- 172.21.15.43:0/1207066457 reap_dead start 2024-09-17T05:32:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 -- 172.21.15.43:0/1207066457 shutdown_connections 2024-09-17T05:32:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 -- 172.21.15.43:0/1207066457 >> 172.21.15.43:0/1207066457 conn(0x7efc100b23d0 msgr2=0x7efc100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 -- 172.21.15.43:0/1207066457 shutdown_connections 2024-09-17T05:32:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.667+0000 7efc16400640 1 -- 172.21.15.43:0/1207066457 wait complete. 2024-09-17T05:32:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 Processor -- start 2024-09-17T05:32:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 -- start start 2024-09-17T05:32:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc100c9830 0x7efc1009db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc101f9da0 0x7efc100cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc1015cb70 con 0x7efbf40051b0 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc1015c9f0 con 0x7efc100c9830 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc16400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efc100ad7c0 con 0x7efc101f9da0 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45720/0 (socket says 172.21.15.43:45720) 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 -- 172.21.15.43:0/2598548902 learned_addr learned my addr 172.21.15.43:0/2598548902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc06a00640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc100c9830 0x7efc1009db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0fe00640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc101f9da0 0x7efc100cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 -- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc101f9da0 msgr2=0x7efc100cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc101f9da0 0x7efc100cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 -- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc100c9830 msgr2=0x7efc1009db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc100c9830 0x7efc1009db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 -- 172.21.15.43:0/2598548902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc100cd360 con 0x7efbf40051b0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.671+0000 7efc0f400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7efc00011a60 tx=0x7efc00011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0001f070 con 0x7efbf40051b0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc0000fdc0 con 0x7efbf40051b0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000126b0 con 0x7efbf40051b0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc100cd640 con 0x7efbf40051b0 2024-09-17T05:32:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101fa1b0 con 0x7efbf40051b0 2024-09-17T05:32:10.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc100add50 con 0x7efbf40051b0 2024-09-17T05:32:10.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc00022070 con 0x7efbf40051b0 2024-09-17T05:32:10.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 0x7efbec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:10.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7efc00095240 con 0x7efbf40051b0 2024-09-17T05:32:10.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.675+0000 7efc06a00640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 0x7efbec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:10.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.679+0000 7efc06a00640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 0x7efbec075740 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efc100c9ec0 tx=0x7efbf80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:10.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.679+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc0005f580 con 0x7efbf40051b0 2024-09-17T05:32:10.818 INFO:tasks.ceph.mds.c.smithi043.stderr: -52> 2024-09-17T05:32:06.063+0000 7f4c81400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:32:10.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc1009e0e0 con 0x7efbf40051b0 2024-09-17T05:32:10.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc07e00640 1 -- 172.21.15.43:0/2598548902 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) ==== 95+0+3557 (secure 0 0 0) 0x7efc00063fb0 con 0x7efbf40051b0 2024-09-17T05:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":210,"btime":"2024-09-17T05:32:10:088901+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:10.088842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:10.897 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 210 2024-09-17T05:32:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 msgr2=0x7efbec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 0x7efbec075740 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efc100c9ec0 tx=0x7efbf80091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 msgr2=0x7efc100ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7efc00011a60 tx=0x7efc00011e60 comp rx=0 tx=0).stop 2024-09-17T05:32:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 shutdown_connections 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efbec073230 0x7efbec075740 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc100c9830 0x7efc1009db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc101f9da0 0x7efc100cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 --2- 172.21.15.43:0/2598548902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efbf40051b0 0x7efc100ccb10 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 >> 172.21.15.43:0/2598548902 conn(0x7efc100b23d0 msgr2=0x7efc100b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 shutdown_connections 2024-09-17T05:32:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:10.895+0000 7efc16400640 1 -- 172.21.15.43:0/2598548902 wait complete. 2024-09-17T05:32:10.920 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:10.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:10.934 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- 172.21.15.43:0/2502244023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 msgr2=0x7f4b800b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 --2- 172.21.15.43:0/2502244023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b800b30d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4b80073040 tx=0x7f4b7001c770 comp rx=0 tx=0).stop 2024-09-17T05:32:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- 172.21.15.43:0/2502244023 shutdown_connections 2024-09-17T05:32:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 --2- 172.21.15.43:0/2502244023 >> v2:172.21.15.62:3300/0 conn(0x7f4b800cb860 0x7f4b800cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 --2- 172.21.15.43:0/2502244023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b800b30d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- 172.21.15.43:0/2502244023 >> 172.21.15.43:0/2502244023 conn(0x7f4b800b23d0 msgr2=0x7f4b800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- 172.21.15.43:0/2502244023 shutdown_connections 2024-09-17T05:32:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- 172.21.15.43:0/2502244023 wait complete. 2024-09-17T05:32:11.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 Processor -- start 2024-09-17T05:32:11.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.083+0000 7f4b86e00640 1 -- start start 2024-09-17T05:32:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b8006ce60 0x7f4b8009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b800cb860 0x7f4b80156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800ce5a0 con 0x7f4b80070620 2024-09-17T05:32:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800ce420 con 0x7f4b800cb860 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4b800ce720 con 0x7f4b8006ce60 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45764/0 (socket says 172.21.15.43:45764) 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 -- 172.21.15.43:0/857464334 learned_addr learned my addr 172.21.15.43:0/857464334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b7fe00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b8006ce60 0x7f4b8009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b84e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b800cb860 0x7f4b80156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 -- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b8006ce60 msgr2=0x7f4b8009d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b8006ce60 0x7f4b8009d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 -- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b800cb860 msgr2=0x7f4b80156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b800cb860 0x7f4b80156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 -- 172.21.15.43:0/857464334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b801544e0 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b77e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f4b68001ec0 tx=0x7f4b6800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68017070 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b68003600 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b801582a0 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68003a50 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b80158820 con 0x7f4b80070620 2024-09-17T05:32:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.087+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b800ced10 con 0x7f4b80070620 2024-09-17T05:32:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.091+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b6801a070 con 0x7f4b80070620 2024-09-17T05:32:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.091+0000 7f4b7cc00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 0x7f4b5c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.091+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7f4b68093b90 con 0x7f4b80070620 2024-09-17T05:32:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.091+0000 7f4b7fe00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 0x7f4b5c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.091+0000 7f4b7fe00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 0x7f4b5c075740 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b700af000 tx=0x7f4b7001afb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.099+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b6805de60 con 0x7f4b80070620 2024-09-17T05:32:11.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.307+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b8009e190 con 0x7f4b80070620 2024-09-17T05:32:11.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.307+0000 7f4b7cc00640 1 -- 172.21.15.43:0/857464334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) ==== 95+0+4380 (secure 0 0 0) 0x7f4b68062890 con 0x7f4b80070620 2024-09-17T05:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":211,"btime":"2024-09-17T05:32:11:091004+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8898,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/324676924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":324676924},{"type":"v1","addr":"172.21.15.43:6835","nonce":324676924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":211}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:10.088842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:11.310 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 211 2024-09-17T05:32:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 msgr2=0x7f4b5c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 0x7f4b5c075740 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b700af000 tx=0x7f4b7001afb0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 msgr2=0x7f4b80155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f4b68001ec0 tx=0x7f4b6800c890 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 shutdown_connections 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4b5c073230 0x7f4b5c075740 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b800cb860 0x7f4b80156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b80070620 0x7f4b80155cd0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 --2- 172.21.15.43:0/857464334 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4b8006ce60 0x7f4b8009d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 >> 172.21.15.43:0/857464334 conn(0x7f4b800b23d0 msgr2=0x7f4b800ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 shutdown_connections 2024-09-17T05:32:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.311+0000 7f4b86e00640 1 -- 172.21.15.43:0/857464334 wait complete. 2024-09-17T05:32:11.333 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-09-17T05:32:11.333 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-09-17T05:32:11.333 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:11.333 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:11.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:11.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:11.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:11.384 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:11.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:11.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 -- 172.21.15.43:0/2852263295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0530003af0 msgr2=0x7f0530003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 --2- 172.21.15.43:0/2852263295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0530003af0 0x7f0530003f60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f053c009b30 tx=0x7f053c01d240 comp rx=0 tx=0).stop 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f054a800640 1 -- 172.21.15.43:0/2852263295 reap_dead start 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 -- 172.21.15.43:0/2852263295 shutdown_connections 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 -- 172.21.15.43:0/2852263295 >> 172.21.15.43:0/2852263295 conn(0x7f054c0b2e80 msgr2=0x7f054c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 -- 172.21.15.43:0/2852263295 shutdown_connections 2024-09-17T05:32:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.639+0000 7f0550e00640 1 -- 172.21.15.43:0/2852263295 wait complete. 2024-09-17T05:32:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 Processor -- start 2024-09-17T05:32:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 -- start start 2024-09-17T05:32:11.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0530003af0 0x7f054c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f054c0ce6f0 0x7f054c0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f054c1669e0 con 0x7f054c1f6a60 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f054c166860 con 0x7f054c0ce6f0 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0550e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f054c166b60 con 0x7f0530003af0 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0541400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0541400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45788/0 (socket says 172.21.15.43:45788) 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0541400640 1 -- 172.21.15.43:0/1009124420 learned_addr learned my addr 172.21.15.43:0/1009124420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f0549e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0530003af0 0x7f054c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.643+0000 7f054a800640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f054c0ce6f0 0x7f054c0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 -- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0530003af0 msgr2=0x7f054c0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0530003af0 0x7f054c0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 -- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f054c0ce6f0 msgr2=0x7f054c0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f054c0ce6f0 0x7f054c0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 -- 172.21.15.43:0/1009124420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c157010 con 0x7f054c1f6a60 2024-09-17T05:32:11.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0541400640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f053400e590 tx=0x7f053400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:11.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534015070 con 0x7f054c1f6a60 2024-09-17T05:32:11.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053400c9e0 con 0x7f054c1f6a60 2024-09-17T05:32:11.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053400cdf0 con 0x7f054c1f6a60 2024-09-17T05:32:11.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c0cd130 con 0x7f054c1f6a60 2024-09-17T05:32:11.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1f7500 con 0x7f054c1f6a60 2024-09-17T05:32:11.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c1f6f90 con 0x7f054c1f6a60 2024-09-17T05:32:11.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0534018070 con 0x7f054c1f6a60 2024-09-17T05:32:11.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 0x7f0528075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:11.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.647+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7f0534094e10 con 0x7f054c1f6a60 2024-09-17T05:32:11.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.651+0000 7f0549e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 0x7f0528075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:11.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.651+0000 7f0549e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 0x7f0528075710 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f053c0ae000 tx=0x7f053c019fe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:11.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.651+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0534098030 con 0x7f054c1f6a60 2024-09-17T05:32:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:11.863+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f054c15f530 con 0x7f054c1f6a60 2024-09-17T05:32:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.095+0000 7f0542a00640 1 -- 172.21.15.43:0/1009124420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v212) ==== 161+0+0 (secure 0 0 0) 0x7f053405f710 con 0x7f054c1f6a60 2024-09-17T05:32:12.099 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 msgr2=0x7f0528075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 0x7f0528075710 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f053c0ae000 tx=0x7f053c019fe0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 msgr2=0x7f054c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f053400e590 tx=0x7f053400ea50 comp rx=0 tx=0).stop 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 shutdown_connections 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0528073200 0x7f0528075710 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f054c0ce6f0 0x7f054c0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f054c1f6a60 0x7f054c156ad0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 --2- 172.21.15.43:0/1009124420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0530003af0 0x7f054c0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 >> 172.21.15.43:0/1009124420 conn(0x7f054c0b2e80 msgr2=0x7f054c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 shutdown_connections 2024-09-17T05:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.099+0000 7f0550e00640 1 -- 172.21.15.43:0/1009124420 wait complete. 2024-09-17T05:32:12.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:12.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 -- 172.21.15.43:0/3152978239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb80b3920 msgr2=0x7f6cb80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 --2- 172.21.15.43:0/3152978239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb80b3920 0x7f6cb80ad0b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80711f0 tx=0x7f6cb001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 -- 172.21.15.43:0/3152978239 shutdown_connections 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 --2- 172.21.15.43:0/3152978239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb80b3920 0x7f6cb80ad0b0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 --2- 172.21.15.43:0/3152978239 >> v2:172.21.15.62:3300/0 conn(0x7f6cb806f110 0x7f6cb80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 --2- 172.21.15.43:0/3152978239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 0x7f6cb815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 -- 172.21.15.43:0/3152978239 >> 172.21.15.43:0/3152978239 conn(0x7f6cb80b23d0 msgr2=0x7f6cb80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 -- 172.21.15.43:0/3152978239 shutdown_connections 2024-09-17T05:32:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.279+0000 7f6cbde00640 1 -- 172.21.15.43:0/3152978239 wait complete. 2024-09-17T05:32:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 Processor -- start 2024-09-17T05:32:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 -- start start 2024-09-17T05:32:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cb80b3920 0x7f6cb815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 0x7f6cb8073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb815f8d0 con 0x7f6cb806f110 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb815f750 con 0x7f6cb80b3920 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cbde00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb815fa50 con 0x7f6cb8155d70 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cb6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cb6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45806/0 (socket says 172.21.15.43:45806) 2024-09-17T05:32:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cb6a00640 1 -- 172.21.15.43:0/4051673533 learned_addr learned my addr 172.21.15.43:0/4051673533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cafe00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cb80b3920 0x7f6cb815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.283+0000 7f6cb7400640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 0x7f6cb8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 -- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 msgr2=0x7f6cb8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 0x7f6cb8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 -- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cb80b3920 msgr2=0x7f6cb815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cb80b3920 0x7f6cb815a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 -- 172.21.15.43:0/4051673533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8160a30 con 0x7f6cb806f110 2024-09-17T05:32:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cb6a00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800d700 tx=0x7f6ca800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:12.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8017070 con 0x7f6cb806f110 2024-09-17T05:32:12.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ca8009b10 con 0x7f6cb806f110 2024-09-17T05:32:12.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb8160d10 con 0x7f6cb806f110 2024-09-17T05:32:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8010460 con 0x7f6cb806f110 2024-09-17T05:32:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb8073270 con 0x7f6cb806f110 2024-09-17T05:32:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb8073090 con 0x7f6cb806f110 2024-09-17T05:32:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6ca801a070 con 0x7f6cb806f110 2024-09-17T05:32:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.287+0000 7f6caf400640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 0x7f6c94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.291+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7f6ca8094a50 con 0x7f6cb806f110 2024-09-17T05:32:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.291+0000 7f6cafe00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 0x7f6c94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.291+0000 7f6cafe00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 0x7f6c94075710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80c3870 tx=0x7f6ca000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:12.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.295+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6cb8073090 con 0x7f6cb806f110 2024-09-17T05:32:12.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:12.531+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f6cb815ff20 con 0x7f6cb806f110 2024-09-17T05:32:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6caf400640 1 -- 172.21.15.43:0/4051673533 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v213) ==== 101+0+0 (secure 0 0 0) 0x7f6ca805ed00 con 0x7f6cb806f110 2024-09-17T05:32:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 msgr2=0x7f6c94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 0x7f6c94075710 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80c3870 tx=0x7f6ca000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 msgr2=0x7f6cb8160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800d700 tx=0x7f6ca800dbc0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 shutdown_connections 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c94073200 0x7f6c94075710 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cb80b3920 0x7f6cb815a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6cb8155d70 0x7f6cb8073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 --2- 172.21.15.43:0/4051673533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb806f110 0x7f6cb8160130 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 >> 172.21.15.43:0/4051673533 conn(0x7f6cb80b23d0 msgr2=0x7f6cb80b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 shutdown_connections 2024-09-17T05:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.099+0000 7f6cbde00640 1 -- 172.21.15.43:0/4051673533 wait complete. 2024-09-17T05:32:13.127 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:13.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 Processor -- start 2024-09-17T05:32:13.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 -- start start 2024-09-17T05:32:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 0x5630d80362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d7fa8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630d7f0bf40 con 0x5630d7fa9280 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5630d7f18d00 con 0x5630d7fa12c0 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fde04546080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5630d7eada00 con 0x5630d7fa1800 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45818/0 (socket says 172.21.15.43:45818) 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 -- 172.21.15.43:0/3098520103 learned_addr learned my addr 172.21.15.43:0/3098520103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfcc00640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d7fa8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfd600640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 0x5630d80362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 -- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 msgr2=0x5630d7fa8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d7fa8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 -- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 msgr2=0x5630d80362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 0x5630d80362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 -- 172.21.15.43:0/3098520103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d7f5f4c0 con 0x5630d7fa9280 2024-09-17T05:32:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfd600640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 0x5630d80362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.187+0000 7fddfe000640 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fddf801c370 tx=0x7fddf8018910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fce454b7f49b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddf7400640 1 -- 172.21.15.43:0/3098520103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80011d0 con 0x5630d7fa9280 2024-09-17T05:32:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddf7400640 1 -- 172.21.15.43:0/3098520103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf8001880 con 0x5630d7fa9280 2024-09-17T05:32:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddf7400640 1 -- 172.21.15.43:0/3098520103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80090d0 con 0x5630d7fa9280 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- 172.21.15.43:0/3098520103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 msgr2=0x5630d8033df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fddf801c370 tx=0x7fddf8018910 comp rx=0 tx=0).stop 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- 172.21.15.43:0/3098520103 shutdown_connections 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa9280 0x5630d8033df0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d7fa8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- 172.21.15.43:0/3098520103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa12c0 0x5630d80362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- 172.21.15.43:0/3098520103 >> 172.21.15.43:0/3098520103 conn(0x5630d7f4bd80 msgr2=0x5630d7fa5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- 172.21.15.43:0/3098520103 shutdown_connections 2024-09-17T05:32:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- 172.21.15.43:0/3098520103 wait complete. 2024-09-17T05:32:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 Processor -- start 2024-09-17T05:32:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- start start 2024-09-17T05:32:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d80cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa9280 0x5630d8007e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630d7f60f20 con 0x5630d7fa12c0 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5630d7f5f760 con 0x5630d7fa9280 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fde04546080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5630d7f611c0 con 0x5630d7fa1800 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddfd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddfd600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45834/0 (socket says 172.21.15.43:45834) 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.191+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 learned_addr learned my addr 172.21.15.43:0/1595748544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d80cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfe000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa9280 0x5630d8007e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 msgr2=0x5630d80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5630d7fa1800 0x5630d80cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa9280 msgr2=0x5630d8007e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630d7fa9280 0x5630d8007e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d80ce530 con 0x5630d7fa12c0 2024-09-17T05:32:13.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fddf0012b20 tx=0x7fddf00103e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddf6000640 1 -- 172.21.15.43:0/1595748544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf000e480 con 0x5630d7fa12c0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddf6000640 1 -- 172.21.15.43:0/1595748544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf000e620 con 0x5630d7fa12c0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddf6000640 1 -- 172.21.15.43:0/1595748544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf000e9d0 con 0x5630d7fa12c0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630d80ce7d0 con 0x5630d7fa12c0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d80ced10 con 0x5630d7fa12c0 2024-09-17T05:32:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.195+0000 7fddf6000640 1 -- 172.21.15.43:0/1595748544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fddf000eb70 con 0x5630d7fa12c0 2024-09-17T05:32:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddf6000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 0x7fdde00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddf6000640 1 -- 172.21.15.43:0/1595748544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7fddf0096c60 con 0x5630d7fa12c0 2024-09-17T05:32:13.200 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 0x7fdde00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5630d80cccb0 0x5630d80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80cf990 con 0x5630d80cccb0 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddfe000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5630d80cccb0 0x5630d80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 0x7fdde00740f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c950 tx=0x7fddec004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.199+0000 7fddfe000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5630d80cccb0 0x5630d80cf450 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v110'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d8032310 con 0x5630d80cccb0 2024-09-17T05:32:13.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v110'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80326e0 con 0x5630d80cccb0 2024-09-17T05:32:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v110'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.207 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:13.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.203+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d8032a80 con 0x5630d80cccb0 2024-09-17T05:32:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d8032e80 con 0x5630d80cccb0 2024-09-17T05:32:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da2d0 con 0x5630d80cccb0 2024-09-17T05:32:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.207+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.211+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v110'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.211+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.211+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v110'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.211+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.215 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:13.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v110'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.215+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.219+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.224 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:13.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.223+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.223+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80cfd30 con 0x5630d80cccb0 2024-09-17T05:32:13.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.223+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.223+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.227+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v110'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v110'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.235 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:13.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v110'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.231+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.235+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.235+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.235+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.235+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v110'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v110'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.239+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v110'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.244 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80db680 con 0x5630d80cccb0 2024-09-17T05:32:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.243+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80da670 con 0x5630d80cccb0 2024-09-17T05:32:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5630d80dab80 0x5630d80dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80dd4e0 con 0x5630d80dab80 2024-09-17T05:32:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5630d80dab80 0x5630d80dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5630d80dab80 0x5630d80dcfa0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'462 uv445 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.247+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80dd880 con 0x5630d80dab80 2024-09-17T05:32:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.251+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'463 uv463 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.251+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ddc20 con 0x5630d80dab80 2024-09-17T05:32:13.254 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:13.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.251+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.251+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de4c0 con 0x5630d80dab80 2024-09-17T05:32:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de860 con 0x5630d80dab80 2024-09-17T05:32:13.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80dec00 con 0x5630d80dab80 2024-09-17T05:32:13.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v110'467 uv450 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.255+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80defa0 con 0x5630d80dab80 2024-09-17T05:32:13.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.259+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v110'468 uv468 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.259+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80df390 con 0x5630d80dab80 2024-09-17T05:32:13.262 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:13.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.259+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v110'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.259+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v110'472 uv455 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.263+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.267+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v110'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.267+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.270 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.267+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v110'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.267+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v110'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v110'477 uv461 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.271+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.275+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v110'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.275+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.275+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v110'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.278 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.275+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80de0f0 con 0x5630d80dab80 2024-09-17T05:32:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80dab80 2024-09-17T05:32:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5630d80dfd00 0x5630d80e2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e2660 con 0x5630d80dfd00 2024-09-17T05:32:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.279+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5630d80dfd00 0x5630d80e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.283+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5630d80dfd00 0x5630d80e2120 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.283+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v110'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.283+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80e2a80 con 0x5630d80dfd00 2024-09-17T05:32:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.283+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v110'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.283+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80df430 con 0x5630d80dfd00 2024-09-17T05:32:13.288 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v110'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e4890 con 0x5630d80dfd00 2024-09-17T05:32:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e4c30 con 0x5630d80dfd00 2024-09-17T05:32:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.287+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e3880 con 0x5630d80dfd00 2024-09-17T05:32:13.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e3c70 con 0x5630d80cccb0 2024-09-17T05:32:13.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'89 uv88 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80e3c70 con 0x5630d80cccb0 2024-09-17T05:32:13.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.291+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e3c70 con 0x5630d80cccb0 2024-09-17T05:32:13.296 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:13.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e3c70 con 0x5630d80cccb0 2024-09-17T05:32:13.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'92 uv92 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5630d80e4fd0 0x5630d80e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e7930 con 0x5630d80e4fd0 2024-09-17T05:32:13.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fddfe000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5630d80e4fd0 0x5630d80e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.295+0000 7fddfe000640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5630d80e4fd0 0x5630d80e73f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.299+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v110'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.299+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80e7930 con 0x5630d80e4fd0 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.299+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v110'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.299+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e7930 con 0x5630d80e4fd0 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.299+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v110'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.303+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e8940 con 0x5630d80e4fd0 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.303+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.303+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e8940 con 0x5630d80e4fd0 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.303+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.303+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80e7930 con 0x5630d80e4fd0 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5630d80e7fb0 0x5630d80ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ea950 con 0x5630d80e7fb0 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5630d80e7fb0 0x5630d80ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fddfd600640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5630d80e7fb0 0x5630d80ea3d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v110'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.307+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v110'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.312 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:13.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v110'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ebd30 con 0x5630d80e7fb0 2024-09-17T05:32:13.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.311+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ebd30 con 0x5630d80e7fb0 2024-09-17T05:32:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v110'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.315+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v110'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.321 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v110'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.319+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ead20 con 0x5630d80e7fb0 2024-09-17T05:32:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fddfd600640 1 -- 172.21.15.43:0/1595748544 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf0022070 con 0x5630d80e7fb0 2024-09-17T05:32:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5630d80eb6e0 0x5630d80edb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee040 con 0x5630d80eb6e0 2024-09-17T05:32:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5630d80eb6e0 0x5630d80edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fddfcc00640 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5630d80eb6e0 0x5630d80edb00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v110'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.323+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee040 con 0x5630d80eb6e0 2024-09-17T05:32:13.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v110'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee0c0 con 0x5630d80eb6e0 2024-09-17T05:32:13.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v110'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.329 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:13.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee4b0 con 0x5630d80eb6e0 2024-09-17T05:32:13.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.327+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee4b0 con 0x5630d80eb6e0 2024-09-17T05:32:13.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee230 con 0x5630d80cccb0 2024-09-17T05:32:13.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v110'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee230 con 0x5630d80cccb0 2024-09-17T05:32:13.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v110'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.331+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee230 con 0x5630d80cccb0 2024-09-17T05:32:13.336 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:13.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.335+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v110'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.335+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ef680 con 0x5630d80cccb0 2024-09-17T05:32:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.335+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.335+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ef350 con 0x5630d80cccb0 2024-09-17T05:32:13.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee340 con 0x5630d80cccb0 2024-09-17T05:32:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80cccb0 2024-09-17T05:32:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee2f0 con 0x5630d80dfd00 2024-09-17T05:32:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v110'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.339+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee2f0 con 0x5630d80dfd00 2024-09-17T05:32:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v110'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee2f0 con 0x5630d80dfd00 2024-09-17T05:32:13.345 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v110'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee700 con 0x5630d80dfd00 2024-09-17T05:32:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.343+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee700 con 0x5630d80dfd00 2024-09-17T05:32:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee700 con 0x5630d80dfd00 2024-09-17T05:32:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v110'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee700 con 0x5630d80dfd00 2024-09-17T05:32:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v110'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.347+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee390 con 0x5630d80dfd00 2024-09-17T05:32:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.351+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v110'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.353 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.351+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee390 con 0x5630d80dfd00 2024-09-17T05:32:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.351+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.351+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee790 con 0x5630d80dfd00 2024-09-17T05:32:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80dfd00 2024-09-17T05:32:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee730 con 0x5630d80e4fd0 2024-09-17T05:32:13.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v110'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee730 con 0x5630d80e4fd0 2024-09-17T05:32:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.355+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee730 con 0x5630d80e4fd0 2024-09-17T05:32:13.360 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:13.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.359+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v110'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.359+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee730 con 0x5630d80e4fd0 2024-09-17T05:32:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.359+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.359+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee710 con 0x5630d80e4fd0 2024-09-17T05:32:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v110'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee710 con 0x5630d80e4fd0 2024-09-17T05:32:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v110'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee710 con 0x5630d80e4fd0 2024-09-17T05:32:13.366 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:13.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v110'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee710 con 0x5630d80e4fd0 2024-09-17T05:32:13.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.363+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80e4fd0 2024-09-17T05:32:13.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.367+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.367+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80e4fd0 2024-09-17T05:32:13.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.367+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8001880 con 0x5630d80e4fd0 2024-09-17T05:32:13.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.367+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v110'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v110'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.374 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:13.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.371+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v110'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.375+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v110'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v110'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.381 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v110'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.379+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v110'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v110'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v110'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v110'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.387 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.383+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6f0 con 0x5630d80eb6e0 2024-09-17T05:32:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v110'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6a0 con 0x5630d80eb6e0 2024-09-17T05:32:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v110'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80ee6a0 con 0x5630d80eb6e0 2024-09-17T05:32:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v110'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.387+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5630d80eeab0 con 0x5630d80eb6e0 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fddfcc00640 1 -- 172.21.15.43:0/1595748544 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v110'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddec00ad80 con 0x5630d80eb6e0 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5630d80dab80 msgr2=0x5630d80dcfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5630d80dab80 0x5630d80dcfa0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5630d80e7fb0 msgr2=0x5630d80ea3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5630d80e7fb0 0x5630d80ea3d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5630d80eb6e0 msgr2=0x5630d80edb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5630d80eb6e0 0x5630d80edb00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5630d80dfd00 msgr2=0x5630d80e2120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5630d80dfd00 0x5630d80e2120 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5630d80cccb0 msgr2=0x5630d80cf450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5630d80cccb0 0x5630d80cf450 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5630d80e4fd0 msgr2=0x5630d80e73f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5630d80e4fd0 0x5630d80e73f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 reap_dead start 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 reap_dead start 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fddfe000640 1 -- 172.21.15.43:0/1595748544 reap_dead start 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 msgr2=0x7fdde00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 0x7fdde00740f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c950 tx=0x7fddec004890 comp rx=0 tx=0).stop 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 msgr2=0x5630d80c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fddf0012b20 tx=0x7fddf00103e0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 shutdown_connections 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdde0071be0 0x7fdde00740f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 --2- 172.21.15.43:0/1595748544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630d7fa12c0 0x5630d80c01e0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 >> 172.21.15.43:0/1595748544 conn(0x5630d7f4bd80 msgr2=0x5630d7fa79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 shutdown_connections 2024-09-17T05:32:13.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.391+0000 7fde04546080 1 -- 172.21.15.43:0/1595748544 wait complete. 2024-09-17T05:32:13.398 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-09-17T05:32:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 Processor -- start 2024-09-17T05:32:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 -- start start 2024-09-17T05:32:13.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 0x5639497f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 0x5639497f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5639497518e0 con 0x563949750e40 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5639495e8a00 con 0x5639496c9cb0 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.495+0000 7f833ddce080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563949751b30 con 0x56394969ad70 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 0x5639497f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8336600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 0x5639497f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:37192/0 (socket says 172.21.15.43:37192) 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337000640 1 -- 172.21.15.43:0/3742083346 learned_addr learned my addr 172.21.15.43:0/3742083346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8336600640 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 0x5639497f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:34282/0 (socket says 172.21.15.43:34282) 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 -- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 msgr2=0x56394976fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 -- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 msgr2=0x5639497f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 0x5639497f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 -- 172.21.15.43:0/3742083346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639495e7a20 con 0x563949750e40 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337000640 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8337a00640 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 0x5639497f6f30 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f83300ad000 tx=0x7f833001aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99ce1918ef1c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8335200640 1 -- 172.21.15.43:0/3742083346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300c7070 con 0x563949750e40 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8335200640 1 -- 172.21.15.43:0/3742083346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83300a7a80 con 0x563949750e40 2024-09-17T05:32:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f8335200640 1 -- 172.21.15.43:0/3742083346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300b77f0 con 0x563949750e40 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- 172.21.15.43:0/3742083346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 msgr2=0x5639497f6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 0x5639497f6f30 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f83300ad000 tx=0x7f833001aab0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- 172.21.15.43:0/3742083346 shutdown_connections 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639496c9cb0 0x5639497f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563949750e40 0x5639497f6f30 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 --2- 172.21.15.43:0/3742083346 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- 172.21.15.43:0/3742083346 >> 172.21.15.43:0/3742083346 conn(0x563949686d80 msgr2=0x5639496e2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- 172.21.15.43:0/3742083346 shutdown_connections 2024-09-17T05:32:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- 172.21.15.43:0/3742083346 wait complete. 2024-09-17T05:32:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 Processor -- start 2024-09-17T05:32:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.499+0000 7f833ddce080 1 -- start start 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 0x56394976cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563949750e40 0x5639496dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5639496dc080 con 0x5639496c9cb0 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56394969d5b0 con 0x563949750e40 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5639496892b0 con 0x56394969ad70 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 0x56394976cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8337a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563949750e40 0x5639496dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 0x56394976cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45856/0 (socket says 172.21.15.43:45856) 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 -- 172.21.15.43:0/1672771190 learned_addr learned my addr 172.21.15.43:0/1672771190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8337000640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 -- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 msgr2=0x56394976b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56394969ad70 0x56394976b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 -- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563949750e40 msgr2=0x5639496dfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563949750e40 0x5639496dfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 -- 172.21.15.43:0/1672771190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639495c1c10 con 0x5639496c9cb0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8337a00640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563949750e40 0x5639496dfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f8336600640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 0x56394976cee0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f83240087b0 tx=0x7f8324008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f832f400640 1 -- 172.21.15.43:0/1672771190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832401a070 con 0x5639496c9cb0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f832f400640 1 -- 172.21.15.43:0/1672771190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f832400abc0 con 0x5639496c9cb0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639496bc230 con 0x5639496c9cb0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f832f400640 1 -- 172.21.15.43:0/1672771190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324015420 con 0x5639496c9cb0 2024-09-17T05:32:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.503+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639496bc770 con 0x5639496c9cb0 2024-09-17T05:32:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f832f400640 1 -- 172.21.15.43:0/1672771190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f832401d070 con 0x5639496c9cb0 2024-09-17T05:32:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f832f400640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8318071be0 0x7f83180740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f832f400640 1 -- 172.21.15.43:0/1672771190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7f83240937f0 con 0x5639496c9cb0 2024-09-17T05:32:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f8337000640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8318071be0 0x7f83180740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f833ddce080 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56394976ead0 0x56394976eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x563949803db0 con 0x56394976ead0 2024-09-17T05:32:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f8337a00640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56394976ead0 0x56394976eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f8337000640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8318071be0 0x7f83180740f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f832800b780 tx=0x7f832800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.507+0000 7f8337a00640 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56394976ead0 0x56394976eee0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f8337a00640 1 -- 172.21.15.43:0/1672771190 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f83300cb040 con 0x56394976ead0 2024-09-17T05:32:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56394976ead0 msgr2=0x56394976eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56394976ead0 0x56394976eee0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8318071be0 msgr2=0x7f83180740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8318071be0 0x7f83180740f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f832800b780 tx=0x7f832800d040 comp rx=0 tx=0).stop 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 msgr2=0x56394976cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 --2- 172.21.15.43:0/1672771190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5639496c9cb0 0x56394976cee0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f83240087b0 tx=0x7f8324008bb0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f8337a00640 1 -- 172.21.15.43:0/1672771190 reap_dead start 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 shutdown_connections 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 >> 172.21.15.43:0/1672771190 conn(0x563949686d80 msgr2=0x56394976c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 shutdown_connections 2024-09-17T05:32:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.511+0000 7f833ddce080 1 -- 172.21.15.43:0/1672771190 wait complete. 2024-09-17T05:32:13.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- 172.21.15.43:0/1498653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0b3af0 msgr2=0x7fd61c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 --2- 172.21.15.43:0/1498653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0b3af0 0x7fd61c15c7e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fd604009900 tx=0x7fd60401c820 comp rx=0 tx=0).stop 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- 172.21.15.43:0/1498653917 shutdown_connections 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 --2- 172.21.15.43:0/1498653917 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 0x7fd61c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 --2- 172.21.15.43:0/1498653917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0b3af0 0x7fd61c15c7e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- 172.21.15.43:0/1498653917 >> 172.21.15.43:0/1498653917 conn(0x7fd61c0b23d0 msgr2=0x7fd61c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- 172.21.15.43:0/1498653917 shutdown_connections 2024-09-17T05:32:13.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- 172.21.15.43:0/1498653917 wait complete. 2024-09-17T05:32:13.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 Processor -- start 2024-09-17T05:32:13.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.719+0000 7fd621000640 1 -- start start 2024-09-17T05:32:13.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 0x7fd61c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61c155d70 0x7fd61c09c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c168bc0 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c168a40 con 0x7fd61c155d70 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd61c168d40 con 0x7fd61c06f110 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49956/0 (socket says 172.21.15.43:49956) 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 -- 172.21.15.43:0/553149626 learned_addr learned my addr 172.21.15.43:0/553149626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd611400640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61c155d70 0x7fd61c09c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd619e00640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 0x7fd61c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 -- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 msgr2=0x7fd61c0b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 0x7fd61c0b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 -- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61c155d70 msgr2=0x7fd61c09c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61c155d70 0x7fd61c09c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 -- 172.21.15.43:0/553149626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c0cd240 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd61a800640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d730 tx=0x7fd61400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd614009b40 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1faba0 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614004020 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c0c48c0 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.723+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c0c6040 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.727+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6140041c0 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.727+0000 7fd612a00640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 0x7fd5f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.727+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7515+0+0 (secure 0 0 0) 0x7fd6140948d0 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.727+0000 7fd619e00640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 0x7fd5f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.727+0000 7fd619e00640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 0x7fd5f8075710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0715c0 tx=0x7fd60c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.731+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd614020030 con 0x7fd61c0cc8f0 2024-09-17T05:32:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:13.943+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd61c0c9010 con 0x7fd61c0cc8f0 2024-09-17T05:32:14.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.111+0000 7fd612a00640 1 -- 172.21.15.43:0/553149626 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v214) ==== 162+0+0 (secure 0 0 0) 0x7fd61c0cd240 con 0x7fd61c0cc8f0 2024-09-17T05:32:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 msgr2=0x7fd5f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 0x7fd5f8075710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0715c0 tx=0x7fd60c009040 comp rx=0 tx=0).stop 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 msgr2=0x7fd61c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d730 tx=0x7fd61400dbf0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 shutdown_connections 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd5f8073200 0x7fd5f8075710 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61c155d70 0x7fd61c09c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c0cc8f0 0x7fd61c0ccd00 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 --2- 172.21.15.43:0/553149626 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd61c06f110 0x7fd61c0b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 >> 172.21.15.43:0/553149626 conn(0x7fd61c0b23d0 msgr2=0x7fd61c0ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 shutdown_connections 2024-09-17T05:32:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.115+0000 7fd621000640 1 -- 172.21.15.43:0/553149626 wait complete. 2024-09-17T05:32:14.125 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:14.123+0000 7f7cdba00640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-17T05:32:14.126 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:14.123+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:14.126 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:14.123+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:14.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 -- 172.21.15.43:0/4248745068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a015fe10 msgr2=0x7ff8a00b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 --2- 172.21.15.43:0/4248745068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a015fe10 0x7ff8a00b4b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff890009cb0 tx=0x7ff89001cc80 comp rx=0 tx=0).stop 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 -- 172.21.15.43:0/4248745068 shutdown_connections 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 --2- 172.21.15.43:0/4248745068 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a00737c0 0x7ff8a00700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 --2- 172.21.15.43:0/4248745068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a015fe10 0x7ff8a00b4b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 -- 172.21.15.43:0/4248745068 >> 172.21.15.43:0/4248745068 conn(0x7ff8a00b23d0 msgr2=0x7ff8a00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 -- 172.21.15.43:0/4248745068 shutdown_connections 2024-09-17T05:32:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.299+0000 7ff8a5400640 1 -- 172.21.15.43:0/4248745068 wait complete. 2024-09-17T05:32:14.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 Processor -- start 2024-09-17T05:32:14.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- start start 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 0x7ff8a01533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 0x7ff8a015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a016a000 con 0x7ff8a015fe10 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0169e80 con 0x7ff8a006fc90 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a016a180 con 0x7ff8a0099bf0 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff895e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff895e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49964/0 (socket says 172.21.15.43:49964) 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff895e00640 1 -- 172.21.15.43:0/2068959823 learned_addr learned my addr 172.21.15.43:0/2068959823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 0x7ff8a015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89e800640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 0x7ff8a01533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 -- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 msgr2=0x7ff8a01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 0x7ff8a01533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 -- 172.21.15.43:0/2068959823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 msgr2=0x7ff8a00995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 -- 172.21.15.43:0/2068959823 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a01f9c90 con 0x7ff8a0099bf0 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff895e00640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89e800640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 0x7ff8a01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:14.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff89f200640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 0x7ff8a015b6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e640 tx=0x7ff89800eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004220 con 0x7ff8a0099bf0 2024-09-17T05:32:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff898005020 con 0x7ff8a0099bf0 2024-09-17T05:32:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a0156550 con 0x7ff8a0099bf0 2024-09-17T05:32:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898005470 con 0x7ff8a0099bf0 2024-09-17T05:32:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.303+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a015b1a0 con 0x7ff8a0099bf0 2024-09-17T05:32:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a01f97c0 con 0x7ff8a0099bf0 2024-09-17T05:32:14.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff89800c810 con 0x7ff8a0099bf0 2024-09-17T05:32:14.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff897400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 0x7ff87c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7ff898094d60 con 0x7ff8a0099bf0 2024-09-17T05:32:14.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff89e800640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 0x7ff87c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.307+0000 7ff89e800640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 0x7ff87c075740 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8900af000 tx=0x7ff89001b4c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.311+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff89805f040 con 0x7ff8a0099bf0 2024-09-17T05:32:14.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.527+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8a00700a0 con 0x7ff8a0099bf0 2024-09-17T05:32:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.527+0000 7ff897400640 1 -- 172.21.15.43:0/2068959823 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) ==== 95+0+3557 (secure 0 0 0) 0x7ff898063a70 con 0x7ff8a0099bf0 2024-09-17T05:32:14.531 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:14.531 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":216,"btime":"2024-09-17T05:32:14:127047+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:14.126919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 216 2024-09-17T05:32:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 msgr2=0x7ff87c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 0x7ff87c075740 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8900af000 tx=0x7ff89001b4c0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 msgr2=0x7ff8a015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 0x7ff8a015b6c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e640 tx=0x7ff89800eb00 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 shutdown_connections 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff87c073230 0x7ff87c075740 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff8a0099bf0 0x7ff8a015b6c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a015fe10 0x7ff8a00995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 --2- 172.21.15.43:0/2068959823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8a006fc90 0x7ff8a01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 >> 172.21.15.43:0/2068959823 conn(0x7ff8a00b23d0 msgr2=0x7ff8a0158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 shutdown_connections 2024-09-17T05:32:14.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.531+0000 7ff8a5400640 1 -- 172.21.15.43:0/2068959823 wait complete. 2024-09-17T05:32:14.553 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:14.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 -- 172.21.15.43:0/229898305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894815fe10 msgr2=0x7f89480b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 --2- 172.21.15.43:0/229898305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894815fe10 0x7f89480b4b80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f893c009870 tx=0x7f893c01c700 comp rx=0 tx=0).stop 2024-09-17T05:32:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 -- 172.21.15.43:0/229898305 shutdown_connections 2024-09-17T05:32:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 --2- 172.21.15.43:0/229898305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894815fe10 0x7f89480b4b80 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 -- 172.21.15.43:0/229898305 >> 172.21.15.43:0/229898305 conn(0x7f89480b23d0 msgr2=0x7f89480d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 -- 172.21.15.43:0/229898305 shutdown_connections 2024-09-17T05:32:14.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.711+0000 7f894fc00640 1 -- 172.21.15.43:0/229898305 wait complete. 2024-09-17T05:32:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 Processor -- start 2024-09-17T05:32:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- start start 2024-09-17T05:32:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f894815fe10 0x7f8948158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 0x7f89481591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f894816aff0 con 0x7f894806fc90 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f894816ae70 con 0x7f89481583a0 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f894816b170 con 0x7f894815fe10 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49994/0 (socket says 172.21.15.43:49994) 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 -- 172.21.15.43:0/2192943990 learned_addr learned my addr 172.21.15.43:0/2192943990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f8947e00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f894815fe10 0x7f8948158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894dc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 0x7f89481591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 -- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f894815fe10 msgr2=0x7f8948158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f894815fe10 0x7f8948158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 -- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 msgr2=0x7f89481591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 0x7f89481591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 -- 172.21.15.43:0/2192943990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89481fbf50 con 0x7f894806fc90 2024-09-17T05:32:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894dc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 0x7f89481591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894d200640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f893c0b0040 tx=0x7f893c0a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0bd030 con 0x7f894806fc90 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f893c0041b0 con 0x7f894806fc90 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0043a0 con 0x7f894806fc90 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89481f9770 con 0x7f894806fc90 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.715+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481fc4f0 con 0x7f894806fc90 2024-09-17T05:32:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.719+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89480b4b80 con 0x7f894806fc90 2024-09-17T05:32:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.719+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f893c0c0070 con 0x7f894806fc90 2024-09-17T05:32:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.719+0000 7f8944c00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 0x7f8928075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.719+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f893c12f330 con 0x7f894806fc90 2024-09-17T05:32:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.719+0000 7f8947e00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 0x7f8928075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.723+0000 7f8947e00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 0x7f8928075710 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8934007fd0 tx=0x7f8934009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.723+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f893c0c7020 con 0x7f894806fc90 2024-09-17T05:32:14.839 INFO:tasks.ceph.mds.d.smithi062.stderr: -43> 2024-09-17T05:32:10.084+0000 7fc21e400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:32:14.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8948072fa0 con 0x7f894806fc90 2024-09-17T05:32:14.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f8944c00640 1 -- 172.21.15.43:0/2192943990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) ==== 95+0+3557 (secure 0 0 0) 0x7f893c016410 con 0x7f894806fc90 2024-09-17T05:32:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:14.938 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":216,"btime":"2024-09-17T05:32:14:127047+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:14.126919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:14.938 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 216 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 msgr2=0x7f8928075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 0x7f8928075710 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8934007fd0 tx=0x7f8934009040 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 msgr2=0x7f8948157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.935+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f893c0b0040 tx=0x7f893c0a66f0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 shutdown_connections 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8928073200 0x7f8928075710 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481583a0 0x7f89481591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f894815fe10 0x7f8948158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 --2- 172.21.15.43:0/2192943990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894806fc90 0x7f8948157d90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 >> 172.21.15.43:0/2192943990 conn(0x7f89480b23d0 msgr2=0x7f89480ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 shutdown_connections 2024-09-17T05:32:14.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:14.939+0000 7f894fc00640 1 -- 172.21.15.43:0/2192943990 wait complete. 2024-09-17T05:32:14.958 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:14.960 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:14.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 -- 172.21.15.43:0/2551085148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 msgr2=0x7f5ebc0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2551085148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc0d0840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005fd0 tx=0x7f5eb001c820 comp rx=0 tx=0).stop 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 -- 172.21.15.43:0/2551085148 shutdown_connections 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2551085148 >> v2:172.21.15.62:3300/0 conn(0x7f5ebc0d0d80 0x7f5ebc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2551085148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc0d0840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 -- 172.21.15.43:0/2551085148 >> 172.21.15.43:0/2551085148 conn(0x7f5ebc0cee80 msgr2=0x7f5ebc0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 -- 172.21.15.43:0/2551085148 shutdown_connections 2024-09-17T05:32:15.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.119+0000 7f5ec4600640 1 -- 172.21.15.43:0/2551085148 wait complete. 2024-09-17T05:32:15.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 Processor -- start 2024-09-17T05:32:15.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- start start 2024-09-17T05:32:15.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc09fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ebc0d0d80 0x7f5ebc153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc169e10 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc169c90 con 0x7f5ebc0d0d80 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc169f90 con 0x7f5ebc0b7ce0 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50010/0 (socket says 172.21.15.43:50010) 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 -- 172.21.15.43:0/2531666443 learned_addr learned my addr 172.21.15.43:0/2531666443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1c00640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc09fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec2600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ebc0d0d80 0x7f5ebc153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 -- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 msgr2=0x7f5ebc09fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc09fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 -- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ebc0d0d80 msgr2=0x7f5ebc153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ebc0d0d80 0x7f5ebc153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 -- 172.21.15.43:0/2531666443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc15b770 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec1200640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e7f0 tx=0x7f5ea800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800ccc0 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea800ce80 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8005280 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc15ba50 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.123+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc06e1c0 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.127+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0d0840 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.127+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ea8005420 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.127+0000 7f5eb9600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 0x7f5e9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.127+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f5ea80952b0 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.127+0000 7f5ec1c00640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 0x7f5e9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.131+0000 7f5ec1c00640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 0x7f5e9c075710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ae000 tx=0x7f5eb001b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.135+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ea805f520 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ebc0a0340 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5eb9600640 1 -- 172.21.15.43:0/2531666443 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) ==== 95+0+4383 (secure 0 0 0) 0x7f5ea8063f50 con 0x7f5ebc0cd3b0 2024-09-17T05:32:15.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:15.345 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":217,"btime":"2024-09-17T05:32:15:128769+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2407777034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2407777034},{"type":"v1","addr":"172.21.15.62:6835","nonce":2407777034}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":217}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:14.126919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 217 2024-09-17T05:32:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 msgr2=0x7f5e9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 0x7f5e9c075710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ae000 tx=0x7f5eb001b060 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 msgr2=0x7f5ebc1fc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800e7f0 tx=0x7f5ea800ecb0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 shutdown_connections 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e9c073200 0x7f5e9c075710 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ebc0d0d80 0x7f5ebc153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ebc0cd3b0 0x7f5ebc1fc2d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 --2- 172.21.15.43:0/2531666443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ebc0b7ce0 0x7f5ebc09fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.343+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 >> 172.21.15.43:0/2531666443 conn(0x7f5ebc0cee80 msgr2=0x7f5ebc0af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.347+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 shutdown_connections 2024-09-17T05:32:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.347+0000 7f5ec4600640 1 -- 172.21.15.43:0/2531666443 wait complete. 2024-09-17T05:32:15.368 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-09-17T05:32:15.368 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-09-17T05:32:15.368 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:15.369 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:15.378 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:15.378 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:15.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:15.378 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:15.427 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:15.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 -- 172.21.15.43:0/2283996282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 msgr2=0x7fa8440b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 --2- 172.21.15.43:0/2283996282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8440b33e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa830009900 tx=0x7fa83001c820 comp rx=0 tx=0).stop 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 -- 172.21.15.43:0/2283996282 shutdown_connections 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 --2- 172.21.15.43:0/2283996282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8440b33e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 --2- 172.21.15.43:0/2283996282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa844155d70 0x7fa84415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 -- 172.21.15.43:0/2283996282 >> 172.21.15.43:0/2283996282 conn(0x7fa8440b23d0 msgr2=0x7fa8440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 -- 172.21.15.43:0/2283996282 shutdown_connections 2024-09-17T05:32:15.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.631+0000 7fa849200640 1 -- 172.21.15.43:0/2283996282 wait complete. 2024-09-17T05:32:15.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 Processor -- start 2024-09-17T05:32:15.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 -- start start 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8440b3920 0x7fa8440ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844155d70 0x7fa84409d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa844168a60 con 0x7fa84406f110 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441688e0 con 0x7fa844155d70 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa849200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa844168be0 con 0x7fa8440b3920 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50032/0 (socket says 172.21.15.43:50032) 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 -- 172.21.15.43:0/2713626302 learned_addr learned my addr 172.21.15.43:0/2713626302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841400640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8440b3920 0x7fa8440ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa842800640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844155d70 0x7fa84409d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 -- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8440b3920 msgr2=0x7fa8440ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8440b3920 0x7fa8440ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 -- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844155d70 msgr2=0x7fa84409d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844155d70 0x7fa84409d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 -- 172.21.15.43:0/2713626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84409dd50 con 0x7fa84406f110 2024-09-17T05:32:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa841e00640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa83400e790 tx=0x7fa83400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83400eea0 con 0x7fa84406f110 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.635+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa83400cce0 con 0x7fa84406f110 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8440c9970 con 0x7fa84406f110 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834004290 con 0x7fa84406f110 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8440c84a0 con 0x7fa84406f110 2024-09-17T05:32:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8440c9e90 con 0x7fa84406f110 2024-09-17T05:32:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa834003600 con 0x7fa84406f110 2024-09-17T05:32:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa83a000640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 0x7fa824075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7fa8340952e0 con 0x7fa84406f110 2024-09-17T05:32:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.639+0000 7fa841400640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 0x7fa824075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.643+0000 7fa841400640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 0x7fa824075740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8300af000 tx=0x7fa83001b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.643+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8440c9e90 con 0x7fa84406f110 2024-09-17T05:32:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:15.855+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa8440b33e0 con 0x7fa84406f110 2024-09-17T05:32:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.135+0000 7fa83a000640 1 -- 172.21.15.43:0/2713626302 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v218) ==== 161+0+0 (secure 0 0 0) 0x7fa83405fb80 con 0x7fa84406f110 2024-09-17T05:32:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.135+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 msgr2=0x7fa824075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.135+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 0x7fa824075740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8300af000 tx=0x7fa83001b060 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 msgr2=0x7fa8441fa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa83400e790 tx=0x7fa83400ec50 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 shutdown_connections 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa824073230 0x7fa824075740 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844155d70 0x7fa84409d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa8440b3920 0x7fa8440ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 --2- 172.21.15.43:0/2713626302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84406f110 0x7fa8441fa930 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 >> 172.21.15.43:0/2713626302 conn(0x7fa8440b23d0 msgr2=0x7fa8440ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 shutdown_connections 2024-09-17T05:32:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.139+0000 7fa849200640 1 -- 172.21.15.43:0/2713626302 wait complete. 2024-09-17T05:32:16.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- 172.21.15.43:0/1652430622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376015f470 msgr2=0x7f37600d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 --2- 172.21.15.43:0/1652430622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376015f470 0x7f37600d0390 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3760071450 tx=0x7f375401c7a0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- 172.21.15.43:0/1652430622 shutdown_connections 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 --2- 172.21.15.43:0/1652430622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b3b10 0x7f37600b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 --2- 172.21.15.43:0/1652430622 >> v2:172.21.15.62:3300/0 conn(0x7f37600b5610 0x7f37600b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 --2- 172.21.15.43:0/1652430622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f376015f470 0x7f37600d0390 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- 172.21.15.43:0/1652430622 >> 172.21.15.43:0/1652430622 conn(0x7f37600b2810 msgr2=0x7f37600b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- 172.21.15.43:0/1652430622 shutdown_connections 2024-09-17T05:32:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- 172.21.15.43:0/1652430622 wait complete. 2024-09-17T05:32:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 Processor -- start 2024-09-17T05:32:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.319+0000 7f3768800640 1 -- start start 2024-09-17T05:32:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 0x7f37601649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 0x7f3760164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3760074530 con 0x7f37600b3b10 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37600743b0 con 0x7f376015f470 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f37600746b0 con 0x7f37600b5610 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50058/0 (socket says 172.21.15.43:50058) 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 -- 172.21.15.43:0/2211990881 learned_addr learned my addr 172.21.15.43:0/2211990881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765400640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 0x7f37601649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 -- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 msgr2=0x7f37601649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3766800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 0x7f3760164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 0x7f37601649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 -- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 msgr2=0x7f3760164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 0x7f3760164ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 -- 172.21.15.43:0/2211990881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37601fd4b0 con 0x7f37600b3b10 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765400640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 0x7f37601649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3766800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 0x7f3760164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3765e00640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f375401cc80 tx=0x7f37540a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540ac070 con 0x7f37600b3b10 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3754004120 con 0x7f37600b3b10 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754005070 con 0x7f37600b3b10 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37600afd10 con 0x7f37600b3b10 2024-09-17T05:32:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37600b02d0 con 0x7f37600b3b10 2024-09-17T05:32:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.323+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37600b5a20 con 0x7f37600b3b10 2024-09-17T05:32:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.327+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37540af070 con 0x7f37600b3b10 2024-09-17T05:32:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.327+0000 7f375a000640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 0x7f3744075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.327+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f375412efe0 con 0x7f37600b3b10 2024-09-17T05:32:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.327+0000 7f3765400640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 0x7f3744075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.327+0000 7f3765400640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 0x7f3744075710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f37601687d0 tx=0x7f3750005d20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.331+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f37540b6070 con 0x7f37600b3b10 2024-09-17T05:32:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:16.567+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3760156200 con 0x7f37600b3b10 2024-09-17T05:32:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.143+0000 7f375a000640 1 -- 172.21.15.43:0/2211990881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v219) ==== 101+0+0 (secure 0 0 0) 0x7f37540164b0 con 0x7f37600b3b10 2024-09-17T05:32:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 msgr2=0x7f3744075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 0x7f3744075710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f37601687d0 tx=0x7f3750005d20 comp rx=0 tx=0).stop 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 msgr2=0x7f37601523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f375401cc80 tx=0x7f37540a6ba0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 shutdown_connections 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3744073200 0x7f3744075710 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f376015f470 0x7f3760164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f37600b5610 0x7f37601649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 --2- 172.21.15.43:0/2211990881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37600b3b10 0x7f37601523e0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 >> 172.21.15.43:0/2211990881 conn(0x7f37600b2810 msgr2=0x7f3760159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 shutdown_connections 2024-09-17T05:32:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.147+0000 7f3768800640 1 -- 172.21.15.43:0/2211990881 wait complete. 2024-09-17T05:32:17.171 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 Processor -- start 2024-09-17T05:32:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 -- start start 2024-09-17T05:32:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 0x5573db122080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573db120200 0x5573db1225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573daff4f40 con 0x7f3dd4002510 2024-09-17T05:32:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573db001d00 con 0x5573db0907d0 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573daf96a00 con 0x5573db120200 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50078/0 (socket says 172.21.15.43:50078) 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 -- 172.21.15.43:0/2708811284 learned_addr learned my addr 172.21.15.43:0/2708811284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dede00640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 0x5573db122080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3def200640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573db120200 0x5573db1225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 -- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573db120200 msgr2=0x5573db1225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573db120200 0x5573db1225c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 -- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 msgr2=0x5573db122080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 0x5573db122080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 -- 172.21.15.43:0/2708811284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573db092150 con 0x7f3dd4002510 2024-09-17T05:32:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dede00640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 0x5573db122080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3dee800640 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f3de0097290 tx=0x7f3dd4004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43e4b31ba7fbca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3deca00640 1 -- 172.21.15.43:0/2708811284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00015c0 con 0x7f3dd4002510 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3deca00640 1 -- 172.21.15.43:0/2708811284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3de0001760 con 0x7f3dd4002510 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3deca00640 1 -- 172.21.15.43:0/2708811284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00a47d0 con 0x7f3dd4002510 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 -- 172.21.15.43:0/2708811284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 msgr2=0x5573db090290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.231+0000 7f3df55c9080 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f3de0097290 tx=0x7f3dd4004130 comp rx=0 tx=0).stop 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- 172.21.15.43:0/2708811284 shutdown_connections 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db0907d0 0x5573db122080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573db120200 0x5573db1225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- 172.21.15.43:0/2708811284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dd4002510 0x5573db090290 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- 172.21.15.43:0/2708811284 >> 172.21.15.43:0/2708811284 conn(0x5573db034d80 msgr2=0x5573db11c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- 172.21.15.43:0/2708811284 shutdown_connections 2024-09-17T05:32:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- 172.21.15.43:0/2708811284 wait complete. 2024-09-17T05:32:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 Processor -- start 2024-09-17T05:32:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- start start 2024-09-17T05:32:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db120200 0x5573db0ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd4002510 0x5573db1b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573db11bd80 con 0x5573db0907d0 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573db120850 con 0x5573db120200 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3df55c9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573db091eb0 con 0x7f3dd4002510 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50084/0 (socket says 172.21.15.43:50084) 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 learned_addr learned my addr 172.21.15.43:0/4128916348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db120200 0x5573db0ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3def200640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd4002510 0x5573db1b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd4002510 msgr2=0x5573db1b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3dd4002510 0x5573db1b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db120200 msgr2=0x5573db0ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573db120200 0x5573db0ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573db1193a0 con 0x5573db0907d0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.235+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3de0019ff0 tx=0x7f3de00a4570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 -- 172.21.15.43:0/4128916348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00b1070 con 0x5573db0907d0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 -- 172.21.15.43:0/4128916348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3de00a4890 con 0x5573db0907d0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573db1b76b0 con 0x5573db0907d0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 -- 172.21.15.43:0/4128916348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00a4ce0 con 0x5573db0907d0 2024-09-17T05:32:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573db1b7bf0 con 0x5573db0907d0 2024-09-17T05:32:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 -- 172.21.15.43:0/4128916348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3de00c3060 con 0x5573db0907d0 2024-09-17T05:32:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 0x7f3dd00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3de7400640 1 -- 172.21.15.43:0/4128916348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f3de012d850 con 0x5573db0907d0 2024-09-17T05:32:17.243 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5573db1b8330 0x5573db1b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1b8c80 con 0x5573db1b8330 2024-09-17T05:32:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.239+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 0x7f3dd00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.243+0000 7f3def200640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5573db1b8330 0x5573db1b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.243+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 0x7f3dd00740f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b720 tx=0x7f3ddc00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.243+0000 7f3def200640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5573db1b8330 0x5573db1b8740 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.243+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v111'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.243+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1b9050 con 0x5573db1b8330 2024-09-17T05:32:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.247+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v111'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.247+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1b9420 con 0x5573db1b8330 2024-09-17T05:32:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.247+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v111'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.250 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:17.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.247+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5e20 con 0x5573db1b8330 2024-09-17T05:32:17.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c61c0 con 0x5573db1b8330 2024-09-17T05:32:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c4e10 con 0x5573db1b8330 2024-09-17T05:32:17.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.251+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c51b0 con 0x5573db1b8330 2024-09-17T05:32:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.255+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v111'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.255+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.255+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v111'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.255+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.260 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v111'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.259+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.263+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.263+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.263+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.263+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.269 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:17.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.267+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.271+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.271+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5550 con 0x5573db1b8330 2024-09-17T05:32:17.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.271+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.271+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c5950 con 0x5573db1b8330 2024-09-17T05:32:17.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.275+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.275+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.275+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v111'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.275+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v111'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:17.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v111'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.279+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.283+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.283+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.283+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.283+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v111'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v111'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.291 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:17.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v111'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.287+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.291+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.291+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.291+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.291+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c57d0 con 0x5573db1b8330 2024-09-17T05:32:17.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5573db1c6560 0x5573db1c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c8ec0 con 0x5573db1c6560 2024-09-17T05:32:17.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5573db1c6560 0x5573db1c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5573db1c6560 0x5573db1c8980 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'483 uv466 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.295+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c9260 con 0x5573db1c6560 2024-09-17T05:32:17.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.299+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'484 uv484 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.299+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c9600 con 0x5573db1c6560 2024-09-17T05:32:17.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.299+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.302 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:17.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.299+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1c9e50 con 0x5573db1c6560 2024-09-17T05:32:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca1f0 con 0x5573db1c6560 2024-09-17T05:32:17.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v111'488 uv471 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.303+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v111'489 uv489 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v111'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.309 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:17.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.307+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v111'493 uv476 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v111'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.311+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.315+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v111'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.317 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:17.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.315+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce760 con 0x5573db1c6560 2024-09-17T05:32:17.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.315+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v111'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.315+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce760 con 0x5573db1c6560 2024-09-17T05:32:17.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v111'498 uv482 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v111'499 uv499 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.319+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v111'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.324 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cb5a0 con 0x5573db1c6560 2024-09-17T05:32:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cb5a0 con 0x5573db1c6560 2024-09-17T05:32:17.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.323+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ca590 con 0x5573db1c6560 2024-09-17T05:32:17.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1c6560 2024-09-17T05:32:17.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5573db1cac10 0x5573db1cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cd640 con 0x5573db1cac10 2024-09-17T05:32:17.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5573db1cac10 0x5573db1cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5573db1cac10 0x5573db1cd030 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v111'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.327+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1c9ab0 con 0x5573db1cac10 2024-09-17T05:32:17.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v111'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cdb00 con 0x5573db1cac10 2024-09-17T05:32:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v111'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.333 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cf930 con 0x5573db1cac10 2024-09-17T05:32:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.331+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1cfcd0 con 0x5573db1cac10 2024-09-17T05:32:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce920 con 0x5573db1cac10 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ced10 con 0x5573db1b8330 2024-09-17T05:32:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'93 uv92 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.335+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ced10 con 0x5573db1b8330 2024-09-17T05:32:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'94 uv94 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ced10 con 0x5573db1b8330 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'95 uv95 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ced10 con 0x5573db1b8330 2024-09-17T05:32:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'96 uv96 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 0x5573db1d2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d29d0 con 0x5573db1d0070 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.339+0000 7f3def200640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 0x5573db1d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.343+0000 7f3def200640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 0x5573db1d2490 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.343+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v111'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.343+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d29d0 con 0x5573db1d0070 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.343+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v111'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.343+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d29d0 con 0x5573db1d0070 2024-09-17T05:32:17.351 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v111'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d39e0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d39e0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.347+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d29d0 con 0x5573db1d0070 2024-09-17T05:32:17.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5573db1d2ee0 0x5573db1d5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5a00 con 0x5573db1d2ee0 2024-09-17T05:32:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5573db1d2ee0 0x5573db1d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3dee800640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5573db1d2ee0 0x5573db1d5300 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v111'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d5ac0 con 0x5573db1d2ee0 2024-09-17T05:32:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v111'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.351+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v111'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.356 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d6ef0 con 0x5573db1d2ee0 2024-09-17T05:32:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d6ef0 con 0x5573db1d2ee0 2024-09-17T05:32:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.355+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v111'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v111'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.359+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v111'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.364 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d5ee0 con 0x5573db1d2ee0 2024-09-17T05:32:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3dee800640 1 -- 172.21.15.43:0/4128916348 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de00f3cf0 con 0x5573db1d2ee0 2024-09-17T05:32:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5573db1d63f0 0x5573db1d8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d8f80 con 0x5573db1d63f0 2024-09-17T05:32:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.363+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5573db1d63f0 0x5573db1d8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.367+0000 7f3dede00640 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5573db1d63f0 0x5573db1d8830 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.367+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v111'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.367+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9350 con 0x5573db1d63f0 2024-09-17T05:32:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.367+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v111'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.367+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1d63f0 2024-09-17T05:32:17.371 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v111'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1d63f0 2024-09-17T05:32:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1d63f0 2024-09-17T05:32:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.371+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1b8330 2024-09-17T05:32:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v111'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1b8330 2024-09-17T05:32:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v111'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1b8330 2024-09-17T05:32:17.378 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v111'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.375+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1da750 con 0x5573db1b8330 2024-09-17T05:32:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1da750 con 0x5573db1b8330 2024-09-17T05:32:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1b8330 2024-09-17T05:32:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1b8330 2024-09-17T05:32:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.379+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.383+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v111'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.383+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.383+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v111'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.383+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.387 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.383+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v111'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.387+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9780 con 0x5573db1cac10 2024-09-17T05:32:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.387+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.387+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.387+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.387+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v111'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v111'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.395 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v111'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.391+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9740 con 0x5573db1cac10 2024-09-17T05:32:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.395+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.395+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1dc990 con 0x5573db1cac10 2024-09-17T05:32:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.395+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1cac10 2024-09-17T05:32:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.395+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v111'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.403 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v111'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.399+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.403+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.403+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.403+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v111'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.403+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v111'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.409 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:17.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v111'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1dae80 con 0x5573db1d0070 2024-09-17T05:32:17.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.407+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1dae80 con 0x5573db1d0070 2024-09-17T05:32:17.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d0070 2024-09-17T05:32:17.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3de80097d0 con 0x5573db1d0070 2024-09-17T05:32:17.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d63f0 2024-09-17T05:32:17.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v111'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.411+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d63f0 2024-09-17T05:32:17.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v111'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.417 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:17.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v111'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.415+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.419+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.419+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.419+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v111'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.419+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v111'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.424 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:17.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v111'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce560 con 0x5573db1d63f0 2024-09-17T05:32:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v111'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.423+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v111'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v111'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.427+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.431 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v111'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v111'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v111'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.431+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1ce500 con 0x5573db1d63f0 2024-09-17T05:32:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v111'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5573db1d9e70 con 0x5573db1d63f0 2024-09-17T05:32:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3dede00640 1 -- 172.21.15.43:0/4128916348 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v111'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3ddc00ad60 con 0x5573db1d63f0 2024-09-17T05:32:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5573db1c6560 msgr2=0x5573db1c8980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5573db1c6560 0x5573db1c8980 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5573db1d2ee0 msgr2=0x5573db1d5300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5573db1d2ee0 0x5573db1d5300 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5573db1d63f0 msgr2=0x5573db1d8830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5573db1d63f0 0x5573db1d8830 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5573db1cac10 msgr2=0x5573db1cd030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5573db1cac10 0x5573db1cd030 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 reap_dead start 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5573db1b8330 msgr2=0x5573db1b8740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5573db1b8330 0x5573db1b8740 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3def200640 1 -- 172.21.15.43:0/4128916348 reap_dead start 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 msgr2=0x5573db1d2490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 0x5573db1d2490 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 msgr2=0x7f3dd00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 0x7f3dd00740f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b720 tx=0x7f3ddc00d040 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 msgr2=0x5573db118e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3de0019ff0 tx=0x7f3de00a4570 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 shutdown_connections 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5573db1d0070 0x5573db1d2490 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3dd0071be0 0x7f3dd00740f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 --2- 172.21.15.43:0/4128916348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573db0907d0 0x5573db118e60 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 >> 172.21.15.43:0/4128916348 conn(0x5573db034d80 msgr2=0x5573db121ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 shutdown_connections 2024-09-17T05:32:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.435+0000 7f3df55c9080 1 -- 172.21.15.43:0/4128916348 wait complete. 2024-09-17T05:32:17.443 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-09-17T05:32:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 Processor -- start 2024-09-17T05:32:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 -- start start 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 0x56355f94c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f9c1e40 0x56355f94cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56355f859a00 con 0x56355f90bd70 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56355f8b7f40 con 0x56355f93acb0 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f44a2869080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56355f9c2890 con 0x56355f9c1e40 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50102/0 (socket says 172.21.15.43:50102) 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 -- 172.21.15.43:0/798984155 learned_addr learned my addr 172.21.15.43:0/798984155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f4493e00640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 0x56355f94c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449c400640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f9c1e40 0x56355f94cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 -- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f9c1e40 msgr2=0x56355f94cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f9c1e40 0x56355f94cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 -- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 msgr2=0x56355f94c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 0x56355f94c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.543+0000 7f449ba00640 1 -- 172.21.15.43:0/798984155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355f832c10 con 0x56355f90bd70 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f4493e00640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 0x56355f94c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f449ba00640 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f448c01a6c0 tx=0x7f448c01a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133512cdb5f72ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f449a600640 1 -- 172.21.15.43:0/798984155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c0a7800 con 0x56355f90bd70 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f449a600640 1 -- 172.21.15.43:0/798984155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f448c0a7b50 con 0x56355f90bd70 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f449a600640 1 -- 172.21.15.43:0/798984155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c0b0470 con 0x56355f90bd70 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- 172.21.15.43:0/798984155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 msgr2=0x56355f950ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f448c01a6c0 tx=0x7f448c01a850 comp rx=0 tx=0).stop 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- 172.21.15.43:0/798984155 shutdown_connections 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f93acb0 0x56355f94c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f9c1e40 0x56355f94cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- 172.21.15.43:0/798984155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f90bd70 0x56355f950ab0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- 172.21.15.43:0/798984155 >> 172.21.15.43:0/798984155 conn(0x56355f8f7d80 msgr2=0x56355f909890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- 172.21.15.43:0/798984155 shutdown_connections 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- 172.21.15.43:0/798984155 wait complete. 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 Processor -- start 2024-09-17T05:32:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- start start 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 0x56355f9dca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f9c1e40 0x56355f9dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f9c18e0 0x56355f9e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56355f951740 con 0x56355f9c18e0 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56355f93a4d0 con 0x56355f9c1e40 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.547+0000 7f44a2869080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56355f9de410 con 0x56355f93acb0 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 0x56355f9dca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f9c18e0 0x56355f9e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4493e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f9c1e40 0x56355f9dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 0x56355f9dca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:54812/0 (socket says 172.21.15.43:54812) 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449ba00640 1 -- 172.21.15.43:0/620680463 learned_addr learned my addr 172.21.15.43:0/620680463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 -- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 msgr2=0x56355f9dca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 0x56355f9dca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 -- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f9c1e40 msgr2=0x56355f9dfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f9c1e40 0x56355f9dfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 -- 172.21.15.43:0/620680463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355fa6be10 con 0x56355f9c18e0 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449ba00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56355f93acb0 0x56355f9dca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4493e00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56355f9c1e40 0x56355f9dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f449c400640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f9c18e0 0x56355f9e0410 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f44940106f0 tx=0x7f449400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4499200640 1 -- 172.21.15.43:0/620680463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494004220 con 0x56355f9c18e0 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4499200640 1 -- 172.21.15.43:0/620680463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f449400ed00 con 0x56355f9c18e0 2024-09-17T05:32:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4499200640 1 -- 172.21.15.43:0/620680463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494003680 con 0x56355f9c18e0 2024-09-17T05:32:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56355fa6c0b0 con 0x56355f9c18e0 2024-09-17T05:32:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355fa6c5f0 con 0x56355f9c18e0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.551+0000 7f4499200640 1 -- 172.21.15.43:0/620680463 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4494003820 con 0x56355f9c18e0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f4499200640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4478071be0 0x7f44780740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f4499200640 1 -- 172.21.15.43:0/620680463 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f44940945a0 con 0x56355f9c18e0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355fa67f30 0x56355fa68340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x56355fa6b9b0 con 0x56355fa67f30 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f449ba00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4478071be0 0x7f44780740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f4493e00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355fa67f30 0x56355fa68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f449ba00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4478071be0 0x7f44780740f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f448c01a6c0 tx=0x7f448c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f4493e00640 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355fa67f30 0x56355fa68340 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f4493e00640 1 -- 172.21.15.43:0/620680463 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v111'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f448400e970 con 0x56355fa67f30 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355fa67f30 msgr2=0x56355fa68340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56355fa67f30 0x56355fa68340 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4478071be0 msgr2=0x7f44780740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4478071be0 0x7f44780740f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f448c01a6c0 tx=0x7f448c0b2040 comp rx=0 tx=0).stop 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f9c18e0 msgr2=0x56355f9e0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f44a2869080 1 --2- 172.21.15.43:0/620680463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56355f9c18e0 0x56355f9e0410 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f44940106f0 tx=0x7f449400df90 comp rx=0 tx=0).stop 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.555+0000 7f449c400640 1 -- 172.21.15.43:0/620680463 reap_dead start 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.559+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 shutdown_connections 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.559+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 >> 172.21.15.43:0/620680463 conn(0x56355f8f7d80 msgr2=0x56355f9e0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.559+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 shutdown_connections 2024-09-17T05:32:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.559+0000 7f44a2869080 1 -- 172.21.15.43:0/620680463 wait complete. 2024-09-17T05:32:17.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:17.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- 172.21.15.43:0/3334327058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e00b3af0 msgr2=0x7f19e015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 --2- 172.21.15.43:0/3334327058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e00b3af0 0x7f19e015c7e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f19e00d0c40 tx=0x7f19c801c740 comp rx=0 tx=0).stop 2024-09-17T05:32:17.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- 172.21.15.43:0/3334327058 shutdown_connections 2024-09-17T05:32:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 --2- 172.21.15.43:0/3334327058 >> v2:172.21.15.62:3300/0 conn(0x7f19e006f110 0x7f19e00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 --2- 172.21.15.43:0/3334327058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e00b3af0 0x7f19e015c7e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- 172.21.15.43:0/3334327058 >> 172.21.15.43:0/3334327058 conn(0x7f19e00b23d0 msgr2=0x7f19e00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- 172.21.15.43:0/3334327058 shutdown_connections 2024-09-17T05:32:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- 172.21.15.43:0/3334327058 wait complete. 2024-09-17T05:32:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 Processor -- start 2024-09-17T05:32:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.767+0000 7f19e4c00640 1 -- start start 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19e006f110 0x7f19e00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e0155d70 0x7f19e0160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0168b90 con 0x7f19e0155340 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0168a10 con 0x7f19e0155d70 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f19e0168d10 con 0x7f19e006f110 2024-09-17T05:32:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50136/0 (socket says 172.21.15.43:50136) 2024-09-17T05:32:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 -- 172.21.15.43:0/1400263923 learned_addr learned my addr 172.21.15.43:0/1400263923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19d7e00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e0155d70 0x7f19e0160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19dde00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19e006f110 0x7f19e00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 -- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19e006f110 msgr2=0x7f19e00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19e006f110 0x7f19e00cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 -- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e0155d70 msgr2=0x7f19e0160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e0155d70 0x7f19e0160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 -- 172.21.15.43:0/1400263923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e00cebb0 con 0x7f19e0155340 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19de800640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e940 tx=0x7f19d800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8004d30 con 0x7f19e0155340 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d800cac0 con 0x7f19e0155340 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8005020 con 0x7f19e0155340 2024-09-17T05:32:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e00cee90 con 0x7f19e0155340 2024-09-17T05:32:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e0155750 con 0x7f19e0155340 2024-09-17T05:32:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.771+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00ad0b0 con 0x7f19e0155340 2024-09-17T05:32:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.775+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19d80051c0 con 0x7f19e0155340 2024-09-17T05:32:17.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.775+0000 7f19d6000640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.775+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7601+0+0 (secure 0 0 0) 0x7f19d8095210 con 0x7f19e0155340 2024-09-17T05:32:17.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.775+0000 7f19dde00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.775+0000 7f19dde00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 0x7f19bc075740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f19d0007fd0 tx=0x7f19d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.779+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19d8016030 con 0x7f19e0155340 2024-09-17T05:32:17.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:17.987+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f19e00ca250 con 0x7f19e0155340 2024-09-17T05:32:18.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.147+0000 7f19d6000640 1 -- 172.21.15.43:0/1400263923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v220) ==== 162+0+0 (secure 0 0 0) 0x7f19d805f3d0 con 0x7f19e0155340 2024-09-17T05:32:18.149 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 msgr2=0x7f19bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 0x7f19bc075740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f19d0007fd0 tx=0x7f19d0009040 comp rx=0 tx=0).stop 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 msgr2=0x7f19e008bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e940 tx=0x7f19d800ee00 comp rx=0 tx=0).stop 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 shutdown_connections 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e0155d70 0x7f19e0160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e0155340 0x7f19e008bfe0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 --2- 172.21.15.43:0/1400263923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f19e006f110 0x7f19e00cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 >> 172.21.15.43:0/1400263923 conn(0x7f19e00b23d0 msgr2=0x7f19e00ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:18.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 shutdown_connections 2024-09-17T05:32:18.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.151+0000 7f19e4c00640 1 -- 172.21.15.43:0/1400263923 wait complete. 2024-09-17T05:32:18.162 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:18.160+0000 7f19aee00640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-17T05:32:18.162 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:18.160+0000 7f19aee00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:32:18.163 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:18.159+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:18.163 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:18.159+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:18.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:18.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 -- 172.21.15.43:0/2493379844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0240 msgr2=0x7f3bd40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2493379844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0240 0x7f3bd40cd040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009840 tx=0x7f3bbc01c820 comp rx=0 tx=0).stop 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 -- 172.21.15.43:0/2493379844 shutdown_connections 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2493379844 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0240 0x7f3bd40cd040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2493379844 >> v2:172.21.15.62:3300/0 conn(0x7f3bd409ba60 0x7f3bd40b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 -- 172.21.15.43:0/2493379844 >> 172.21.15.43:0/2493379844 conn(0x7f3bd40cee60 msgr2=0x7f3bd40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 -- 172.21.15.43:0/2493379844 shutdown_connections 2024-09-17T05:32:18.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.331+0000 7f3bd9200640 1 -- 172.21.15.43:0/2493379844 wait complete. 2024-09-17T05:32:18.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 Processor -- start 2024-09-17T05:32:18.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 -- start start 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd409ba60 0x7f3bd406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0650 0x7f3bd406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40b2b80 con 0x7f3bd40d0240 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40b2a00 con 0x7f3bd409ba60 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bd9200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd4169e30 con 0x7f3bd40d0650 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.335+0000 7f3bc9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50142/0 (socket says 172.21.15.43:50142) 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 -- 172.21.15.43:0/2246572525 learned_addr learned my addr 172.21.15.43:0/2246572525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:18.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bd1e00640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd409ba60 0x7f3bd406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bd2800640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0650 0x7f3bd406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 -- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0650 msgr2=0x7f3bd406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0650 0x7f3bd406ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 -- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd409ba60 msgr2=0x7f3bd406fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd409ba60 0x7f3bd406fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 -- 172.21.15.43:0/2246572525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd41fcb00 con 0x7f3bd40d0240 2024-09-17T05:32:18.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bc9400640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000c10 tx=0x7f3bbc0a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0a8ca0 con 0x7f3bd40d0240 2024-09-17T05:32:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bbc0a8e40 con 0x7f3bd40d0240 2024-09-17T05:32:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc005270 con 0x7f3bd40d0240 2024-09-17T05:32:18.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41fcde0 con 0x7f3bd40d0240 2024-09-17T05:32:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd406e010 con 0x7f3bd40d0240 2024-09-17T05:32:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd40b3130 con 0x7f3bd40d0240 2024-09-17T05:32:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.339+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bbc0b8030 con 0x7f3bd40d0240 2024-09-17T05:32:18.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.343+0000 7f3bcaa00640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.343+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f3bbc12ec00 con 0x7f3bd40d0240 2024-09-17T05:32:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.343+0000 7f3bd1e00640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.343+0000 7f3bd1e00640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 0x7f3bb0075710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009b40 tx=0x7f3bc4009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:18.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.347+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3bbc0ac030 con 0x7f3bd40d0240 2024-09-17T05:32:18.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.559+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bd40b75f0 con 0x7f3bd40d0240 2024-09-17T05:32:18.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.559+0000 7f3bcaa00640 1 -- 172.21.15.43:0/2246572525 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 222 v222) ==== 95+0+3557 (secure 0 0 0) 0x7f3bbc016410 con 0x7f3bd40d0240 2024-09-17T05:32:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:18.562 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":222,"btime":"2024-09-17T05:32:18:163844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:18.163779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:18.562 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 222 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 msgr2=0x7f3bb0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 0x7f3bb0075710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009b40 tx=0x7f3bc4009260 comp rx=0 tx=0).stop 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 msgr2=0x7f3bd406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000c10 tx=0x7f3bbc0a8590 comp rx=0 tx=0).stop 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 shutdown_connections 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3bd40d0650 0x7f3bd406ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40d0240 0x7f3bd406e530 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 --2- 172.21.15.43:0/2246572525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bd409ba60 0x7f3bd406fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 >> 172.21.15.43:0/2246572525 conn(0x7f3bd40cee60 msgr2=0x7f3bd40717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 shutdown_connections 2024-09-17T05:32:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.563+0000 7f3bd9200640 1 -- 172.21.15.43:0/2246572525 wait complete. 2024-09-17T05:32:18.584 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:18.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:18.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- 172.21.15.43:0/2301133457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c06fc90 msgr2=0x7f9c5c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 --2- 172.21.15.43:0/2301133457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c06fc90 0x7f9c5c0700a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009840 tx=0x7f9c4801c700 comp rx=0 tx=0).stop 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- 172.21.15.43:0/2301133457 shutdown_connections 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 --2- 172.21.15.43:0/2301133457 >> v2:172.21.15.62:3300/0 conn(0x7f9c5c0730b0 0x7f9c5c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 --2- 172.21.15.43:0/2301133457 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c06fc90 0x7f9c5c0700a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- 172.21.15.43:0/2301133457 >> 172.21.15.43:0/2301133457 conn(0x7f9c5c0b23d0 msgr2=0x7f9c5c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- 172.21.15.43:0/2301133457 shutdown_connections 2024-09-17T05:32:18.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- 172.21.15.43:0/2301133457 wait complete. 2024-09-17T05:32:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 Processor -- start 2024-09-17T05:32:18.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.743+0000 7f9c64400640 1 -- start start 2024-09-17T05:32:18.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c5c06fc90 0x7f9c5c098dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c0730b0 0x7f9c5c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c169f70 con 0x7f9c5c15fe10 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c169df0 con 0x7f9c5c06fc90 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c16a0f0 con 0x7f9c5c0730b0 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50156/0 (socket says 172.21.15.43:50156) 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 -- 172.21.15.43:0/1361939329 learned_addr learned my addr 172.21.15.43:0/1361939329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c61a00640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c5c06fc90 0x7f9c5c098dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c61000640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c0730b0 0x7f9c5c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 -- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c0730b0 msgr2=0x7f9c5c0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c0730b0 0x7f9c5c0ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 -- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c5c06fc90 msgr2=0x7f9c5c098dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c5c06fc90 0x7f9c5c098dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 -- 172.21.15.43:0/1361939329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c074d60 con 0x7f9c5c15fe10 2024-09-17T05:32:18.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c62400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ea80 tx=0x7f9c5400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5400ce40 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c54004b30 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54005020 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c0aea80 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c099550 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.747+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c0700a0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.751+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c540051c0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.751+0000 7f9c59600640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 0x7f9c3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:18.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.751+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f9c540954c0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.751+0000 7f9c61a00640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 0x7f9c3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.751+0000 7f9c61a00640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 0x7f9c3c075740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007d70 tx=0x7f9c50007d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:18.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.755+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c5405f6c0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.877 INFO:tasks.ceph.mds.c.smithi043.stderr: -49> 2024-09-17T05:32:14.123+0000 7f7cdba00640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-17T05:32:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.967+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c5c15f5d0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.967+0000 7f9c59600640 1 -- 172.21.15.43:0/1361939329 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 222 v222) ==== 95+0+3557 (secure 0 0 0) 0x7f9c540640f0 con 0x7f9c5c15fe10 2024-09-17T05:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":222,"btime":"2024-09-17T05:32:18:163844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:18.163779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 222 2024-09-17T05:32:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 msgr2=0x7f9c3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 0x7f9c3c075740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007d70 tx=0x7f9c50007d00 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 msgr2=0x7f9c5c158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ea80 tx=0x7f9c5400ef40 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 shutdown_connections 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9c3c073230 0x7f9c3c075740 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c5c15fe10 0x7f9c5c158e00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9c5c0730b0 0x7f9c5c0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 --2- 172.21.15.43:0/1361939329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c5c06fc90 0x7f9c5c098dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 >> 172.21.15.43:0/1361939329 conn(0x7f9c5c0b23d0 msgr2=0x7f9c5c15ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 shutdown_connections 2024-09-17T05:32:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:18.971+0000 7f9c64400640 1 -- 172.21.15.43:0/1361939329 wait complete. 2024-09-17T05:32:18.994 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:18.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:18.998 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:19.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- 172.21.15.43:0/3499273943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39240b3af0 msgr2=0x7f39240ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 --2- 172.21.15.43:0/3499273943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39240b3af0 0x7f39240ad0b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f391c009bc0 tx=0x7f391c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- 172.21.15.43:0/3499273943 shutdown_connections 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 --2- 172.21.15.43:0/3499273943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39240b3af0 0x7f39240ad0b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 --2- 172.21.15.43:0/3499273943 >> v2:172.21.15.62:3300/0 conn(0x7f3924155d70 0x7f392415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- 172.21.15.43:0/3499273943 >> 172.21.15.43:0/3499273943 conn(0x7f39240b23d0 msgr2=0x7f39240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- 172.21.15.43:0/3499273943 shutdown_connections 2024-09-17T05:32:19.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- 172.21.15.43:0/3499273943 wait complete. 2024-09-17T05:32:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 Processor -- start 2024-09-17T05:32:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.155+0000 7f392b200640 1 -- start start 2024-09-17T05:32:19.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f392406f110 0x7f39240cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 0x7f39240748b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3924168cb0 con 0x7f3924155540 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3924168b30 con 0x7f3924155d70 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3924168e30 con 0x7f392406f110 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50186/0 (socket says 172.21.15.43:50186) 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 -- 172.21.15.43:0/3897939454 learned_addr learned my addr 172.21.15.43:0/3897939454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3923e00640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f392406f110 0x7f39240cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f391b400640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 0x7f39240748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 -- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f392406f110 msgr2=0x7f39240cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f392406f110 0x7f39240cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 -- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 msgr2=0x7f39240748b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 0x7f39240748b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 -- 172.21.15.43:0/3897939454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240cc5a0 con 0x7f3924155540 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f391b400640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 0x7f39240748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3929200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f391c01b190 tx=0x7f391c0a5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c0b2040 con 0x7f3924155540 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f391c003b30 con 0x7f3924155540 2024-09-17T05:32:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39240cc880 con 0x7f3924155540 2024-09-17T05:32:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c0a6e60 con 0x7f3924155540 2024-09-17T05:32:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39240cce40 con 0x7f3924155540 2024-09-17T05:32:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.159+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39240750c0 con 0x7f3924155540 2024-09-17T05:32:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.163+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f391c0bd030 con 0x7f3924155540 2024-09-17T05:32:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.163+0000 7f3920c00640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 0x7f3900075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.163+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f391c12e990 con 0x7f3924155540 2024-09-17T05:32:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.163+0000 7f3923e00640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 0x7f3900075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.163+0000 7f3923e00640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 0x7f3900075740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f39240715c0 tx=0x7f39140091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.167+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39240750c0 con 0x7f3924155540 2024-09-17T05:32:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.375+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39240aff90 con 0x7f3924155540 2024-09-17T05:32:19.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f3920c00640 1 -- 172.21.15.43:0/3897939454 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) ==== 95+0+4383 (secure 0 0 0) 0x7f391c0c4060 con 0x7f3924155540 2024-09-17T05:32:19.380 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:19.380 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":223,"btime":"2024-09-17T05:32:19:169551+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3572479007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3572479007},{"type":"v1","addr":"172.21.15.43:6835","nonce":3572479007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":223}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:18.163779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:19.380 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 223 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 msgr2=0x7f3900075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 0x7f3900075740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f39240715c0 tx=0x7f39140091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 msgr2=0x7f392415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f391c01b190 tx=0x7f391c0a5a40 comp rx=0 tx=0).stop 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 shutdown_connections 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3900073230 0x7f3900075740 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3924155d70 0x7f39240748b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3924155540 0x7f392415a710 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 --2- 172.21.15.43:0/3897939454 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f392406f110 0x7f39240cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 >> 172.21.15.43:0/3897939454 conn(0x7f39240b23d0 msgr2=0x7f39240d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 shutdown_connections 2024-09-17T05:32:19.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.379+0000 7f392b200640 1 -- 172.21.15.43:0/3897939454 wait complete. 2024-09-17T05:32:19.402 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-09-17T05:32:19.402 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-09-17T05:32:19.402 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:19.402 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:19.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:19.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:19.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:19.411 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:19.463 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:19.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:19.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- 172.21.15.43:0/737487246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f8005340 msgr2=0x7f57f800b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 --2- 172.21.15.43:0/737487246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f8005340 0x7f57f800b9f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f580c001dc0 tx=0x7f580c01cb30 comp rx=0 tx=0).stop 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- 172.21.15.43:0/737487246 shutdown_connections 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 --2- 172.21.15.43:0/737487246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57f8005340 0x7f57f800b9f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- 172.21.15.43:0/737487246 >> 172.21.15.43:0/737487246 conn(0x7f58140cee10 msgr2=0x7f58140cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- 172.21.15.43:0/737487246 shutdown_connections 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- 172.21.15.43:0/737487246 wait complete. 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 Processor -- start 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- start start 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 0x7f58141f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58141586a0 0x7f5814158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5814160dd0 con 0x7f58140cbb20 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.671+0000 7f5818e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f58140ceb40 con 0x7f57f8005340 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5818e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5814160f50 con 0x7f58141586a0 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50208/0 (socket says 172.21.15.43:50208) 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580be00640 1 -- 172.21.15.43:0/3328778374 learned_addr learned my addr 172.21.15.43:0/3328778374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 0x7f58141f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5812800640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58141586a0 0x7f5814158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 -- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58141586a0 msgr2=0x7f5814158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58141586a0 0x7f5814158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 -- 172.21.15.43:0/3328778374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 msgr2=0x7f581406f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 -- 172.21.15.43:0/3328778374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814158ff0 con 0x7f57f8005340 2024-09-17T05:32:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580be00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5811e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 0x7f58141f5960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5804011770 tx=0x7f5804011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580400fe70 con 0x7f57f8005340 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5804004020 con 0x7f57f8005340 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5814159df0 con 0x7f57f8005340 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004430 con 0x7f57f8005340 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140ca960 con 0x7f57f8005340 2024-09-17T05:32:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.675+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58140d01f0 con 0x7f57f8005340 2024-09-17T05:32:19.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.679+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5804003600 con 0x7f57f8005340 2024-09-17T05:32:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.679+0000 7f580a000640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 0x7f57f0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.679+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f5804095eb0 con 0x7f57f8005340 2024-09-17T05:32:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.679+0000 7f580be00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 0x7f57f0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.679+0000 7f580be00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 0x7f57f0075740 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f58140cb500 tx=0x7f57fc0067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.683+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5814158ff0 con 0x7f57f8005340 2024-09-17T05:32:19.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:19.895+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5814072670 con 0x7f57f8005340 2024-09-17T05:32:20.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f580a000640 1 -- 172.21.15.43:0/3328778374 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v224) ==== 161+0+0 (secure 0 0 0) 0x7f5804060140 con 0x7f57f8005340 2024-09-17T05:32:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 msgr2=0x7f57f0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 0x7f57f0075740 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f58140cb500 tx=0x7f57fc0067b0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 msgr2=0x7f58141f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 0x7f58141f5960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5804011770 tx=0x7f5804011c30 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 shutdown_connections 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f57f0073230 0x7f57f0075740 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f58141586a0 0x7f5814158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58140cbb20 0x7f581406f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 --2- 172.21.15.43:0/3328778374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f8005340 0x7f58141f5960 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 >> 172.21.15.43:0/3328778374 conn(0x7f58140cee10 msgr2=0x7f58140b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:20.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 shutdown_connections 2024-09-17T05:32:20.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.179+0000 7f5818e00640 1 -- 172.21.15.43:0/3328778374 wait complete. 2024-09-17T05:32:20.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- 172.21.15.43:0/3380506676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc644003c20 msgr2=0x7fc644004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 --2- 172.21.15.43:0/3380506676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc644003c20 0x7fc644004070 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc650001f50 tx=0x7fc65001cc40 comp rx=0 tx=0).stop 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65f400640 1 -- 172.21.15.43:0/3380506676 reap_dead start 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- 172.21.15.43:0/3380506676 shutdown_connections 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- 172.21.15.43:0/3380506676 >> 172.21.15.43:0/3380506676 conn(0x7fc6600b26c0 msgr2=0x7fc6600b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- 172.21.15.43:0/3380506676 shutdown_connections 2024-09-17T05:32:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- 172.21.15.43:0/3380506676 wait complete. 2024-09-17T05:32:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 Processor -- start 2024-09-17T05:32:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.363+0000 7fc65fe00640 1 -- start start 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644003c20 0x7fc6601503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6601593e0 0x7fc6601f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6601675b0 con 0x7fc660156060 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc660167430 con 0x7fc644003c20 2024-09-17T05:32:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc660167730 con 0x7fc6601593e0 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50234/0 (socket says 172.21.15.43:50234) 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 -- 172.21.15.43:0/1792343188 learned_addr learned my addr 172.21.15.43:0/1792343188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc656000640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6601593e0 0x7fc6601f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65ea00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644003c20 0x7fc6601503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 -- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6601593e0 msgr2=0x7fc6601f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6601593e0 0x7fc6601f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 -- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644003c20 msgr2=0x7fc6601503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644003c20 0x7fc6601503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 -- 172.21.15.43:0/1792343188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6601f9080 con 0x7fc660156060 2024-09-17T05:32:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65f400640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc658010910 tx=0x7fc658010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:20.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65801c070 con 0x7fc660156060 2024-09-17T05:32:20.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc65800ed50 con 0x7fc660156060 2024-09-17T05:32:20.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6601f9360 con 0x7fc660156060 2024-09-17T05:32:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658004290 con 0x7fc660156060 2024-09-17T05:32:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc660158be0 con 0x7fc660156060 2024-09-17T05:32:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.367+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66015ba50 con 0x7fc660156060 2024-09-17T05:32:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.371+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc65801f070 con 0x7fc660156060 2024-09-17T05:32:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.371+0000 7fc657400640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 0x7fc63c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.371+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7fc6580946b0 con 0x7fc660156060 2024-09-17T05:32:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.371+0000 7fc65ea00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 0x7fc63c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.371+0000 7fc65ea00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 0x7fc63c075740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc6500a9000 tx=0x7fc650019a80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.375+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc65805ee70 con 0x7fc660156060 2024-09-17T05:32:20.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:20.615+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc660164840 con 0x7fc660156060 2024-09-17T05:32:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.179+0000 7fc657400640 1 -- 172.21.15.43:0/1792343188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v225) ==== 101+0+0 (secure 0 0 0) 0x7fc65805e770 con 0x7fc660156060 2024-09-17T05:32:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 msgr2=0x7fc63c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 0x7fc63c075740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc6500a9000 tx=0x7fc650019a80 comp rx=0 tx=0).stop 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 msgr2=0x7fc660156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc658010910 tx=0x7fc658010dd0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 shutdown_connections 2024-09-17T05:32:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc63c073230 0x7fc63c075740 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660156060 0x7fc660156470 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6601593e0 0x7fc6601f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1792343188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc644003c20 0x7fc6601503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 >> 172.21.15.43:0/1792343188 conn(0x7fc6600b26c0 msgr2=0x7fc660151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 shutdown_connections 2024-09-17T05:32:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.183+0000 7fc65fe00640 1 -- 172.21.15.43:0/1792343188 wait complete. 2024-09-17T05:32:21.210 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.267+0000 7f3f23b55080 1 Processor -- start 2024-09-17T05:32:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.267+0000 7f3f23b55080 1 -- start start 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 0x557f22533dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22532380 0x557f2252be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f223a9a00 con 0x557f225148b0 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557f22407f40 con 0x557f22532380 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557f2245e760 con 0x557f22513010 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50258/0 (socket says 172.21.15.43:50258) 2024-09-17T05:32:21.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 -- 172.21.15.43:0/1153725597 learned_addr learned my addr 172.21.15.43:0/1153725597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 0x557f22533dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f1d600640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22532380 0x557f2252be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 -- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 msgr2=0x557f22533dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 0x557f22533dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 -- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22532380 msgr2=0x557f2252be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22532380 0x557f2252be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 -- 172.21.15.43:0/1153725597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f22382c10 con 0x557f225148b0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 0x557f22533dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f17e00640 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004f50 tx=0x7f3f08017f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ec481dcb4bad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f16a00640 1 -- 172.21.15.43:0/1153725597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080ab5f0 con 0x557f225148b0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f16a00640 1 -- 172.21.15.43:0/1153725597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f080b9070 con 0x557f225148b0 2024-09-17T05:32:21.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f16a00640 1 -- 172.21.15.43:0/1153725597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080abc90 con 0x557f225148b0 2024-09-17T05:32:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- 172.21.15.43:0/1153725597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 msgr2=0x557f22531e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004f50 tx=0x7f3f08017f10 comp rx=0 tx=0).stop 2024-09-17T05:32:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- 172.21.15.43:0/1153725597 shutdown_connections 2024-09-17T05:32:21.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22532380 0x557f2252be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f22531e40 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 --2- 172.21.15.43:0/1153725597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22513010 0x557f22533dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- 172.21.15.43:0/1153725597 >> 172.21.15.43:0/1153725597 conn(0x557f22447d80 msgr2=0x557f225334b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- 172.21.15.43:0/1153725597 shutdown_connections 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 -- 172.21.15.43:0/1153725597 wait complete. 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.271+0000 7f3f23b55080 1 Processor -- start 2024-09-17T05:32:21.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- start start 2024-09-17T05:32:21.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22513010 0x557f2252ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22532380 0x557f2245b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557f22512bb0 con 0x557f225148b0 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557f22534fd0 con 0x557f22513010 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557f2252e740 con 0x557f22532380 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50264/0 (socket says 172.21.15.43:50264) 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 learned_addr learned my addr 172.21.15.43:0/2725305490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22513010 0x557f2252ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f1d600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22532380 0x557f2245b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22532380 msgr2=0x557f2245b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557f22532380 0x557f2245b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22513010 msgr2=0x557f2252ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f22513010 0x557f2252ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2249e4f0 con 0x557f225148b0 2024-09-17T05:32:21.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3f080b6040 tx=0x7f3f080abaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f15600640 1 -- 172.21.15.43:0/2725305490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080abce0 con 0x557f225148b0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f15600640 1 -- 172.21.15.43:0/2725305490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f080c4070 con 0x557f225148b0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557f2249e790 con 0x557f225148b0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f15600640 1 -- 172.21.15.43:0/2725305490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080b4660 con 0x557f225148b0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.275+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2248b620 con 0x557f225148b0 2024-09-17T05:32:21.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f15600640 1 -- 172.21.15.43:0/2725305490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f080b3070 con 0x557f225148b0 2024-09-17T05:32:21.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f15600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 0x7f3efc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f15600640 1 -- 172.21.15.43:0/2725305490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f3f0812f1a0 con 0x557f225148b0 2024-09-17T05:32:21.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 0x7f3efc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.282 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557f2245bde0 0x557f225d7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225341a0 con 0x557f2245bde0 2024-09-17T05:32:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f1d600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557f2245bde0 0x557f225d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 0x7f3efc0740f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3f10003f00 tx=0x7f3f10003840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.279+0000 7f3f1d600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557f2245bde0 0x557f225d7300 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.283+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v112'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.283+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225d7840 con 0x557f2245bde0 2024-09-17T05:32:21.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.283+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v112'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.283+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d7be0 con 0x557f2245bde0 2024-09-17T05:32:21.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v112'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.288 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:21.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8f90 con 0x557f2245bde0 2024-09-17T05:32:21.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d9330 con 0x557f2245bde0 2024-09-17T05:32:21.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.287+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d7f80 con 0x557f2245bde0 2024-09-17T05:32:21.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8370 con 0x557f2245bde0 2024-09-17T05:32:21.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v112'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225d8760 con 0x557f2245bde0 2024-09-17T05:32:21.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v112'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.291+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8760 con 0x557f2245bde0 2024-09-17T05:32:21.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v112'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.296 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:21.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.295+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.299+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.299+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.299+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.299+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.299+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.303+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.305 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:21.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.303+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.303+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.303+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.303+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.307+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v112'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v112'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v112'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.315 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.311+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.315+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v112'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v112'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v112'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.323 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:21.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.319+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.323+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.323+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.323+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.323+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f2245bde0 2024-09-17T05:32:21.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557f225d8b10 0x557f225dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225d8710 con 0x557f225d8b10 2024-09-17T05:32:21.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557f225d8b10 0x557f225dbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557f225d8b10 0x557f225dbbf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'504 uv487 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.327+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225dc130 con 0x557f225d8b10 2024-09-17T05:32:21.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'505 uv505 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225dc4d0 con 0x557f225d8b10 2024-09-17T05:32:21.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.333 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:21.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225dcd20 con 0x557f225d8b10 2024-09-17T05:32:21.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.331+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225dd0c0 con 0x557f225d8b10 2024-09-17T05:32:21.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.335+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.335+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225dd460 con 0x557f225d8b10 2024-09-17T05:32:21.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.335+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v112'509 uv492 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.335+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225dd800 con 0x557f225d8b10 2024-09-17T05:32:21.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v112'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v112'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.342 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:21.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.339+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.343+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.343+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.343+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v112'514 uv497 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.343+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.347+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v112'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.347+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.347+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v112'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.350 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.347+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1d70 con 0x557f225d8b10 2024-09-17T05:32:21.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v112'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1d70 con 0x557f225d8b10 2024-09-17T05:32:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v112'519 uv503 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.351+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.355+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v112'520 uv520 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.355+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.355+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v112'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.358 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:21.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.355+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225debb0 con 0x557f225d8b10 2024-09-17T05:32:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225debb0 con 0x557f225d8b10 2024-09-17T05:32:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ddba0 con 0x557f225d8b10 2024-09-17T05:32:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225d8b10 2024-09-17T05:32:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557f225de220 0x557f225e0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e0bd0 con 0x557f225de220 2024-09-17T05:32:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.359+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557f225de220 0x557f225e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557f225de220 0x557f225e0640 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v112'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e1090 con 0x557f225de220 2024-09-17T05:32:21.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v112'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1570 con 0x557f225de220 2024-09-17T05:32:21.367 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:21.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v112'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.363+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e2e20 con 0x557f225de220 2024-09-17T05:32:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e31c0 con 0x557f225de220 2024-09-17T05:32:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1e10 con 0x557f225de220 2024-09-17T05:32:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.367+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e2200 con 0x557f2245bde0 2024-09-17T05:32:21.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.371+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'97 uv96 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.371+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e2200 con 0x557f2245bde0 2024-09-17T05:32:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.371+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'98 uv98 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.371+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e2200 con 0x557f2245bde0 2024-09-17T05:32:21.376 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'99 uv99 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e2200 con 0x557f2245bde0 2024-09-17T05:32:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'100 uv100 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 0x557f225e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e5ec0 con 0x557f225e3560 2024-09-17T05:32:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f1d600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 0x557f225e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.375+0000 7f3f1d600640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 0x557f225e5980 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v112'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e5ec0 con 0x557f225e3560 2024-09-17T05:32:21.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v112'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e5ec0 con 0x557f225e3560 2024-09-17T05:32:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v112'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.383 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.379+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e6ed0 con 0x557f225e3560 2024-09-17T05:32:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e6ed0 con 0x557f225e3560 2024-09-17T05:32:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e5ec0 con 0x557f225e3560 2024-09-17T05:32:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557f225e63d0 0x557f225e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.383+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e8e00 con 0x557f225e63d0 2024-09-17T05:32:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557f225e63d0 0x557f225e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f1cc00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557f225e63d0 0x557f225e87f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v112'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v112'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.387+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.391 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v112'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ea1e0 con 0x557f225e63d0 2024-09-17T05:32:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ea1e0 con 0x557f225e63d0 2024-09-17T05:32:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.391+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v112'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v112'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.395+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.399 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v112'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e91d0 con 0x557f225e63d0 2024-09-17T05:32:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f1cc00640 1 -- 172.21.15.43:0/2725305490 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1001b040 con 0x557f225e63d0 2024-09-17T05:32:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557f225e9850 0x557f225ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ec330 con 0x557f225e9850 2024-09-17T05:32:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557f225e9850 0x557f225ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.399+0000 7f3f17e00640 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557f225e9850 0x557f225ebc70 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v112'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ec700 con 0x557f225e9850 2024-09-17T05:32:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v112'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225e9850 2024-09-17T05:32:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v112'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.407 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.403+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225e9850 2024-09-17T05:32:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225e9850 2024-09-17T05:32:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f2245bde0 2024-09-17T05:32:21.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v112'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.407+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f2245bde0 2024-09-17T05:32:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.411+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v112'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.411+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f2245bde0 2024-09-17T05:32:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.411+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v112'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.414 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.411+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edb30 con 0x557f2245bde0 2024-09-17T05:32:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edb30 con 0x557f2245bde0 2024-09-17T05:32:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f2245bde0 2024-09-17T05:32:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f2245bde0 2024-09-17T05:32:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.415+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225de220 2024-09-17T05:32:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v112'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225de220 2024-09-17T05:32:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v112'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecb20 con 0x557f225de220 2024-09-17T05:32:21.423 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v112'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.419+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1a20 con 0x557f225de220 2024-09-17T05:32:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1a20 con 0x557f225de220 2024-09-17T05:32:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1a00 con 0x557f225de220 2024-09-17T05:32:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v112'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.423+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225e1a00 con 0x557f225de220 2024-09-17T05:32:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v112'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1a00 con 0x557f225de220 2024-09-17T05:32:21.429 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v112'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225e1a00 con 0x557f225de220 2024-09-17T05:32:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.427+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ef930 con 0x557f225de220 2024-09-17T05:32:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.431+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225de220 2024-09-17T05:32:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.431+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.431+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.431+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.437 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:21.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v112'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.435+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v112'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v112'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.443 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v112'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.439+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edfe0 con 0x557f225e3560 2024-09-17T05:32:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edfe0 con 0x557f225e3560 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e3560 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f1800b040 con 0x557f225e3560 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.443+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v112'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v112'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v112'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.451 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.447+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.451+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.451+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.451+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.451+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v112'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v112'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.458 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v112'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.455+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v112'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v112'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.459+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v112'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v112'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.466 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edfe0 con 0x557f225e9850 2024-09-17T05:32:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.463+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225edfe0 con 0x557f225e9850 2024-09-17T05:32:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ecfd0 con 0x557f225e9850 2024-09-17T05:32:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x557f225ed240 con 0x557f225e9850 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f17e00640 1 -- 172.21.15.43:0/2725305490 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f080f5620 con 0x557f225e9850 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557f225d8b10 msgr2=0x557f225dbbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557f225d8b10 0x557f225dbbf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557f225e63d0 msgr2=0x557f225e87f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557f225e63d0 0x557f225e87f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557f225e9850 msgr2=0x557f225ebc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557f225e9850 0x557f225ebc70 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557f225de220 msgr2=0x557f225e0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557f225de220 0x557f225e0640 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 reap_dead start 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557f2245bde0 msgr2=0x557f225d7300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557f2245bde0 0x557f225d7300 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f1d600640 1 -- 172.21.15.43:0/2725305490 reap_dead start 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 msgr2=0x557f225e5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.467+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 0x557f225e5980 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 msgr2=0x7f3efc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 0x7f3efc0740f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3f10003f00 tx=0x7f3f10003840 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 msgr2=0x557f2245b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3f080b6040 tx=0x7f3f080abaf0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 shutdown_connections 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557f225e3560 0x557f225e5980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3efc071be0 0x7f3efc0740f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 --2- 172.21.15.43:0/2725305490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557f225148b0 0x557f2245b360 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 >> 172.21.15.43:0/2725305490 conn(0x557f22447d80 msgr2=0x557f22532790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 shutdown_connections 2024-09-17T05:32:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.471+0000 7f3f23b55080 1 -- 172.21.15.43:0/2725305490 wait complete. 2024-09-17T05:32:21.477 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-09-17T05:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 Processor -- start 2024-09-17T05:32:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 -- start start 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b922402e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b92232ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b9222daf40 con 0x55b9223e4e40 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9222e7d00 con 0x55b92235dcb0 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25de24b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b92227ca00 con 0x55b9223314d0 2024-09-17T05:32:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.575+0000 7f25d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50272/0 (socket says 172.21.15.43:50272) 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 -- 172.21.15.43:0/3673647649 learned_addr learned my addr 172.21.15.43:0/3673647649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d5e00640 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b92232ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d6800640 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b922402e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 -- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 msgr2=0x55b922402e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b922402e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 -- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 msgr2=0x55b92232ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b92232ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 -- 172.21.15.43:0/3673647649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9224046f0 con 0x55b9223e4e40 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d7200640 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001600 tx=0x7f25d0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30e172f57f5059 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d4a00640 1 -- 172.21.15.43:0/3673647649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0016070 con 0x55b9223e4e40 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d4a00640 1 -- 172.21.15.43:0/3673647649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d000f800 con 0x55b9223e4e40 2024-09-17T05:32:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25d4a00640 1 -- 172.21.15.43:0/3673647649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d000fc30 con 0x55b9223e4e40 2024-09-17T05:32:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- 172.21.15.43:0/3673647649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 msgr2=0x55b92236f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001600 tx=0x7f25d0001ac0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- 172.21.15.43:0/3673647649 shutdown_connections 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b92232ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b92236f560 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 --2- 172.21.15.43:0/3673647649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b922402e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- 172.21.15.43:0/3673647649 >> 172.21.15.43:0/3673647649 conn(0x55b92231ad80 msgr2=0x55b92235d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- 172.21.15.43:0/3673647649 shutdown_connections 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- 172.21.15.43:0/3673647649 wait complete. 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 Processor -- start 2024-09-17T05:32:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.579+0000 7f25de24b080 1 -- start start 2024-09-17T05:32:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b92235aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b922350190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b922403a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b922404990 con 0x55b9223e4e40 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b9224026c0 con 0x55b92235dcb0 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b922403340 con 0x55b9223314d0 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b922403a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b92235aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b922403a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50276/0 (socket says 172.21.15.43:50276) 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d7200640 1 -- 172.21.15.43:0/829803729 learned_addr learned my addr 172.21.15.43:0/829803729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d5e00640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b922350190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 -- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 msgr2=0x55b922350190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b92235dcb0 0x55b922350190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 -- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 msgr2=0x55b922403a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b922403a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 -- 172.21.15.43:0/829803729 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b922350b80 con 0x55b9223314d0 2024-09-17T05:32:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d7200640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9223e4e40 0x55b922403a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25d6800640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b92235aa30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25c8002f40 tx=0x7f25c800c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25cf400640 1 -- 172.21.15.43:0/829803729 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c80019c0 con 0x55b9223314d0 2024-09-17T05:32:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25cf400640 1 -- 172.21.15.43:0/829803729 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c8001b60 con 0x55b9223314d0 2024-09-17T05:32:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b922350e20 con 0x55b9223314d0 2024-09-17T05:32:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25cf400640 1 -- 172.21.15.43:0/829803729 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8001d90 con 0x55b9223314d0 2024-09-17T05:32:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.583+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55b922374e90 con 0x55b9223314d0 2024-09-17T05:32:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25cf400640 1 -- 172.21.15.43:0/829803729 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25c800f040 con 0x55b9223314d0 2024-09-17T05:32:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25cf400640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25b8071be0 0x7f25b80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25cf400640 1 -- 172.21.15.43:0/829803729 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f25c8093210 con 0x55b9223314d0 2024-09-17T05:32:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25d5e00640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25b8071be0 0x7f25b80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25de24b080 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b9223506d0 0x55b922375ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b9223ffde0 con 0x55b9223506d0 2024-09-17T05:32:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25d7200640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b9223506d0 0x55b922375ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25d5e00640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25b8071be0 0x7f25b80740f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f25c40af000 tx=0x7f25c40b4040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.587+0000 7f25d7200640 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b9223506d0 0x55b922375ae0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25d7200640 1 -- 172.21.15.43:0/829803729 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v112'525 uv525 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f25d0019040 con 0x55b9223506d0 2024-09-17T05:32:21.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b9223506d0 msgr2=0x55b922375ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b9223506d0 0x55b922375ae0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25b8071be0 msgr2=0x7f25b80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25b8071be0 0x7f25b80740f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f25c40af000 tx=0x7f25c40b4040 comp rx=0 tx=0).stop 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 msgr2=0x55b92235aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 --2- 172.21.15.43:0/829803729 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b9223314d0 0x55b92235aa30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f25c8002f40 tx=0x7f25c800c610 comp rx=0 tx=0).stop 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25d7200640 1 -- 172.21.15.43:0/829803729 reap_dead start 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 shutdown_connections 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 >> 172.21.15.43:0/829803729 conn(0x55b92231ad80 msgr2=0x55b92232ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 shutdown_connections 2024-09-17T05:32:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.591+0000 7f25de24b080 1 -- 172.21.15.43:0/829803729 wait complete. 2024-09-17T05:32:21.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 -- 172.21.15.43:0/391020752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d806e390 msgr2=0x7f64d806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 --2- 172.21.15.43:0/391020752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d806e390 0x7f64d806ea70 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098f0 tx=0x7f64c801c850 comp rx=0 tx=0).stop 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 -- 172.21.15.43:0/391020752 shutdown_connections 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 --2- 172.21.15.43:0/391020752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d806e390 0x7f64d806ea70 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 -- 172.21.15.43:0/391020752 >> 172.21.15.43:0/391020752 conn(0x7f64d80cee80 msgr2=0x7f64d80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 -- 172.21.15.43:0/391020752 shutdown_connections 2024-09-17T05:32:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.755+0000 7f64dce00640 1 -- 172.21.15.43:0/391020752 wait complete. 2024-09-17T05:32:21.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 Processor -- start 2024-09-17T05:32:21.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 -- start start 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 0x7f64d80c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80b1db0 0x7f64d81f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64d80b7500 con 0x7f64d80b19a0 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f64d806f170 con 0x7f64d806e390 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64dce00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f64d80b7680 con 0x7f64d80b1db0 2024-09-17T05:32:21.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50296/0 (socket says 172.21.15.43:50296) 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64cd400640 1 -- 172.21.15.43:0/2910114404 learned_addr learned my addr 172.21.15.43:0/2910114404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64d5e00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 0x7f64d80c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.759+0000 7f64d6800640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80b1db0 0x7f64d81f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 -- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80b1db0 msgr2=0x7f64d81f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80b1db0 0x7f64d81f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 -- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 msgr2=0x7f64d80c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 0x7f64d80c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 -- 172.21.15.43:0/2910114404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d80cba00 con 0x7f64d80b19a0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64d5e00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 0x7f64d80c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cd400640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b3c0 tx=0x7f64c000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0011830 con 0x7f64d80b19a0 2024-09-17T05:32:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64c0016070 con 0x7f64d80b19a0 2024-09-17T05:32:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0011d40 con 0x7f64d80b19a0 2024-09-17T05:32:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d80cbce0 con 0x7f64d80b19a0 2024-09-17T05:32:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d80cc2a0 con 0x7f64d80b19a0 2024-09-17T05:32:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d80cb750 con 0x7f64d80b19a0 2024-09-17T05:32:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64c001b460 con 0x7f64d80b19a0 2024-09-17T05:32:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 0x7f64b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.763+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7687+0+0 (secure 0 0 0) 0x7f64c0094590 con 0x7f64d80b19a0 2024-09-17T05:32:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.767+0000 7f64d5e00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 0x7f64b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.767+0000 7f64d5e00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 0x7f64b4075710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64c80af000 tx=0x7f64c801b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.771+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f64c0020020 con 0x7f64d80b19a0 2024-09-17T05:32:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:21.979+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f64d806ea70 con 0x7f64d80b19a0 2024-09-17T05:32:21.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:22.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.187+0000 7f64cea00640 1 -- 172.21.15.43:0/2910114404 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v226) ==== 162+0+0 (secure 0 0 0) 0x7f64c005ed50 con 0x7f64d80b19a0 2024-09-17T05:32:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 msgr2=0x7f64b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 0x7f64b4075710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f64c80af000 tx=0x7f64c801b090 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 msgr2=0x7f64d80c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f64c000b3c0 tx=0x7f64c000b880 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 shutdown_connections 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f64b4073200 0x7f64b4075710 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f64d80b1db0 0x7f64d81f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d80b19a0 0x7f64d80c99f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 --2- 172.21.15.43:0/2910114404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64d806e390 0x7f64d80c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 >> 172.21.15.43:0/2910114404 conn(0x7f64d80cee80 msgr2=0x7f64d80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 shutdown_connections 2024-09-17T05:32:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.191+0000 7f64dce00640 1 -- 172.21.15.43:0/2910114404 wait complete. 2024-09-17T05:32:22.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:22.222 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:22.219+0000 7fbf17400640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-17T05:32:22.222 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:22.219+0000 7fbf17400640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:32:22.223 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:22.219+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:22.223 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:22.219+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:22.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- 172.21.15.43:0/1495993340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc160cf0 msgr2=0x7fcfdc15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/1495993340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15ce90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009c80 tx=0x7fcfd401cd70 comp rx=0 tx=0).stop 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- 172.21.15.43:0/1495993340 shutdown_connections 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/1495993340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15ce90 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/1495993340 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 0x7fcfdc0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- 172.21.15.43:0/1495993340 >> 172.21.15.43:0/1495993340 conn(0x7fcfdc068fe0 msgr2=0x7fcfdc069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- 172.21.15.43:0/1495993340 shutdown_connections 2024-09-17T05:32:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- 172.21.15.43:0/1495993340 wait complete. 2024-09-17T05:32:22.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 Processor -- start 2024-09-17T05:32:22.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.379+0000 7fcfe1e00640 1 -- start start 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 0x7fcfdc13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1536d0 con 0x7fcfdc15ca80 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc153550 con 0x7fcfdc160cf0 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc153850 con 0x7fcfdc0d1cb0 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50316/0 (socket says 172.21.15.43:50316) 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 -- 172.21.15.43:0/4019823079 learned_addr learned my addr 172.21.15.43:0/4019823079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1400640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe0a00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 0x7fcfdc13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 -- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 msgr2=0x7fcfdc13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 0x7fcfdc13e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 -- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc160cf0 msgr2=0x7fcfdc15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 -- 172.21.15.43:0/4019823079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc1fc180 con 0x7fcfdc15ca80 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfdbe00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c3c0 tx=0x7fcfc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8017070 con 0x7fcfdc15ca80 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfc80053c0 con 0x7fcfdc15ca80 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc14c060 con 0x7fcfdc15ca80 2024-09-17T05:32:22.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8003860 con 0x7fcfdc15ca80 2024-09-17T05:32:22.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1fc740 con 0x7fcfdc15ca80 2024-09-17T05:32:22.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.383+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc1fce00 con 0x7fcfdc15ca80 2024-09-17T05:32:22.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.387+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfc801a070 con 0x7fcfdc15ca80 2024-09-17T05:32:22.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.387+0000 7fcfd8c00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.387+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7fcfc8093da0 con 0x7fcfdc15ca80 2024-09-17T05:32:22.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.387+0000 7fcfe0a00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.387+0000 7fcfe0a00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 0x7fcfbc075710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009b40 tx=0x7fcfd00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.391+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfdc1fce00 con 0x7fcfdc15ca80 2024-09-17T05:32:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.599+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfdc0ccc40 con 0x7fcfdc15ca80 2024-09-17T05:32:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfd8c00640 1 -- 172.21.15.43:0/4019823079 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 228 v228) ==== 95+0+3557 (secure 0 0 0) 0x7fcfc805df50 con 0x7fcfdc15ca80 2024-09-17T05:32:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:22.604 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":228,"btime":"2024-09-17T05:32:22:223782+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:22.223692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:22.604 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 228 2024-09-17T05:32:22.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 msgr2=0x7fcfbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 0x7fcfbc075710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009b40 tx=0x7fcfd00091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 msgr2=0x7fcfdc1675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c3c0 tx=0x7fcfc800c880 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 shutdown_connections 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfdc160cf0 0x7fcfdc15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfdc15ca80 0x7fcfdc1675a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 --2- 172.21.15.43:0/4019823079 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcfdc0d1cb0 0x7fcfdc13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 >> 172.21.15.43:0/4019823079 conn(0x7fcfdc068fe0 msgr2=0x7fcfdc0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 shutdown_connections 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.603+0000 7fcfe1e00640 1 -- 172.21.15.43:0/4019823079 wait complete. 2024-09-17T05:32:22.625 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:22.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- 172.21.15.43:0/2459622155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 msgr2=0x7fa0440734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 --2- 172.21.15.43:0/2459622155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa0440734c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa0440d0190 tx=0x7fa03c01cc60 comp rx=0 tx=0).stop 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- 172.21.15.43:0/2459622155 shutdown_connections 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 --2- 172.21.15.43:0/2459622155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa0440734c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 --2- 172.21.15.43:0/2459622155 >> v2:172.21.15.62:3300/0 conn(0x7fa04406fc90 0x7fa0440700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- 172.21.15.43:0/2459622155 >> 172.21.15.43:0/2459622155 conn(0x7fa0440b23d0 msgr2=0x7fa0440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- 172.21.15.43:0/2459622155 shutdown_connections 2024-09-17T05:32:22.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- 172.21.15.43:0/2459622155 wait complete. 2024-09-17T05:32:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 Processor -- start 2024-09-17T05:32:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.823+0000 7fa049800640 1 -- start start 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 0x7fa044074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 0x7fa0441fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa04416a070 con 0x7fa0440730b0 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa044169ef0 con 0x7fa04415fe10 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa04416a1f0 con 0x7fa04406fc90 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50332/0 (socket says 172.21.15.43:50332) 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 -- 172.21.15.43:0/2900743037 learned_addr learned my addr 172.21.15.43:0/2900743037 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:22.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa042a00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 0x7fa044074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 -- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 msgr2=0x7fa044074f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa043400640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 0x7fa0441fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 0x7fa044074f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 -- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 msgr2=0x7fa0441fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 0x7fa0441fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 -- 172.21.15.43:0/2900743037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04415db20 con 0x7fa0440730b0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa042a00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 0x7fa044074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa043400640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 0x7fa0441fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03be00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e820 tx=0x7fa02c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c00cc60 con 0x7fa0440730b0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa02c00ce00 con 0x7fa0440730b0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c005280 con 0x7fa0440730b0 2024-09-17T05:32:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa044156bd0 con 0x7fa0440730b0 2024-09-17T05:32:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440cc1a0 con 0x7fa0440730b0 2024-09-17T05:32:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.827+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440700a0 con 0x7fa0440730b0 2024-09-17T05:32:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.831+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa02c005420 con 0x7fa0440730b0 2024-09-17T05:32:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.831+0000 7fa03aa00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.831+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7fa02c0953f0 con 0x7fa0440730b0 2024-09-17T05:32:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.831+0000 7fa042a00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.831+0000 7fa042a00640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 0x7fa020075740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa034001630 tx=0x7fa034009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:22.835+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa02c05fbf0 con 0x7fa0440730b0 2024-09-17T05:32:22.915 INFO:tasks.ceph.mds.d.smithi062.stderr: -48> 2024-09-17T05:32:18.160+0000 7f19aee00640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-17T05:32:22.915 INFO:tasks.ceph.mds.d.smithi062.stderr: -38> 2024-09-17T05:32:18.160+0000 7f19aee00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:32:23.039 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.043+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0440ca2f0 con 0x7fa0440730b0 2024-09-17T05:32:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.043+0000 7fa03aa00640 1 -- 172.21.15.43:0/2900743037 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 228 v228) ==== 95+0+3557 (secure 0 0 0) 0x7fa02c05f4f0 con 0x7fa0440730b0 2024-09-17T05:32:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:23.047 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":228,"btime":"2024-09-17T05:32:22:223782+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:22.223692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 228 2024-09-17T05:32:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 msgr2=0x7fa020075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 0x7fa020075740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa034001630 tx=0x7fa034009040 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 msgr2=0x7fa04415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00e820 tx=0x7fa02c00ece0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 shutdown_connections 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa04415fe10 0x7fa0441fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0440730b0 0x7fa04415e580 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 --2- 172.21.15.43:0/2900743037 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa04406fc90 0x7fa044074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 >> 172.21.15.43:0/2900743037 conn(0x7fa0440b23d0 msgr2=0x7fa044158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 shutdown_connections 2024-09-17T05:32:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.047+0000 7fa049800640 1 -- 172.21.15.43:0/2900743037 wait complete. 2024-09-17T05:32:23.069 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:23.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 -- 172.21.15.43:0/3124193205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 msgr2=0x7f75e80d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 --2- 172.21.15.43:0/3124193205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e80d0840 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f75e809bbe0 tx=0x7f75d801c5e0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 -- 172.21.15.43:0/3124193205 shutdown_connections 2024-09-17T05:32:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 --2- 172.21.15.43:0/3124193205 >> v2:172.21.15.62:3300/0 conn(0x7f75e80cd580 0x7f75e81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 --2- 172.21.15.43:0/3124193205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e80d0840 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 -- 172.21.15.43:0/3124193205 >> 172.21.15.43:0/3124193205 conn(0x7f75e80cee80 msgr2=0x7f75e80cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 -- 172.21.15.43:0/3124193205 shutdown_connections 2024-09-17T05:32:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.227+0000 7f75ed400640 1 -- 172.21.15.43:0/3124193205 wait complete. 2024-09-17T05:32:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 Processor -- start 2024-09-17T05:32:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 -- start start 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e80cd580 0x7f75e81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 0x7f75e8145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75e816a020 con 0x7f75e80b7ce0 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f75e8169ea0 con 0x7f75e80d0d80 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f75e816a1a0 con 0x7f75e80cd580 2024-09-17T05:32:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50358/0 (socket says 172.21.15.43:50358) 2024-09-17T05:32:23.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 -- 172.21.15.43:0/469688858 learned_addr learned my addr 172.21.15.43:0/469688858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:23.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75dde00640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e80cd580 0x7f75e81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e7200640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 0x7f75e8145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 -- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e80cd580 msgr2=0x7f75e81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e80cd580 0x7f75e81563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 -- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 msgr2=0x7f75e8145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 0x7f75e8145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 -- 172.21.15.43:0/469688858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e8154d80 con 0x7f75e80b7ce0 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e7200640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 0x7f75e8145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75e6800640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c10 tx=0x7f75d80a9720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d80ad070 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75d80a9ce0 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e8155060 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.231+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8005290 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e8155620 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e80d0840 con 0x7f75e80b7ce0 2024-09-17T05:32:23.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75d80b0070 con 0x7f75e80b7ce0 2024-09-17T05:32:23.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75df400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 0x7f75c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7f75d812eab0 con 0x7f75e80b7ce0 2024-09-17T05:32:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.235+0000 7f75dde00640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 0x7f75c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.239+0000 7f75dde00640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 0x7f75c4075710 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75e8140440 tx=0x7f75d0009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.239+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f75d80b7070 con 0x7f75e80b7ce0 2024-09-17T05:32:23.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.455+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75e80ceab0 con 0x7f75e80b7ce0 2024-09-17T05:32:23.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.455+0000 7f75df400640 1 -- 172.21.15.43:0/469688858 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) ==== 95+0+4383 (secure 0 0 0) 0x7f75d8016490 con 0x7f75e80b7ce0 2024-09-17T05:32:23.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:23.460 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":229,"btime":"2024-09-17T05:32:23:225717+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8599,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3432953722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3432953722},{"type":"v1","addr":"172.21.15.62:6835","nonce":3432953722}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":229}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:22.223692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:23.460 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 229 2024-09-17T05:32:23.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 msgr2=0x7f75c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 0x7f75c4075710 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75e8140440 tx=0x7f75d0009060 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 msgr2=0x7f75e8155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c10 tx=0x7f75d80a9720 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 shutdown_connections 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f75c4073200 0x7f75c4075710 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75e80d0d80 0x7f75e8145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f75e80cd580 0x7f75e81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 --2- 172.21.15.43:0/469688858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e80b7ce0 0x7f75e8155e70 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 >> 172.21.15.43:0/469688858 conn(0x7f75e80cee80 msgr2=0x7f75e80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 shutdown_connections 2024-09-17T05:32:23.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.459+0000 7f75ed400640 1 -- 172.21.15.43:0/469688858 wait complete. 2024-09-17T05:32:23.482 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-09-17T05:32:23.482 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-09-17T05:32:23.482 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:23.482 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:23.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:23.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:23.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:23.493 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:23.543 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:23.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:23.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 -- 172.21.15.43:0/3825153763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c070620 msgr2=0x7fcd5c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 --2- 172.21.15.43:0/3825153763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c070620 0x7fcd5c0b30d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000a950 tx=0x7fcd5001dae0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 -- 172.21.15.43:0/3825153763 shutdown_connections 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 --2- 172.21.15.43:0/3825153763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c070620 0x7fcd5c0b30d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 -- 172.21.15.43:0/3825153763 >> 172.21.15.43:0/3825153763 conn(0x7fcd5c0b23d0 msgr2=0x7fcd5c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 -- 172.21.15.43:0/3825153763 shutdown_connections 2024-09-17T05:32:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.747+0000 7fcd63e00640 1 -- 172.21.15.43:0/3825153763 wait complete. 2024-09-17T05:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 Processor -- start 2024-09-17T05:32:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- start start 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd5c06ce60 0x7fcd5c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd5c070620 0x7fcd5c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c0ce7a0 con 0x7fcd5c0cb860 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c0ce620 con 0x7fcd5c070620 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c0ce920 con 0x7fcd5c06ce60 2024-09-17T05:32:23.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd5c06ce60 0x7fcd5c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45686/0 (socket says 172.21.15.43:45686) 2024-09-17T05:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 -- 172.21.15.43:0/1105390331 learned_addr learned my addr 172.21.15.43:0/1105390331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd60a00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd5c070620 0x7fcd5c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 -- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd5c06ce60 msgr2=0x7fcd5c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd5c06ce60 0x7fcd5c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 -- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd5c070620 msgr2=0x7fcd5c09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd5c070620 0x7fcd5c09d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 -- 172.21.15.43:0/1105390331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c0c77d0 con 0x7fcd5c0cb860 2024-09-17T05:32:23.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd61e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400cb40 tx=0x7fcd5400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54016020 con 0x7fcd5c0cb860 2024-09-17T05:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd5400ab60 con 0x7fcd5c0cb860 2024-09-17T05:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54012420 con 0x7fcd5c0cb860 2024-09-17T05:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c0c7ab0 con 0x7fcd5c0cb860 2024-09-17T05:32:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.751+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c09da30 con 0x7fcd5c0cb860 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c1fd310 con 0x7fcd5c0cb860 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd54019020 con 0x7fcd5c0cb860 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd59600640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 0x7fcd3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7fcd54094c20 con 0x7fcd5c0cb860 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd61400640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 0x7fcd3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.755+0000 7fcd61400640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 0x7fcd3c075740 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b0000 tx=0x7fcd5001c320 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.759+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd54020020 con 0x7fcd5c0cb860 2024-09-17T05:32:23.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:23.971+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd5c0b30d0 con 0x7fcd5c0cb860 2024-09-17T05:32:24.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.231+0000 7fcd59600640 1 -- 172.21.15.43:0/1105390331 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v230) ==== 161+0+0 (secure 0 0 0) 0x7fcd5405ed40 con 0x7fcd5c0cb860 2024-09-17T05:32:24.235 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 msgr2=0x7fcd3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 0x7fcd3c075740 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b0000 tx=0x7fcd5001c320 comp rx=0 tx=0).stop 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 msgr2=0x7fcd5c1fc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400cb40 tx=0x7fcd5400cc30 comp rx=0 tx=0).stop 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 shutdown_connections 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcd3c073230 0x7fcd3c075740 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd5c070620 0x7fcd5c09d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5c0cb860 0x7fcd5c1fc5e0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 --2- 172.21.15.43:0/1105390331 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcd5c06ce60 0x7fcd5c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 >> 172.21.15.43:0/1105390331 conn(0x7fcd5c0b23d0 msgr2=0x7fcd5c073070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 shutdown_connections 2024-09-17T05:32:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.235+0000 7fcd63e00640 1 -- 172.21.15.43:0/1105390331 wait complete. 2024-09-17T05:32:24.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:24.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- 172.21.15.43:0/1691044670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240cdfb0 msgr2=0x7f52240cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 --2- 172.21.15.43:0/1691044670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240cdfb0 0x7f52240cd370 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5218009840 tx=0x7f521801c790 comp rx=0 tx=0).stop 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- 172.21.15.43:0/1691044670 shutdown_connections 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 --2- 172.21.15.43:0/1691044670 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240ce3c0 0x7f52240cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 --2- 172.21.15.43:0/1691044670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240cdfb0 0x7f52240cd370 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- 172.21.15.43:0/1691044670 >> 172.21.15.43:0/1691044670 conn(0x7f52240b2810 msgr2=0x7f52240b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- 172.21.15.43:0/1691044670 shutdown_connections 2024-09-17T05:32:24.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- 172.21.15.43:0/1691044670 wait complete. 2024-09-17T05:32:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 Processor -- start 2024-09-17T05:32:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.415+0000 7f522c400640 1 -- start start 2024-09-17T05:32:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240cdfb0 0x7f522414dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 0x7f5224155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f522416a150 con 0x7f52240ce3c0 2024-09-17T05:32:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5224169fd0 con 0x7f52241603e0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f522416a2d0 con 0x7f52240cdfb0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45706/0 (socket says 172.21.15.43:45706) 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 -- 172.21.15.43:0/3688801448 learned_addr learned my addr 172.21.15.43:0/3688801448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229a00640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240cdfb0 0x7f522414dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522a400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 0x7f5224155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 -- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240cdfb0 msgr2=0x7f522414dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240cdfb0 0x7f522414dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 -- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 msgr2=0x7f5224155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 0x7f5224155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 -- 172.21.15.43:0/3688801448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5224159250 con 0x7f52240ce3c0 2024-09-17T05:32:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522a400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 0x7f5224155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5229000640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f521000d780 tx=0x7f521000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210017070 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f521000de90 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52240b5c20 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52100099d0 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5224158490 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.419+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52241607f0 con 0x7f52240ce3c0 2024-09-17T05:32:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.423+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5210009b70 con 0x7f52240ce3c0 2024-09-17T05:32:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.423+0000 7f5221600640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 0x7f5204075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.423+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7f5210094e00 con 0x7f52240ce3c0 2024-09-17T05:32:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.423+0000 7f5229a00640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 0x7f5204075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.423+0000 7f5229a00640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 0x7f5204075740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f52180ae000 tx=0x7f52180b3040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.427+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5210020020 con 0x7f52240ce3c0 2024-09-17T05:32:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:24.663+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5224071300 con 0x7f52240ce3c0 2024-09-17T05:32:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.239+0000 7f5221600640 1 -- 172.21.15.43:0/3688801448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v231) ==== 101+0+0 (secure 0 0 0) 0x7f521005f570 con 0x7f52240ce3c0 2024-09-17T05:32:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 msgr2=0x7f5204075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 0x7f5204075740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f52180ae000 tx=0x7f52180b3040 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 msgr2=0x7f5224157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f521000d780 tx=0x7f521000dc40 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 shutdown_connections 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5204073230 0x7f5204075740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52241603e0 0x7f5224155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52240ce3c0 0x7f5224157480 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 --2- 172.21.15.43:0/3688801448 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52240cdfb0 0x7f522414dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 >> 172.21.15.43:0/3688801448 conn(0x7f52240b2810 msgr2=0x7f522406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 shutdown_connections 2024-09-17T05:32:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.243+0000 7f522c400640 1 -- 172.21.15.43:0/3688801448 wait complete. 2024-09-17T05:32:25.268 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:25.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 Processor -- start 2024-09-17T05:32:25.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 -- start start 2024-09-17T05:32:25.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee9c0 0x55774f738460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55774f7f1990 con 0x55774f7ee5b0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55774f685a00 con 0x55774f73c9c0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30cfec3080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55774f7f1b10 con 0x55774f7ee9c0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45734/0 (socket says 172.21.15.43:45734) 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30c3e00640 1 -- 172.21.15.43:0/889284911 learned_addr learned my addr 172.21.15.43:0/889284911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30c9000640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.327+0000 7f30c9a00640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee9c0 0x55774f738460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 -- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee9c0 msgr2=0x55774f738460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee9c0 0x55774f738460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 -- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 msgr2=0x55774f7eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7eda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 -- 172.21.15.43:0/889284911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774f684a20 con 0x55774f7ee5b0 2024-09-17T05:32:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c9000640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c3e00640 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000c10 tx=0x7f30b4017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a510d419537bcac server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c2a00640 1 -- 172.21.15.43:0/889284911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40ba070 con 0x55774f7ee5b0 2024-09-17T05:32:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c2a00640 1 -- 172.21.15.43:0/889284911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30b40ab980 con 0x55774f7ee5b0 2024-09-17T05:32:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30c2a00640 1 -- 172.21.15.43:0/889284911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b40abdd0 con 0x55774f7ee5b0 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- 172.21.15.43:0/889284911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 msgr2=0x55774f8131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f30b4000c10 tx=0x7f30b4017f90 comp rx=0 tx=0).stop 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- 172.21.15.43:0/889284911 shutdown_connections 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee9c0 0x55774f738460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee5b0 0x55774f8131c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 --2- 172.21.15.43:0/889284911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- 172.21.15.43:0/889284911 >> 172.21.15.43:0/889284911 conn(0x55774f723d80 msgr2=0x55774f781b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- 172.21.15.43:0/889284911 shutdown_connections 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- 172.21.15.43:0/889284911 wait complete. 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 Processor -- start 2024-09-17T05:32:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.331+0000 7f30cfec3080 1 -- start start 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7817a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee5b0 0x55774f89e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55774f80e110 con 0x55774f7ee9c0 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55774f739390 con 0x55774f73c9c0 2024-09-17T05:32:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55774f738e50 con 0x55774f7ee5b0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45742/0 (socket says 172.21.15.43:45742) 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 learned_addr learned my addr 172.21.15.43:0/3028164960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee5b0 0x55774f89e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee5b0 msgr2=0x55774f89e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55774f7ee5b0 0x55774f89e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 msgr2=0x55774f7817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55774f73c9c0 0x55774f7817a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774f65ec10 con 0x55774f7ee9c0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c9a00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f30c40107c0 tx=0x7f30c4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c1600640 1 -- 172.21.15.43:0/3028164960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c400ed30 con 0x55774f7ee9c0 2024-09-17T05:32:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c1600640 1 -- 172.21.15.43:0/3028164960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30c4019440 con 0x55774f7ee9c0 2024-09-17T05:32:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30c1600640 1 -- 172.21.15.43:0/3028164960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40198b0 con 0x55774f7ee9c0 2024-09-17T05:32:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55774f780b20 con 0x55774f7ee9c0 2024-09-17T05:32:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.335+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55774f781060 con 0x55774f7ee9c0 2024-09-17T05:32:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c1600640 1 -- 172.21.15.43:0/3028164960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30c40223a0 con 0x55774f7ee9c0 2024-09-17T05:32:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c1600640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 0x7f30a80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c1600640 1 -- 172.21.15.43:0/3028164960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7f30c4094af0 con 0x55774f7ee9c0 2024-09-17T05:32:25.341 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 0x7f30a80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55774f80d7b0 0x55774f7ec360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f7ec8a0 con 0x55774f80d7b0 2024-09-17T05:32:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55774f80d7b0 0x55774f7ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 0x7f30a80740f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c6f0 tx=0x7f30bc00f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.339+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55774f80d7b0 0x55774f7ec360 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.343+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v113'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.343+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f7ecc40 con 0x55774f80d7b0 2024-09-17T05:32:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.343+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v113'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.343+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f7ed010 con 0x55774f80d7b0 2024-09-17T05:32:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v113'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.347 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b47d0 con 0x55774f80d7b0 2024-09-17T05:32:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b4b70 con 0x55774f80d7b0 2024-09-17T05:32:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.347+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b37c0 con 0x55774f80d7b0 2024-09-17T05:32:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3bd0 con 0x55774f80d7b0 2024-09-17T05:32:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v113'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v113'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.351+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.357 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.355+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v113'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.355+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.355+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.355+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.359+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.365 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.363+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.367+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.367+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.367+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.367+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v113'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v113'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.371+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.376 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v113'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.375+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.379+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.379+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.379+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v113'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.379+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v113'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.385 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v113'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.383+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b3f70 con 0x55774f80d7b0 2024-09-17T05:32:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55774f8b4360 0x55774f8b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b7970 con 0x55774f8b4360 2024-09-17T05:32:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55774f8b4360 0x55774f8b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.387+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55774f8b4360 0x55774f8b7430 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.391+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'526 uv525 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.391+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b7d10 con 0x55774f8b4360 2024-09-17T05:32:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.391+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'527 uv527 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.391+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b80b0 con 0x55774f8b4360 2024-09-17T05:32:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.391+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.395 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.395+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b8900 con 0x55774f8b4360 2024-09-17T05:32:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.395+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.395+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b8ca0 con 0x55774f8b4360 2024-09-17T05:32:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.395+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.395+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9040 con 0x55774f8b4360 2024-09-17T05:32:25.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v113'531 uv513 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b93e0 con 0x55774f8b4360 2024-09-17T05:32:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v113'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9780 con 0x55774f8b4360 2024-09-17T05:32:25.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v113'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:25.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.399+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9b70 con 0x55774f8b4360 2024-09-17T05:32:25.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9b70 con 0x55774f8b4360 2024-09-17T05:32:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9890 con 0x55774f8b4360 2024-09-17T05:32:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v113'536 uv518 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.403+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b9890 con 0x55774f8b4360 2024-09-17T05:32:25.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.407+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v113'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.407+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9890 con 0x55774f8b4360 2024-09-17T05:32:25.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.407+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v113'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.410 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.407+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bda30 con 0x55774f8b4360 2024-09-17T05:32:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v113'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bda30 con 0x55774f8b4360 2024-09-17T05:32:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9860 con 0x55774f8b4360 2024-09-17T05:32:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v113'541 uv524 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.411+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8b9860 con 0x55774f8b4360 2024-09-17T05:32:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.415+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v113'542 uv542 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.415+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9860 con 0x55774f8b4360 2024-09-17T05:32:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.415+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v113'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.418 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.415+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9860 con 0x55774f8b4360 2024-09-17T05:32:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.415+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8ba870 con 0x55774f8b4360 2024-09-17T05:32:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b9860 con 0x55774f8b4360 2024-09-17T05:32:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8b4360 2024-09-17T05:32:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55774f8ba160 0x55774f8ba570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f8ba160 2024-09-17T05:32:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55774f8ba160 0x55774f8ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.419+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55774f8ba160 0x55774f8ba570 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v113'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f8ba160 2024-09-17T05:32:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v113'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f8ba160 2024-09-17T05:32:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v113'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.423+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bdce0 con 0x55774f8ba160 2024-09-17T05:32:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bdca0 con 0x55774f8ba160 2024-09-17T05:32:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bdca0 con 0x55774f8ba160 2024-09-17T05:32:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.427+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f80d7b0 2024-09-17T05:32:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'101 uv100 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f80d7b0 2024-09-17T05:32:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'102 uv102 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bcc70 con 0x55774f80d7b0 2024-09-17T05:32:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'103 uv103 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.435 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.431+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bd0e0 con 0x55774f80d7b0 2024-09-17T05:32:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 0x55774f8c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8bd0e0 con 0x55774f8bda20 2024-09-17T05:32:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30c9a00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 0x55774f8c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30c9a00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 0x55774f8c00f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v113'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.435+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c0630 con 0x55774f8bda20 2024-09-17T05:32:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v113'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c09d0 con 0x55774f8bda20 2024-09-17T05:32:25.441 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:25.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v113'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8b8530 con 0x55774f8bda20 2024-09-17T05:32:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.439+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c2370 con 0x55774f8bda20 2024-09-17T05:32:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c1220 con 0x55774f8bda20 2024-09-17T05:32:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55774f8c1ce0 0x55774f8c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c4d50 con 0x55774f8c1ce0 2024-09-17T05:32:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55774f8c1ce0 0x55774f8c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.443+0000 7f30c9000640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55774f8c1ce0 0x55774f8c4770 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v113'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v113'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v113'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.451 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:25.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.447+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c6130 con 0x55774f8c1ce0 2024-09-17T05:32:25.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c6130 con 0x55774f8c1ce0 2024-09-17T05:32:25.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.451+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v113'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v113'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v113'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.459 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.455+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c5120 con 0x55774f8c1ce0 2024-09-17T05:32:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30c9000640 1 -- 172.21.15.43:0/3028164960 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30bc00a730 con 0x55774f8c1ce0 2024-09-17T05:32:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55774f8c57a0 0x55774f8c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8300 con 0x55774f8c57a0 2024-09-17T05:32:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55774f8c57a0 0x55774f8c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.459+0000 7f30c3e00640 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55774f8c57a0 0x55774f8c7c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v113'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8300 con 0x55774f8c57a0 2024-09-17T05:32:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v113'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8c57a0 2024-09-17T05:32:25.466 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v113'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.463+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8c57a0 2024-09-17T05:32:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8c57a0 2024-09-17T05:32:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f80d7b0 2024-09-17T05:32:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v113'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.467+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f80d7b0 2024-09-17T05:32:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v113'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f80d7b0 2024-09-17T05:32:25.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v113'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.474 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:25.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9760 con 0x55774f80d7b0 2024-09-17T05:32:25.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.471+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9760 con 0x55774f80d7b0 2024-09-17T05:32:25.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.475+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.475+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f80d7b0 2024-09-17T05:32:25.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.475+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f80d7b0 2024-09-17T05:32:25.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.475+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.475+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v113'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v113'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.481 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v113'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.479+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v113'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v113'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.483+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.488 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v113'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8750 con 0x55774f8ba160 2024-09-17T05:32:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8cb560 con 0x55774f8ba160 2024-09-17T05:32:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8ba160 2024-09-17T05:32:25.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.487+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8c00 con 0x55774f8bda20 2024-09-17T05:32:25.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.491+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v113'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.491+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.491+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.491+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.495 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:25.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v113'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v113'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.495+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v113'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.501 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v113'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.499+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9fb0 con 0x55774f8bda20 2024-09-17T05:32:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8bda20 2024-09-17T05:32:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c4002070 con 0x55774f8bda20 2024-09-17T05:32:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v113'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.503+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.507+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v113'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.507+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.510 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.507+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v113'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.507+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v113'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.511+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v113'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.517 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v113'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.515+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v113'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v113'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.519+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.524 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v113'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9fb0 con 0x55774f8c57a0 2024-09-17T05:32:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9fb0 con 0x55774f8c57a0 2024-09-17T05:32:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.523+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c8fa0 con 0x55774f8c57a0 2024-09-17T05:32:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55774f8c9450 con 0x55774f8c57a0 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30c3e00640 1 -- 172.21.15.43:0/3028164960 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f30b40bd040 con 0x55774f8c57a0 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55774f8b4360 msgr2=0x55774f8b7430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55774f8b4360 0x55774f8b7430 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55774f8c1ce0 msgr2=0x55774f8c4770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55774f8c1ce0 0x55774f8c4770 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55774f8c57a0 msgr2=0x55774f8c7c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55774f8c57a0 0x55774f8c7c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55774f8ba160 msgr2=0x55774f8ba570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55774f8ba160 0x55774f8ba570 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 reap_dead start 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55774f80d7b0 msgr2=0x55774f7ec360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55774f80d7b0 0x55774f7ec360 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30c9a00640 1 -- 172.21.15.43:0/3028164960 reap_dead start 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 msgr2=0x55774f8c00f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 0x55774f8c00f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 msgr2=0x7f30a80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 0x7f30a80740f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c6f0 tx=0x7f30bc00f040 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 msgr2=0x55774f7ebe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f30c40107c0 tx=0x7f30c4010c80 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 shutdown_connections 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55774f8bda20 0x55774f8c00f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30a8071be0 0x7f30a80740f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 --2- 172.21.15.43:0/3028164960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55774f7ee9c0 0x55774f7ebe20 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 >> 172.21.15.43:0/3028164960 conn(0x55774f723d80 msgr2=0x55774f779e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 shutdown_connections 2024-09-17T05:32:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.527+0000 7f30cfec3080 1 -- 172.21.15.43:0/3028164960 wait complete. 2024-09-17T05:32:25.535 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-09-17T05:32:25.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 Processor -- start 2024-09-17T05:32:25.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 -- start start 2024-09-17T05:32:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967af04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967af59c0 0x55b967a7f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b967af89e0 con 0x55b967a439c0 2024-09-17T05:32:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.635+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b96798ca00 con 0x55b967af59c0 2024-09-17T05:32:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b967af8b60 con 0x55b967af55b0 2024-09-17T05:32:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45768/0 (socket says 172.21.15.43:45768) 2024-09-17T05:32:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 -- 172.21.15.43:0/2247014975 learned_addr learned my addr 172.21.15.43:0/2247014975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a6400640 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967af59c0 0x55b967a7f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5000640 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967af04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 -- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 msgr2=0x55b967af04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967af04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 -- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967af59c0 msgr2=0x55b967a7f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967af59c0 0x55b967a7f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 -- 172.21.15.43:0/2247014975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b96798ba20 con 0x55b967a439c0 2024-09-17T05:32:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe198004f60 tx=0x7fe198017fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574b3bd5bb5f2e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe19f400640 1 -- 172.21.15.43:0/2247014975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1980ba070 con 0x55b967a439c0 2024-09-17T05:32:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe19f400640 1 -- 172.21.15.43:0/2247014975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1980ab980 con 0x55b967a439c0 2024-09-17T05:32:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe19f400640 1 -- 172.21.15.43:0/2247014975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1980abdd0 con 0x55b967a439c0 2024-09-17T05:32:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/2247014975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 msgr2=0x55b967b1a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe198004f60 tx=0x7fe198017fb0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/2247014975 shutdown_connections 2024-09-17T05:32:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967af59c0 0x55b967a7f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967af04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/2247014975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967a439c0 0x55b967b1a1c0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/2247014975 >> 172.21.15.43:0/2247014975 conn(0x55b967a2ad80 msgr2=0x55b967a88b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/2247014975 shutdown_connections 2024-09-17T05:32:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/2247014975 wait complete. 2024-09-17T05:32:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 Processor -- start 2024-09-17T05:32:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.639+0000 7fe1ac7a9080 1 -- start start 2024-09-17T05:32:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967a439c0 0x55b967b11c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967b1b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 0x55b967b1b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b967b150d0 con 0x55b967af59c0 2024-09-17T05:32:25.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b967b14e30 con 0x55b967a439c0 2024-09-17T05:32:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55b967a7f220 con 0x55b967af55b0 2024-09-17T05:32:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 0x55b967b1b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 0x55b967b1b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45780/0 (socket says 172.21.15.43:45780) 2024-09-17T05:32:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 -- 172.21.15.43:0/3973448175 learned_addr learned my addr 172.21.15.43:0/3973448175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:25.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967a439c0 0x55b967b11c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a5000640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967b1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 -- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 msgr2=0x55b967b1b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55b967af55b0 0x55b967b1b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 -- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967a439c0 msgr2=0x55b967b11c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b967a439c0 0x55b967b11c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 -- 172.21.15.43:0/3973448175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b967965c10 con 0x55b967af59c0 2024-09-17T05:32:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1a6400640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 0x55b967b1b880 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0010860 tx=0x7fe1a0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe19e000640 1 -- 172.21.15.43:0/3973448175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000ed30 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe19e000640 1 -- 172.21.15.43:0/3973448175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1a0019440 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b967a88b10 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe19e000640 1 -- 172.21.15.43:0/3973448175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00198d0 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.643+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b967af0600 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe19e000640 1 -- 172.21.15.43:0/3973448175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe1a0022440 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe19e000640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe188071be0 0x7fe1880740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe19e000640 1 -- 172.21.15.43:0/3973448175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7fe1a0094b80 con 0x55b967af59c0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b967a86a60 0x55b967a86e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe188071be0 0x7fe1880740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55b967a873b0 con 0x55b967a86a60 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1a5000640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b967a86a60 0x55b967a86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1a5a00640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe188071be0 0x7fe1880740f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe19801bab0 tx=0x7fe198004950 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.647+0000 7fe1a5000640 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b967a86a60 0x55b967a86e70 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1a5000640 1 -- 172.21.15.43:0/3973448175 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v113'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe19400c6b0 con 0x55b967a86a60 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b967a86a60 msgr2=0x55b967a86e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55b967a86a60 0x55b967a86e70 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe188071be0 msgr2=0x7fe1880740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe188071be0 0x7fe1880740f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe19801bab0 tx=0x7fe198004950 comp rx=0 tx=0).stop 2024-09-17T05:32:25.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 msgr2=0x55b967b1b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 --2- 172.21.15.43:0/3973448175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b967af59c0 0x55b967b1b880 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0010860 tx=0x7fe1a0010d20 comp rx=0 tx=0).stop 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1a6400640 1 -- 172.21.15.43:0/3973448175 reap_dead start 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 shutdown_connections 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 >> 172.21.15.43:0/3973448175 conn(0x55b967a2ad80 msgr2=0x55b967a7e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 shutdown_connections 2024-09-17T05:32:25.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.651+0000 7fe1ac7a9080 1 -- 172.21.15.43:0/3973448175 wait complete. 2024-09-17T05:32:25.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 -- 172.21.15.43:0/1448975617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f406ce60 msgr2=0x7f41f406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 --2- 172.21.15.43:0/1448975617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f406ce60 0x7f41f406d270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009840 tx=0x7f41e001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 -- 172.21.15.43:0/1448975617 shutdown_connections 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 --2- 172.21.15.43:0/1448975617 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 --2- 172.21.15.43:0/1448975617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f406ce60 0x7f41f406d270 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 -- 172.21.15.43:0/1448975617 >> 172.21.15.43:0/1448975617 conn(0x7f41f40b23d0 msgr2=0x7f41f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 -- 172.21.15.43:0/1448975617 shutdown_connections 2024-09-17T05:32:25.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.827+0000 7f41fc200640 1 -- 172.21.15.43:0/1448975617 wait complete. 2024-09-17T05:32:25.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 Processor -- start 2024-09-17T05:32:25.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 -- start start 2024-09-17T05:32:25.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 0x7f41f4073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40ce630 con 0x7f41f406ce60 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40ce4b0 con 0x7f41f4070620 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fc200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f41f40ce7b0 con 0x7f41f40cb860 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41f9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41f9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45790/0 (socket says 172.21.15.43:45790) 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41f9800640 1 -- 172.21.15.43:0/2915332654 learned_addr learned my addr 172.21.15.43:0/2915332654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41fa200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.831+0000 7f41f8e00640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 0x7f41f4073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 -- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 msgr2=0x7f41f40c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 -- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 msgr2=0x7f41f4073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 0x7f41f4073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 -- 172.21.15.43:0/2915332654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f40c6650 con 0x7f41f406ce60 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41fa200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f8e00640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 0x7f41f4073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f9800640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c500 tx=0x7f41e800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8011790 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e8017070 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f40c5db0 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8011ce0 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f40c6370 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f40ced80 con 0x7f41f406ce60 2024-09-17T05:32:25.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.835+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f41e801c460 con 0x7f41f406ce60 2024-09-17T05:32:25.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.839+0000 7f41f1600640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 0x7f41d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.839+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7773+0+0 (secure 0 0 0) 0x7f41e8094820 con 0x7f41f406ce60 2024-09-17T05:32:25.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.839+0000 7f41f8e00640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 0x7f41d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.839+0000 7f41f8e00640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 0x7f41d4075710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001aef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:25.843+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41f40c6650 con 0x7f41f406ce60 2024-09-17T05:32:26.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.055+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f41f40c5970 con 0x7f41f406ce60 2024-09-17T05:32:26.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41f1600640 1 -- 172.21.15.43:0/2915332654 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v232) ==== 162+0+0 (secure 0 0 0) 0x7f41f406d7b0 con 0x7f41f406ce60 2024-09-17T05:32:26.245 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 msgr2=0x7f41d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 0x7f41d4075710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001aef0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 msgr2=0x7f41f40cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c500 tx=0x7f41e800c900 comp rx=0 tx=0).stop 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 shutdown_connections 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41d4073200 0x7f41d4075710 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f4070620 0x7f41f4073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f41f40cb860 0x7f41f40c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 --2- 172.21.15.43:0/2915332654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41f406ce60 0x7f41f40cad30 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 >> 172.21.15.43:0/2915332654 conn(0x7f41f40b23d0 msgr2=0x7f41f4073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 shutdown_connections 2024-09-17T05:32:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.243+0000 7f41fc200640 1 -- 172.21.15.43:0/2915332654 wait complete. 2024-09-17T05:32:26.258 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:26.256+0000 7fb652600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:32:26.259 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:26.259+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:26.259 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:26.259+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:26.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- 172.21.15.43:0/2739997862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 msgr2=0x7f28780b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 --2- 172.21.15.43:0/2739997862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f28780b33e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2860009900 tx=0x7f286001c820 comp rx=0 tx=0).stop 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- 172.21.15.43:0/2739997862 shutdown_connections 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 --2- 172.21.15.43:0/2739997862 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f28780b33e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 --2- 172.21.15.43:0/2739997862 >> v2:172.21.15.62:3300/0 conn(0x7f2878155d70 0x7f287815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- 172.21.15.43:0/2739997862 >> 172.21.15.43:0/2739997862 conn(0x7f28780b23d0 msgr2=0x7f28780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- 172.21.15.43:0/2739997862 shutdown_connections 2024-09-17T05:32:26.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- 172.21.15.43:0/2739997862 wait complete. 2024-09-17T05:32:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 Processor -- start 2024-09-17T05:32:26.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.427+0000 7f287e600640 1 -- start start 2024-09-17T05:32:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f287809d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2878155d70 0x7f2878164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2878168a60 con 0x7f28780b3920 2024-09-17T05:32:26.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f28781688e0 con 0x7f2878155d70 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2878168be0 con 0x7f287806f110 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45792/0 (socket says 172.21.15.43:45792) 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 -- 172.21.15.43:0/550586153 learned_addr learned my addr 172.21.15.43:0/550586153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f2877400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f287809d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f2877e00640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2878155d70 0x7f2878164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 -- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 msgr2=0x7f287809d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f287809d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 -- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2878155d70 msgr2=0x7f2878164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2878155d70 0x7f2878164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 -- 172.21.15.43:0/550586153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28781f8ba0 con 0x7f28780b3920 2024-09-17T05:32:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286fe00640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2860009850 tx=0x7f2860005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28600b2040 con 0x7f28780b3920 2024-09-17T05:32:26.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28600bd070 con 0x7f28780b3920 2024-09-17T05:32:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860003600 con 0x7f28780b3920 2024-09-17T05:32:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781f8e60 con 0x7f28780b3920 2024-09-17T05:32:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781f9420 con 0x7f28780b3920 2024-09-17T05:32:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.431+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28780b33e0 con 0x7f28780b3920 2024-09-17T05:32:26.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.435+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f28600037a0 con 0x7f28780b3920 2024-09-17T05:32:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.435+0000 7f286f400640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.435+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7f2860130000 con 0x7f28780b3920 2024-09-17T05:32:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.435+0000 7f2877400640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.435+0000 7f2877400640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 0x7f2854075710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28780715c0 tx=0x7f28680091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.439+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f28600164d0 con 0x7f28780b3920 2024-09-17T05:32:26.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.647+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28780af170 con 0x7f28780b3920 2024-09-17T05:32:26.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.647+0000 7f286f400640 1 -- 172.21.15.43:0/550586153 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 234 v234) ==== 95+0+3557 (secure 0 0 0) 0x7f28600fa1d0 con 0x7f28780b3920 2024-09-17T05:32:26.650 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:26.650 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":234,"btime":"2024-09-17T05:32:26:260139+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:26.260050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:26.650 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 234 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 msgr2=0x7f2854075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 0x7f2854075710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28780715c0 tx=0x7f28680091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 msgr2=0x7f287806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2860009850 tx=0x7f2860005e70 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 shutdown_connections 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2878155d70 0x7f2878164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28780b3920 0x7f287806d230 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 --2- 172.21.15.43:0/550586153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f287806f110 0x7f287809d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 >> 172.21.15.43:0/550586153 conn(0x7f28780b23d0 msgr2=0x7f28780ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 shutdown_connections 2024-09-17T05:32:26.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.651+0000 7f287e600640 1 -- 172.21.15.43:0/550586153 wait complete. 2024-09-17T05:32:26.672 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:26.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- 172.21.15.43:0/2507804973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba80003d00 msgr2=0x7fba80004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2507804973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba80003d00 0x7fba80004150 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001830 tx=0x7fba8c01c8a0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fba9b400640 1 -- 172.21.15.43:0/2507804973 reap_dead start 2024-09-17T05:32:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- 172.21.15.43:0/2507804973 shutdown_connections 2024-09-17T05:32:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- 172.21.15.43:0/2507804973 >> 172.21.15.43:0/2507804973 conn(0x7fba9c0b23d0 msgr2=0x7fba9c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- 172.21.15.43:0/2507804973 shutdown_connections 2024-09-17T05:32:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- 172.21.15.43:0/2507804973 wait complete. 2024-09-17T05:32:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 Processor -- start 2024-09-17T05:32:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.831+0000 7fbaa1800640 1 -- start start 2024-09-17T05:32:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba80003d00 0x7fba9c165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba9c08fde0 0x7fba9c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c167540 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1673c0 con 0x7fba9c08fde0 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c1676c0 con 0x7fba80003d00 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45810/0 (socket says 172.21.15.43:45810) 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 -- 172.21.15.43:0/2822465807 learned_addr learned my addr 172.21.15.43:0/2822465807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9aa00640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba80003d00 0x7fba9c165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba92000640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba9c08fde0 0x7fba9c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 -- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba80003d00 msgr2=0x7fba9c165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba80003d00 0x7fba9c165e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 -- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba9c08fde0 msgr2=0x7fba9c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba9c08fde0 0x7fba9c161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 -- 172.21.15.43:0/2822465807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0b1520 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba9b400640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fba94000f90 tx=0x7fba94006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94011900 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba94017070 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c0ca8e0 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94011d40 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c08f300 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.835+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c0ad0b0 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.839+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba9401c460 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.839+0000 7fba93400640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.839+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7fba940953f0 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.839+0000 7fba9aa00640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.839+0000 7fba9aa00640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 0x7fba78075740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0715c0 tx=0x7fba8c0196e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:26.843+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba9c0b1520 con 0x7fba9c0b0bd0 2024-09-17T05:32:26.974 INFO:tasks.ceph.mds.c.smithi043.stderr: -40> 2024-09-17T05:32:22.219+0000 7fbf17400640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-17T05:32:26.975 INFO:tasks.ceph.mds.c.smithi043.stderr: -38> 2024-09-17T05:32:22.219+0000 7fbf17400640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:32:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba9c0c6c70 con 0x7fba9c0b0bd0 2024-09-17T05:32:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fba93400640 1 -- 172.21.15.43:0/2822465807 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 234 v234) ==== 95+0+3557 (secure 0 0 0) 0x7fba9405fba0 con 0x7fba9c0b0bd0 2024-09-17T05:32:27.057 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:27.057 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":234,"btime":"2024-09-17T05:32:26:260139+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:26.260050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 234 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 msgr2=0x7fba78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 0x7fba78075740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0715c0 tx=0x7fba8c0196e0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 msgr2=0x7fba9c0b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fba94000f90 tx=0x7fba94006430 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 shutdown_connections 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba9c08fde0 0x7fba9c161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba9c0b0bd0 0x7fba9c0b0fe0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 --2- 172.21.15.43:0/2822465807 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fba80003d00 0x7fba9c165e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 >> 172.21.15.43:0/2822465807 conn(0x7fba9c0b23d0 msgr2=0x7fba9c15e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 shutdown_connections 2024-09-17T05:32:27.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.055+0000 7fbaa1800640 1 -- 172.21.15.43:0/2822465807 wait complete. 2024-09-17T05:32:27.080 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:27.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:27.094 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 -- 172.21.15.43:0/2205714648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c0099c0 msgr2=0x7f558c00bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 --2- 172.21.15.43:0/2205714648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c0099c0 0x7f558c00bde0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f55a815bc80 tx=0x7f55a001ce40 comp rx=0 tx=0).stop 2024-09-17T05:32:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 -- 172.21.15.43:0/2205714648 shutdown_connections 2024-09-17T05:32:27.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 --2- 172.21.15.43:0/2205714648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c0099c0 0x7f558c00bde0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 -- 172.21.15.43:0/2205714648 >> 172.21.15.43:0/2205714648 conn(0x7f55a80b23d0 msgr2=0x7f55a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 -- 172.21.15.43:0/2205714648 shutdown_connections 2024-09-17T05:32:27.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.239+0000 7f55ad800640 1 -- 172.21.15.43:0/2205714648 wait complete. 2024-09-17T05:32:27.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 Processor -- start 2024-09-17T05:32:27.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 -- start start 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558c0099c0 0x7f55a80ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a81f96e0 0x7f55a80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55a806f5b0 con 0x7f55a80ca300 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55a806f430 con 0x7f558c0099c0 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.243+0000 7f55ad800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f55a806f730 con 0x7f55a81f96e0 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45846/0 (socket says 172.21.15.43:45846) 2024-09-17T05:32:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 -- 172.21.15.43:0/1634522086 learned_addr learned my addr 172.21.15.43:0/1634522086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a6a00640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558c0099c0 0x7f55a80ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f559e000640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a81f96e0 0x7f55a80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 -- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a81f96e0 msgr2=0x7f55a80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a81f96e0 0x7f55a80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 -- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558c0099c0 msgr2=0x7f55a80ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558c0099c0 0x7f55a80ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 -- 172.21.15.43:0/1634522086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a80753c0 con 0x7f55a80ca300 2024-09-17T05:32:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55a7400640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f55a001d6e0 tx=0x7f55a0005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00ab040 con 0x7f55a80ca300 2024-09-17T05:32:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55a0003b20 con 0x7f55a80ca300 2024-09-17T05:32:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a80cb130 con 0x7f55a80ca300 2024-09-17T05:32:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00b8440 con 0x7f55a80ca300 2024-09-17T05:32:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81f8c00 con 0x7f55a80ca300 2024-09-17T05:32:27.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.247+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a80b3750 con 0x7f55a80ca300 2024-09-17T05:32:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.251+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55a00a5d10 con 0x7f55a80ca300 2024-09-17T05:32:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.251+0000 7f559f400640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 0x7f5584075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.251+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7f55a012efe0 con 0x7f55a80ca300 2024-09-17T05:32:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.251+0000 7f55a6a00640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 0x7f5584075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.251+0000 7f55a6a00640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 0x7f5584075710 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f55a80715c0 tx=0x7f5598006b30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.255+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f55a0016b50 con 0x7f55a80ca300 2024-09-17T05:32:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.463+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55a815b4a0 con 0x7f55a80ca300 2024-09-17T05:32:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.463+0000 7f559f400640 1 -- 172.21.15.43:0/1634522086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) ==== 95+0+4383 (secure 0 0 0) 0x7f55a00f90a0 con 0x7f55a80ca300 2024-09-17T05:32:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:27.467 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":235,"btime":"2024-09-17T05:32:27:261976+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1404316783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1404316783},{"type":"v1","addr":"172.21.15.43:6835","nonce":1404316783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":235}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:26.260050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 235 2024-09-17T05:32:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 msgr2=0x7f5584075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 0x7f5584075710 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f55a80715c0 tx=0x7f5598006b30 comp rx=0 tx=0).stop 2024-09-17T05:32:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 msgr2=0x7f55a81f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f55a001d6e0 tx=0x7f55a0005d00 comp rx=0 tx=0).stop 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 shutdown_connections 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5584073200 0x7f5584075710 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55a80ca300 0x7f55a81f5570 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f55a81f96e0 0x7f55a80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 --2- 172.21.15.43:0/1634522086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558c0099c0 0x7f55a80ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 >> 172.21.15.43:0/1634522086 conn(0x7f55a80b23d0 msgr2=0x7f55a806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 shutdown_connections 2024-09-17T05:32:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.467+0000 7f55ad800640 1 -- 172.21.15.43:0/1634522086 wait complete. 2024-09-17T05:32:27.489 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-09-17T05:32:27.489 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-09-17T05:32:27.490 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:27.490 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:27.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:27.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:27.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:27.544 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:27.594 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:27.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:27.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 -- 172.21.15.43:0/3335132225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f815fe10 msgr2=0x7ff6f80b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 --2- 172.21.15.43:0/3335132225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f815fe10 0x7ff6f80b4b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009870 tx=0x7ff6ec01c660 comp rx=0 tx=0).stop 2024-09-17T05:32:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 -- 172.21.15.43:0/3335132225 shutdown_connections 2024-09-17T05:32:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 --2- 172.21.15.43:0/3335132225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f815fe10 0x7ff6f80b4b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 -- 172.21.15.43:0/3335132225 >> 172.21.15.43:0/3335132225 conn(0x7ff6f80b23d0 msgr2=0x7ff6f80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 -- 172.21.15.43:0/3335132225 shutdown_connections 2024-09-17T05:32:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.799+0000 7ff6ff800640 1 -- 172.21.15.43:0/3335132225 wait complete. 2024-09-17T05:32:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 Processor -- start 2024-09-17T05:32:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 -- start start 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f806fc90 0x7ff6f80c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f80b0ec0 0x7ff6f80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f816aff0 con 0x7ff6f815fe10 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f816ae70 con 0x7ff6f80b0ec0 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6ff800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f816b170 con 0x7ff6f806fc90 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45852/0 (socket says 172.21.15.43:45852) 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 -- 172.21.15.43:0/2935393649 learned_addr learned my addr 172.21.15.43:0/2935393649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6fce00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f806fc90 0x7ff6f80c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6fd800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f80b0ec0 0x7ff6f80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 -- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f806fc90 msgr2=0x7ff6f80c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f806fc90 0x7ff6f80c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 -- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f80b0ec0 msgr2=0x7ff6f80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f80b0ec0 0x7ff6f80b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.803+0000 7ff6f7e00640 1 -- 172.21.15.43:0/2935393649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80b1810 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f7e00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001230 tx=0x7ff6e400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4016020 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6e400a8d0 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f80c1d50 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400ad20 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f81fa8c0 con 0x7ff6f815fe10 2024-09-17T05:32:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f80b4b80 con 0x7ff6f815fe10 2024-09-17T05:32:27.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6e4019020 con 0x7ff6f815fe10 2024-09-17T05:32:27.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 0x7ff6d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.807+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7ff6e4094c60 con 0x7ff6f815fe10 2024-09-17T05:32:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.811+0000 7ff6fce00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 0x7ff6d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.811+0000 7ff6fce00640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 0x7ff6d8075740 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0af000 tx=0x7ff6ec0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:27.815+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6e4020020 con 0x7ff6f815fe10 2024-09-17T05:32:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.023+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff6f80ca0d0 con 0x7ff6f815fe10 2024-09-17T05:32:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.267+0000 7ff6f4c00640 1 -- 172.21.15.43:0/2935393649 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v236) ==== 161+0+0 (secure 0 0 0) 0x7ff6f80b1810 con 0x7ff6f815fe10 2024-09-17T05:32:28.269 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 msgr2=0x7ff6d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 0x7ff6d8075740 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0af000 tx=0x7ff6ec0b2040 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 msgr2=0x7ff6f80c1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001230 tx=0x7ff6e400cb70 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 shutdown_connections 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6d8073230 0x7ff6d8075740 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6f80b0ec0 0x7ff6f80b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f815fe10 0x7ff6f80c1540 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 --2- 172.21.15.43:0/2935393649 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff6f806fc90 0x7ff6f80c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 >> 172.21.15.43:0/2935393649 conn(0x7ff6f80b23d0 msgr2=0x7ff6f815b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 shutdown_connections 2024-09-17T05:32:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.271+0000 7ff6ff800640 1 -- 172.21.15.43:0/2935393649 wait complete. 2024-09-17T05:32:28.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- 172.21.15.43:0/230760852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff240b3af0 msgr2=0x7eff240ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 --2- 172.21.15.43:0/230760852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff240b3af0 0x7eff240ad0b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009900 tx=0x7eff1c01c820 comp rx=0 tx=0).stop 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- 172.21.15.43:0/230760852 shutdown_connections 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 --2- 172.21.15.43:0/230760852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff240b3af0 0x7eff240ad0b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 --2- 172.21.15.43:0/230760852 >> v2:172.21.15.62:3300/0 conn(0x7eff24155d70 0x7eff2415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- 172.21.15.43:0/230760852 >> 172.21.15.43:0/230760852 conn(0x7eff240b23d0 msgr2=0x7eff240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- 172.21.15.43:0/230760852 shutdown_connections 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- 172.21.15.43:0/230760852 wait complete. 2024-09-17T05:32:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 Processor -- start 2024-09-17T05:32:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.451+0000 7eff2b000640 1 -- start start 2024-09-17T05:32:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff2406f110 0x7eff24160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 0x7eff241f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff24168c10 con 0x7eff24155d70 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff24168a90 con 0x7eff2406f110 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7eff24168d90 con 0x7eff241f9550 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45868/0 (socket says 172.21.15.43:45868) 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 -- 172.21.15.43:0/2892630124 learned_addr learned my addr 172.21.15.43:0/2892630124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff29000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 0x7eff241f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff23e00640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff2406f110 0x7eff24160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 -- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 msgr2=0x7eff241f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 0x7eff241f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 -- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff2406f110 msgr2=0x7eff24160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff2406f110 0x7eff24160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 -- 172.21.15.43:0/2892630124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff240cf070 con 0x7eff24155d70 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff29000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 0x7eff241f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff1b400640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c017070 con 0x7eff24155d70 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff0c0053c0 con 0x7eff24155d70 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003860 con 0x7eff24155d70 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff240cf350 con 0x7eff24155d70 2024-09-17T05:32:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff240cf910 con 0x7eff24155d70 2024-09-17T05:32:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.455+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff240ad0b0 con 0x7eff24155d70 2024-09-17T05:32:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.459+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7eff0c01a070 con 0x7eff24155d70 2024-09-17T05:32:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.459+0000 7eff20c00640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 0x7eff00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.459+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7eff0c093e10 con 0x7eff24155d70 2024-09-17T05:32:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.459+0000 7eff23e00640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 0x7eff00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.459+0000 7eff23e00640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 0x7eff00075710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7eff240715c0 tx=0x7eff14009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.463+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7eff0c097030 con 0x7eff24155d70 2024-09-17T05:32:28.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:28.703+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7eff240b07a0 con 0x7eff24155d70 2024-09-17T05:32:29.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff20c00640 1 -- 172.21.15.43:0/2892630124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v237) ==== 101+0+0 (secure 0 0 0) 0x7eff0c05dee0 con 0x7eff24155d70 2024-09-17T05:32:29.281 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 msgr2=0x7eff00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 0x7eff00075710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7eff240715c0 tx=0x7eff14009500 comp rx=0 tx=0).stop 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 msgr2=0x7eff24154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3c0 tx=0x7eff0c00c880 comp rx=0 tx=0).stop 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 shutdown_connections 2024-09-17T05:32:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff00073200 0x7eff00075710 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff241f9550 0x7eff241f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff24155d70 0x7eff24154d40 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 --2- 172.21.15.43:0/2892630124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff2406f110 0x7eff24160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 >> 172.21.15.43:0/2892630124 conn(0x7eff240b23d0 msgr2=0x7eff240b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 shutdown_connections 2024-09-17T05:32:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.279+0000 7eff2b000640 1 -- 172.21.15.43:0/2892630124 wait complete. 2024-09-17T05:32:29.305 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 Processor -- start 2024-09-17T05:32:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 -- start start 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2dddccb0 0x55cd2ddf12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2ddf1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd2dcfba00 con 0x55cd2ddab890 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cd2dd59f40 con 0x55cd2de69da0 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.363+0000 7f7216ebd080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cd2de68920 con 0x55cd2dddccb0 2024-09-17T05:32:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45896/0 (socket says 172.21.15.43:45896) 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 -- 172.21.15.43:0/1144753295 learned_addr learned my addr 172.21.15.43:0/1144753295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f720f600640 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2dddccb0 0x55cd2ddf12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210a00640 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2ddf1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 -- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2dddccb0 msgr2=0x55cd2ddf12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2dddccb0 0x55cd2ddf12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 -- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 msgr2=0x55cd2ddf1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2ddf1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 -- 172.21.15.43:0/1144753295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2dcd4c10 con 0x55cd2ddab890 2024-09-17T05:32:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7210000640 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7200018cc0 tx=0x7f7200017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854d3b018d872736 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f720e200640 1 -- 172.21.15.43:0/1144753295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000ba070 con 0x55cd2ddab890 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f720e200640 1 -- 172.21.15.43:0/1144753295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72000ab980 con 0x55cd2ddab890 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f720e200640 1 -- 172.21.15.43:0/1144753295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000abd90 con 0x55cd2ddab890 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- 172.21.15.43:0/1144753295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 msgr2=0x55cd2de67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7200018cc0 tx=0x7f7200017e30 comp rx=0 tx=0).stop 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- 172.21.15.43:0/1144753295 shutdown_connections 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2ddf1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2dddccb0 0x55cd2ddf12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 --2- 172.21.15.43:0/1144753295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2ddab890 0x55cd2de67700 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- 172.21.15.43:0/1144753295 >> 172.21.15.43:0/1144753295 conn(0x55cd2dd99d80 msgr2=0x55cd2ddf3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- 172.21.15.43:0/1144753295 shutdown_connections 2024-09-17T05:32:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- 172.21.15.43:0/1144753295 wait complete. 2024-09-17T05:32:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 Processor -- start 2024-09-17T05:32:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.367+0000 7f7216ebd080 1 -- start start 2024-09-17T05:32:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2ddab890 0x55cd2df0e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2de66020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cd2de837b0 con 0x55cd2dddccb0 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cd2df094c0 con 0x55cd2de69da0 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55cd2df09720 con 0x55cd2ddab890 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45910/0 (socket says 172.21.15.43:45910) 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 learned_addr learned my addr 172.21.15.43:0/360681949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2ddab890 0x55cd2df0e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7210a00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2de66020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2ddab890 msgr2=0x55cd2df0e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55cd2ddab890 0x55cd2df0e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 msgr2=0x55cd2de66020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd2de69da0 0x55cd2de66020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2de728d0 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a6f0 tx=0x7f71fc004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720ce00640 1 -- 172.21.15.43:0/360681949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0089a0 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720ce00640 1 -- 172.21.15.43:0/360681949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71fc008b40 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f720ce00640 1 -- 172.21.15.43:0/360681949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc016420 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd2de72b70 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.371+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2de730b0 con 0x55cd2dddccb0 2024-09-17T05:32:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f720ce00640 1 -- 172.21.15.43:0/360681949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71fc01f440 con 0x55cd2dddccb0 2024-09-17T05:32:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f720ce00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 0x7f71f00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f720ce00640 1 -- 172.21.15.43:0/360681949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7f71fc093800 con 0x55cd2dddccb0 2024-09-17T05:32:29.377 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 0x7f71f00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 0x55cd2de73c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2de74140 con 0x55cd2de737f0 2024-09-17T05:32:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7210a00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 0x55cd2de73c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 0x7f71f00740f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55cd2ddf2960 tx=0x7f72000ad040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.375+0000 7f7210a00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 0x55cd2de73c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.379+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v114'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.379+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2de74510 con 0x55cd2de737f0 2024-09-17T05:32:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.379+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v114'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.379+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df26a60 con 0x55cd2de737f0 2024-09-17T05:32:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v114'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.383 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27e10 con 0x55cd2de737f0 2024-09-17T05:32:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df281b0 con 0x55cd2de737f0 2024-09-17T05:32:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.383+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df26e00 con 0x55cd2de737f0 2024-09-17T05:32:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.387+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.387+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df271f0 con 0x55cd2de737f0 2024-09-17T05:32:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.387+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v114'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.387+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v114'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.393 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v114'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.391+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.395+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.399+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.399+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.402 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.399+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.399+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.399+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.403+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.403+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.403+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27590 con 0x55cd2de737f0 2024-09-17T05:32:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.403+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.403+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df279e0 con 0x55cd2de737f0 2024-09-17T05:32:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27950 con 0x55cd2de737f0 2024-09-17T05:32:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v114'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v114'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.407+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.412 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:29.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v114'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.411+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v114'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v114'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.415+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.420 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v114'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.419+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2de737f0 2024-09-17T05:32:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.423+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.423+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cd2df28a60 0x55cd2df2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.423+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df27810 con 0x55cd2df28a60 2024-09-17T05:32:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.423+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cd2df28a60 0x55cd2df2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.423+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cd2df28a60 0x55cd2df2ae80 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'548 uv530 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df2b3c0 con 0x55cd2df28a60 2024-09-17T05:32:29.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'549 uv549 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2b760 con 0x55cd2df28a60 2024-09-17T05:32:29.430 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.427+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2bfb0 con 0x55cd2df28a60 2024-09-17T05:32:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.431+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.431+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2c350 con 0x55cd2df28a60 2024-09-17T05:32:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.431+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.431+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2c6f0 con 0x55cd2df28a60 2024-09-17T05:32:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v114'553 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df2ca90 con 0x55cd2df28a60 2024-09-17T05:32:29.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v114'554 uv554 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.438 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v114'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.435+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.439+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.439+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.439+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.439+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v114'558 uv540 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v114'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v114'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.447 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:29.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.443+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df31000 con 0x55cd2df28a60 2024-09-17T05:32:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.447+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v114'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.447+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df31000 con 0x55cd2df28a60 2024-09-17T05:32:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.447+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.447+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v114'563 uv546 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v114'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v114'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.455 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:29.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.451+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2de40 con 0x55cd2df28a60 2024-09-17T05:32:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2de40 con 0x55cd2df28a60 2024-09-17T05:32:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2ce30 con 0x55cd2df28a60 2024-09-17T05:32:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df28a60 2024-09-17T05:32:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cd2df2d4b0 0x55cd2df2f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df2fe10 con 0x55cd2df2d4b0 2024-09-17T05:32:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.455+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cd2df2d4b0 0x55cd2df2f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.459+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cd2df2d4b0 0x55cd2df2f8d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.459+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v114'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.459+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df30230 con 0x55cd2df2d4b0 2024-09-17T05:32:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.459+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v114'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.459+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30710 con 0x55cd2df2d4b0 2024-09-17T05:32:29.463 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v114'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df31fc0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df32360 con 0x55cd2df2d4b0 2024-09-17T05:32:29.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.463+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30fb0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30fb0 con 0x55cd2de737f0 2024-09-17T05:32:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df30fb0 con 0x55cd2de737f0 2024-09-17T05:32:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'106 uv106 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.467+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30fb0 con 0x55cd2de737f0 2024-09-17T05:32:29.473 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'107 uv107 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30fb0 con 0x55cd2de737f0 2024-09-17T05:32:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'108 uv108 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 0x55cd2df34710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df34c50 con 0x55cd2df318b0 2024-09-17T05:32:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7210a00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 0x55cd2df34710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.471+0000 7f7210a00640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 0x55cd2df34710 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v114'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df34c50 con 0x55cd2df318b0 2024-09-17T05:32:29.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v114'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df34c50 con 0x55cd2df318b0 2024-09-17T05:32:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v114'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.475+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df35c60 con 0x55cd2df318b0 2024-09-17T05:32:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df35c60 con 0x55cd2df318b0 2024-09-17T05:32:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df34c50 con 0x55cd2df318b0 2024-09-17T05:32:29.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cd2df352d0 0x55cd2df376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df37d00 con 0x55cd2df352d0 2024-09-17T05:32:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.479+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cd2df352d0 0x55cd2df376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.483+0000 7f7210000640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cd2df352d0 0x55cd2df376f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.483+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v114'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.483+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df37dc0 con 0x55cd2df352d0 2024-09-17T05:32:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.483+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v114'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.483+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.488 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v114'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df391f0 con 0x55cd2df352d0 2024-09-17T05:32:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df391f0 con 0x55cd2df352d0 2024-09-17T05:32:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.487+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v114'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v114'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.491+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.495 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v114'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df381e0 con 0x55cd2df352d0 2024-09-17T05:32:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7210000640 1 -- 172.21.15.43:0/360681949 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f72000a3d70 con 0x55cd2df352d0 2024-09-17T05:32:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cd2df38860 0x55cd2df3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b2e0 con 0x55cd2df38860 2024-09-17T05:32:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cd2df38860 0x55cd2df3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.495+0000 7f720f600640 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cd2df38860 0x55cd2df3ac80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.499+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v114'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.499+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3b2e0 con 0x55cd2df38860 2024-09-17T05:32:29.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.499+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v114'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.499+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df38860 2024-09-17T05:32:29.503 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.499+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v114'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.503+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df38860 2024-09-17T05:32:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.503+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.503+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df38860 2024-09-17T05:32:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.503+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.503+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2de737f0 2024-09-17T05:32:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.507+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v114'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.507+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2de737f0 2024-09-17T05:32:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.507+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v114'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.507+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2de737f0 2024-09-17T05:32:29.511 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.507+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v114'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.511+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3c740 con 0x55cd2de737f0 2024-09-17T05:32:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.511+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.511+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3c740 con 0x55cd2de737f0 2024-09-17T05:32:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.511+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.511+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2de737f0 2024-09-17T05:32:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2de737f0 2024-09-17T05:32:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df2d4b0 2024-09-17T05:32:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v114'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df2d4b0 2024-09-17T05:32:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v114'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.515+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3b730 con 0x55cd2df2d4b0 2024-09-17T05:32:29.520 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v114'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.519+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.523+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v114'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.523+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.523+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v114'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.523+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.523+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v114'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.527+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df30ba0 con 0x55cd2df2d4b0 2024-09-17T05:32:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.527+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.527+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3e580 con 0x55cd2df2d4b0 2024-09-17T05:32:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.527+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df2d4b0 2024-09-17T05:32:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.527+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.531+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v114'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.531+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.531+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.531+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.531+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v114'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.535+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.535+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.535+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.535+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v114'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.535+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v114'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.541 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:29.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v114'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3cbf0 con 0x55cd2df318b0 2024-09-17T05:32:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.539+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3cbf0 con 0x55cd2df318b0 2024-09-17T05:32:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df318b0 2024-09-17T05:32:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7208009ae0 con 0x55cd2df318b0 2024-09-17T05:32:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v114'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.543+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v114'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v114'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.547+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.551+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.551+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.551+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v114'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.551+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v114'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v114'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.557 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v114'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.555+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v114'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v114'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.559+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v114'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.564 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.563+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bbe0 con 0x55cd2df38860 2024-09-17T05:32:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55cd2df3bff0 con 0x55cd2df38860 2024-09-17T05:32:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f720f600640 1 -- 172.21.15.43:0/360681949 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f71fc01c020 con 0x55cd2df38860 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cd2df28a60 msgr2=0x55cd2df2ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55cd2df28a60 0x55cd2df2ae80 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cd2df352d0 msgr2=0x55cd2df376f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55cd2df352d0 0x55cd2df376f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cd2df38860 msgr2=0x55cd2df3ac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55cd2df38860 0x55cd2df3ac80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cd2df2d4b0 msgr2=0x55cd2df2f8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55cd2df2d4b0 0x55cd2df2f8d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 reap_dead start 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 msgr2=0x55cd2de73c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 0x55cd2de73c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7210a00640 1 -- 172.21.15.43:0/360681949 reap_dead start 2024-09-17T05:32:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 msgr2=0x55cd2df34710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 0x55cd2df34710 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 msgr2=0x7f71f00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 0x7f71f00740f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55cd2ddf2960 tx=0x7f72000ad040 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 msgr2=0x55cd2de721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a6f0 tx=0x7f71fc004f90 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 shutdown_connections 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55cd2df318b0 0x55cd2df34710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f71f0071be0 0x7f71f00740f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55cd2de737f0 0x55cd2de73c00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 --2- 172.21.15.43:0/360681949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cd2dddccb0 0x55cd2de721c0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 >> 172.21.15.43:0/360681949 conn(0x55cd2dd99d80 msgr2=0x55cd2de67700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 shutdown_connections 2024-09-17T05:32:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.567+0000 7f7216ebd080 1 -- 172.21.15.43:0/360681949 wait complete. 2024-09-17T05:32:29.574 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-09-17T05:32:29.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.671+0000 7f802373f080 1 Processor -- start 2024-09-17T05:32:29.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.671+0000 7f802373f080 1 -- start start 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645de4590 0x558645e9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e12cb0 0x558645eb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558645d31a00 con 0x558645e9fda0 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558645d8ff40 con 0x558645de4590 2024-09-17T05:32:29.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558645e9ede0 con 0x558645e12cb0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45932/0 (socket says 172.21.15.43:45932) 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 -- 172.21.15.43:0/3351115144 learned_addr learned my addr 172.21.15.43:0/3351115144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801c800640 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645de4590 0x558645e9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801be00640 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e12cb0 0x558645eb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 -- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e12cb0 msgr2=0x558645eb8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e12cb0 0x558645eb8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 -- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645de4590 msgr2=0x558645e9b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645de4590 0x558645e9b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 -- 172.21.15.43:0/3351115144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558645d0ac10 con 0x558645e9fda0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801d200640 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f801400a550 tx=0x7f8014018240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf41dc51fce62a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801aa00640 1 -- 172.21.15.43:0/3351115144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140af070 con 0x558645e9fda0 2024-09-17T05:32:29.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801aa00640 1 -- 172.21.15.43:0/3351115144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80140ba070 con 0x558645e9fda0 2024-09-17T05:32:29.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f801aa00640 1 -- 172.21.15.43:0/3351115144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140b5420 con 0x558645e9fda0 2024-09-17T05:32:29.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- 172.21.15.43:0/3351115144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 msgr2=0x558645e2b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f801400a550 tx=0x7f8014018240 comp rx=0 tx=0).stop 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- 172.21.15.43:0/3351115144 shutdown_connections 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645e9fda0 0x558645e2b5f0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e12cb0 0x558645eb8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 --2- 172.21.15.43:0/3351115144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645de4590 0x558645e9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- 172.21.15.43:0/3351115144 >> 172.21.15.43:0/3351115144 conn(0x558645dcfd80 msgr2=0x558645de5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- 172.21.15.43:0/3351115144 shutdown_connections 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 -- 172.21.15.43:0/3351115144 wait complete. 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.675+0000 7f802373f080 1 Processor -- start 2024-09-17T05:32:29.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- start start 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 0x558645e98570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645e12cb0 0x558645ec28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e9fda0 0x558645e9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x558645e9bce0 con 0x558645de4590 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558645eb9f20 con 0x558645e12cb0 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x558645eba180 con 0x558645e9fda0 2024-09-17T05:32:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 0x558645e98570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 0x558645e98570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45946/0 (socket says 172.21.15.43:45946) 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 -- 172.21.15.43:0/1681730149 learned_addr learned my addr 172.21.15.43:0/1681730149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801be00640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645e12cb0 0x558645ec28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801d200640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e9fda0 0x558645e9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 -- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e9fda0 msgr2=0x558645e9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x558645e9fda0 0x558645e9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 -- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645e12cb0 msgr2=0x558645ec28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645e12cb0 0x558645ec28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 -- 172.21.15.43:0/1681730149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558645ec1980 con 0x558645de4590 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801be00640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558645e12cb0 0x558645ec28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f801c800640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 0x558645e98570 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8010004fd0 tx=0x7f801000abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f8019600640 1 -- 172.21.15.43:0/1681730149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010009dc0 con 0x558645de4590 2024-09-17T05:32:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f8019600640 1 -- 172.21.15.43:0/1681730149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f801000ae10 con 0x558645de4590 2024-09-17T05:32:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558645ec1c20 con 0x558645de4590 2024-09-17T05:32:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f8019600640 1 -- 172.21.15.43:0/1681730149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010008680 con 0x558645de4590 2024-09-17T05:32:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.679+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558645ec2160 con 0x558645de4590 2024-09-17T05:32:29.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f8019600640 1 -- 172.21.15.43:0/1681730149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8010008820 con 0x558645de4590 2024-09-17T05:32:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f8019600640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ffc071be0 0x7f7ffc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f8019600640 1 -- 172.21.15.43:0/1681730149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7f80100943b0 con 0x558645de4590 2024-09-17T05:32:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f802373f080 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558645ec14a0 0x558645ebbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x558645ebc220 con 0x558645ec14a0 2024-09-17T05:32:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f801be00640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ffc071be0 0x7f7ffc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f801d200640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558645ec14a0 0x558645ebbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f801d200640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558645ec14a0 0x558645ebbce0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.683+0000 7f801be00640 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ffc071be0 0x7f7ffc0740f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8008002ef0 tx=0x7f8008002a80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f801d200640 1 -- 172.21.15.43:0/1681730149 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v114'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8014002050 con 0x558645ec14a0 2024-09-17T05:32:29.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558645ec14a0 msgr2=0x558645ebbce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x558645ec14a0 0x558645ebbce0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ffc071be0 msgr2=0x7f7ffc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ffc071be0 0x7f7ffc0740f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8008002ef0 tx=0x7f8008002a80 comp rx=0 tx=0).stop 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 msgr2=0x558645e98570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 --2- 172.21.15.43:0/1681730149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558645de4590 0x558645e98570 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8010004fd0 tx=0x7f801000abc0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f801d200640 1 -- 172.21.15.43:0/1681730149 reap_dead start 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 shutdown_connections 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 >> 172.21.15.43:0/1681730149 conn(0x558645dcfd80 msgr2=0x558645e28ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 shutdown_connections 2024-09-17T05:32:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.687+0000 7f802373f080 1 -- 172.21.15.43:0/1681730149 wait complete. 2024-09-17T05:32:29.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.851+0000 7fd80e000640 1 -- 172.21.15.43:0/3212523630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8080b3af0 msgr2=0x7fd80815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.851+0000 7fd80e000640 1 --2- 172.21.15.43:0/3212523630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8080b3af0 0x7fd80815c7e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fd8080d0c40 tx=0x7fd7f001c740 comp rx=0 tx=0).stop 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- 172.21.15.43:0/3212523630 shutdown_connections 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 --2- 172.21.15.43:0/3212523630 >> v2:172.21.15.62:3300/0 conn(0x7fd80806f110 0x7fd8080ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 --2- 172.21.15.43:0/3212523630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8080b3af0 0x7fd80815c7e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- 172.21.15.43:0/3212523630 >> 172.21.15.43:0/3212523630 conn(0x7fd8080b23d0 msgr2=0x7fd8080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- 172.21.15.43:0/3212523630 shutdown_connections 2024-09-17T05:32:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- 172.21.15.43:0/3212523630 wait complete. 2024-09-17T05:32:29.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 Processor -- start 2024-09-17T05:32:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- start start 2024-09-17T05:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80806f110 0x7fd8081551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808155d70 0x7fd8080cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168c30 con 0x7fd808165e30 2024-09-17T05:32:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168ab0 con 0x7fd808155d70 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.855+0000 7fd80e000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd808168db0 con 0x7fd80806f110 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808155d70 0x7fd8080cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45952/0 (socket says 172.21.15.43:45952) 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 -- 172.21.15.43:0/367709081 learned_addr learned my addr 172.21.15.43:0/367709081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807400640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80806f110 0x7fd8081551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 -- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80806f110 msgr2=0x7fd8081551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80806f110 0x7fd8081551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 -- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808155d70 msgr2=0x7fd8080cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808155d70 0x7fd8080cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 -- 172.21.15.43:0/367709081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081f66f0 con 0x7fd808165e30 2024-09-17T05:32:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd807e00640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd80000ea80 tx=0x7fd80000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000cea0 con 0x7fd808165e30 2024-09-17T05:32:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd800004b30 con 0x7fd808165e30 2024-09-17T05:32:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800005020 con 0x7fd808165e30 2024-09-17T05:32:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081f6990 con 0x7fd808165e30 2024-09-17T05:32:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd808165460 con 0x7fd808165e30 2024-09-17T05:32:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8080ad0b0 con 0x7fd808165e30 2024-09-17T05:32:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.859+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8000051c0 con 0x7fd808165e30 2024-09-17T05:32:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.863+0000 7fd7ff400640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 0x7fd7e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.863+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7859+0+0 (secure 0 0 0) 0x7fd8000954f0 con 0x7fd808165e30 2024-09-17T05:32:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.863+0000 7fd807400640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 0x7fd7e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.863+0000 7fd807400640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 0x7fd7e4075710 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8007fd0 tx=0x7fd7f8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:29.867+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd80005f6d0 con 0x7fd808165e30 2024-09-17T05:32:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.079+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd80806d950 con 0x7fd808165e30 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.283+0000 7fd7ff400640 1 -- 172.21.15.43:0/367709081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v238) ==== 162+0+0 (secure 0 0 0) 0x7fd800064100 con 0x7fd808165e30 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 msgr2=0x7fd7e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 0x7fd7e4075710 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8007fd0 tx=0x7fd7f8009040 comp rx=0 tx=0).stop 2024-09-17T05:32:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 msgr2=0x7fd8080cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd80000ea80 tx=0x7fd80000ef40 comp rx=0 tx=0).stop 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 shutdown_connections 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd7e4073200 0x7fd7e4075710 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd808155d70 0x7fd8080cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd808165e30 0x7fd8080cfc30 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 --2- 172.21.15.43:0/367709081 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd80806f110 0x7fd8081551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 >> 172.21.15.43:0/367709081 conn(0x7fd8080b23d0 msgr2=0x7fd8080afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 shutdown_connections 2024-09-17T05:32:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.287+0000 7fd80e000640 1 -- 172.21.15.43:0/367709081 wait complete. 2024-09-17T05:32:30.298 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:30.295+0000 7f26fa000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-17T05:32:30.299 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:30.299+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:30.299 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:30.299+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:30.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.471+0000 7f238e600640 1 -- 172.21.15.43:0/2709705940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 msgr2=0x7f23880700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.471+0000 7f238e600640 1 --2- 172.21.15.43:0/2709705940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f23880700a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2374009870 tx=0x7f237401c660 comp rx=0 tx=0).stop 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- 172.21.15.43:0/2709705940 shutdown_connections 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 --2- 172.21.15.43:0/2709705940 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f23880700a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- 172.21.15.43:0/2709705940 >> 172.21.15.43:0/2709705940 conn(0x7f23880b23d0 msgr2=0x7f23880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- 172.21.15.43:0/2709705940 shutdown_connections 2024-09-17T05:32:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- 172.21.15.43:0/2709705940 wait complete. 2024-09-17T05:32:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 Processor -- start 2024-09-17T05:32:30.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- start start 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f238809d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23880730b0 0x7f23880c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f238816b610 con 0x7f238815fe10 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f238816b490 con 0x7f23880730b0 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.475+0000 7f238e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f238816b790 con 0x7f238806fc90 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f238809d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2386a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23880730b0 0x7f23880c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f238809d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:49250/0 (socket says 172.21.15.43:49250) 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387400640 1 -- 172.21.15.43:0/4279302601 learned_addr learned my addr 172.21.15.43:0/4279302601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45954/0 (socket says 172.21.15.43:45954) 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 -- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 msgr2=0x7f238809d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f238809d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 -- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23880730b0 msgr2=0x7f23880c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23880730b0 0x7f23880c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 -- 172.21.15.43:0/4279302601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238815d990 con 0x7f238815fe10 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f2387e00640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f23880d01d0 tx=0x7f238000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380003b80 con 0x7f238815fe10 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2380003d20 con 0x7f238815fe10 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238815dc70 con 0x7f238815fe10 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380005330 con 0x7f238815fe10 2024-09-17T05:32:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238809de00 con 0x7f238815fe10 2024-09-17T05:32:30.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23880700a0 con 0x7f238815fe10 2024-09-17T05:32:30.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.479+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23800054d0 con 0x7f238815fe10 2024-09-17T05:32:30.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.483+0000 7f237f400640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 0x7f2368075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.483+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f23800945f0 con 0x7f238815fe10 2024-09-17T05:32:30.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.483+0000 7f2387400640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 0x7f2368075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.483+0000 7f2387400640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 0x7f2368075710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2378000c10 tx=0x7f2378001800 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.487+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f238005e800 con 0x7f238815fe10 2024-09-17T05:32:30.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.695+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23880b00c0 con 0x7f238815fe10 2024-09-17T05:32:30.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.695+0000 7f237f400640 1 -- 172.21.15.43:0/4279302601 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 240 v240) ==== 95+0+3557 (secure 0 0 0) 0x7f2380063230 con 0x7f238815fe10 2024-09-17T05:32:30.698 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:30.698 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":240,"btime":"2024-09-17T05:32:30:300279+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:30.300188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 240 2024-09-17T05:32:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 msgr2=0x7f2368075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 0x7f2368075710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2378000c10 tx=0x7f2378001800 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 msgr2=0x7f23880c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f23880d01d0 tx=0x7f238000cad0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 shutdown_connections 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2368073200 0x7f2368075710 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23880730b0 0x7f23880c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f238815fe10 0x7f23880c8fb0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 --2- 172.21.15.43:0/4279302601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f238806fc90 0x7f238809d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 >> 172.21.15.43:0/4279302601 conn(0x7f23880b23d0 msgr2=0x7f23880b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 shutdown_connections 2024-09-17T05:32:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.699+0000 7f238e600640 1 -- 172.21.15.43:0/4279302601 wait complete. 2024-09-17T05:32:30.721 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:30.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:30.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- 172.21.15.43:0/1646490726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f406f110 msgr2=0x7fd8f40b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/1646490726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f406f110 0x7fd8f40b33e0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009840 tx=0x7fd8dc01c760 comp rx=0 tx=0).stop 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- 172.21.15.43:0/1646490726 shutdown_connections 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/1646490726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f406f110 0x7fd8f40b33e0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/1646490726 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f4155d70 0x7fd8f415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- 172.21.15.43:0/1646490726 >> 172.21.15.43:0/1646490726 conn(0x7fd8f40b23d0 msgr2=0x7fd8f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- 172.21.15.43:0/1646490726 shutdown_connections 2024-09-17T05:32:30.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- 172.21.15.43:0/1646490726 wait complete. 2024-09-17T05:32:30.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 Processor -- start 2024-09-17T05:32:30.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.879+0000 7fd8f8a00640 1 -- start start 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f406f110 0x7fd8f40c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f40b3920 0x7fd8f40cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4168b10 con 0x7fd8f4155d70 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4168990 con 0x7fd8f406f110 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f4168c90 con 0x7fd8f40b3920 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45968/0 (socket says 172.21.15.43:45968) 2024-09-17T05:32:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 -- 172.21.15.43:0/4293149523 learned_addr learned my addr 172.21.15.43:0/4293149523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8ebe00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f40b3920 0x7fd8f40cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f406f110 0x7fd8f40c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 -- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f40b3920 msgr2=0x7fd8f40cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f40b3920 0x7fd8f40cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 -- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f406f110 msgr2=0x7fd8f40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f406f110 0x7fd8f40c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 -- 172.21.15.43:0/4293149523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f41f9220 con 0x7fd8f4155d70 2024-09-17T05:32:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f2800640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c760 tx=0x7fd8ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec003db0 con 0x7fd8f4155d70 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8ec00ce10 con 0x7fd8f4155d70 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0052d0 con 0x7fd8f4155d70 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f41f9500 con 0x7fd8f4155d70 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41f9ac0 con 0x7fd8f4155d70 2024-09-17T05:32:30.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.883+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f40b33e0 con 0x7fd8f4155d70 2024-09-17T05:32:30.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.887+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8ec005470 con 0x7fd8f4155d70 2024-09-17T05:32:30.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.887+0000 7fd8ea000640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 0x7fd8d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.887+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7fd8ec094780 con 0x7fd8f4155d70 2024-09-17T05:32:30.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.887+0000 7fd8f1e00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 0x7fd8d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.887+0000 7fd8f1e00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 0x7fd8d0075740 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40715c0 tx=0x7fd8e40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:30.891+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8f40b3d30 con 0x7fd8f4155d70 2024-09-17T05:32:31.011 INFO:tasks.ceph.mds.d.smithi062.stderr: -38> 2024-09-17T05:32:26.256+0000 7fb652600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-17T05:32:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.099+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8f40ad250 con 0x7fd8f4155d70 2024-09-17T05:32:31.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.099+0000 7fd8ea000640 1 -- 172.21.15.43:0/4293149523 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 240 v240) ==== 95+0+3557 (secure 0 0 0) 0x7fd8ec05e880 con 0x7fd8f4155d70 2024-09-17T05:32:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:31.103 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":240,"btime":"2024-09-17T05:32:30:300279+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:30.300188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:31.103 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 240 2024-09-17T05:32:31.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 msgr2=0x7fd8d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 0x7fd8d0075740 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40715c0 tx=0x7fd8e40091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 msgr2=0x7fd8f40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c760 tx=0x7fd8ec00cc20 comp rx=0 tx=0).stop 2024-09-17T05:32:31.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 shutdown_connections 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8d0073230 0x7fd8d0075740 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4155d70 0x7fd8f40cfc30 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd8f40b3920 0x7fd8f40cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 --2- 172.21.15.43:0/4293149523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8f406f110 0x7fd8f40c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 >> 172.21.15.43:0/4293149523 conn(0x7fd8f40b23d0 msgr2=0x7fd8f4165a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 shutdown_connections 2024-09-17T05:32:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.103+0000 7fd8f8a00640 1 -- 172.21.15.43:0/4293149523 wait complete. 2024-09-17T05:32:31.126 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:31.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:31.132 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- 172.21.15.43:0/3877079300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc155d70 msgr2=0x7f77cc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 --2- 172.21.15.43:0/3877079300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc155d70 0x7f77cc15c7e0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0d0c40 tx=0x7f77c001c740 comp rx=0 tx=0).stop 2024-09-17T05:32:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- 172.21.15.43:0/3877079300 shutdown_connections 2024-09-17T05:32:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 --2- 172.21.15.43:0/3877079300 >> v2:172.21.15.62:3300/0 conn(0x7f77cc0b3af0 0x7f77cc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 --2- 172.21.15.43:0/3877079300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc155d70 0x7f77cc15c7e0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- 172.21.15.43:0/3877079300 >> 172.21.15.43:0/3877079300 conn(0x7f77cc0b23d0 msgr2=0x7f77cc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- 172.21.15.43:0/3877079300 shutdown_connections 2024-09-17T05:32:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- 172.21.15.43:0/3877079300 wait complete. 2024-09-17T05:32:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 Processor -- start 2024-09-17T05:32:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.283+0000 7f77d3e00640 1 -- start start 2024-09-17T05:32:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 0x7f77cc163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 0x7f77cc160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc168c30 con 0x7f77cc06f110 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc168ab0 con 0x7f77cc09c5f0 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc168db0 con 0x7f77cc155d70 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45982/0 (socket says 172.21.15.43:45982) 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 -- 172.21.15.43:0/1207978315 learned_addr learned my addr 172.21.15.43:0/1207978315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 0x7f77cc160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d0a00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 0x7f77cc163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 -- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 msgr2=0x7f77cc163ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 0x7f77cc163ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 -- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 msgr2=0x7f77cc160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 0x7f77cc160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 -- 172.21.15.43:0/1207978315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc162200 con 0x7f77cc06f110 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d0a00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 0x7f77cc163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 0x7f77cc160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d1400640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c10 tx=0x7f77c00a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c00bb070 con 0x7f77cc06f110 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c0004470 con 0x7f77cc06f110 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c00b2040 con 0x7f77cc06f110 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc1624e0 con 0x7f77cc06f110 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.287+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc160890 con 0x7f77cc06f110 2024-09-17T05:32:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77cc0ad0b0 con 0x7f77cc06f110 2024-09-17T05:32:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77c0005230 con 0x7f77cc06f110 2024-09-17T05:32:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77c9600640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 0x7f77ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f77c012e4e0 con 0x7f77cc06f110 2024-09-17T05:32:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77d0a00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 0x7f77ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.291+0000 7f77d0a00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 0x7f77ac075740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000f70 tx=0x7f77b8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.295+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77c00163f0 con 0x7f77cc06f110 2024-09-17T05:32:31.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77cc0ca630 con 0x7f77cc06f110 2024-09-17T05:32:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77c9600640 1 -- 172.21.15.43:0/1207978315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 241 v241) ==== 95+0+4383 (secure 0 0 0) 0x7f77c00ff030 con 0x7f77cc06f110 2024-09-17T05:32:31.514 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:31.514 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":241,"btime":"2024-09-17T05:32:31:302167+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8719,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2354591248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2354591248},{"type":"v1","addr":"172.21.15.62:6835","nonce":2354591248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:30.300188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 241 2024-09-17T05:32:31.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 msgr2=0x7f77ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 0x7f77ac075740 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000f70 tx=0x7f77b8009040 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 msgr2=0x7f77cc09bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c10 tx=0x7f77c00a5680 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 shutdown_connections 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f77ac073230 0x7f77ac075740 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77cc09c5f0 0x7f77cc160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f77cc155d70 0x7f77cc163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 --2- 172.21.15.43:0/1207978315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc06f110 0x7f77cc09bee0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.511+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 >> 172.21.15.43:0/1207978315 conn(0x7f77cc0b23d0 msgr2=0x7f77cc15e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.515+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 shutdown_connections 2024-09-17T05:32:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.515+0000 7f77d3e00640 1 -- 172.21.15.43:0/1207978315 wait complete. 2024-09-17T05:32:31.536 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-09-17T05:32:31.536 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-09-17T05:32:31.536 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:31.536 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:31.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:31.545 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:31.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:31.545 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:31.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:31.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 -- 172.21.15.43:0/817475519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0b69f0 msgr2=0x7fbe8c0ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 --2- 172.21.15.43:0/817475519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c0ae160 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0b0130 tx=0x7fbe7401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 -- 172.21.15.43:0/817475519 shutdown_connections 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 --2- 172.21.15.43:0/817475519 >> v2:172.21.15.62:3300/0 conn(0x7fbe8c0ae6a0 0x7fbe8c0d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 --2- 172.21.15.43:0/817475519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c0ae160 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 -- 172.21.15.43:0/817475519 >> 172.21.15.43:0/817475519 conn(0x7fbe8c0b2e80 msgr2=0x7fbe8c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 -- 172.21.15.43:0/817475519 shutdown_connections 2024-09-17T05:32:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.795+0000 7fbe91a00640 1 -- 172.21.15.43:0/817475519 wait complete. 2024-09-17T05:32:31.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 Processor -- start 2024-09-17T05:32:31.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 -- start start 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 0x7fbe8c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c16c3f0 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c16c270 con 0x7fbe8c0b69f0 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe91a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c16c570 con 0x7fbe8c15ee30 2024-09-17T05:32:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe8aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe8aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46002/0 (socket says 172.21.15.43:46002) 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.799+0000 7fbe8aa00640 1 -- 172.21.15.43:0/2453543880 learned_addr learned my addr 172.21.15.43:0/2453543880 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8b400640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 0x7fbe8c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe83e00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 -- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 msgr2=0x7fbe8c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 0x7fbe8c0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 -- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 msgr2=0x7fbe8c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 -- 172.21.15.43:0/2453543880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c15aa50 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe83e00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8b400640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 0x7fbe8c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe8aa00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c3c0 tx=0x7fbe7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017070 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe7c0053c0 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c003860 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c15d390 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c15d950 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.803+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c0d0470 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.807+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe7c01a070 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.807+0000 7fbe82a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 0x7fbe68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.807+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7fbe7c094200 con 0x7fbe8c0ae6a0 2024-09-17T05:32:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.807+0000 7fbe83e00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 0x7fbe68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.807+0000 7fbe83e00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 0x7fbe68075710 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbe740af000 tx=0x7fbe7401b100 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:31.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:31.811+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe7c021020 con 0x7fbe8c0ae6a0 2024-09-17T05:32:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.019+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbe8c152d10 con 0x7fbe8c0ae6a0 2024-09-17T05:32:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe82a00640 1 -- 172.21.15.43:0/2453543880 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v242) ==== 161+0+0 (secure 0 0 0) 0x7fbe8c15aa50 con 0x7fbe8c0ae6a0 2024-09-17T05:32:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 msgr2=0x7fbe68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 0x7fbe68075710 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbe740af000 tx=0x7fbe7401b100 comp rx=0 tx=0).stop 2024-09-17T05:32:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 msgr2=0x7fbe8c152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c3c0 tx=0x7fbe7c00c880 comp rx=0 tx=0).stop 2024-09-17T05:32:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 shutdown_connections 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbe68073200 0x7fbe68075710 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe8c0b69f0 0x7fbe8c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbe8c15ee30 0x7fbe8c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 --2- 172.21.15.43:0/2453543880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0ae6a0 0x7fbe8c152770 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 >> 172.21.15.43:0/2453543880 conn(0x7fbe8c0b2e80 msgr2=0x7fbe8c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 shutdown_connections 2024-09-17T05:32:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.307+0000 7fbe91a00640 1 -- 172.21.15.43:0/2453543880 wait complete. 2024-09-17T05:32:32.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 -- 172.21.15.43:0/2634974485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 msgr2=0x7faca4004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 --2- 172.21.15.43:0/2634974485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7faca4004150 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7facb0001860 tx=0x7facb001c880 comp rx=0 tx=0).stop 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facbf200640 1 -- 172.21.15.43:0/2634974485 reap_dead start 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 -- 172.21.15.43:0/2634974485 shutdown_connections 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 -- 172.21.15.43:0/2634974485 >> 172.21.15.43:0/2634974485 conn(0x7facc00b23d0 msgr2=0x7facc00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 -- 172.21.15.43:0/2634974485 shutdown_connections 2024-09-17T05:32:32.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.491+0000 7facc5400640 1 -- 172.21.15.43:0/2634974485 wait complete. 2024-09-17T05:32:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 Processor -- start 2024-09-17T05:32:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- start start 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 0x7facc0162d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc00b08f0 0x7facc00b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7facc006d700 con 0x7faca4003d00 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7facc006d580 con 0x7facc00b08f0 2024-09-17T05:32:32.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7facc00ce100 con 0x7facc0162930 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46020/0 (socket says 172.21.15.43:46020) 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 -- 172.21.15.43:0/864272419 learned_addr learned my addr 172.21.15.43:0/864272419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facb7e00640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 0x7facc0162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 -- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 msgr2=0x7facc0162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 0x7facc0162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 -- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc00b08f0 msgr2=0x7facc00b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc00b08f0 0x7facc00b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 -- 172.21.15.43:0/864272419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc00b3810 con 0x7faca4003d00 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facb7e00640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 0x7facc0162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facbe800640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7facb0001860 tx=0x7facb0003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0001480 con 0x7faca4003d00 2024-09-17T05:32:32.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb0006090 con 0x7faca4003d00 2024-09-17T05:32:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc00c9ff0 con 0x7faca4003d00 2024-09-17T05:32:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0006420 con 0x7faca4003d00 2024-09-17T05:32:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.495+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01f6450 con 0x7faca4003d00 2024-09-17T05:32:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc01f99b0 con 0x7faca4003d00 2024-09-17T05:32:32.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7facb00a6510 con 0x7faca4003d00 2024-09-17T05:32:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facb6a00640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 0x7fac9c075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7facb012edd0 con 0x7faca4003d00 2024-09-17T05:32:32.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facb7e00640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 0x7fac9c075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.499+0000 7facb7e00640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 0x7fac9c075730 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7facc00cb9d0 tx=0x7faca80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.503+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7facb00aa070 con 0x7faca4003d00 2024-09-17T05:32:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:32.743+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7facc00c80f0 con 0x7faca4003d00 2024-09-17T05:32:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.307+0000 7facb6a00640 1 -- 172.21.15.43:0/864272419 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v243) ==== 101+0+0 (secure 0 0 0) 0x7facb0016690 con 0x7faca4003d00 2024-09-17T05:32:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 msgr2=0x7fac9c075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 0x7fac9c075730 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7facc00cb9d0 tx=0x7faca80091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 msgr2=0x7facc01f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7facb0001860 tx=0x7facb0003be0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 shutdown_connections 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fac9c073220 0x7fac9c075730 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc00b08f0 0x7facc00b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7facc0162930 0x7facc0162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 --2- 172.21.15.43:0/864272419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca4003d00 0x7facc01f5f10 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 >> 172.21.15.43:0/864272419 conn(0x7facc00b23d0 msgr2=0x7facc00ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 shutdown_connections 2024-09-17T05:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.311+0000 7facc5400640 1 -- 172.21.15.43:0/864272419 wait complete. 2024-09-17T05:32:33.338 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.395+0000 7f22a8469080 1 Processor -- start 2024-09-17T05:32:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.395+0000 7f22a8469080 1 -- start start 2024-09-17T05:32:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56101514a9c0 0x56101521ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x56101521b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610151ff9e0 con 0x5610151fc9c0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561015093a00 con 0x5610151fc5b0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5610151ffb60 con 0x56101514a9c0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46050/0 (socket says 172.21.15.43:46050) 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 -- 172.21.15.43:0/1011321156 learned_addr learned my addr 172.21.15.43:0/1011321156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a1600640 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56101514a9c0 0x56101521ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a0c00640 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x56101521b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 -- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56101514a9c0 msgr2=0x56101521ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56101514a9c0 0x56101521ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 -- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 msgr2=0x56101521b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x56101521b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 -- 172.21.15.43:0/1011321156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561015092a20 con 0x5610151fc9c0 2024-09-17T05:32:33.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a2000640 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f229c00a550 tx=0x7f229c018230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e697b0f647733e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f229b400640 1 -- 172.21.15.43:0/1011321156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0ba020 con 0x5610151fc9c0 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f229b400640 1 -- 172.21.15.43:0/1011321156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f229c0b0040 con 0x5610151fc9c0 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f229b400640 1 -- 172.21.15.43:0/1011321156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0af070 con 0x5610151fc9c0 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- 172.21.15.43:0/1011321156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 msgr2=0x5610151f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f229c00a550 tx=0x7f229c018230 comp rx=0 tx=0).stop 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- 172.21.15.43:0/1011321156 shutdown_connections 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x56101521b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610151fc9c0 0x5610151f72e0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 --2- 172.21.15.43:0/1011321156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56101514a9c0 0x56101521ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- 172.21.15.43:0/1011321156 >> 172.21.15.43:0/1011321156 conn(0x561015131d80 msgr2=0x561015187dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- 172.21.15.43:0/1011321156 shutdown_connections 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.399+0000 7f22a8469080 1 -- 172.21.15.43:0/1011321156 wait complete. 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 Processor -- start 2024-09-17T05:32:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- start start 2024-09-17T05:32:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x5610152b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610151fc9c0 0x561015218980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610151fbdc0 con 0x56101514a9c0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561015188690 con 0x5610151fc5b0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561015186ac0 con 0x5610151fc9c0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46060/0 (socket says 172.21.15.43:46060) 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 learned_addr learned my addr 172.21.15.43:0/2981569586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x5610152b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a2000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610151fc9c0 0x561015218980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610151fc9c0 msgr2=0x561015218980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610151fc9c0 0x561015218980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 msgr2=0x5610152b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x5610152b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101506cc10 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610151fc5b0 0x5610152b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2294008780 tx=0x7f2294008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f229a000640 1 -- 172.21.15.43:0/2981569586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229400abc0 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f229a000640 1 -- 172.21.15.43:0/2981569586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f229400ad60 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56101518f5b0 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f229a000640 1 -- 172.21.15.43:0/2981569586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294015680 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.403+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101518d570 con 0x56101514a9c0 2024-09-17T05:32:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f229a000640 1 -- 172.21.15.43:0/2981569586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2294015820 con 0x56101514a9c0 2024-09-17T05:32:33.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f229a000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 0x7f2284074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f229a000640 1 -- 172.21.15.43:0/2981569586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f2294093b90 con 0x56101514a9c0 2024-09-17T05:32:33.411 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:33.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 0x7f2284074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56101518dcb0 0x56101518e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.407+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56101518e600 con 0x56101518dcb0 2024-09-17T05:32:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.411+0000 7f22a2000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56101518dcb0 0x56101518e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.411+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 0x7f2284074120 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2290004fd0 tx=0x7f229000e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.411+0000 7f22a2000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56101518dcb0 0x56101518e0c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.411+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v115'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.411+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56101518e9d0 con 0x56101518dcb0 2024-09-17T05:32:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v115'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152bf4c0 con 0x56101518dcb0 2024-09-17T05:32:33.418 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:33.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v115'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0870 con 0x56101518dcb0 2024-09-17T05:32:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.415+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0c10 con 0x56101518dcb0 2024-09-17T05:32:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.419+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.419+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152bf860 con 0x56101518dcb0 2024-09-17T05:32:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.419+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.419+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152bfc00 con 0x56101518dcb0 2024-09-17T05:32:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.423+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v115'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.423+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152bffa0 con 0x56101518dcb0 2024-09-17T05:32:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.423+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v115'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.423+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v115'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.428 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:33.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.427+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.431+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.431+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.431+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.431+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.435+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.435+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.435+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.438 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:33.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.435+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.439+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.443+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.443+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.443+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v115'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.443+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.447+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v115'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.447+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.449 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.447+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v115'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.447+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.447+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.451+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.451+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.451+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.451+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.451+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.455+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v115'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.455+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.455+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v115'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.455+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.459 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v115'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.459+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x56101518dcb0 2024-09-17T05:32:33.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5610152c14c0 0x5610152c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c0340 con 0x5610152c14c0 2024-09-17T05:32:33.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5610152c14c0 0x5610152c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5610152c14c0 0x5610152c38e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'570 uv552 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.463+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c3e20 con 0x5610152c14c0 2024-09-17T05:32:33.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.467+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'571 uv571 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.467+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c41c0 con 0x5610152c14c0 2024-09-17T05:32:33.470 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:33.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.467+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.467+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c4a10 con 0x5610152c14c0 2024-09-17T05:32:33.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.471+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.471+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c4db0 con 0x5610152c14c0 2024-09-17T05:32:33.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.471+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.471+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5150 con 0x5610152c14c0 2024-09-17T05:32:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v115'575 uv557 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c54f0 con 0x5610152c14c0 2024-09-17T05:32:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v115'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5890 con 0x5610152c14c0 2024-09-17T05:32:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v115'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.475+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5c80 con 0x5610152c14c0 2024-09-17T05:32:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.479+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.479+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5c80 con 0x5610152c14c0 2024-09-17T05:32:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.479+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.479+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v115'580 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v115'581 uv581 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.486 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v115'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.483+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c9b40 con 0x5610152c14c0 2024-09-17T05:32:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v115'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c9b40 con 0x5610152c14c0 2024-09-17T05:32:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v115'585 uv568 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.487+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v115'586 uv586 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c5970 con 0x5610152c14c0 2024-09-17T05:32:33.493 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v115'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c69a0 con 0x5610152c14c0 2024-09-17T05:32:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.491+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c69a0 con 0x5610152c14c0 2024-09-17T05:32:33.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c69a0 con 0x5610152c14c0 2024-09-17T05:32:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152c14c0 2024-09-17T05:32:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5610152c5e80 0x5610152c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c8db0 con 0x5610152c5e80 2024-09-17T05:32:33.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.495+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5610152c5e80 0x5610152c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.499+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5610152c5e80 0x5610152c6290 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.499+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v115'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.499+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152c9150 con 0x5610152c5e80 2024-09-17T05:32:33.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.499+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v115'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.499+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c4650 con 0x5610152c5e80 2024-09-17T05:32:33.503 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:33.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v115'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152caf30 con 0x5610152c5e80 2024-09-17T05:32:33.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cb2d0 con 0x5610152c5e80 2024-09-17T05:32:33.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.503+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152c9f20 con 0x5610152c5e80 2024-09-17T05:32:33.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152ca310 con 0x56101518dcb0 2024-09-17T05:32:33.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'109 uv108 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152ca310 con 0x56101518dcb0 2024-09-17T05:32:33.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.507+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152ca310 con 0x56101518dcb0 2024-09-17T05:32:33.511 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:33.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152ca310 con 0x56101518dcb0 2024-09-17T05:32:33.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 0x5610152cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cdfd0 con 0x5610152cb670 2024-09-17T05:32:33.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a2000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 0x5610152cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.511+0000 7f22a2000640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 0x5610152cda90 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v115'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152cdfd0 con 0x5610152cb670 2024-09-17T05:32:33.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v115'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cdfd0 con 0x5610152cb670 2024-09-17T05:32:33.518 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v115'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.515+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cefe0 con 0x5610152cb670 2024-09-17T05:32:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cefe0 con 0x5610152cb670 2024-09-17T05:32:33.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152cdfd0 con 0x5610152cb670 2024-09-17T05:32:33.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610152ce650 0x5610152d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d0fb0 con 0x5610152ce650 2024-09-17T05:32:33.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610152ce650 0x5610152d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.519+0000 7f22a1600640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610152ce650 0x5610152d0a70 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v115'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v115'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v115'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.523+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d2390 con 0x5610152ce650 2024-09-17T05:32:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d2390 con 0x5610152ce650 2024-09-17T05:32:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.527+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.531+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v115'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.531+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.531+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v115'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.531+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.535 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:33.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v115'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d1380 con 0x5610152ce650 2024-09-17T05:32:33.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a1600640 1 -- 172.21.15.43:0/2981569586 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2294059f90 con 0x5610152ce650 2024-09-17T05:32:33.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610152d1a00 0x5610152d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4490 con 0x5610152d1a00 2024-09-17T05:32:33.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610152d1a00 0x5610152d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.535+0000 7f22a0c00640 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610152d1a00 0x5610152d3e20 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.539+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v115'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.539+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d4860 con 0x5610152d1a00 2024-09-17T05:32:33.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.539+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v115'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.539+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152d1a00 2024-09-17T05:32:33.543 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v115'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152d1a00 2024-09-17T05:32:33.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152d1a00 2024-09-17T05:32:33.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.543+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x56101518dcb0 2024-09-17T05:32:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.547+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v115'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.547+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d4c80 con 0x56101518dcb0 2024-09-17T05:32:33.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.547+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v115'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.547+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x56101518dcb0 2024-09-17T05:32:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v115'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d5c90 con 0x56101518dcb0 2024-09-17T05:32:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d5c90 con 0x56101518dcb0 2024-09-17T05:32:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.551+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x56101518dcb0 2024-09-17T05:32:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x56101518dcb0 2024-09-17T05:32:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152c5e80 2024-09-17T05:32:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v115'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152c5e80 2024-09-17T05:32:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v115'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.555+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4c80 con 0x5610152c5e80 2024-09-17T05:32:33.560 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v115'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d50c0 con 0x5610152c5e80 2024-09-17T05:32:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4d70 con 0x5610152c5e80 2024-09-17T05:32:33.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.559+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4d70 con 0x5610152c5e80 2024-09-17T05:32:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.563+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v115'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.563+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d4d20 con 0x5610152c5e80 2024-09-17T05:32:33.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.563+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v115'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.563+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4d20 con 0x5610152c5e80 2024-09-17T05:32:33.567 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.567+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v115'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.567+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d4d20 con 0x5610152c5e80 2024-09-17T05:32:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.567+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.567+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7b10 con 0x5610152c5e80 2024-09-17T05:32:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152c5e80 2024-09-17T05:32:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.571+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.577 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.575+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v115'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.575+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.575+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.575+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v115'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v115'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v115'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.583 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.579+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ad0 con 0x5610152cb670 2024-09-17T05:32:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152cb670 2024-09-17T05:32:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152cb670 2024-09-17T05:32:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f229c002050 con 0x5610152cb670 2024-09-17T05:32:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.583+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v115'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v115'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.590 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v115'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.587+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.591+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.591+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.591+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.591+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v115'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v115'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.598 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v115'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.595+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.599+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.599+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.599+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.599+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v115'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v115'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d7ab0 con 0x5610152d1a00 2024-09-17T05:32:33.606 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v115'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.603+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d6090 con 0x5610152d1a00 2024-09-17T05:32:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d6090 con 0x5610152d1a00 2024-09-17T05:32:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d5080 con 0x5610152d1a00 2024-09-17T05:32:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.607+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x5610152d5470 con 0x5610152d1a00 2024-09-17T05:32:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a0c00640 1 -- 172.21.15.43:0/2981569586 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2290008c50 con 0x5610152d1a00 2024-09-17T05:32:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5610152c14c0 msgr2=0x5610152c38e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x5610152c14c0 0x5610152c38e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610152ce650 msgr2=0x5610152d0a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610152ce650 0x5610152d0a70 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610152d1a00 msgr2=0x5610152d3e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610152d1a00 0x5610152d3e20 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5610152c5e80 msgr2=0x5610152c6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5610152c5e80 0x5610152c6290 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 reap_dead start 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56101518dcb0 msgr2=0x56101518e0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56101518dcb0 0x56101518e0c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a2000640 1 -- 172.21.15.43:0/2981569586 reap_dead start 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 msgr2=0x5610152cda90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 0x5610152cda90 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 msgr2=0x7f2284074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 0x7f2284074120 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2290004fd0 tx=0x7f229000e040 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 msgr2=0x56101518ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2294008780 tx=0x7f2294008c40 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 shutdown_connections 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610152cb670 0x5610152cda90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2284071c10 0x7f2284074120 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 --2- 172.21.15.43:0/2981569586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56101514a9c0 0x56101518ef10 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 >> 172.21.15.43:0/2981569586 conn(0x561015131d80 msgr2=0x561015185810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 shutdown_connections 2024-09-17T05:32:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.611+0000 7f22a8469080 1 -- 172.21.15.43:0/2981569586 wait complete. 2024-09-17T05:32:33.617 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-09-17T05:32:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 Processor -- start 2024-09-17T05:32:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 -- start start 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b720cc360 0x556b721ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 0x556b720cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.719+0000 7f592b4da080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556b7201aa00 con 0x556b721854d0 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556b72078f40 con 0x556b720cc360 2024-09-17T05:32:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556b72187b00 con 0x556b721169e0 2024-09-17T05:32:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5923c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 0x556b720cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5923c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 0x556b720cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:46730/0 (socket says 172.21.15.43:46730) 2024-09-17T05:32:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5923c00640 1 -- 172.21.15.43:0/1965021338 learned_addr learned my addr 172.21.15.43:0/1965021338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48880/0 (socket says 172.21.15.43:48880) 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5924600640 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b720cc360 0x556b721ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 -- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 msgr2=0x556b720cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 0x556b720cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 -- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b720cc360 msgr2=0x556b721ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b720cc360 0x556b721ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 -- 172.21.15.43:0/1965021338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b71ff3c10 con 0x556b721854d0 2024-09-17T05:32:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5925000640 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f591c00a9d0 tx=0x7f591c00ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b973b46b50c7f4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5922800640 1 -- 172.21.15.43:0/1965021338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c00dbf0 con 0x556b721854d0 2024-09-17T05:32:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5922800640 1 -- 172.21.15.43:0/1965021338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c00dd90 con 0x556b721854d0 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f5922800640 1 -- 172.21.15.43:0/1965021338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c00f690 con 0x556b721854d0 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- 172.21.15.43:0/1965021338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 msgr2=0x556b72182cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f591c00a9d0 tx=0x7f591c00ae90 comp rx=0 tx=0).stop 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- 172.21.15.43:0/1965021338 shutdown_connections 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b72182cf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b721169e0 0x556b720cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 --2- 172.21.15.43:0/1965021338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b720cc360 0x556b721ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- 172.21.15.43:0/1965021338 >> 172.21.15.43:0/1965021338 conn(0x556b720b8d80 msgr2=0x556b721849a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- 172.21.15.43:0/1965021338 shutdown_connections 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- 172.21.15.43:0/1965021338 wait complete. 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 Processor -- start 2024-09-17T05:32:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.723+0000 7f592b4da080 1 -- start start 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b720cc360 0x556b721826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b721169e0 0x556b7217db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b7223f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556b72114b70 con 0x556b721854d0 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556b721acc10 con 0x556b721169e0 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556b72114e10 con 0x556b720cc360 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b7223f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b7223f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48882/0 (socket says 172.21.15.43:48882) 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 -- 172.21.15.43:0/1473116570 learned_addr learned my addr 172.21.15.43:0/1473116570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5923c00640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b721169e0 0x556b7217db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5924600640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b720cc360 0x556b721826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 -- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b720cc360 msgr2=0x556b721826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556b720cc360 0x556b721826b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 -- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b721169e0 msgr2=0x556b7217db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b721169e0 0x556b7217db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 -- 172.21.15.43:0/1473116570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b72240240 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5923c00640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556b721169e0 0x556b7217db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5925000640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b7223f470 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f591c00f6e0 tx=0x7f591c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5921400640 1 -- 172.21.15.43:0/1473116570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c023020 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5921400640 1 -- 172.21.15.43:0/1473116570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c00fc50 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f5921400640 1 -- 172.21.15.43:0/1473116570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c01f660 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b7218bc10 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.727+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7218beb0 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5921400640 1 -- 172.21.15.43:0/1473116570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f591c026020 con 0x556b721854d0 2024-09-17T05:32:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5921400640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5904071be0 0x7f59040740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5921400640 1 -- 172.21.15.43:0/1473116570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f591c0962b0 con 0x556b721854d0 2024-09-17T05:32:33.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5924600640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5904071be0 0x7f59040740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f592b4da080 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556b7217ff70 0x556b72180380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x556b721808c0 con 0x556b7217ff70 2024-09-17T05:32:33.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5923c00640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556b7217ff70 0x556b72180380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5924600640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5904071be0 0x7f59040740f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5914000c10 tx=0x7f5914001780 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.731+0000 7f5923c00640 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556b7217ff70 0x556b72180380 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f5923c00640 1 -- 172.21.15.43:0/1473116570 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v115'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59100aba80 con 0x556b7217ff70 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556b7217ff70 msgr2=0x556b72180380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556b7217ff70 0x556b72180380 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5904071be0 msgr2=0x7f59040740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5904071be0 0x7f59040740f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5914000c10 tx=0x7f5914001780 comp rx=0 tx=0).stop 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 msgr2=0x556b7223f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 --2- 172.21.15.43:0/1473116570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556b721854d0 0x556b7223f470 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f591c00f6e0 tx=0x7f591c00f710 comp rx=0 tx=0).stop 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f5925000640 1 -- 172.21.15.43:0/1473116570 reap_dead start 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 shutdown_connections 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 >> 172.21.15.43:0/1473116570 conn(0x556b720b8d80 msgr2=0x556b720cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 shutdown_connections 2024-09-17T05:32:33.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.735+0000 7f592b4da080 1 -- 172.21.15.43:0/1473116570 wait complete. 2024-09-17T05:32:33.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 -- 172.21.15.43:0/2293574912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec0b5d20 msgr2=0x7fc4ec0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 --2- 172.21.15.43:0/2293574912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec0b5d20 0x7fc4ec0d0390 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009840 tx=0x7fc4d801c700 comp rx=0 tx=0).stop 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 -- 172.21.15.43:0/2293574912 shutdown_connections 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 --2- 172.21.15.43:0/2293574912 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec06fce0 0x7fc4ec0700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 --2- 172.21.15.43:0/2293574912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec0b5d20 0x7fc4ec0d0390 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 -- 172.21.15.43:0/2293574912 >> 172.21.15.43:0/2293574912 conn(0x7fc4ec0b2810 msgr2=0x7fc4ec0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 -- 172.21.15.43:0/2293574912 shutdown_connections 2024-09-17T05:32:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.899+0000 7fc4f4000640 1 -- 172.21.15.43:0/2293574912 wait complete. 2024-09-17T05:32:33.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 Processor -- start 2024-09-17T05:32:33.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 -- start start 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4ec06fce0 0x7fc4ec161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec15b200 0x7fc4ec13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec169580 con 0x7fc4ec15f470 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec169400 con 0x7fc4ec06fce0 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f4000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec169700 con 0x7fc4ec15b200 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48898/0 (socket says 172.21.15.43:48898) 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 -- 172.21.15.43:0/56103341 learned_addr learned my addr 172.21.15.43:0/56103341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f1600640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4ec06fce0 0x7fc4ec161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f2000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec15b200 0x7fc4ec13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 -- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec15b200 msgr2=0x7fc4ec13ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec15b200 0x7fc4ec13ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 -- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4ec06fce0 msgr2=0x7fc4ec161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4ec06fce0 0x7fc4ec161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 -- 172.21.15.43:0/56103341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec162560 con 0x7fc4ec15f470 2024-09-17T05:32:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4f0c00640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc4d801cbe0 tx=0x7fc4d80a8430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.903+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80be070 con 0x7fc4ec15f470 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d80a8a70 con 0x7fc4ec15f470 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec162840 con 0x7fc4ec15f470 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8005040 con 0x7fc4ec15f470 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec162e00 con 0x7fc4ec15f470 2024-09-17T05:32:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec0700f0 con 0x7fc4ec15f470 2024-09-17T05:32:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4d80c1070 con 0x7fc4ec15f470 2024-09-17T05:32:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4e9600640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 0x7fc4cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7fc4d812ecb0 con 0x7fc4ec15f470 2024-09-17T05:32:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.907+0000 7fc4f1600640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 0x7fc4cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.911+0000 7fc4f1600640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 0x7fc4cc075710 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec069610 tx=0x7fc4e00013a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:33.911+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4d80ac030 con 0x7fc4ec15f470 2024-09-17T05:32:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.123+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc4ec1622f0 con 0x7fc4ec15f470 2024-09-17T05:32:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.319+0000 7fc4e9600640 1 -- 172.21.15.43:0/56103341 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v244) ==== 162+0+0 (secure 0 0 0) 0x7fc4d8016410 con 0x7fc4ec15f470 2024-09-17T05:32:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 msgr2=0x7fc4cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 0x7fc4cc075710 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec069610 tx=0x7fc4e00013a0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 msgr2=0x7fc4ec15acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fc4d801cbe0 tx=0x7fc4d80a8430 comp rx=0 tx=0).stop 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 shutdown_connections 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4cc073200 0x7fc4cc075710 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4ec15b200 0x7fc4ec13ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ec15f470 0x7fc4ec15acc0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 --2- 172.21.15.43:0/56103341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4ec06fce0 0x7fc4ec161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 >> 172.21.15.43:0/56103341 conn(0x7fc4ec0b2810 msgr2=0x7fc4ec152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 shutdown_connections 2024-09-17T05:32:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.323+0000 7fc4f4000640 1 -- 172.21.15.43:0/56103341 wait complete. 2024-09-17T05:32:34.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- 172.21.15.43:0/3456470432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f44003d00 msgr2=0x7f9f44004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 --2- 172.21.15.43:0/3456470432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f44003d00 0x7f9f44004150 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001890 tx=0x7f9f5001c880 comp rx=0 tx=0).stop 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f61e00640 1 -- 172.21.15.43:0/3456470432 reap_dead start 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- 172.21.15.43:0/3456470432 shutdown_connections 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- 172.21.15.43:0/3456470432 >> 172.21.15.43:0/3456470432 conn(0x7f9f5c0cee80 msgr2=0x7f9f5c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- 172.21.15.43:0/3456470432 shutdown_connections 2024-09-17T05:32:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- 172.21.15.43:0/3456470432 wait complete. 2024-09-17T05:32:34.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 Processor -- start 2024-09-17T05:32:34.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.911+0000 7f9f63e00640 1 -- start start 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 0x7f9f5c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 0x7f9f5c1627d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c168e10 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c168c90 con 0x7f9f44003d00 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c168f90 con 0x7f9f5c1623c0 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48916/0 (socket says 172.21.15.43:48916) 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 -- 172.21.15.43:0/2447548175 learned_addr learned my addr 172.21.15.43:0/2447548175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f61e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 0x7f9f5c1627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 -- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 msgr2=0x7f9f5c1627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f61400640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 0x7f9f5c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 0x7f9f5c1627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 -- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 msgr2=0x7f9f5c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 0x7f9f5c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 -- 172.21.15.43:0/2447548175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c162d10 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f61e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 0x7f9f5c1627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f61400640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 0x7f9f5c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f60a00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9f48011ae0 tx=0x7f9f48009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4801e070 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f48004460 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c06e870 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800fbe0 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c06ee30 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.915+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c06e5e0 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.919+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f48021070 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.919+0000 7f9f59600640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 0x7f9f3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.919+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f9f48095510 con 0x7f9f5c1f90e0 2024-09-17T05:32:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.919+0000 7f9f61400640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 0x7f9f3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.919+0000 7f9f61400640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 0x7f9f3c075710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ac000 tx=0x7f9f500a8040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:34.923+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9f5c162d10 con 0x7f9f5c1f90e0 2024-09-17T05:32:35.053 INFO:tasks.ceph.mds.c.smithi043.stderr: -40> 2024-09-17T05:32:30.295+0000 7f26fa000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-17T05:32:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.155+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5c0d0840 con 0x7f9f5c1f90e0 2024-09-17T05:32:35.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.155+0000 7f9f59600640 1 -- 172.21.15.43:0/2447548175 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 245 v245) ==== 95+0+4392 (secure 0 0 0) 0x7f9f4805f610 con 0x7f9f5c1f90e0 2024-09-17T05:32:35.159 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:35.159 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":245,"btime":"2024-09-17T05:32:34:325909+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":245,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:34.325892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8719":{"gid":8719,"name":"d","rank":0,"incarnation":245,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/2354591248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2354591248},{"type":"v1","addr":"172.21.15.62:6835","nonce":2354591248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 245 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 msgr2=0x7f9f3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 0x7f9f3c075710 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ac000 tx=0x7f9f500a8040 comp rx=0 tx=0).stop 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 msgr2=0x7f9f5c1f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9f48011ae0 tx=0x7f9f48009e50 comp rx=0 tx=0).stop 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 shutdown_connections 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f3c073200 0x7f9f3c075710 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f5c1623c0 0x7f9f5c1627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f5c1f90e0 0x7f9f5c1f9950 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 --2- 172.21.15.43:0/2447548175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f44003d00 0x7f9f5c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 >> 172.21.15.43:0/2447548175 conn(0x7f9f5c0cee80 msgr2=0x7f9f5c15c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 shutdown_connections 2024-09-17T05:32:35.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:35.159+0000 7f9f63e00640 1 -- 172.21.15.43:0/2447548175 wait complete. 2024-09-17T05:32:35.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 245, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:32:34.325892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8719': {'gid': 8719, 'name': 'd', 'rank': 0, 'incarnation': 245, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/2354591248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2354591248}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2354591248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:32:35.183 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8719:up:replay 2024-09-17T05:32:35.183 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:32:35.191 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:40.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:40.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 -- 172.21.15.43:0/3023634655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200b52a0 msgr2=0x7fd9200b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 --2- 172.21.15.43:0/3023634655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200b52a0 0x7fd9200b3a20 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b2e90 tx=0x7fd91801cb80 comp rx=0 tx=0).stop 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 -- 172.21.15.43:0/3023634655 shutdown_connections 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 --2- 172.21.15.43:0/3023634655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200b52a0 0x7fd9200b3a20 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 --2- 172.21.15.43:0/3023634655 >> v2:172.21.15.62:3300/0 conn(0x7fd9200cf390 0x7fd9200b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.755+0000 7fd927c00640 1 -- 172.21.15.43:0/3023634655 >> 172.21.15.43:0/3023634655 conn(0x7fd9200b1dd0 msgr2=0x7fd9200b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 -- 172.21.15.43:0/3023634655 shutdown_connections 2024-09-17T05:32:40.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 -- 172.21.15.43:0/3023634655 wait complete. 2024-09-17T05:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 Processor -- start 2024-09-17T05:32:40.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 -- start start 2024-09-17T05:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9200b3610 0x7fd9201542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 0x7fd92015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.759+0000 7fd927c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd920168df0 con 0x7fd9200cf390 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd927c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9200b2ca0 con 0x7fd9200b52a0 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd927c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd920168f70 con 0x7fd9200b3610 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48954/0 (socket says 172.21.15.43:48954) 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 -- 172.21.15.43:0/911665157 learned_addr learned my addr 172.21.15.43:0/911665157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925200640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9200b3610 0x7fd9201542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd917e00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 0x7fd92015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 -- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9200b3610 msgr2=0x7fd9201542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9200b3610 0x7fd9201542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 -- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 msgr2=0x7fd92015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 0x7fd92015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 -- 172.21.15.43:0/911665157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9200c49c0 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd917e00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 0x7fd92015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd925c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fd91801a7b0 tx=0x7fd918009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180bc070 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9180042f0 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180b2040 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9200c4ca0 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9201f8c50 con 0x7fd9200cf390 2024-09-17T05:32:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9200b3a20 con 0x7fd9200cf390 2024-09-17T05:32:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.763+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd9180bf070 con 0x7fd9200cf390 2024-09-17T05:32:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.767+0000 7fd91d600640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 0x7fd8fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.767+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7fd91812e840 con 0x7fd9200cf390 2024-09-17T05:32:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.767+0000 7fd925200640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 0x7fd8fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.767+0000 7fd925200640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 0x7fd8fc075740 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd908009b40 tx=0x7fd9080091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.771+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd9180f3db0 con 0x7fd9200cf390 2024-09-17T05:32:40.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd92006ec90 con 0x7fd9200cf390 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd91d600640 1 -- 172.21.15.43:0/911665157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) ==== 95+0+5225 (secure 0 0 0) 0x7fd918016770 con 0x7fd9200cf390 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":248,"btime":"2024-09-17T05:32:37:332561+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3812598362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3812598362},{"type":"v1","addr":"172.21.15.43:6835","nonce":3812598362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":246}],"filesystems":[{"mdsmap":{"epoch":248,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:37.332550+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8719":{"gid":8719,"name":"d","rank":0,"incarnation":245,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/2354591248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2354591248},{"type":"v1","addr":"172.21.15.62:6835","nonce":2354591248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8719,"qdb_cluster":[8719]},"id":6}]} 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 248 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 msgr2=0x7fd8fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 0x7fd8fc075740 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd908009b40 tx=0x7fd9080091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 msgr2=0x7fd9201f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.979+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fd91801a7b0 tx=0x7fd918009260 comp rx=0 tx=0).stop 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 shutdown_connections 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd8fc073230 0x7fd8fc075740 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9200b52a0 0x7fd92015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9200cf390 0x7fd9201f9170 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 --2- 172.21.15.43:0/911665157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd9200b3610 0x7fd9201542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 >> 172.21.15.43:0/911665157 conn(0x7fd9200b1dd0 msgr2=0x7fd92015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 shutdown_connections 2024-09-17T05:32:40.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:40.983+0000 7fd927c00640 1 -- 172.21.15.43:0/911665157 wait complete. 2024-09-17T05:32:41.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 248, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:32:37.332550+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8719': {'gid': 8719, 'name': 'd', 'rank': 0, 'incarnation': 245, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6835/2354591248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2354591248}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2354591248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8719, 'qdb_cluster': [8719]} 2024-09-17T05:32:41.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:32:41.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:32:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 -- 172.21.15.43:0/266898333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913806fc90 msgr2=0x7f91380700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 --2- 172.21.15.43:0/266898333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913806fc90 0x7f91380700a0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9120009840 tx=0x7f912001c700 comp rx=0 tx=0).stop 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 -- 172.21.15.43:0/266898333 shutdown_connections 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 --2- 172.21.15.43:0/266898333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913806fc90 0x7f91380700a0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 --2- 172.21.15.43:0/266898333 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 0x7f91380b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.159+0000 7f913da00640 1 -- 172.21.15.43:0/266898333 >> 172.21.15.43:0/266898333 conn(0x7f91380b23d0 msgr2=0x7f91380d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- 172.21.15.43:0/266898333 shutdown_connections 2024-09-17T05:32:41.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- 172.21.15.43:0/266898333 wait complete. 2024-09-17T05:32:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 Processor -- start 2024-09-17T05:32:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- start start 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913806fc90 0x7f9138155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 0x7f91381550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9138169fd0 con 0x7f91380730b0 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9138169e50 con 0x7f913806fc90 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.163+0000 7f913da00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f913816a150 con 0x7f913815fe10 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48974/0 (socket says 172.21.15.43:48974) 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 -- 172.21.15.43:0/738152514 learned_addr learned my addr 172.21.15.43:0/738152514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f9136a00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913806fc90 0x7f9138155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f9137400640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 0x7f91381550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 -- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 msgr2=0x7f91381550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 0x7f91381550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 -- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913806fc90 msgr2=0x7f9138155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913806fc90 0x7f9138155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 -- 172.21.15.43:0/738152514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138156370 con 0x7f91380730b0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912e000640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9120009790 tx=0x7f91200a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912f400640 1 -- 172.21.15.43:0/738152514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91200bd020 con 0x7f91380730b0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912f400640 1 -- 172.21.15.43:0/738152514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91200a66d0 con 0x7f91380730b0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912f400640 1 -- 172.21.15.43:0/738152514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120003600 con 0x7f91380730b0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9138156650 con 0x7f91380730b0 2024-09-17T05:32:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9138156c10 con 0x7f91380730b0 2024-09-17T05:32:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.167+0000 7f912f400640 1 -- 172.21.15.43:0/738152514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91200c0020 con 0x7f91380730b0 2024-09-17T05:32:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f912f400640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 0x7f9114075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f912f400640 1 -- 172.21.15.43:0/738152514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f912012f990 con 0x7f91380730b0 2024-09-17T05:32:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9136a00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 0x7f9114075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 Processor -- start 2024-09-17T05:32:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 -- start start 2024-09-17T05:32:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9136a00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 0x7f9114075710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9128004660 tx=0x7f912800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd618460 0x563acdb596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdb59be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563acda2e410 con 0x563acd9f1ca0 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563acd96bb40 con 0x563acd618460 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563acd95af70 con 0x563acda08950 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9137400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9137400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48980/0 (socket says 172.21.15.43:48980) 2024-09-17T05:32:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9137400640 1 -- 172.21.15.43:0/2554648781 learned_addr learned my addr 172.21.15.43:0/2554648781 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f9136a00640 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd618460 0x563acdb596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.171+0000 7f912e000640 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdb59be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 -- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 msgr2=0x563acdb59be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdb59be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 -- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd618460 msgr2=0x563acdb596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd618460 0x563acdb596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 -- 172.21.15.43:0/2554648781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acd9225b0 con 0x563acd9f1ca0 2024-09-17T05:32:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9137400640 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9130002230 tx=0x7f913000f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4f19f3c7282df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9127400640 1 -- 172.21.15.43:0/2554648781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130018020 con 0x563acd9f1ca0 2024-09-17T05:32:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9127400640 1 -- 172.21.15.43:0/2554648781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f913000fde0 con 0x563acd9f1ca0 2024-09-17T05:32:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9127400640 1 -- 172.21.15.43:0/2554648781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91300146d0 con 0x563acd9f1ca0 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- 172.21.15.43:0/2554648781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 msgr2=0x563acdb5a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9130002230 tx=0x7f913000f290 comp rx=0 tx=0).stop 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- 172.21.15.43:0/2554648781 shutdown_connections 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd9f1ca0 0x563acdb5a120 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdb59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2554648781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd618460 0x563acdb596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- 172.21.15.43:0/2554648781 >> 172.21.15.43:0/2554648781 conn(0x563acdaded90 msgr2=0x563acdb0e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- 172.21.15.43:0/2554648781 shutdown_connections 2024-09-17T05:32:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- 172.21.15.43:0/2554648781 wait complete. 2024-09-17T05:32:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 -- start start 2024-09-17T05:32:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.175+0000 7f9144f7b000 1 Processor -- start 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd9f1ca0 0x563acdb60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acda08950 0x563acdb613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563acd8f55f0 con 0x563acd618460 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563acd95ed80 con 0x563acda08950 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563acd901520 con 0x563acd9f1ca0 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 10 client.0 _ll_register_callbacks cb 0x563acd905830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48994/0 (socket says 172.21.15.43:48994) 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 -- 172.21.15.43:0/2090396286 learned_addr learned my addr 172.21.15.43:0/2090396286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f912e000640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd9f1ca0 0x563acdb60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9137400640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acda08950 0x563acdb613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 -- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd9f1ca0 msgr2=0x563acdb60e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd9f1ca0 0x563acdb60e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 -- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acda08950 msgr2=0x563acdb613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acda08950 0x563acdb613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 -- 172.21.15.43:0/2090396286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acd8f9190 con 0x563acd618460 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9136a00640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f912800bd20 tx=0x7f9128004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2270a3679859b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:32:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128028670 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563acd8f3110 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f9128028810 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=248) ==== 24+0+0 (secure 0 0 0) 0x7f9128030020 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 10 client.9306 fetch_fsmap learned FSMap version 248 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=248,monmap=2+}) -- 0x563acd9dff10 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563acd8f3110 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 248) ==== 3502+0+0 (secure 0 0 0) 0x7f9128028b00 con 0x563acd618460 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 10 client.9306 fetch_fsmap finished waiting for FSMap version 248 2024-09-17T05:32:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 10 client.9306 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 20 client.9306 populate_metadata read hostname 'smithi043' 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5fd7d0 0x563acdb6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 4 client.9306 mds_command: new command op to 8719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 --> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563acd96da90 con 0x563acd5fd7d0 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f912e000640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5fd7d0 0x563acdb6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f911be00640 10 client.9306 ms_handle_connect on v2:172.21.15.62:6834/2354591248 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.179+0000 7f912e000640 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5fd7d0 0x563acdb6ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738d02398ee7cfe4 server_cookie=78b6fe9c893172ce in_seq=0 out_seq=0 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f911be00640 1 -- 172.21.15.43:0/2090396286 <== mds.0 v2:172.21.15.62:6834/2354591248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563acd96da90 con 0x563acd5fd7d0 2024-09-17T05:32:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f911be00640 10 client.9306 handle_command_reply: tid=0 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 client.9306 shutdown 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 msgr2=0x563acdb60920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f912800bd20 tx=0x7f9128004740 comp rx=0 tx=0).stop 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 shutdown_connections 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5fd7d0 0x563acdb6ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acda08950 0x563acdb613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd9f1ca0 0x563acdb60e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f911be00640 0 client.9306 ms_handle_reset on v2:172.21.15.62:6834/2354591248 2024-09-17T05:32:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 --2- 172.21.15.43:0/2090396286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdb60920 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 >> 172.21.15.43:0/2090396286 conn(0x563acdaded90 msgr2=0x563acdb15440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 shutdown_connections 2024-09-17T05:32:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 1 -- 172.21.15.43:0/2090396286 wait complete. 2024-09-17T05:32:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.183+0000 7f9144f7b000 20 client.9306 trim_cache size 0 max 16384 2024-09-17T05:32:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 Processor -- start 2024-09-17T05:32:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- start start 2024-09-17T05:32:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdbcb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9138206910 con 0x563acd618460 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91382059b0 con 0x563acd9f1ca0 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f913815d0d0 con 0x563acda08950 2024-09-17T05:32:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49010/0 (socket says 172.21.15.43:49010) 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 -- 172.21.15.43:0/1952263582 learned_addr learned my addr 172.21.15.43:0/1952263582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f912e000640 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9137400640 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdbcb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 -- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 msgr2=0x563acdbcb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdbcb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 -- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 msgr2=0x563acdbcb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 -- 172.21.15.43:0/1952263582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acd9b6db0 con 0x563acd618460 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9136a00640 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f912800b5e0 tx=0x7f912800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685db7289fd25e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9126a00640 1 -- 172.21.15.43:0/1952263582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128005220 con 0x563acd618460 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9126a00640 1 -- 172.21.15.43:0/1952263582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9128023070 con 0x563acd618460 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9126a00640 1 -- 172.21.15.43:0/1952263582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91280047b0 con 0x563acd618460 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- 172.21.15.43:0/1952263582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 msgr2=0x563acdbcac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f912800b5e0 tx=0x7f912800b610 comp rx=0 tx=0).stop 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- 172.21.15.43:0/1952263582 shutdown_connections 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acda08950 0x563acdbcb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 --2- 172.21.15.43:0/1952263582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acd618460 0x563acdbcac50 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- 172.21.15.43:0/1952263582 >> 172.21.15.43:0/1952263582 conn(0x563acdaded90 msgr2=0x563acdb1b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- 172.21.15.43:0/1952263582 shutdown_connections 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.203+0000 7f9144f7b000 1 -- 172.21.15.43:0/1952263582 wait complete. 2024-09-17T05:32:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 -- start start 2024-09-17T05:32:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 Processor -- start 2024-09-17T05:32:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd618460 0x563acdbcec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9138205360 con 0x563acda08950 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563acd96a910 con 0x563acd9f1ca0 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563acd926ee0 con 0x563acd618460 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 10 client.0 _ll_register_callbacks cb 0x7f9128004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49018/0 (socket says 172.21.15.43:49018) 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 -- 172.21.15.43:0/758293097 learned_addr learned my addr 172.21.15.43:0/758293097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9136a00640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd618460 0x563acdbcec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f912e000640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 -- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd618460 msgr2=0x563acdbcec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd618460 0x563acdbcec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 -- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 msgr2=0x563acdbcf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 -- 172.21.15.43:0/758293097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acd921ab0 con 0x563acda08950 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9137400640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9130000f90 tx=0x7f9130014b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd4f0258a72fa6b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f911be00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913002b060 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f9130026420 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.207+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9128023070 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=248) ==== 24+0+0 (secure 0 0 0) 0x7f913002e020 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 10 client.9312 fetch_fsmap learned FSMap version 248 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=248,monmap=2+}) -- 0x563acd900070 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9128023070 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 248) ==== 3502+0+0 (secure 0 0 0) 0x7f9130026620 con 0x563acda08950 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 10 client.9312 fetch_fsmap finished waiting for FSMap version 248 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 10 client.9312 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 20 client.9312 populate_metadata read hostname 'smithi043' 2024-09-17T05:32:41.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5eb830 0x563acdb8b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 4 client.9312 mds_command: new command op to 8719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 --> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] -- command(tid 0: {"prefix": "status"}) -- 0x563acd969540 con 0x563acd5eb830 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9136a00640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5eb830 0x563acdb8b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9136a00640 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5eb830 0x563acdb8b500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0dce6996a16c4a server_cookie=c510205b373b45ee in_seq=0 out_seq=0 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 10 client.9312 ms_handle_connect on v2:172.21.15.62:6834/2354591248 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 1 -- 172.21.15.43:0/758293097 <== mds.0 v2:172.21.15.62:6834/2354591248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x563acd969540 con 0x563acd5eb830 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 10 client.9312 handle_command_reply: tid=0 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 client.9312 shutdown 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 msgr2=0x563acdbcf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9130000f90 tx=0x7f9130014b50 comp rx=0 tx=0).stop 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 shutdown_connections 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:6834/2354591248,v1:172.21.15.62:6835/2354591248] conn(0x563acd5eb830 0x563acdb8b500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acd9f1ca0 0x563acdbcf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563acda08950 0x563acdbcf690 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 --2- 172.21.15.43:0/758293097 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563acd618460 0x563acdbcec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f911be00640 0 client.9312 ms_handle_reset on v2:172.21.15.62:6834/2354591248 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 >> 172.21.15.43:0/758293097 conn(0x563acdaded90 msgr2=0x563acdb6a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 shutdown_connections 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 1 -- 172.21.15.43:0/758293097 wait complete. 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f9144f7b000 20 client.9312 trim_cache size 0 max 16384 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:32:41.223 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8719, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8854558350000001, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 248, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 115, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 115, 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.078092154 2024-09-17T05:32:41.224 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:32:41.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 msgr2=0x7f9114075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 0x7f9114075710 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9128004660 tx=0x7f912800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 msgr2=0x7f9138157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9120009790 tx=0x7f91200a64e0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 shutdown_connections 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9114073200 0x7f9114075710 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f913815fe10 0x7f91381550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91380730b0 0x7f9138157410 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 --2- 172.21.15.43:0/738152514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913806fc90 0x7f9138155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 >> 172.21.15.43:0/738152514 conn(0x7f91380b23d0 msgr2=0x7f91380ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 shutdown_connections 2024-09-17T05:32:41.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.211+0000 7f913da00640 1 -- 172.21.15.43:0/738152514 wait complete. 2024-09-17T05:32:41.234 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:32:41.234 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-09-17T05:32:41.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:32:41.310 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:41.308+0000 7f021c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:32:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 -- 172.21.15.43:0/2803702846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 msgr2=0x7f28880734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 --2- 172.21.15.43:0/2803702846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f28880734c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2880009bc0 tx=0x7f288001ca80 comp rx=0 tx=0).stop 2024-09-17T05:32:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 -- 172.21.15.43:0/2803702846 shutdown_connections 2024-09-17T05:32:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 --2- 172.21.15.43:0/2803702846 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f28880734c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 --2- 172.21.15.43:0/2803702846 >> v2:172.21.15.62:3300/0 conn(0x7f288806fc90 0x7f28880700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 -- 172.21.15.43:0/2803702846 >> 172.21.15.43:0/2803702846 conn(0x7f28880b23d0 msgr2=0x7f28880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 -- 172.21.15.43:0/2803702846 shutdown_connections 2024-09-17T05:32:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.391+0000 7f288d200640 1 -- 172.21.15.43:0/2803702846 wait complete. 2024-09-17T05:32:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 Processor -- start 2024-09-17T05:32:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- start start 2024-09-17T05:32:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f288806fc90 0x7f28881fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 0x7f288815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2888169fc0 con 0x7f288815fe10 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2888169e40 con 0x7f288806fc90 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f288816a140 con 0x7f28880730b0 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f2886800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 0x7f288815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:46882/0 (socket says 172.21.15.43:46882) 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f2885e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f288806fc90 0x7f28881fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 -- 172.21.15.43:0/707199006 learned_addr learned my addr 172.21.15.43:0/707199006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f2886800640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 0x7f288815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49026/0 (socket says 172.21.15.43:49026) 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 -- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f288806fc90 msgr2=0x7f28881fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f288806fc90 0x7f28881fd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 -- 172.21.15.43:0/707199006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 msgr2=0x7f288815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 0x7f288815b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 -- 172.21.15.43:0/707199006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288814fd50 con 0x7f28880730b0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287d400640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f287000e770 tx=0x7f287000ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287000ccc0 con 0x7f28880730b0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f287000ee80 con 0x7f28880730b0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287000ce60 con 0x7f28880730b0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2888150030 con 0x7f28880730b0 2024-09-17T05:32:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.395+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28881fcbe0 con 0x7f28880730b0 2024-09-17T05:32:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28880700a0 con 0x7f28880730b0 2024-09-17T05:32:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2870005280 con 0x7f28880730b0 2024-09-17T05:32:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f287ea00640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 0x7f2864075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f2870095870 con 0x7f28880730b0 2024-09-17T05:32:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f2885e00640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 0x7f2864075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.399+0000 7f2885e00640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 0x7f2864075710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2878007d70 tx=0x7f2878007d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:41.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.403+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f287005f960 con 0x7f28880730b0 2024-09-17T05:32:41.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.615+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28880b15f0 con 0x7f28880730b0 2024-09-17T05:32:41.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.615+0000 7f287ea00640 1 -- 172.21.15.43:0/707199006 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v248) ==== 53+0+73 (secure 0 0 0) 0x7f2870064390 con 0x7f28880730b0 2024-09-17T05:32:41.618 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:32:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 msgr2=0x7f2864075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 0x7f2864075710 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2878007d70 tx=0x7f2878007d00 comp rx=0 tx=0).stop 2024-09-17T05:32:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 msgr2=0x7f2888099790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f287000e770 tx=0x7f287000ec30 comp rx=0 tx=0).stop 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 shutdown_connections 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2864073200 0x7f2864075710 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f288815fe10 0x7f288815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28880730b0 0x7f2888099790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 --2- 172.21.15.43:0/707199006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f288806fc90 0x7f28881fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 >> 172.21.15.43:0/707199006 conn(0x7f28880b23d0 msgr2=0x7f28880702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 shutdown_connections 2024-09-17T05:32:41.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41.619+0000 7f288d200640 1 -- 172.21.15.43:0/707199006 wait complete. 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:32:41.641 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:32:41.642 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:32:41.642 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:41.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:41.642 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:32:41.651 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41882sec preferred_lft 41882sec 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.654 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:32:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:32:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:41.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:41.655 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:32:41.713 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:32:41.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:41.715 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:32:41.769 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:32:41.770 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:32:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:32:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:41.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:41.907 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:32:41.916 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:32:41.916 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:32:41.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:41.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:41.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:32:41.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:41.979 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:42.031 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:42.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:42.031 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:42.082 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:32:42.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:42.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:42.189 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:42.189 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:42.189 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:42.207 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:42.207 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:42.207 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:42.207 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:32:42.207 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:42.218 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:42.218 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:32:42.219 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:32:42.222 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:32:42.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:42.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:42.242 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:32:42.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:32:42.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:42.346 INFO:teuthology.orchestra.run.smithi043.stdout:client9318 2024-09-17T05:32:42.346 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:42.346 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:42.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:42.356 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:32:42.402 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:32:42.411 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:32:42.411 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:32:42.411 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:42.411 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:32:42.411 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:32:42.412 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-09-17T05:32:42.412 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-09-17T05:32:42.412 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:42.412 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:42.463 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:42.463 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:42.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:32:42.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 -- 172.21.15.43:0/189557587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 msgr2=0x7faa600d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 --2- 172.21.15.43:0/189557587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa600d0840 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faa50005fd0 tx=0x7faa5001c820 comp rx=0 tx=0).stop 2024-09-17T05:32:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 -- 172.21.15.43:0/189557587 shutdown_connections 2024-09-17T05:32:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 --2- 172.21.15.43:0/189557587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa600d0840 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 -- 172.21.15.43:0/189557587 >> 172.21.15.43:0/189557587 conn(0x7faa600cee80 msgr2=0x7faa600cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 -- 172.21.15.43:0/189557587 shutdown_connections 2024-09-17T05:32:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.715+0000 7faa66000640 1 -- 172.21.15.43:0/189557587 wait complete. 2024-09-17T05:32:42.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 Processor -- start 2024-09-17T05:32:42.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 -- start start 2024-09-17T05:32:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa6014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa600cd580 0x7faa601545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa6016b030 con 0x7faa600d0d80 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faa6016aeb0 con 0x7faa600b7ce0 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa66000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faa6016b1b0 con 0x7faa600cd580 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa6014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49032/0 (socket says 172.21.15.43:49032) 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 -- 172.21.15.43:0/2337401520 learned_addr learned my addr 172.21.15.43:0/2337401520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa56a00640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa600cd580 0x7faa601545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 -- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa600cd580 msgr2=0x7faa601545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa600cd580 0x7faa601545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 -- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 msgr2=0x7faa6014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa6014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 -- 172.21.15.43:0/2337401520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601571a0 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.719+0000 7faa5fe00640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c730 tx=0x7faa5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58003b80 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa58003d20 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800aa50 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa60153db0 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa60154af0 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa600d0840 con 0x7faa600d0d80 2024-09-17T05:32:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa5800abf0 con 0x7faa600d0d80 2024-09-17T05:32:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 0x7faa3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7faa58094860 con 0x7faa600d0d80 2024-09-17T05:32:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.723+0000 7faa5f400640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 0x7faa3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.727+0000 7faa5f400640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 0x7faa3c075740 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faa500ae000 tx=0x7faa5001b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.727+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faa5805e8d0 con 0x7faa600d0d80 2024-09-17T05:32:42.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faa60155180 con 0x7faa600d0d80 2024-09-17T05:32:42.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa57e00640 1 -- 172.21.15.43:0/2337401520 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v115) ==== 81+0+3162 (secure 0 0 0) 0x7faa58063300 con 0x7faa600d0d80 2024-09-17T05:32:42.916 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:32:42.916 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:32:42.917 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:32:42.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:32:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:32:42.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:32:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:listed 52 entries 2024-09-17T05:32:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 msgr2=0x7faa3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 0x7faa3c075740 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faa500ae000 tx=0x7faa5001b060 comp rx=0 tx=0).stop 2024-09-17T05:32:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 msgr2=0x7faa6015d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c730 tx=0x7faa5800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 shutdown_connections 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faa3c073230 0x7faa3c075740 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa600d0d80 0x7faa6015d450 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faa600cd580 0x7faa601545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 --2- 172.21.15.43:0/2337401520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa600b7ce0 0x7faa6014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 >> 172.21.15.43:0/2337401520 conn(0x7faa600cee80 msgr2=0x7faa60072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 shutdown_connections 2024-09-17T05:32:42.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:42.915+0000 7faa66000640 1 -- 172.21.15.43:0/2337401520 wait complete. 2024-09-17T05:32:42.939 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:32:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:42.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:32:42.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:42.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:42.963 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:43.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 -- 172.21.15.43:0/946395863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c80b3af0 msgr2=0x7f26c815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 --2- 172.21.15.43:0/946395863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c80b3af0 0x7f26c815c7e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009840 tx=0x7f26b001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 -- 172.21.15.43:0/946395863 shutdown_connections 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 --2- 172.21.15.43:0/946395863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c80b3af0 0x7f26c815c7e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 -- 172.21.15.43:0/946395863 >> 172.21.15.43:0/946395863 conn(0x7f26c80b23d0 msgr2=0x7f26c80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 -- 172.21.15.43:0/946395863 shutdown_connections 2024-09-17T05:32:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.215+0000 7f26cf000640 1 -- 172.21.15.43:0/946395863 wait complete. 2024-09-17T05:32:43.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 Processor -- start 2024-09-17T05:32:43.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 -- start start 2024-09-17T05:32:43.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c8155d70 0x7f26c8098b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26c806ce60 0x7f26c806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8169dc0 con 0x7f26c806f110 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8169c40 con 0x7f26c8155d70 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cf000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f26c8169f40 con 0x7f26c806ce60 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49038/0 (socket says 172.21.15.43:49038) 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 -- 172.21.15.43:0/1019949257 learned_addr learned my addr 172.21.15.43:0/1019949257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26bf400640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c8155d70 0x7f26c8098b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26cd000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26c806ce60 0x7f26c806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 -- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26c806ce60 msgr2=0x7f26c806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26c806ce60 0x7f26c806d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 -- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c8155d70 msgr2=0x7f26c8098b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c8155d70 0x7f26c8098b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.219+0000 7f26c7e00640 1 -- 172.21.15.43:0/1019949257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c806d7b0 con 0x7f26c806f110 2024-09-17T05:32:43.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26c7e00640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c460 tx=0x7f26b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80052d0 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26b8005470 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c8099070 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8012870 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c81fbc80 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c815c7e0 con 0x7f26c806f110 2024-09-17T05:32:43.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.223+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26b8003600 con 0x7f26c806f110 2024-09-17T05:32:43.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.227+0000 7f26c4c00640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 0x7f26a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.227+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7945+0+0 (secure 0 0 0) 0x7f26b80944e0 con 0x7f26c806f110 2024-09-17T05:32:43.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.227+0000 7f26bf400640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 0x7f26a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.227+0000 7f26bf400640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 0x7f26a4075740 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f26b00ae000 tx=0x7f26b001aef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.231+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26c806d7b0 con 0x7f26c806f110 2024-09-17T05:32:43.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.443+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f26c80af690 con 0x7f26c806f110 2024-09-17T05:32:43.444 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:43.443+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:43.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.443+0000 7f26c4c00640 1 -- 172.21.15.43:0/1019949257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v249) ==== 161+0+0 (secure 0 0 0) 0x7f26b805e5e0 con 0x7f26c806f110 2024-09-17T05:32:43.446 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:43.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 msgr2=0x7f26a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 0x7f26a4075740 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f26b00ae000 tx=0x7f26b001aef0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 msgr2=0x7f26c80cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c460 tx=0x7f26b800c920 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 shutdown_connections 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26a4073230 0x7f26a4075740 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26c8155d70 0x7f26c8098b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26c806ce60 0x7f26c806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 --2- 172.21.15.43:0/1019949257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c806f110 0x7f26c80cbab0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 >> 172.21.15.43:0/1019949257 conn(0x7f26c80b23d0 msgr2=0x7f26c80ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 shutdown_connections 2024-09-17T05:32:43.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.447+0000 7f26cf000640 1 -- 172.21.15.43:0/1019949257 wait complete. 2024-09-17T05:32:43.457 INFO:tasks.ceph.mds.d.smithi062.stderr: -563> 2024-09-17T05:32:41.308+0000 7f021c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:32:43.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:43.580 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 -- 172.21.15.43:0/2917782583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e080b4570 msgr2=0x7f8e080b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 --2- 172.21.15.43:0/2917782583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e080b4570 0x7f8e080b4980 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8e08074cd0 tx=0x7f8df001c830 comp rx=0 tx=0).stop 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 -- 172.21.15.43:0/2917782583 shutdown_connections 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 --2- 172.21.15.43:0/2917782583 >> v2:172.21.15.62:3300/0 conn(0x7f8e08160210 0x7f8e08160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 --2- 172.21.15.43:0/2917782583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e080b4570 0x7f8e080b4980 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 -- 172.21.15.43:0/2917782583 >> 172.21.15.43:0/2917782583 conn(0x7f8e080b2810 msgr2=0x7f8e080b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 -- 172.21.15.43:0/2917782583 shutdown_connections 2024-09-17T05:32:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.627+0000 7f8e0f000640 1 -- 172.21.15.43:0/2917782583 wait complete. 2024-09-17T05:32:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 Processor -- start 2024-09-17T05:32:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 -- start start 2024-09-17T05:32:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 0x7f8e08153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e08160210 0x7f8e0814e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0816bea0 con 0x7f8e08160210 2024-09-17T05:32:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0816c020 con 0x7f8e0806f940 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0f000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0816c1a0 con 0x7f8e080b4570 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e0d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e08160210 0x7f8e0814e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:59362/0 (socket says 172.21.15.43:59362) 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8e07e00640 1 -- 172.21.15.43:0/3078490882 learned_addr learned my addr 172.21.15.43:0/3078490882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.631+0000 7f8dff400640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 0x7f8e08153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 -- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 msgr2=0x7f8e08153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 0x7f8e08153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 -- 172.21.15.43:0/3078490882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e08160210 msgr2=0x7f8e0814e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e08160210 0x7f8e0814e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 -- 172.21.15.43:0/3078490882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e08149240 con 0x7f8e0806f940 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8dff400640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 0x7f8e08153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e07e00640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c3f0 tx=0x7f8df800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8017070 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8df80053c0 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8003860 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0814ac40 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0814d330 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e080b4980 con 0x7f8e0806f940 2024-09-17T05:32:43.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.635+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8df801a070 con 0x7f8e0806f940 2024-09-17T05:32:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.639+0000 7f8e04c00640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 0x7f8de4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.639+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8031+0+0 (secure 0 0 0) 0x7f8df8093f60 con 0x7f8e0806f940 2024-09-17T05:32:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.639+0000 7f8dff400640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 0x7f8de4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.639+0000 7f8dff400640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 0x7f8de4075710 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8df00af000 tx=0x7f8df001b070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.639+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8df8021080 con 0x7f8e0806f940 2024-09-17T05:32:43.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:43.879+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8e081536d0 con 0x7f8e0806f940 2024-09-17T05:32:44.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e04c00640 1 -- 172.21.15.43:0/3078490882 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v250) ==== 107+0+0 (secure 0 0 0) 0x7f8e080b4ec0 con 0x7f8e0806f940 2024-09-17T05:32:44.456 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:32:44.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 msgr2=0x7f8de4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 0x7f8de4075710 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8df00af000 tx=0x7f8df001b070 comp rx=0 tx=0).stop 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 msgr2=0x7f8e08148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c3f0 tx=0x7f8df800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 shutdown_connections 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8de4073200 0x7f8de4075710 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e08160210 0x7f8e0814e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e080b4570 0x7f8e08153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 --2- 172.21.15.43:0/3078490882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e0806f940 0x7f8e08148d00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 >> 172.21.15.43:0/3078490882 conn(0x7f8e080b2810 msgr2=0x7f8e080d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 shutdown_connections 2024-09-17T05:32:44.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.455+0000 7f8e0f000640 1 -- 172.21.15.43:0/3078490882 wait complete. 2024-09-17T05:32:44.480 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:44.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 Processor -- start 2024-09-17T05:32:44.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 -- start start 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c816636750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c8165c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c8164cda00 con 0x55c8165aecb0 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c81652bf40 con 0x55c8166388b0 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1760e76080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c8165825b0 con 0x55c816637010 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36246/0 (socket says 172.21.15.43:36246) 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 -- 172.21.15.43:0/859587618 learned_addr learned my addr 172.21.15.43:0/859587618 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175aa00640 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c8165c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1759600640 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c816636750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 -- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 msgr2=0x55c816636750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c816636750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 -- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 msgr2=0x55c8165c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c8165c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 -- 172.21.15.43:0/859587618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8164a6c10 con 0x55c8165aecb0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f175a000640 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f174c01baf0 tx=0x7f174c017ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9c980f531753b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1753e00640 1 -- 172.21.15.43:0/859587618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0ba070 con 0x55c8165aecb0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1753e00640 1 -- 172.21.15.43:0/859587618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f174c0ab980 con 0x55c8165aecb0 2024-09-17T05:32:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.539+0000 7f1753e00640 1 -- 172.21.15.43:0/859587618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0abdb0 con 0x55c8165aecb0 2024-09-17T05:32:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- 172.21.15.43:0/859587618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 msgr2=0x55c816636210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f174c01baf0 tx=0x7f174c017ff0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- 172.21.15.43:0/859587618 shutdown_connections 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c8165c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c816636750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- 172.21.15.43:0/859587618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c816636210 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- 172.21.15.43:0/859587618 >> 172.21.15.43:0/859587618 conn(0x55c81656bd80 msgr2=0x55c816652e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- 172.21.15.43:0/859587618 shutdown_connections 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- 172.21.15.43:0/859587618 wait complete. 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 Processor -- start 2024-09-17T05:32:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- start start 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c81664eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c816652db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c8166551d0 con 0x55c8165aecb0 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c816655ec0 con 0x55c8166388b0 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1760e76080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55c816656750 con 0x55c816637010 2024-09-17T05:32:44.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36250/0 (socket says 172.21.15.43:36250) 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 learned_addr learned my addr 172.21.15.43:0/1689930438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c81664eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175aa00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c816652db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 msgr2=0x55c81664eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55c816637010 0x55c81664eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 msgr2=0x55c816652db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8166388b0 0x55c816652db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8166e5430 con 0x55c8165aecb0 2024-09-17T05:32:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.543+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b8040 tx=0x7f174c0acaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 -- 172.21.15.43:0/1689930438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0ba040 con 0x55c8165aecb0 2024-09-17T05:32:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 -- 172.21.15.43:0/1689930438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f174c0b5610 con 0x55c8165aecb0 2024-09-17T05:32:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 -- 172.21.15.43:0/1689930438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0b5860 con 0x55c8165aecb0 2024-09-17T05:32:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8166e56d0 con 0x55c8165aecb0 2024-09-17T05:32:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c816650f70 con 0x55c8165aecb0 2024-09-17T05:32:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 -- 172.21.15.43:0/1689930438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f174c0c5020 con 0x55c8165aecb0 2024-09-17T05:32:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 0x7f173c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1752a00640 1 -- 172.21.15.43:0/1689930438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8031+0+0 (secure 0 0 0) 0x7f174c12efe0 con 0x55c8165aecb0 2024-09-17T05:32:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 0x7f173c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.551 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c816651410 0x55c816651820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.547+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816651d60 con 0x55c816651410 2024-09-17T05:32:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.551+0000 7f175aa00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c816651410 0x55c816651820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.551+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 0x7f173c0740f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f174800c670 tx=0x7f174800e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.551+0000 7f175aa00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c816651410 0x55c816651820 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.551+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v116'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.551+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816652100 con 0x55c816651410 2024-09-17T05:32:44.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v116'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166524a0 con 0x55c816651410 2024-09-17T05:32:44.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v116'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:44.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc650 con 0x55c816651410 2024-09-17T05:32:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.555+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc9f0 con 0x55c816651410 2024-09-17T05:32:44.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.559+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.559+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fb640 con 0x55c816651410 2024-09-17T05:32:44.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.559+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.559+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fb9e0 con 0x55c816651410 2024-09-17T05:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v116'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8166fbdd0 con 0x55c816651410 2024-09-17T05:32:44.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v116'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbdd0 con 0x55c816651410 2024-09-17T05:32:44.566 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v116'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.563+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.567+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.574 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.571+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.575+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.575+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.575+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.575+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fbd80 con 0x55c816651410 2024-09-17T05:32:44.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc140 con 0x55c816651410 2024-09-17T05:32:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc140 con 0x55c816651410 2024-09-17T05:32:44.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v116'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.579+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v116'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.585 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:44.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v116'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.583+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v116'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.587+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.591+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v116'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.591+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.593 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.591+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v116'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.591+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c816651410 2024-09-17T05:32:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c8166fd2a0 0x55c8166ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fc000 con 0x55c8166fd2a0 2024-09-17T05:32:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c8166fd2a0 0x55c8166ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.595+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c8166fd2a0 0x55c8166ff6c0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.599+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'595 uv574 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.599+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8166ffc00 con 0x55c8166fd2a0 2024-09-17T05:32:44.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.599+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'596 uv596 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.599+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8166fffa0 con 0x55c8166fd2a0 2024-09-17T05:32:44.604 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:44.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167007f0 con 0x55c8166fd2a0 2024-09-17T05:32:44.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700b90 con 0x55c8166fd2a0 2024-09-17T05:32:44.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.603+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700f30 con 0x55c8166fd2a0 2024-09-17T05:32:44.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v116'600 uv593 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c8167012d0 con 0x55c8166fd2a0 2024-09-17T05:32:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v116'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167012d0 con 0x55c8166fd2a0 2024-09-17T05:32:44.611 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:44.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v116'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.607+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.611+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.611+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.611+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.611+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v116'605 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v116'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.619 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v116'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.615+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.619+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v116'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.619+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.619+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.619+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v116'610 uv590 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v116'611 uv611 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v116'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.627 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:44.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.623+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816700420 con 0x55c8166fd2a0 2024-09-17T05:32:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c8166fd2a0 2024-09-17T05:32:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c816701c90 0x55c8167020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.627+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816704810 con 0x55c816701c90 2024-09-17T05:32:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c816701c90 0x55c8167020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c816701c90 0x55c8167020a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v116'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816704bb0 con 0x55c816701c90 2024-09-17T05:32:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v116'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.631+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816704fd0 con 0x55c816701c90 2024-09-17T05:32:44.636 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v116'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167068b0 con 0x55c816701c90 2024-09-17T05:32:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816706c50 con 0x55c816701c90 2024-09-17T05:32:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.635+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167058a0 con 0x55c816701c90 2024-09-17T05:32:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816651410 2024-09-17T05:32:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'114 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816651410 2024-09-17T05:32:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'115 uv115 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.639+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816651410 2024-09-17T05:32:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'116 uv116 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.645 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816651410 2024-09-17T05:32:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 0x55c816709410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816706ff0 2024-09-17T05:32:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f175aa00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 0x55c816709410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.643+0000 7f175aa00640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 0x55c816709410 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v116'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816706ff0 2024-09-17T05:32:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v116'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816705cc0 con 0x55c816706ff0 2024-09-17T05:32:44.650 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:44.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v116'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.647+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670a9b0 con 0x55c816706ff0 2024-09-17T05:32:44.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670a9b0 con 0x55c816706ff0 2024-09-17T05:32:44.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167099a0 con 0x55c816706ff0 2024-09-17T05:32:44.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c816709eb0 0x55c81670c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670c8e0 con 0x55c816709eb0 2024-09-17T05:32:44.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c816709eb0 0x55c81670c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.651+0000 7f175a000640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c816709eb0 0x55c81670c2d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.655+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v116'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.655+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.655+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v116'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.655+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.659 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:44.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v116'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670dcc0 con 0x55c816709eb0 2024-09-17T05:32:44.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670dcc0 con 0x55c816709eb0 2024-09-17T05:32:44.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.659+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v116'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v116'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.667 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v116'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.663+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670ccb0 con 0x55c816709eb0 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f175a000640 1 -- 172.21.15.43:0/1689930438 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f174c0a3d70 con 0x55c816709eb0 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c81670d330 0x55c81670f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c81670fe90 con 0x55c81670d330 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c81670d330 0x55c81670f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.667+0000 7f1759600640 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c81670d330 0x55c81670f7b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v116'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710230 con 0x55c81670d330 2024-09-17T05:32:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v116'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c81670d330 2024-09-17T05:32:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v116'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.676 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.671+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c81670d330 2024-09-17T05:32:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c81670d330 2024-09-17T05:32:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816651410 2024-09-17T05:32:44.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v116'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.675+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816651410 2024-09-17T05:32:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v116'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816651410 2024-09-17T05:32:44.681 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v116'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711660 con 0x55c816651410 2024-09-17T05:32:44.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.679+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711a50 con 0x55c816651410 2024-09-17T05:32:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816651410 2024-09-17T05:32:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816651410 2024-09-17T05:32:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816701c90 2024-09-17T05:32:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v116'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.683+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816701c90 2024-09-17T05:32:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v116'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710650 con 0x55c816701c90 2024-09-17T05:32:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v116'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.690 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.687+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.691+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.691+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.691+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v116'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.691+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v116'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.697 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v116'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710a60 con 0x55c816701c90 2024-09-17T05:32:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.695+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816713870 con 0x55c816701c90 2024-09-17T05:32:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.699+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c816701c90 2024-09-17T05:32:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.699+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710f10 con 0x55c816706ff0 2024-09-17T05:32:44.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.699+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v116'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.699+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710f10 con 0x55c816706ff0 2024-09-17T05:32:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.703+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.703+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.706 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.703+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v116'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.703+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v116'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v116'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.707+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.713 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:44.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.711+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v116'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.711+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711d40 con 0x55c816706ff0 2024-09-17T05:32:44.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.711+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.711+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711d40 con 0x55c816706ff0 2024-09-17T05:32:44.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c816706ff0 2024-09-17T05:32:44.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f175400d850 con 0x55c816706ff0 2024-09-17T05:32:44.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v116'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.715+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v116'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167054a0 con 0x55c81670d330 2024-09-17T05:32:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v116'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.721 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167054a0 con 0x55c81670d330 2024-09-17T05:32:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.719+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c8167054a0 con 0x55c81670d330 2024-09-17T05:32:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v116'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v116'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.723+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v116'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.728 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:44.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.727+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v116'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v116'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v116'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.735 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:44.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.731+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711d40 con 0x55c81670d330 2024-09-17T05:32:44.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.735+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.735+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816711d40 con 0x55c81670d330 2024-09-17T05:32:44.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.735+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.735+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.735+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55c816710d30 con 0x55c81670d330 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1759600640 1 -- 172.21.15.43:0/1689930438 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1748008ac0 con 0x55c81670d330 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c8166fd2a0 msgr2=0x55c8166ff6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55c8166fd2a0 0x55c8166ff6c0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c816709eb0 msgr2=0x55c81670c2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55c816709eb0 0x55c81670c2d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c81670d330 msgr2=0x55c81670f7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55c81670d330 0x55c81670f7b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c816701c90 msgr2=0x55c8167020a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55c816701c90 0x55c8167020a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 reap_dead start 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c816651410 msgr2=0x55c816651820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55c816651410 0x55c816651820 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f175aa00640 1 -- 172.21.15.43:0/1689930438 reap_dead start 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 msgr2=0x55c816709410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 0x55c816709410 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 msgr2=0x7f173c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 0x7f173c0740f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f174800c670 tx=0x7f174800e040 comp rx=0 tx=0).stop 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 msgr2=0x55c8165c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b8040 tx=0x7f174c0acaa0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 shutdown_connections 2024-09-17T05:32:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55c816706ff0 0x55c816709410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f173c071be0 0x7f173c0740f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 --2- 172.21.15.43:0/1689930438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c8165aecb0 0x55c8165c6c90 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 >> 172.21.15.43:0/1689930438 conn(0x55c81656bd80 msgr2=0x55c816655380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 shutdown_connections 2024-09-17T05:32:44.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.739+0000 7f1760e76080 1 -- 172.21.15.43:0/1689930438 wait complete. 2024-09-17T05:32:44.746 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-09-17T05:32:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.843+0000 7f34cfc49080 1 Processor -- start 2024-09-17T05:32:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.843+0000 7f34cfc49080 1 -- start start 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c0e79e0 0x561f5c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c1564d0 0x561f5c14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f5c0a2a60 con 0x561f5c1564d0 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561f5bfeba00 con 0x561f5c09d360 2024-09-17T05:32:44.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561f5c158b00 con 0x561f5c0e79e0 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c1564d0 0x561f5c14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:33764/0 (socket says 172.21.15.43:33764) 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 -- 172.21.15.43:0/600043441 learned_addr learned my addr 172.21.15.43:0/600043441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c7800640 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c0e79e0 0x561f5c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 -- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c0e79e0 msgr2=0x561f5c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c0e79e0 0x561f5c154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 -- 172.21.15.43:0/600043441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c1564d0 msgr2=0x561f5c14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c1564d0 0x561f5c14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 -- 172.21.15.43:0/600043441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5bfeaa20 con 0x561f5c09d360 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c8200640 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b580 tx=0x7f34b800ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f308c3dfbde262 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c6400640 1 -- 172.21.15.43:0/600043441 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b800a870 con 0x561f5c09d360 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c6400640 1 -- 172.21.15.43:0/600043441 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34b800aa30 con 0x561f5c09d360 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34c6400640 1 -- 172.21.15.43:0/600043441 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8015460 con 0x561f5c09d360 2024-09-17T05:32:44.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 msgr2=0x561f5c152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b580 tx=0x7f34b800ba40 comp rx=0 tx=0).stop 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- 172.21.15.43:0/600043441 shutdown_connections 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c1564d0 0x561f5c14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c0e79e0 0x561f5c154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 --2- 172.21.15.43:0/600043441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c09d360 0x561f5c152d70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- 172.21.15.43:0/600043441 >> 172.21.15.43:0/600043441 conn(0x561f5c089d80 msgr2=0x561f5c0e70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- 172.21.15.43:0/600043441 shutdown_connections 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 -- 172.21.15.43:0/600043441 wait complete. 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.847+0000 7f34cfc49080 1 Processor -- start 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- start start 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c09d360 0x561f5c13a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 0x561f5c2109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c1564d0 0x561f5c210f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561f5c151570 con 0x561f5c0e79e0 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561f5c0e25c0 con 0x561f5c1564d0 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561f5c174310 con 0x561f5c09d360 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 0x561f5c2109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 0x561f5c2109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36292/0 (socket says 172.21.15.43:36292) 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 -- 172.21.15.43:0/3704464636 learned_addr learned my addr 172.21.15.43:0/3704464636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c8200640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c09d360 0x561f5c13a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c8c00640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c1564d0 0x561f5c210f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 -- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c09d360 msgr2=0x561f5c13a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561f5c09d360 0x561f5c13a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 -- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c1564d0 msgr2=0x561f5c210f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f5c1564d0 0x561f5c210f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 -- 172.21.15.43:0/3704464636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5bfc4c10 con 0x561f5c0e79e0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c7800640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 0x561f5c2109c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f34b4004fd0 tx=0x7f34b400abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c5000640 1 -- 172.21.15.43:0/3704464636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4009dc0 con 0x561f5c0e79e0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c5000640 1 -- 172.21.15.43:0/3704464636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34b400ae10 con 0x561f5c0e79e0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f5c175590 con 0x561f5c0e79e0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34c5000640 1 -- 172.21.15.43:0/3704464636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4008680 con 0x561f5c0e79e0 2024-09-17T05:32:44.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.851+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5c175ad0 con 0x561f5c0e79e0 2024-09-17T05:32:44.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c5000640 1 -- 172.21.15.43:0/3704464636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34b4008820 con 0x561f5c0e79e0 2024-09-17T05:32:44.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c5000640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34a8071be0 0x7f34a80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c5000640 1 -- 172.21.15.43:0/3704464636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8031+0+0 (secure 0 0 0) 0x7f34b40944f0 con 0x561f5c0e79e0 2024-09-17T05:32:44.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c8200640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34a8071be0 0x7f34a80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34cfc49080 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f5c20d3d0 0x561f5c20d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:44.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561f5c176bd0 con 0x561f5c20d3d0 2024-09-17T05:32:44.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c8c00640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f5c20d3d0 0x561f5c20d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:44.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c8200640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34a8071be0 0x7f34a80740f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f34b8000c10 tx=0x7f34b8019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.855+0000 7f34c8c00640 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f5c20d3d0 0x561f5c20d7e0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34c8c00640 1 -- 172.21.15.43:0/3704464636 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v116'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f34c0002050 con 0x561f5c20d3d0 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f5c20d3d0 msgr2=0x561f5c20d7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x561f5c20d3d0 0x561f5c20d7e0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34a8071be0 msgr2=0x7f34a80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f34a8071be0 0x7f34a80740f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f34b8000c10 tx=0x7f34b8019040 comp rx=0 tx=0).stop 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 msgr2=0x561f5c2109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 --2- 172.21.15.43:0/3704464636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561f5c0e79e0 0x561f5c2109c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f34b4004fd0 tx=0x7f34b400abc0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34c8c00640 1 -- 172.21.15.43:0/3704464636 reap_dead start 2024-09-17T05:32:44.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 shutdown_connections 2024-09-17T05:32:44.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 >> 172.21.15.43:0/3704464636 conn(0x561f5c089d80 msgr2=0x561f5c0e70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 shutdown_connections 2024-09-17T05:32:44.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:44.859+0000 7f34cfc49080 1 -- 172.21.15.43:0/3704464636 wait complete. 2024-09-17T05:32:44.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 -- 172.21.15.43:0/1150173939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155d70 msgr2=0x7f5ac415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 --2- 172.21.15.43:0/1150173939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155d70 0x7f5ac415c7e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40af0a0 tx=0x7f5ab801c800 comp rx=0 tx=0).stop 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 -- 172.21.15.43:0/1150173939 shutdown_connections 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 --2- 172.21.15.43:0/1150173939 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 0x7f5ac40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 --2- 172.21.15.43:0/1150173939 >> v2:172.21.15.62:3300/0 conn(0x7f5ac406f110 0x7f5ac40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 --2- 172.21.15.43:0/1150173939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155d70 0x7f5ac415c7e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 -- 172.21.15.43:0/1150173939 >> 172.21.15.43:0/1150173939 conn(0x7f5ac40b23d0 msgr2=0x7f5ac40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 -- 172.21.15.43:0/1150173939 shutdown_connections 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.023+0000 7f5acc000640 1 -- 172.21.15.43:0/1150173939 wait complete. 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 Processor -- start 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 -- start start 2024-09-17T05:32:45.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 0x7f5ac4155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4155d70 0x7f5ac408bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac415f8c0 con 0x7f5ac406f110 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac415f740 con 0x7f5ac4155d70 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5acc000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac415fa40 con 0x7f5ac40b3920 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36306/0 (socket says 172.21.15.43:36306) 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 -- 172.21.15.43:0/959424732 learned_addr learned my addr 172.21.15.43:0/959424732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5aca000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4155d70 0x7f5ac408bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac8c00640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 0x7f5ac4155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 -- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 msgr2=0x7f5ac4155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 0x7f5ac4155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 -- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4155d70 msgr2=0x7f5ac408bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4155d70 0x7f5ac408bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.027+0000 7f5ac9600640 1 -- 172.21.15.43:0/959424732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac4155980 con 0x7f5ac406f110 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac9600640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5ab801aeb0 tx=0x7f5ab8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b2040 con 0x7f5ac406f110 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ab8004340 con 0x7f5ac406f110 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80052d0 con 0x7f5ac406f110 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac41f9220 con 0x7f5ac406f110 2024-09-17T05:32:45.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41f97e0 con 0x7f5ac406f110 2024-09-17T05:32:45.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac415ff30 con 0x7f5ac406f110 2024-09-17T05:32:45.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ab80bd060 con 0x7f5ac406f110 2024-09-17T05:32:45.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.031+0000 7f5ac1600640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 0x7f5aa4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.035+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8031+0+0 (secure 0 0 0) 0x7f5ab812dd50 con 0x7f5ac406f110 2024-09-17T05:32:45.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.035+0000 7f5ac8c00640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 0x7f5aa4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.035+0000 7f5ac8c00640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 0x7f5aa4075740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ac409e140 tx=0x7f5ab00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.039+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ab8131030 con 0x7f5ac406f110 2024-09-17T05:32:45.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.247+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5ac415e710 con 0x7f5ac406f110 2024-09-17T05:32:45.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.451+0000 7f5ac1600640 1 -- 172.21.15.43:0/959424732 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v251) ==== 162+0+0 (secure 0 0 0) 0x7f5ab80164b0 con 0x7f5ac406f110 2024-09-17T05:32:45.454 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 msgr2=0x7f5aa4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 0x7f5aa4075740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ac409e140 tx=0x7f5ab00091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 msgr2=0x7f5ac40cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5ab801aeb0 tx=0x7f5ab8005e70 comp rx=0 tx=0).stop 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 shutdown_connections 2024-09-17T05:32:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5aa4073230 0x7f5aa4075740 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ac4155d70 0x7f5ac408bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5ac40b3920 0x7f5ac4155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 --2- 172.21.15.43:0/959424732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac406f110 0x7f5ac40cd380 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 >> 172.21.15.43:0/959424732 conn(0x7f5ac40b23d0 msgr2=0x7f5ac4074880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 shutdown_connections 2024-09-17T05:32:45.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.455+0000 7f5acc000640 1 -- 172.21.15.43:0/959424732 wait complete. 2024-09-17T05:32:45.465 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:45.463+0000 7f31e3200640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-17T05:32:45.466 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:45.463+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:45.466 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:45.463+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:45.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:45.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- 172.21.15.43:0/471828643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 msgr2=0x7fcff40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 --2- 172.21.15.43:0/471828643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff40ad0b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fcff40af0a0 tx=0x7fcfec01cac0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- 172.21.15.43:0/471828643 shutdown_connections 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 --2- 172.21.15.43:0/471828643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff40ad0b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 --2- 172.21.15.43:0/471828643 >> v2:172.21.15.62:3300/0 conn(0x7fcff40b3af0 0x7fcff415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- 172.21.15.43:0/471828643 >> 172.21.15.43:0/471828643 conn(0x7fcff40b23d0 msgr2=0x7fcff40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- 172.21.15.43:0/471828643 shutdown_connections 2024-09-17T05:32:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- 172.21.15.43:0/471828643 wait complete. 2024-09-17T05:32:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 Processor -- start 2024-09-17T05:32:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.639+0000 7fcffc400640 1 -- start start 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcff4155d70 0x7fcff409be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcff409c630 0x7fcff40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4168b90 con 0x7fcff406f110 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4168a10 con 0x7fcff4155d70 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcff4168d10 con 0x7fcff409c630 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36320/0 (socket says 172.21.15.43:36320) 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 -- 172.21.15.43:0/30950418 learned_addr learned my addr 172.21.15.43:0/30950418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9000640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcff4155d70 0x7fcff409be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffa400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcff409c630 0x7fcff40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 -- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcff409c630 msgr2=0x7fcff40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcff409c630 0x7fcff40cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 -- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcff4155d70 msgr2=0x7fcff409be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcff4155d70 0x7fcff409be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 -- 172.21.15.43:0/30950418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff4073400 con 0x7fcff406f110 2024-09-17T05:32:45.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff9a00640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c3a0 tx=0x7fcfe800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8017070 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe8003600 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003a50 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff40736e0 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41fbb10 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.643+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff40ad0b0 con 0x7fcff406f110 2024-09-17T05:32:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.647+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfe801a070 con 0x7fcff406f110 2024-09-17T05:32:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.647+0000 7fcff1600640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 0x7fcfd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.647+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7fcfe8093e50 con 0x7fcff406f110 2024-09-17T05:32:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.647+0000 7fcff9000640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 0x7fcfd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.647+0000 7fcff9000640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 0x7fcfd4075710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7640 tx=0x7fcfe000b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.651+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfe805de20 con 0x7fcff406f110 2024-09-17T05:32:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.859+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff409c3b0 con 0x7fcff406f110 2024-09-17T05:32:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.859+0000 7fcff1600640 1 -- 172.21.15.43:0/30950418 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4383 (secure 0 0 0) 0x7fcfe8062850 con 0x7fcff406f110 2024-09-17T05:32:45.863 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:45.863 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":253,"btime":"2024-09-17T05:32:45:467037+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3412634924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3412634924},{"type":"v1","addr":"172.21.15.62:6835","nonce":3412634924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":250}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:45.466948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 253 2024-09-17T05:32:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 msgr2=0x7fcfd4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 0x7fcfd4075710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7640 tx=0x7fcfe000b390 comp rx=0 tx=0).stop 2024-09-17T05:32:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 msgr2=0x7fcff41fc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c3a0 tx=0x7fcfe800c860 comp rx=0 tx=0).stop 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 shutdown_connections 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcfd4073200 0x7fcfd4075710 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcff4155d70 0x7fcff409be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcff409c630 0x7fcff40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 --2- 172.21.15.43:0/30950418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcff406f110 0x7fcff41fc310 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 >> 172.21.15.43:0/30950418 conn(0x7fcff40b23d0 msgr2=0x7fcff415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 shutdown_connections 2024-09-17T05:32:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:45.863+0000 7fcffc400640 1 -- 172.21.15.43:0/30950418 wait complete. 2024-09-17T05:32:45.885 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:45.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:46.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 -- 172.21.15.43:0/65969521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d00b3af0 msgr2=0x7fe6d00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 --2- 172.21.15.43:0/65969521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d00b3af0 0x7fe6d00ad0b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009bc0 tx=0x7fe6c801cae0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 -- 172.21.15.43:0/65969521 shutdown_connections 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 --2- 172.21.15.43:0/65969521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d00b3af0 0x7fe6d00ad0b0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 --2- 172.21.15.43:0/65969521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 0x7fe6d015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 -- 172.21.15.43:0/65969521 >> 172.21.15.43:0/65969521 conn(0x7fe6d00b23d0 msgr2=0x7fe6d00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 -- 172.21.15.43:0/65969521 shutdown_connections 2024-09-17T05:32:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.043+0000 7fe6d5200640 1 -- 172.21.15.43:0/65969521 wait complete. 2024-09-17T05:32:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 Processor -- start 2024-09-17T05:32:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- start start 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6d006f110 0x7fe6d01f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 0x7fe6d009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0168be0 con 0x7fe6d015a710 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0168a60 con 0x7fe6d006f110 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d0168d60 con 0x7fe6d0155d70 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36338/0 (socket says 172.21.15.43:36338) 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 -- 172.21.15.43:0/3466476243 learned_addr learned my addr 172.21.15.43:0/3466476243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6c7e00640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 0x7fe6d009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6d006f110 0x7fe6d01f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 -- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 msgr2=0x7fe6d009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 0x7fe6d009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 -- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6d006f110 msgr2=0x7fe6d01f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6d006f110 0x7fe6d01f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 -- 172.21.15.43:0/3466476243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d00d0740 con 0x7fe6d015a710 2024-09-17T05:32:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6ce800640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6c801c550 tx=0x7fe6c8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80b2040 con 0x7fe6d015a710 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c80bb070 con 0x7fe6d015a710 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d00ad3c0 con 0x7fe6d015a710 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004370 con 0x7fe6d015a710 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.047+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d015b060 con 0x7fe6d015a710 2024-09-17T05:32:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.051+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d00ad0b0 con 0x7fe6d015a710 2024-09-17T05:32:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.051+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6c8004510 con 0x7fe6d015a710 2024-09-17T05:32:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.051+0000 7fe6c6000640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 0x7fe6ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.051+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7fe6c81300d0 con 0x7fe6d015a710 2024-09-17T05:32:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.051+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 0x7fe6ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.055+0000 7fe6cde00640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 0x7fe6ac075710 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00715c0 tx=0x7fe6c00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.055+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe6d015b060 con 0x7fe6d015a710 2024-09-17T05:32:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.271+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d00b1200 con 0x7fe6d015a710 2024-09-17T05:32:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.271+0000 7fe6c6000640 1 -- 172.21.15.43:0/3466476243 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4383 (secure 0 0 0) 0x7fe6c8016790 con 0x7fe6d015a710 2024-09-17T05:32:46.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:46.274 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":253,"btime":"2024-09-17T05:32:45:467037+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3412634924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3412634924},{"type":"v1","addr":"172.21.15.62:6835","nonce":3412634924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":250}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:45.466948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 253 2024-09-17T05:32:46.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 msgr2=0x7fe6ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 0x7fe6ac075710 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00715c0 tx=0x7fe6c00091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 msgr2=0x7fe6d015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6c801c550 tx=0x7fe6c8003f40 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 shutdown_connections 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe6ac073200 0x7fe6ac075710 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d015a710 0x7fe6d015ab20 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6d0155d70 0x7fe6d009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 --2- 172.21.15.43:0/3466476243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6d006f110 0x7fe6d01f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 >> 172.21.15.43:0/3466476243 conn(0x7fe6d00b23d0 msgr2=0x7fe6d00b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 shutdown_connections 2024-09-17T05:32:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.275+0000 7fe6d5200640 1 -- 172.21.15.43:0/3466476243 wait complete. 2024-09-17T05:32:46.296 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:46.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- 172.21.15.43:0/1674675110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 msgr2=0x7f30d40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 --2- 172.21.15.43:0/1674675110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d40734c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009c80 tx=0x7f30cc01caa0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- 172.21.15.43:0/1674675110 shutdown_connections 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 --2- 172.21.15.43:0/1674675110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d40734c0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 --2- 172.21.15.43:0/1674675110 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d406fc90 0x7f30d40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- 172.21.15.43:0/1674675110 >> 172.21.15.43:0/1674675110 conn(0x7f30d40b23d0 msgr2=0x7f30d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- 172.21.15.43:0/1674675110 shutdown_connections 2024-09-17T05:32:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- 172.21.15.43:0/1674675110 wait complete. 2024-09-17T05:32:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 Processor -- start 2024-09-17T05:32:46.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.455+0000 7f30db200640 1 -- start start 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 0x7f30d4153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d415fe10 0x7f30d41fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30d416a060 con 0x7f30d40730b0 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4169ee0 con 0x7f30d406fc90 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30db200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f30d416a1e0 con 0x7f30d415fe10 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36364/0 (socket says 172.21.15.43:36364) 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 -- 172.21.15.43:0/3714014727 learned_addr learned my addr 172.21.15.43:0/3714014727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30d9200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d415fe10 0x7f30d41fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30d3e00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 0x7f30d4153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 -- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d415fe10 msgr2=0x7f30d41fd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d415fe10 0x7f30d41fd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 -- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 msgr2=0x7f30d4153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 0x7f30d4153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 -- 172.21.15.43:0/3714014727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d41516f0 con 0x7f30d40730b0 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30d3e00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 0x7f30d4153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:46.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.459+0000 7f30cbe00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b410 tx=0x7f30bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc015020 con 0x7f30d40730b0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30bc011920 con 0x7f30d40730b0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41519d0 con 0x7f30d40730b0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc011d70 con 0x7f30d40730b0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d4151f90 con 0x7f30d40730b0 2024-09-17T05:32:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d40700a0 con 0x7f30d40730b0 2024-09-17T05:32:46.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30bc018020 con 0x7f30d40730b0 2024-09-17T05:32:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7f30bc094b00 con 0x7f30d40730b0 2024-09-17T05:32:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.463+0000 7f30d3e00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.467+0000 7f30d3e00640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 0x7f30b0075740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009890 tx=0x7f30c4008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.467+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30bc01f020 con 0x7f30d40730b0 2024-09-17T05:32:46.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.679+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30d4151450 con 0x7f30d40730b0 2024-09-17T05:32:46.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.679+0000 7f30d0c00640 1 -- 172.21.15.43:0/3714014727 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4383 (secure 0 0 0) 0x7f30bc094e20 con 0x7f30d40730b0 2024-09-17T05:32:46.682 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:46.682 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":253,"btime":"2024-09-17T05:32:45:467037+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8815,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3412634924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3412634924},{"type":"v1","addr":"172.21.15.62:6835","nonce":3412634924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":250}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:45.466948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:46.682 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 253 2024-09-17T05:32:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 msgr2=0x7f30b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 0x7f30b0075740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009890 tx=0x7f30c4008040 comp rx=0 tx=0).stop 2024-09-17T05:32:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 msgr2=0x7f30d4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b410 tx=0x7f30bc00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 shutdown_connections 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f30d415fe10 0x7f30d41fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30d40730b0 0x7f30d4156570 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 --2- 172.21.15.43:0/3714014727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d406fc90 0x7f30d4153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 >> 172.21.15.43:0/3714014727 conn(0x7f30d40b23d0 msgr2=0x7f30d40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 shutdown_connections 2024-09-17T05:32:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.683+0000 7f30db200640 1 -- 172.21.15.43:0/3714014727 wait complete. 2024-09-17T05:32:46.704 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-09-17T05:32:46.704 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-09-17T05:32:46.704 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:46.705 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:46.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:46.713 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:46.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:46.713 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:46.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:46.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 -- 172.21.15.43:0/4259198766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 msgr2=0x7fb1580cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 --2- 172.21.15.43:0/4259198766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1580cd370 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb1580afb40 tx=0x7fb14801c770 comp rx=0 tx=0).stop 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 -- 172.21.15.43:0/4259198766 shutdown_connections 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 --2- 172.21.15.43:0/4259198766 >> v2:172.21.15.62:3300/0 conn(0x7fb1580ce3c0 0x7fb1580cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 --2- 172.21.15.43:0/4259198766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1580cd370 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 -- 172.21.15.43:0/4259198766 >> 172.21.15.43:0/4259198766 conn(0x7fb1580b2810 msgr2=0x7fb1580b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 -- 172.21.15.43:0/4259198766 shutdown_connections 2024-09-17T05:32:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.963+0000 7fb15e200640 1 -- 172.21.15.43:0/4259198766 wait complete. 2024-09-17T05:32:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 Processor -- start 2024-09-17T05:32:46.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- start start 2024-09-17T05:32:46.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 0x7fb15814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 0x7fb158165860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb15816a150 con 0x7fb1580cdfb0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb158169fd0 con 0x7fb1581603e0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb15816a2d0 con 0x7fb1580ce3c0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36378/0 (socket says 172.21.15.43:36378) 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 -- 172.21.15.43:0/1164017961 learned_addr learned my addr 172.21.15.43:0/1164017961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb156a00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 0x7fb15814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157e00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 0x7fb158165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 -- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 msgr2=0x7fb15814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 0x7fb15814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 -- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 msgr2=0x7fb158165860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 0x7fb158165860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 -- 172.21.15.43:0/1164017961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb158150610 con 0x7fb1580cdfb0 2024-09-17T05:32:46.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb156a00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 0x7fb15814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157e00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 0x7fb158165860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb157400640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb148098150 tx=0x7fb1480a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1480be070 con 0x7fb1580cdfb0 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1480a6b40 con 0x7fb1580cdfb0 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148004040 con 0x7fb1580cdfb0 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1581508f0 con 0x7fb1580cdfb0 2024-09-17T05:32:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.967+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb158166330 con 0x7fb1580cdfb0 2024-09-17T05:32:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1581607f0 con 0x7fb1580cdfb0 2024-09-17T05:32:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb1480041e0 con 0x7fb1580cdfb0 2024-09-17T05:32:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb14f400640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 0x7fb138075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7fb14812e5e0 con 0x7fb1580cdfb0 2024-09-17T05:32:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb156a00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 0x7fb138075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.971+0000 7fb156a00640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 0x7fb138075710 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb144005fd0 tx=0x7fb144005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:46.975+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb1480ac030 con 0x7fb1580cdfb0 2024-09-17T05:32:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.183+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb15815adc0 con 0x7fb1580cdfb0 2024-09-17T05:32:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.471+0000 7fb14f400640 1 -- 172.21.15.43:0/1164017961 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v254) ==== 161+0+0 (secure 0 0 0) 0x7fb1480163f0 con 0x7fb1580cdfb0 2024-09-17T05:32:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 msgr2=0x7fb138075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 0x7fb138075710 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb144005fd0 tx=0x7fb144005de0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 msgr2=0x7fb1581513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb148098150 tx=0x7fb1480a65a0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 shutdown_connections 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb138073200 0x7fb138075710 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1581603e0 0x7fb158165860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb1580ce3c0 0x7fb15814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 --2- 172.21.15.43:0/1164017961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1580cdfb0 0x7fb1581513d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 >> 172.21.15.43:0/1164017961 conn(0x7fb1580b2810 msgr2=0x7fb15815e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 shutdown_connections 2024-09-17T05:32:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.475+0000 7fb15e200640 1 -- 172.21.15.43:0/1164017961 wait complete. 2024-09-17T05:32:47.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 -- 172.21.15.43:0/3532740147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa415fe10 msgr2=0x7fafa40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 --2- 172.21.15.43:0/3532740147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa415fe10 0x7fafa40b4b80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf9401c700 comp rx=0 tx=0).stop 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 -- 172.21.15.43:0/3532740147 shutdown_connections 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 --2- 172.21.15.43:0/3532740147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa415fe10 0x7fafa40b4b80 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 -- 172.21.15.43:0/3532740147 >> 172.21.15.43:0/3532740147 conn(0x7fafa40b23d0 msgr2=0x7fafa40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 -- 172.21.15.43:0/3532740147 shutdown_connections 2024-09-17T05:32:47.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.655+0000 7fafaa800640 1 -- 172.21.15.43:0/3532740147 wait complete. 2024-09-17T05:32:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 Processor -- start 2024-09-17T05:32:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- start start 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa406fc90 0x7fafa40c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa415fe10 0x7fafa415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafa416aff0 con 0x7fafa4099c70 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fafa416ae70 con 0x7fafa406fc90 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fafa416b170 con 0x7fafa415fe10 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36400/0 (socket says 172.21.15.43:36400) 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 -- 172.21.15.43:0/1441357135 learned_addr learned my addr 172.21.15.43:0/1441357135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3400640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa406fc90 0x7fafa40c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7faf9aa00640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa415fe10 0x7fafa415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 -- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa415fe10 msgr2=0x7fafa415d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa415fe10 0x7fafa415d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 -- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa406fc90 msgr2=0x7fafa40c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa406fc90 0x7fafa40c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 -- 172.21.15.43:0/1441357135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa41564d0 con 0x7fafa4099c70 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafa3e00640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c730 tx=0x7faf9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c003b80 con 0x7fafa4099c70 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c003d20 con 0x7fafa4099c70 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00aab0 con 0x7fafa4099c70 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa41567b0 con 0x7fafa4099c70 2024-09-17T05:32:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.659+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa4156d70 con 0x7fafa4099c70 2024-09-17T05:32:47.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.663+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa40b4b80 con 0x7fafa4099c70 2024-09-17T05:32:47.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.663+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf9c00ac50 con 0x7fafa4099c70 2024-09-17T05:32:47.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.663+0000 7faf9be00640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 0x7faf80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.663+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7faf9c094880 con 0x7fafa4099c70 2024-09-17T05:32:47.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.663+0000 7fafa3400640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 0x7faf80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.667+0000 7fafa3400640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 0x7faf80075710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faf940af000 tx=0x7faf9401af40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.667+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf9c05e8d0 con 0x7fafa4099c70 2024-09-17T05:32:47.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:47.907+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fafa415bbe0 con 0x7fafa4099c70 2024-09-17T05:32:48.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.483+0000 7faf9be00640 1 -- 172.21.15.43:0/1441357135 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v255) ==== 101+0+0 (secure 0 0 0) 0x7faf9c063300 con 0x7fafa4099c70 2024-09-17T05:32:48.487 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 msgr2=0x7faf80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 0x7faf80075710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faf940af000 tx=0x7faf9401af40 comp rx=0 tx=0).stop 2024-09-17T05:32:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 msgr2=0x7fafa415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c730 tx=0x7faf9c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 shutdown_connections 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faf80073200 0x7faf80075710 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa4099c70 0x7fafa415deb0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fafa415fe10 0x7fafa415d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 --2- 172.21.15.43:0/1441357135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafa406fc90 0x7fafa40c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 >> 172.21.15.43:0/1441357135 conn(0x7fafa40b23d0 msgr2=0x7fafa4157b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 shutdown_connections 2024-09-17T05:32:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.487+0000 7fafaa800640 1 -- 172.21.15.43:0/1441357135 wait complete. 2024-09-17T05:32:48.512 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 Processor -- start 2024-09-17T05:32:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 -- start start 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 0x563bd16a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd15ffe40 0x563bd16a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563bd1497a00 con 0x563bd1549d70 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563bd14f5f40 con 0x563bd1578cb0 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563bd1600890 con 0x563bd15ffe40 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36406/0 (socket says 172.21.15.43:36406) 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 -- 172.21.15.43:0/216024476 learned_addr learned my addr 172.21.15.43:0/216024476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3e800640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd15ffe40 0x563bd16a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3d400640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 0x563bd16a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 -- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd15ffe40 msgr2=0x563bd16a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd15ffe40 0x563bd16a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 -- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 msgr2=0x563bd16a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 0x563bd16a6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 -- 172.21.15.43:0/216024476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd1470c10 con 0x563bd1549d70 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3d400640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 0x563bd16a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c3de00640 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c590 tx=0x7f6c3000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf975291d13ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c37e00640 1 -- 172.21.15.43:0/216024476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000aa70 con 0x563bd1549d70 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c37e00640 1 -- 172.21.15.43:0/216024476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c3000ac10 con 0x563bd1549d70 2024-09-17T05:32:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.571+0000 7f6c37e00640 1 -- 172.21.15.43:0/216024476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30008430 con 0x563bd1549d70 2024-09-17T05:32:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- 172.21.15.43:0/216024476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 msgr2=0x563bd161ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c590 tx=0x7f6c3000ca50 comp rx=0 tx=0).stop 2024-09-17T05:32:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- 172.21.15.43:0/216024476 shutdown_connections 2024-09-17T05:32:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd1578cb0 0x563bd16a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd15ffe40 0x563bd16a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/216024476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd161ee70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- 172.21.15.43:0/216024476 >> 172.21.15.43:0/216024476 conn(0x563bd1535d80 msgr2=0x563bd15783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- 172.21.15.43:0/216024476 shutdown_connections 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- 172.21.15.43:0/216024476 wait complete. 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 Processor -- start 2024-09-17T05:32:48.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- start start 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd1578cb0 0x563bd158f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd15ffe40 0x563bd16b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563bd158a8e0 con 0x563bd1549d70 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563bd1591500 con 0x563bd15ffe40 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c44c4e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563bd15917a0 con 0x563bd1578cb0 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36422/0 (socket says 172.21.15.43:36422) 2024-09-17T05:32:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 learned_addr learned my addr 172.21.15.43:0/3222848913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd15ffe40 0x563bd16b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd1578cb0 0x563bd158f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd1578cb0 msgr2=0x563bd158f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563bd1578cb0 0x563bd158f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd15ffe40 msgr2=0x563bd16b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd15ffe40 0x563bd16b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd16b8460 con 0x563bd1549d70 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bd15ffe40 0x563bd16b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.575+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6c300086e0 tx=0x7f6c30008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 -- 172.21.15.43:0/3222848913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30008b40 con 0x563bd1549d70 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 -- 172.21.15.43:0/3222848913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c30008df0 con 0x563bd1549d70 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 -- 172.21.15.43:0/3222848913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3001d690 con 0x563bd1549d70 2024-09-17T05:32:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bd1609840 con 0x563bd1549d70 2024-09-17T05:32:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd16a96c0 con 0x563bd1549d70 2024-09-17T05:32:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 -- 172.21.15.43:0/3222848913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c30024020 con 0x563bd1549d70 2024-09-17T05:32:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 0x7f6c200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c36a00640 1 -- 172.21.15.43:0/3222848913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7f6c30094b00 con 0x563bd1549d70 2024-09-17T05:32:48.583 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563bd16b4a20 0x563bd16a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 0x7f6c200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.579+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16aa340 con 0x563bd16b4a20 2024-09-17T05:32:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.583+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563bd16b4a20 0x563bd16a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.583+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 0x7f6c200740f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c10 tx=0x7f6c2c001810 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.583+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563bd16b4a20 0x563bd16a9e00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.583+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v117'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.583+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16aa710 con 0x563bd16b4a20 2024-09-17T05:32:48.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.587+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v117'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.587+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c0790 con 0x563bd16b4a20 2024-09-17T05:32:48.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.587+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v117'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.590 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:48.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.587+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1b40 con 0x563bd16b4a20 2024-09-17T05:32:48.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1ee0 con 0x563bd16b4a20 2024-09-17T05:32:48.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c0b30 con 0x563bd16b4a20 2024-09-17T05:32:48.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.591+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c0f20 con 0x563bd16b4a20 2024-09-17T05:32:48.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.595+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v117'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.595+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c1310 con 0x563bd16b4a20 2024-09-17T05:32:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.595+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v117'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.595+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1310 con 0x563bd16b4a20 2024-09-17T05:32:48.599 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:48.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v117'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.599+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.603+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.607+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.609 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.607+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.607+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.607+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c12c0 con 0x563bd16b4a20 2024-09-17T05:32:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1720 con 0x563bd16b4a20 2024-09-17T05:32:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.611+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.615+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v117'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.615+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.615+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v117'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.615+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.619+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v117'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.620 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.619+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.619+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.619+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.619+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.623+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.623+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.623+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.623+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v117'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.623+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v117'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v117'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.630 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:48.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.627+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16b4a20 2024-09-17T05:32:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563bd16c2790 0x563bd16c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c1540 con 0x563bd16c2790 2024-09-17T05:32:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563bd16c2790 0x563bd16c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.631+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563bd16c2790 0x563bd16c4bb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.635+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'616 uv599 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.635+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c50f0 con 0x563bd16c2790 2024-09-17T05:32:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.635+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'617 uv617 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.635+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c5490 con 0x563bd16c2790 2024-09-17T05:32:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.640 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c5ce0 con 0x563bd16c2790 2024-09-17T05:32:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6080 con 0x563bd16c2790 2024-09-17T05:32:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.639+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6420 con 0x563bd16c2790 2024-09-17T05:32:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.643+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v117'621 uv604 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.643+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c67c0 con 0x563bd16c2790 2024-09-17T05:32:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.643+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v117'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.643+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.647 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v117'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.647+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v117'626 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v117'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v117'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.655 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:48.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.651+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cad30 con 0x563bd16c2790 2024-09-17T05:32:48.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.655+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v117'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.655+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cad30 con 0x563bd16c2790 2024-09-17T05:32:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.655+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.655+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v117'631 uv615 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v117'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v117'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.663 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.659+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c7b70 con 0x563bd16c2790 2024-09-17T05:32:48.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.663+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.663+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c7b70 con 0x563bd16c2790 2024-09-17T05:32:48.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.663+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.663+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c6b60 con 0x563bd16c2790 2024-09-17T05:32:48.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16c2790 2024-09-17T05:32:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563bd16c7070 0x563bd16c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c99d0 con 0x563bd16c7070 2024-09-17T05:32:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563bd16c7070 0x563bd16c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563bd16c7070 0x563bd16c9490 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v117'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.667+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16c9e30 con 0x563bd16c7070 2024-09-17T05:32:48.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v117'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16c5920 con 0x563bd16c7070 2024-09-17T05:32:48.673 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:48.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v117'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cbc10 con 0x563bd16c7070 2024-09-17T05:32:48.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.671+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cbfb0 con 0x563bd16c7070 2024-09-17T05:32:48.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cac00 con 0x563bd16c7070 2024-09-17T05:32:48.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16caff0 con 0x563bd16b4a20 2024-09-17T05:32:48.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'118 uv117 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.675+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16caff0 con 0x563bd16b4a20 2024-09-17T05:32:48.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'119 uv119 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16caff0 con 0x563bd16b4a20 2024-09-17T05:32:48.681 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'120 uv120 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16caff0 con 0x563bd16b4a20 2024-09-17T05:32:48.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'121 uv121 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 0x563bd16ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cecb0 con 0x563bd16cc350 2024-09-17T05:32:48.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.679+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 0x563bd16ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.683+0000 7f6c3e800640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 0x563bd16ce770 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.683+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v117'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.683+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16cecb0 con 0x563bd16cc350 2024-09-17T05:32:48.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.683+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v117'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.683+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cecb0 con 0x563bd16cc350 2024-09-17T05:32:48.688 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:48.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v117'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cfcc0 con 0x563bd16cc350 2024-09-17T05:32:48.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cfcc0 con 0x563bd16cc350 2024-09-17T05:32:48.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.687+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16cecb0 con 0x563bd16cc350 2024-09-17T05:32:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563bd16cf1c0 0x563bd16d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d1c70 con 0x563bd16cf1c0 2024-09-17T05:32:48.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563bd16cf1c0 0x563bd16d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c3de00640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563bd16cf1c0 0x563bd16d15e0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v117'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.691+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v117'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.697 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v117'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d3050 con 0x563bd16cf1c0 2024-09-17T05:32:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.695+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d3050 con 0x563bd16cf1c0 2024-09-17T05:32:48.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v117'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v117'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.699+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.703+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v117'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.703+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.703+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.703+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d2040 con 0x563bd16cf1c0 2024-09-17T05:32:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c3de00640 1 -- 172.21.15.43:0/3222848913 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c3005ada0 con 0x563bd16cf1c0 2024-09-17T05:32:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563bd16d26c0 0x563bd16d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5230 con 0x563bd16d26c0 2024-09-17T05:32:48.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563bd16d26c0 0x563bd16d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c3d400640 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563bd16d26c0 0x563bd16d4ae0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v117'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5600 con 0x563bd16d26c0 2024-09-17T05:32:48.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v117'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.707+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16d26c0 2024-09-17T05:32:48.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v117'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.712 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:48.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16d26c0 2024-09-17T05:32:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16d26c0 2024-09-17T05:32:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.711+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16b4a20 2024-09-17T05:32:48.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.715+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v117'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.715+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16b4a20 2024-09-17T05:32:48.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.715+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v117'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.715+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16b4a20 2024-09-17T05:32:48.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v117'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.719 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:48.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6a30 con 0x563bd16b4a20 2024-09-17T05:32:48.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6a30 con 0x563bd16b4a20 2024-09-17T05:32:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.719+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16b4a20 2024-09-17T05:32:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16b4a20 2024-09-17T05:32:48.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v117'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v117'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.723+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.727 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v117'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.727+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.731+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v117'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.731+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.731+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v117'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.731+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.735 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v117'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5a20 con 0x563bd16c7070 2024-09-17T05:32:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca800 con 0x563bd16c7070 2024-09-17T05:32:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16c7070 2024-09-17T05:32:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.735+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca7c0 con 0x563bd16cc350 2024-09-17T05:32:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v117'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16ca7c0 con 0x563bd16cc350 2024-09-17T05:32:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca7c0 con 0x563bd16cc350 2024-09-17T05:32:48.743 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v117'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.739+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca7c0 con 0x563bd16cc350 2024-09-17T05:32:48.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.743+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.743+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca780 con 0x563bd16cc350 2024-09-17T05:32:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.743+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v117'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.743+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16ca780 con 0x563bd16cc350 2024-09-17T05:32:48.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v117'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16ca780 con 0x563bd16cc350 2024-09-17T05:32:48.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v117'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.749 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:48.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6e40 con 0x563bd16cc350 2024-09-17T05:32:48.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.747+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6e40 con 0x563bd16cc350 2024-09-17T05:32:48.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5e30 con 0x563bd16cc350 2024-09-17T05:32:48.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c38002050 con 0x563bd16cc350 2024-09-17T05:32:48.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5e30 con 0x563bd16d26c0 2024-09-17T05:32:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v117'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.751+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v117'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v117'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.757 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:48.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.755+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v117'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v117'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.759+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.763+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v117'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.765 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.763+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.763+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.763+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v117'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v117'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.767+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v117'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.772 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:48.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6df0 con 0x563bd16d26c0 2024-09-17T05:32:48.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d6df0 con 0x563bd16d26c0 2024-09-17T05:32:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.771+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d5de0 con 0x563bd16d26c0 2024-09-17T05:32:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x563bd16d61a0 con 0x563bd16d26c0 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c3d400640 1 -- 172.21.15.43:0/3222848913 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6c2c01a040 con 0x563bd16d26c0 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563bd16c2790 msgr2=0x563bd16c4bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563bd16c2790 0x563bd16c4bb0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563bd16cf1c0 msgr2=0x563bd16d15e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563bd16cf1c0 0x563bd16d15e0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563bd16d26c0 msgr2=0x563bd16d4ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563bd16d26c0 0x563bd16d4ae0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563bd16c7070 msgr2=0x563bd16c9490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563bd16c7070 0x563bd16c9490 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563bd16b4a20 msgr2=0x563bd16a9e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563bd16b4a20 0x563bd16a9e00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 reap_dead start 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 msgr2=0x563bd16ce770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 0x563bd16ce770 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c3e800640 1 -- 172.21.15.43:0/3222848913 reap_dead start 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 msgr2=0x7f6c200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 0x7f6c200740f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c10 tx=0x7f6c2c001810 comp rx=0 tx=0).stop 2024-09-17T05:32:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 msgr2=0x563bd1606890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6c300086e0 tx=0x7f6c30008710 comp rx=0 tx=0).stop 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 shutdown_connections 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563bd16cc350 0x563bd16ce770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c20071be0 0x7f6c200740f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 --2- 172.21.15.43:0/3222848913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563bd1549d70 0x563bd1606890 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 >> 172.21.15.43:0/3222848913 conn(0x563bd1535d80 msgr2=0x563bd1547890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 shutdown_connections 2024-09-17T05:32:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.775+0000 7f6c44c4e080 1 -- 172.21.15.43:0/3222848913 wait complete. 2024-09-17T05:32:48.783 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.883+0000 7f80f31fc080 1 Processor -- start 2024-09-17T05:32:48.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.883+0000 7f80f31fc080 1 -- start start 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65ee5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65ee2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c65e90f40 con 0x564c65ee2fb0 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564c65e9dd00 con 0x564c65fba470 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564c65e32a00 con 0x7f80d0003100 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36430/0 (socket says 172.21.15.43:36430) 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 -- 172.21.15.43:0/462411093 learned_addr learned my addr 172.21.15.43:0/462411093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ec400640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65ee5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80eba00640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65ee2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 -- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 msgr2=0x564c65ee5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65ee5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 -- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 msgr2=0x564c65ee2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65ee2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 -- 172.21.15.43:0/462411093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c65f2a1d0 con 0x564c65ee2fb0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ec400640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65ee5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ece00640 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x564c65ed3dd0 tx=0x7f80e4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab05678933b649b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ea600640 1 -- 172.21.15.43:0/462411093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40042d0 con 0x564c65ee2fb0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ea600640 1 -- 172.21.15.43:0/462411093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80e4004470 con 0x564c65ee2fb0 2024-09-17T05:32:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80ea600640 1 -- 172.21.15.43:0/462411093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4004880 con 0x564c65ee2fb0 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- 172.21.15.43:0/462411093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 msgr2=0x564c65fb7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x564c65ed3dd0 tx=0x7f80e4001ac0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- 172.21.15.43:0/462411093 shutdown_connections 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65ee2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65fb7a50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 --2- 172.21.15.43:0/462411093 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65ee5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- 172.21.15.43:0/462411093 >> 172.21.15.43:0/462411093 conn(0x564c65ed0d80 msgr2=0x564c65fb91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- 172.21.15.43:0/462411093 shutdown_connections 2024-09-17T05:32:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- 172.21.15.43:0/462411093 wait complete. 2024-09-17T05:32:48.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 Processor -- start 2024-09-17T05:32:48.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.887+0000 7f80f31fc080 1 -- start start 2024-09-17T05:32:48.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65f265b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65fab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65fb5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564c65f2a470 con 0x564c65ee2fb0 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564c65f2a710 con 0x564c65fba470 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x564c66043480 con 0x7f80d0003100 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ec400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65f265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65fb5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80eba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65fab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65fb5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:44296/0 (socket says 172.21.15.43:44296) 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 -- 172.21.15.43:0/289471921 learned_addr learned my addr 172.21.15.43:0/289471921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 -- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 msgr2=0x564c65fab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c65fba470 0x564c65fab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 -- 172.21.15.43:0/289471921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 msgr2=0x564c65f265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65f265b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 -- 172.21.15.43:0/289471921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c65f26ec0 con 0x7f80d0003100 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ec400640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564c65ee2fb0 0x564c65f265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80ece00640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65fb5c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009a10 tx=0x7f80e4002060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80e9200640 1 -- 172.21.15.43:0/289471921 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40049d0 con 0x7f80d0003100 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80e9200640 1 -- 172.21.15.43:0/289471921 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80e4017660 con 0x7f80d0003100 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80e9200640 1 -- 172.21.15.43:0/289471921 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4017a90 con 0x7f80d0003100 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c65f27160 con 0x7f80d0003100 2024-09-17T05:32:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.891+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x564c65f276a0 con 0x7f80d0003100 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80e9200640 1 -- 172.21.15.43:0/289471921 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f80e4020030 con 0x7f80d0003100 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80e9200640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f80cc071d80 0x7f80cc074290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80e9200640 1 -- 172.21.15.43:0/289471921 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7f80e4096b40 con 0x7f80d0003100 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80ec400640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f80cc071d80 0x7f80cc074290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80f31fc080 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564c65f282f0 0x564c65f2ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x564c65f28cc0 con 0x564c65f282f0 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80eba00640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564c65f282f0 0x564c65f2ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80ec400640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f80cc071d80 0x7f80cc074290 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x564c65ed1ef0 tx=0x7f80dc003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.895+0000 7f80eba00640 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564c65f282f0 0x564c65f2ae90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80eba00640 1 -- 172.21.15.43:0/289471921 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v117'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f80d80a77c0 con 0x564c65f282f0 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564c65f282f0 msgr2=0x564c65f2ae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x564c65f282f0 0x564c65f2ae90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f80cc071d80 msgr2=0x7f80cc074290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f80cc071d80 0x7f80cc074290 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x564c65ed1ef0 tx=0x7f80dc003040 comp rx=0 tx=0).stop 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 msgr2=0x564c65fb5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 --2- 172.21.15.43:0/289471921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80d0003100 0x564c65fb5c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009a10 tx=0x7f80e4002060 comp rx=0 tx=0).stop 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80ece00640 1 -- 172.21.15.43:0/289471921 reap_dead start 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 shutdown_connections 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 >> 172.21.15.43:0/289471921 conn(0x564c65ed0d80 msgr2=0x564c65fb6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 shutdown_connections 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:48.899+0000 7f80f31fc080 1 -- 172.21.15.43:0/289471921 wait complete. 2024-09-17T05:32:48.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 -- 172.21.15.43:0/1344074222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d415fe10 msgr2=0x7f33d40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 --2- 172.21.15.43:0/1344074222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d415fe10 0x7f33d40b4b80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009930 tx=0x7f33c401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 -- 172.21.15.43:0/1344074222 shutdown_connections 2024-09-17T05:32:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 --2- 172.21.15.43:0/1344074222 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d40737c0 0x7f33d40700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 --2- 172.21.15.43:0/1344074222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d415fe10 0x7f33d40b4b80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 -- 172.21.15.43:0/1344074222 >> 172.21.15.43:0/1344074222 conn(0x7f33d40b23d0 msgr2=0x7f33d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 -- 172.21.15.43:0/1344074222 shutdown_connections 2024-09-17T05:32:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.063+0000 7f33db000640 1 -- 172.21.15.43:0/1344074222 wait complete. 2024-09-17T05:32:49.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 Processor -- start 2024-09-17T05:32:49.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 -- start start 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 0x7f33d40ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d4098270 0x7f33d40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f33d416a0a0 con 0x7f33d415fe10 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4169f20 con 0x7f33d406fc90 2024-09-17T05:32:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33db000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f33d416a220 con 0x7f33d4098270 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 0x7f33d40ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:33952/0 (socket says 172.21.15.43:33952) 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33d3e00640 1 -- 172.21.15.43:0/26404161 learned_addr learned my addr 172.21.15.43:0/26404161 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.067+0000 7f33d9000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d4098270 0x7f33d40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 -- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d4098270 msgr2=0x7f33d40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d4098270 0x7f33d40cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 -- 172.21.15.43:0/26404161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 msgr2=0x7f33d40ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 0x7f33d40ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 -- 172.21.15.43:0/26404161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d4098920 con 0x7f33d406fc90 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33cbe00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 0x7f33d40ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d3e00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33c40b0040 tx=0x7f33c4005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c40b2040 con 0x7f33d406fc90 2024-09-17T05:32:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33c4004230 con 0x7f33d406fc90 2024-09-17T05:32:49.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4005030 con 0x7f33d406fc90 2024-09-17T05:32:49.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d4098c00 con 0x7f33d406fc90 2024-09-17T05:32:49.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d40991c0 con 0x7f33d406fc90 2024-09-17T05:32:49.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d40700a0 con 0x7f33d406fc90 2024-09-17T05:32:49.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33c40051d0 con 0x7f33d406fc90 2024-09-17T05:32:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.071+0000 7f33d0c00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.075+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8117+0+0 (secure 0 0 0) 0x7f33c412e820 con 0x7f33d406fc90 2024-09-17T05:32:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.075+0000 7f33cbe00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.075+0000 7f33cbe00640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 0x7f33b0075740 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b4e10 tx=0x7f33bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.075+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33c40164d0 con 0x7f33d406fc90 2024-09-17T05:32:49.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.287+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f33d4159840 con 0x7f33d406fc90 2024-09-17T05:32:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.487+0000 7f33d0c00640 1 -- 172.21.15.43:0/26404161 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v256) ==== 162+0+0 (secure 0 0 0) 0x7f33c40f8eb0 con 0x7f33d406fc90 2024-09-17T05:32:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 msgr2=0x7f33b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 0x7f33b0075740 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b4e10 tx=0x7f33bc009040 comp rx=0 tx=0).stop 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 msgr2=0x7f33d41530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33c40b0040 tx=0x7f33c4005d60 comp rx=0 tx=0).stop 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 shutdown_connections 2024-09-17T05:32:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f33d4098270 0x7f33d40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d415fe10 0x7f33d40ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 --2- 172.21.15.43:0/26404161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33d406fc90 0x7f33d41530b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 >> 172.21.15.43:0/26404161 conn(0x7f33d40b23d0 msgr2=0x7f33d4075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 shutdown_connections 2024-09-17T05:32:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.491+0000 7f33db000640 1 -- 172.21.15.43:0/26404161 wait complete. 2024-09-17T05:32:49.497 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:49.496+0000 7fdffd600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-17T05:32:49.498 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:49.495+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:49.498 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:49.495+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:49.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:49.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- 172.21.15.43:0/769091976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d006f110 msgr2=0x7f82d00ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 --2- 172.21.15.43:0/769091976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d006f110 0x7f82d00ad0b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f82d00af0a0 tx=0x7f82c801cac0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- 172.21.15.43:0/769091976 shutdown_connections 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 --2- 172.21.15.43:0/769091976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d006f110 0x7f82d00ad0b0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 --2- 172.21.15.43:0/769091976 >> v2:172.21.15.62:3300/0 conn(0x7f82d00b3af0 0x7f82d015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- 172.21.15.43:0/769091976 >> 172.21.15.43:0/769091976 conn(0x7f82d00b23d0 msgr2=0x7f82d00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- 172.21.15.43:0/769091976 shutdown_connections 2024-09-17T05:32:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- 172.21.15.43:0/769091976 wait complete. 2024-09-17T05:32:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 Processor -- start 2024-09-17T05:32:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.675+0000 7f82d7400640 1 -- start start 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d006f110 0x7f82d00c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82d00c9630 0x7f82d00c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0168b90 con 0x7f82d0155d70 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0168a10 con 0x7f82d006f110 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f82d0168d10 con 0x7f82d00c9630 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36472/0 (socket says 172.21.15.43:36472) 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 -- 172.21.15.43:0/4271377045 learned_addr learned my addr 172.21.15.43:0/4271377045 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d4a00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d006f110 0x7f82d00c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d5400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82d00c9630 0x7f82d00c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 -- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82d00c9630 msgr2=0x7f82d00c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82d00c9630 0x7f82d00c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 -- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d006f110 msgr2=0x7f82d00c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d006f110 0x7f82d00c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 -- 172.21.15.43:0/4271377045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0165a60 con 0x7f82d0155d70 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82cfe00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00d720 tx=0x7f82bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc017070 con 0x7f82d0155d70 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82bc009d20 con 0x7f82d0155d70 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0106c0 con 0x7f82d0155d70 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d0165d40 con 0x7f82d0155d70 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d00c4600 con 0x7f82d0155d70 2024-09-17T05:32:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.679+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d00ad0b0 con 0x7f82d0155d70 2024-09-17T05:32:49.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.683+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82bc01a070 con 0x7f82d0155d70 2024-09-17T05:32:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.683+0000 7f82ccc00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 0x7f82b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.683+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7f82bc094cc0 con 0x7f82d0155d70 2024-09-17T05:32:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.683+0000 7f82d4a00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 0x7f82b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.683+0000 7f82d4a00640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 0x7f82b0075710 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f82c4007fd0 tx=0x7f82c4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.687+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82bc09b080 con 0x7f82d0155d70 2024-09-17T05:32:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.899+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82d00c43f0 con 0x7f82d0155d70 2024-09-17T05:32:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.899+0000 7f82ccc00640 1 -- 172.21.15.43:0/4271377045 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) ==== 95+0+3557 (secure 0 0 0) 0x7f82bc05ecb0 con 0x7f82d0155d70 2024-09-17T05:32:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:49.902 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":258,"btime":"2024-09-17T05:32:49:498784+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:49.498712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 258 2024-09-17T05:32:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 msgr2=0x7f82b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 0x7f82b0075710 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f82c4007fd0 tx=0x7f82c4009040 comp rx=0 tx=0).stop 2024-09-17T05:32:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 msgr2=0x7f82d00c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00d720 tx=0x7f82bc00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 shutdown_connections 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82b0073200 0x7f82b0075710 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82d00c9630 0x7f82d00c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d0155d70 0x7f82d00c4b20 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 --2- 172.21.15.43:0/4271377045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d006f110 0x7f82d00c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 >> 172.21.15.43:0/4271377045 conn(0x7f82d00b23d0 msgr2=0x7f82d015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 shutdown_connections 2024-09-17T05:32:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:49.903+0000 7f82d7400640 1 -- 172.21.15.43:0/4271377045 wait complete. 2024-09-17T05:32:49.924 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:49.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- 172.21.15.43:0/3366658016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 msgr2=0x7f2cd80b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 --2- 172.21.15.43:0/3366658016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd80b4540 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009900 tx=0x7f2cc001c850 comp rx=0 tx=0).stop 2024-09-17T05:32:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- 172.21.15.43:0/3366658016 shutdown_connections 2024-09-17T05:32:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 --2- 172.21.15.43:0/3366658016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd80b4540 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 --2- 172.21.15.43:0/3366658016 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8072160 0x7f2cd8072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- 172.21.15.43:0/3366658016 >> 172.21.15.43:0/3366658016 conn(0x7f2cd80d1100 msgr2=0x7f2cd80d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- 172.21.15.43:0/3366658016 shutdown_connections 2024-09-17T05:32:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- 172.21.15.43:0/3366658016 wait complete. 2024-09-17T05:32:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 Processor -- start 2024-09-17T05:32:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.083+0000 7f2cde600640 1 -- start start 2024-09-17T05:32:50.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd8072160 0x7f2cd8165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8073f50 0x7f2cd815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd80d0680 con 0x7f2cd80b4130 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd80d0500 con 0x7f2cd8072160 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd80d0800 con 0x7f2cd8073f50 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36492/0 (socket says 172.21.15.43:36492) 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 -- 172.21.15.43:0/2982770369 learned_addr learned my addr 172.21.15.43:0/2982770369 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2ccfe00640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8073f50 0x7f2cd815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cdd200640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd8072160 0x7f2cd8165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 -- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8073f50 msgr2=0x7f2cd815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8073f50 0x7f2cd815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 -- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd8072160 msgr2=0x7f2cd8165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd8072160 0x7f2cd8165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 -- 172.21.15.43:0/2982770369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd815a840 con 0x7f2cd80b4130 2024-09-17T05:32:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cddc00640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c760 tx=0x7f2cd000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0003b80 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cd0003d20 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000aab0 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd80cc930 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd80ccbd0 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.087+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd8166070 con 0x7f2cd80b4130 2024-09-17T05:32:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.091+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2cd000ac50 con 0x7f2cd80b4130 2024-09-17T05:32:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.091+0000 7f2cd5600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 0x7f2cb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.091+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7f2cd0094a00 con 0x7f2cd80b4130 2024-09-17T05:32:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.091+0000 7f2cdd200640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 0x7f2cb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.091+0000 7f2cdd200640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 0x7f2cb4075710 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009ab0 tx=0x7f2cc80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.095+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2cd005f010 con 0x7f2cd80b4130 2024-09-17T05:32:50.218 INFO:tasks.ceph.mds.c.smithi043.stderr: -42> 2024-09-17T05:32:45.463+0000 7f31e3200640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-17T05:32:50.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd8072570 con 0x7f2cd80b4130 2024-09-17T05:32:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cd5600640 1 -- 172.21.15.43:0/2982770369 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) ==== 95+0+3557 (secure 0 0 0) 0x7f2cd005e910 con 0x7f2cd80b4130 2024-09-17T05:32:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:50.305 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":258,"btime":"2024-09-17T05:32:49:498784+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:49.498712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 258 2024-09-17T05:32:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 msgr2=0x7f2cb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 0x7f2cb4075710 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009ab0 tx=0x7f2cc80091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 msgr2=0x7f2cd8153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c760 tx=0x7f2cd000cc20 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.303+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 shutdown_connections 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2cb4073200 0x7f2cb4075710 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd80b4130 0x7f2cd8153760 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2cd8073f50 0x7f2cd815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 --2- 172.21.15.43:0/2982770369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cd8072160 0x7f2cd8165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 >> 172.21.15.43:0/2982770369 conn(0x7f2cd80d1100 msgr2=0x7f2cd80b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 shutdown_connections 2024-09-17T05:32:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.307+0000 7f2cde600640 1 -- 172.21.15.43:0/2982770369 wait complete. 2024-09-17T05:32:50.328 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:50.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:50.337 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:50.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.483+0000 7fd4eee00640 1 -- 172.21.15.43:0/1242426380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e8155d70 msgr2=0x7fd4e815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.483+0000 7fd4eee00640 1 --2- 172.21.15.43:0/1242426380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e8155d70 0x7fd4e815c7e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009840 tx=0x7fd4d801c790 comp rx=0 tx=0).stop 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.483+0000 7fd4eee00640 1 -- 172.21.15.43:0/1242426380 shutdown_connections 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.483+0000 7fd4eee00640 1 --2- 172.21.15.43:0/1242426380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e8155d70 0x7fd4e815c7e0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.483+0000 7fd4eee00640 1 -- 172.21.15.43:0/1242426380 >> 172.21.15.43:0/1242426380 conn(0x7fd4e80b23d0 msgr2=0x7fd4e80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.487+0000 7fd4eee00640 1 -- 172.21.15.43:0/1242426380 shutdown_connections 2024-09-17T05:32:50.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.487+0000 7fd4eee00640 1 -- 172.21.15.43:0/1242426380 wait complete. 2024-09-17T05:32:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.487+0000 7fd4eee00640 1 Processor -- start 2024-09-17T05:32:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.487+0000 7fd4eee00640 1 -- start start 2024-09-17T05:32:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4e8155d70 0x7fd4e809d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e80c92f0 0x7fd4e809d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8169e10 con 0x7fd4e806f110 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8169c90 con 0x7fd4e80c92f0 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e8169f90 con 0x7fd4e8155d70 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36526/0 (socket says 172.21.15.43:36526) 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 -- 172.21.15.43:0/2879001616 learned_addr learned my addr 172.21.15.43:0/2879001616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4df400640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4e8155d70 0x7fd4e809d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4ece00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e80c92f0 0x7fd4e809d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 -- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4e8155d70 msgr2=0x7fd4e809d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4e8155d70 0x7fd4e809d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 -- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e80c92f0 msgr2=0x7fd4e809d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e80c92f0 0x7fd4e809d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 -- 172.21.15.43:0/2879001616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e815b650 con 0x7fd4e806f110 2024-09-17T05:32:50.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e7e00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd4d801d030 tx=0x7fd4d80a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80bb070 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4d8004470 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80b2040 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e80c9a20 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81f90e0 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.491+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e81f8bf0 con 0x7fd4e806f110 2024-09-17T05:32:50.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.495+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4d80be070 con 0x7fd4e806f110 2024-09-17T05:32:50.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.495+0000 7fd4e4c00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 0x7fd4c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.495+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7fd4d812e4b0 con 0x7fd4e806f110 2024-09-17T05:32:50.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.495+0000 7fd4df400640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 0x7fd4c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.495+0000 7fd4df400640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 0x7fd4c4075740 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8166570 tx=0x7fd4d0008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.499+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd4d8016410 con 0x7fd4e806f110 2024-09-17T05:32:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.715+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e815c7e0 con 0x7fd4e806f110 2024-09-17T05:32:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.715+0000 7fd4e4c00640 1 -- 172.21.15.43:0/2879001616 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 259 v259) ==== 95+0+4383 (secure 0 0 0) 0x7fd4d80f8b00 con 0x7fd4e806f110 2024-09-17T05:32:50.722 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:50.722 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":259,"btime":"2024-09-17T05:32:50:502596+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1653179583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1653179583},{"type":"v1","addr":"172.21.15.43:6835","nonce":1653179583}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":259}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:49.498712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 259 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 msgr2=0x7fd4c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 0x7fd4c4075740 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8166570 tx=0x7fd4d0008040 comp rx=0 tx=0).stop 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 msgr2=0x7fd4e809b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd4d801d030 tx=0x7fd4d80a55a0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 shutdown_connections 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd4c4073230 0x7fd4c4075740 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e80c92f0 0x7fd4e809d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4e8155d70 0x7fd4e809d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 --2- 172.21.15.43:0/2879001616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e806f110 0x7fd4e809b5f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 >> 172.21.15.43:0/2879001616 conn(0x7fd4e80b23d0 msgr2=0x7fd4e80b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:50.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 shutdown_connections 2024-09-17T05:32:50.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:50.719+0000 7fd4eee00640 1 -- 172.21.15.43:0/2879001616 wait complete. 2024-09-17T05:32:50.741 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-09-17T05:32:50.742 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-09-17T05:32:50.742 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:50.742 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:50.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:50.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:50.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:50.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:50.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 -- 172.21.15.43:0/2082442335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386006fc90 msgr2=0x7f38600700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 --2- 172.21.15.43:0/2082442335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386006fc90 0x7f38600700a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f38600ccb60 tx=0x7f384c01c7a0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 -- 172.21.15.43:0/2082442335 shutdown_connections 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 --2- 172.21.15.43:0/2082442335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386006fc90 0x7f38600700a0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 --2- 172.21.15.43:0/2082442335 >> v2:172.21.15.62:3300/0 conn(0x7f386015fe10 0x7f38600b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 -- 172.21.15.43:0/2082442335 >> 172.21.15.43:0/2082442335 conn(0x7f38600b23d0 msgr2=0x7f38600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 -- 172.21.15.43:0/2082442335 shutdown_connections 2024-09-17T05:32:51.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.043+0000 7f3867e00640 1 -- 172.21.15.43:0/2082442335 wait complete. 2024-09-17T05:32:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 Processor -- start 2024-09-17T05:32:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- start start 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 0x7f3860155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f386015fe10 0x7f38600990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f386016a050 con 0x7f38600730b0 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3860169ed0 con 0x7f386006fc90 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f386016a1d0 con 0x7f386015fe10 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36540/0 (socket says 172.21.15.43:36540) 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 -- 172.21.15.43:0/2627309568 learned_addr learned my addr 172.21.15.43:0/2627309568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3865e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f386015fe10 0x7f38600990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3865400640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 0x7f3860155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 -- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f386015fe10 msgr2=0x7f38600990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f386015fe10 0x7f38600990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 -- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 msgr2=0x7f3860155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 0x7f3860155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 -- 172.21.15.43:0/2627309568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38601572f0 con 0x7f38600730b0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3865400640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 0x7f3860155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3864a00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f384c0b0040 tx=0x7f384c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0b2040 con 0x7f38600730b0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f384c004330 con 0x7f38600730b0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38601575d0 con 0x7f38600730b0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c005280 con 0x7f38600730b0 2024-09-17T05:32:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.047+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3860157b90 con 0x7f38600730b0 2024-09-17T05:32:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3860099db0 con 0x7f38600730b0 2024-09-17T05:32:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f384c005420 con 0x7f38600730b0 2024-09-17T05:32:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f385d600640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 0x7f3840075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7f384c12e550 con 0x7f38600730b0 2024-09-17T05:32:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f3865400640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 0x7f3840075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.051+0000 7f3865400640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 0x7f3840075740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3854009b90 tx=0x7f38540091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.055+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f384c0164b0 con 0x7f38600730b0 2024-09-17T05:32:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.263+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3860074d90 con 0x7f38600730b0 2024-09-17T05:32:51.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.511+0000 7f385d600640 1 -- 172.21.15.43:0/2627309568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v260) ==== 161+0+0 (secure 0 0 0) 0x7f384c0f8b00 con 0x7f38600730b0 2024-09-17T05:32:51.515 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 msgr2=0x7f3840075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 0x7f3840075740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3854009b90 tx=0x7f38540091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 msgr2=0x7f3860155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f384c0b0040 tx=0x7f384c005d60 comp rx=0 tx=0).stop 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 shutdown_connections 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3840073230 0x7f3840075740 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f386015fe10 0x7f38600990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600730b0 0x7f3860155d70 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 --2- 172.21.15.43:0/2627309568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386006fc90 0x7f3860155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 >> 172.21.15.43:0/2627309568 conn(0x7f38600b23d0 msgr2=0x7f3860073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 shutdown_connections 2024-09-17T05:32:51.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.515+0000 7f3867e00640 1 -- 172.21.15.43:0/2627309568 wait complete. 2024-09-17T05:32:51.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- 172.21.15.43:0/1183903344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c070d60 msgr2=0x7fa12c0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 --2- 172.21.15.43:0/1183903344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c070d60 0x7fa12c0b1530 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa12c13eb40 tx=0x7fa11801c800 comp rx=0 tx=0).stop 2024-09-17T05:32:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- 172.21.15.43:0/1183903344 shutdown_connections 2024-09-17T05:32:51.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 --2- 172.21.15.43:0/1183903344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b1a70 0x7fa12c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 --2- 172.21.15.43:0/1183903344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c070d60 0x7fa12c0b1530 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- 172.21.15.43:0/1183903344 >> 172.21.15.43:0/1183903344 conn(0x7fa12c0b24c0 msgr2=0x7fa12c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- 172.21.15.43:0/1183903344 shutdown_connections 2024-09-17T05:32:51.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- 172.21.15.43:0/1183903344 wait complete. 2024-09-17T05:32:51.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 Processor -- start 2024-09-17T05:32:51.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.695+0000 7fa133a00640 1 -- start start 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c0b1a70 0x7fa12c13d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b5910 0x7fa12c1f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c13faf0 con 0x7fa12c15fbb0 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c13f970 con 0x7fa12c0b1a70 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c13fc70 con 0x7fa12c0b5910 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36554/0 (socket says 172.21.15.43:36554) 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 -- 172.21.15.43:0/2961892590 learned_addr learned my addr 172.21.15.43:0/2961892590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa131a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b5910 0x7fa12c1f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa131000640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c0b1a70 0x7fa12c13d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 -- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b5910 msgr2=0x7fa12c1f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b5910 0x7fa12c1f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 -- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c0b1a70 msgr2=0x7fa12c13d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c0b1a70 0x7fa12c13d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 -- 172.21.15.43:0/2961892590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c0b37c0 con 0x7fa12c15fbb0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa12be00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa1180b0040 tx=0x7fa1180a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b2040 con 0x7fa12c15fbb0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa118004430 con 0x7fa12c15fbb0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c1f9160 con 0x7fa12c15fbb0 2024-09-17T05:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118005290 con 0x7fa12c15fbb0 2024-09-17T05:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.699+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c0b39d0 con 0x7fa12c15fbb0 2024-09-17T05:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c140210 con 0x7fa12c15fbb0 2024-09-17T05:32:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1180bd060 con 0x7fa12c15fbb0 2024-09-17T05:32:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa128c00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 0x7fa10c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7fa11812e480 con 0x7fa12c15fbb0 2024-09-17T05:32:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa131000640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 0x7fa10c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.703+0000 7fa131000640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 0x7fa10c075740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098e0 tx=0x7fa120009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.707+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa1180164b0 con 0x7fa12c15fbb0 2024-09-17T05:32:51.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:51.943+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa12c050fc0 con 0x7fa12c15fbb0 2024-09-17T05:32:52.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:52.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:52.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.519+0000 7fa128c00640 1 -- 172.21.15.43:0/2961892590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v261) ==== 101+0+0 (secure 0 0 0) 0x7fa1180f8480 con 0x7fa12c15fbb0 2024-09-17T05:32:52.521 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.519+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 msgr2=0x7fa10c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.519+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 0x7fa10c075740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098e0 tx=0x7fa120009060 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.519+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 msgr2=0x7fa12c1f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.519+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa1180b0040 tx=0x7fa1180a57c0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 shutdown_connections 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa10c073230 0x7fa10c075740 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa12c0b5910 0x7fa12c1f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa12c15fbb0 0x7fa12c1f8380 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 --2- 172.21.15.43:0/2961892590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa12c0b1a70 0x7fa12c13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 >> 172.21.15.43:0/2961892590 conn(0x7fa12c0b24c0 msgr2=0x7fa12c15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 shutdown_connections 2024-09-17T05:32:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.523+0000 7fa133a00640 1 -- 172.21.15.43:0/2961892590 wait complete. 2024-09-17T05:32:52.547 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 Processor -- start 2024-09-17T05:32:52.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 -- start start 2024-09-17T05:32:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6acb2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ac250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563e6ab2da00 con 0x563e6ac26bd0 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563e6ab8bf40 con 0x563e6ac24ce0 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1cd593080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563e6ac96dd0 con 0x7fa1ac003100 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36580/0 (socket says 172.21.15.43:36580) 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 -- 172.21.15.43:0/1017993061 learned_addr learned my addr 172.21.15.43:0/1017993061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ac250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c6800640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6acb2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 -- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 msgr2=0x563e6acb2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6acb2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 -- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 msgr2=0x563e6ac250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ac250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 -- 172.21.15.43:0/1017993061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6ab06c10 con 0x563e6ac26bd0 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c6800640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6acb2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ac250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c7200640 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x563e6ac292a0 tx=0x7fa1c0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d520b01d55956c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c4a00640 1 -- 172.21.15.43:0/1017993061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007590 con 0x563e6ac26bd0 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c4a00640 1 -- 172.21.15.43:0/1017993061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0007730 con 0x563e6ac26bd0 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.607+0000 7fa1c4a00640 1 -- 172.21.15.43:0/1017993061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007b80 con 0x563e6ac26bd0 2024-09-17T05:32:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- 172.21.15.43:0/1017993061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 msgr2=0x563e6ac26fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x563e6ac292a0 tx=0x7fa1c0009940 comp rx=0 tx=0).stop 2024-09-17T05:32:52.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- 172.21.15.43:0/1017993061 shutdown_connections 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ac250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac26fe0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- 172.21.15.43:0/1017993061 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6acb2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- 172.21.15.43:0/1017993061 >> 172.21.15.43:0/1017993061 conn(0x563e6abcbd80 msgr2=0x563e6abdd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- 172.21.15.43:0/1017993061 shutdown_connections 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- 172.21.15.43:0/1017993061 wait complete. 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 Processor -- start 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- start start 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ad4c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac01cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6ac26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563e6abce280 con 0x563e6ac26bd0 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563e6ac0f0c0 con 0x563e6ac24ce0 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1cd593080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563e6ad3bc70 con 0x7fa1ac003100 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac01cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1c5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac01cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36586/0 (socket says 172.21.15.43:36586) 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 learned_addr learned my addr 172.21.15.43:0/2728890384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ad4c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.611+0000 7fa1c7200640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6ac26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 msgr2=0x563e6ac26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa1ac003100 0x563e6ac26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 msgr2=0x563e6ad4c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e6ac24ce0 0x563e6ad4c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6ac02a30 con 0x563e6ac26bd0 2024-09-17T05:32:52.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac01cf0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa1b401b5c0 tx=0x7fa1b40a4f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1bf400640 1 -- 172.21.15.43:0/2728890384 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b9070 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1bf400640 1 -- 172.21.15.43:0/2728890384 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1b40a4590 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1bf400640 1 -- 172.21.15.43:0/2728890384 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40b5850 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e6ac02cd0 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6ad3c140 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.615+0000 7fa1bf400640 1 -- 172.21.15.43:0/2728890384 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1b40bc070 con 0x563e6ac26bd0 2024-09-17T05:32:52.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1bf400640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a8071be0 0x7fa1a80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1bf400640 1 -- 172.21.15.43:0/2728890384 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7fa1b412e5f0 con 0x563e6ac26bd0 2024-09-17T05:32:52.620 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563e6ad3d5f0 0x563e6ad587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ac02590 con 0x563e6ad3d5f0 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a8071be0 0x7fa1a80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1c7200640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563e6ad3d5f0 0x563e6ad587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a8071be0 0x7fa1a80740f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c8b0 tx=0x7fa1b8004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.619+0000 7fa1c7200640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563e6ad3d5f0 0x563e6ad587b0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v118'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c0008620 con 0x563e6ad3d5f0 2024-09-17T05:32:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad58cf0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v118'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59090 con 0x563e6ad3d5f0 2024-09-17T05:32:52.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v118'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.627 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:52.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.623+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5a440 con 0x563e6ad3d5f0 2024-09-17T05:32:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5a7e0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59430 con 0x563e6ad3d5f0 2024-09-17T05:32:52.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.627+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59820 con 0x563e6ad3d5f0 2024-09-17T05:32:52.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.631+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v118'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.631+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad59c10 con 0x563e6ad3d5f0 2024-09-17T05:32:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.631+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v118'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.631+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59c10 con 0x563e6ad3d5f0 2024-09-17T05:32:52.635 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:52.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v118'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.635+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.639+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.644 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.643+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad59bc0 con 0x563e6ad3d5f0 2024-09-17T05:32:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v118'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.647+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.651+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v118'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.651+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.651+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v118'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.654 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.651+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5bb90 con 0x563e6ad3d5f0 2024-09-17T05:32:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5bb90 con 0x563e6ad3d5f0 2024-09-17T05:32:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.655+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v118'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v118'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v118'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.663 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.659+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5bb90 con 0x563e6ad3d5f0 2024-09-17T05:32:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.663+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.663+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5bb90 con 0x563e6ad3d5f0 2024-09-17T05:32:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.663+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.663+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ab80 con 0x563e6ad3d5f0 2024-09-17T05:32:52.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563e6ad5b090 0x563e6ad5d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5d9f0 con 0x563e6ad5b090 2024-09-17T05:32:52.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563e6ad5b090 0x563e6ad5d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563e6ad5b090 0x563e6ad5d4b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'637 uv620 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.667+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5dd90 con 0x563e6ad5b090 2024-09-17T05:32:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.671+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'638 uv638 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.671+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5e130 con 0x563e6ad5b090 2024-09-17T05:32:52.673 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:52.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.671+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.671+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5e980 con 0x563e6ad5b090 2024-09-17T05:32:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5ed20 con 0x563e6ad5b090 2024-09-17T05:32:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f0c0 con 0x563e6ad5b090 2024-09-17T05:32:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v118'642 uv625 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.675+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.679+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v118'643 uv643 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.679+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.679+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v118'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.682 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.679+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v118'647 uv630 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.683+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.687+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v118'648 uv648 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.687+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.690 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.687+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v118'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.687+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63630 con 0x563e6ad5b090 2024-09-17T05:32:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v118'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63630 con 0x563e6ad5b090 2024-09-17T05:32:52.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v118'652 uv636 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.691+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.695+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v118'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.695+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.698 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.695+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v118'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.695+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad60470 con 0x563e6ad5b090 2024-09-17T05:32:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad60470 con 0x563e6ad5b090 2024-09-17T05:32:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad5f460 con 0x563e6ad5b090 2024-09-17T05:32:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad5b090 2024-09-17T05:32:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563e6ad5f970 0x563e6ad61d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad623a0 con 0x563e6ad5f970 2024-09-17T05:32:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.699+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563e6ad5f970 0x563e6ad61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.703+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563e6ad5f970 0x563e6ad61d90 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.703+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v118'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.703+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad62890 con 0x563e6ad5f970 2024-09-17T05:32:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.703+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v118'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.703+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad62d70 con 0x563e6ad5f970 2024-09-17T05:32:52.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v118'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.707 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad64620 con 0x563e6ad5f970 2024-09-17T05:32:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad649c0 con 0x563e6ad5f970 2024-09-17T05:32:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.707+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63610 con 0x563e6ad5f970 2024-09-17T05:32:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63a00 con 0x563e6ad3d5f0 2024-09-17T05:32:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'122 uv121 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad63a00 con 0x563e6ad3d5f0 2024-09-17T05:32:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'123 uv123 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.711+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63a00 con 0x563e6ad3d5f0 2024-09-17T05:32:52.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'124 uv124 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad63a00 con 0x563e6ad3d5f0 2024-09-17T05:32:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563e6ad64d60 0x563e6ad67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad676c0 con 0x563e6ad64d60 2024-09-17T05:32:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1c7200640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563e6ad64d60 0x563e6ad67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.715+0000 7fa1c7200640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563e6ad64d60 0x563e6ad67180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v118'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad676c0 con 0x563e6ad64d60 2024-09-17T05:32:52.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v118'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad676c0 con 0x563e6ad64d60 2024-09-17T05:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v118'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.719+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad686d0 con 0x563e6ad64d60 2024-09-17T05:32:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad686d0 con 0x563e6ad64d60 2024-09-17T05:32:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad676c0 con 0x563e6ad64d60 2024-09-17T05:32:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563e6ad67bd0 0x563e6ad69ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.723+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6a680 con 0x563e6ad67bd0 2024-09-17T05:32:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563e6ad67bd0 0x563e6ad69ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1c6800640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563e6ad67bd0 0x563e6ad69ff0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v118'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v118'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.727+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v118'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6ba60 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6ba60 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.731+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v118'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v118'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.735+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.739+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v118'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.741 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.739+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.739+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.739+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6aa50 con 0x563e6ad67bd0 2024-09-17T05:32:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1c6800640 1 -- 172.21.15.43:0/2728890384 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b8017420 con 0x563e6ad67bd0 2024-09-17T05:32:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563e6ad6b0d0 0x563e6ad6d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6dc40 con 0x563e6ad6b0d0 2024-09-17T05:32:52.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563e6ad6b0d0 0x563e6ad6d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1c5e00640 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563e6ad6b0d0 0x563e6ad6d4f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v118'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e060 con 0x563e6ad6b0d0 2024-09-17T05:32:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v118'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.743+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad6b0d0 2024-09-17T05:32:52.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.747+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v118'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.748 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.747+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad6b0d0 2024-09-17T05:32:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.747+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.747+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad6b0d0 2024-09-17T05:32:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad3d5f0 2024-09-17T05:32:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v118'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad3d5f0 2024-09-17T05:32:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v118'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.751+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad3d5f0 2024-09-17T05:32:52.755 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v118'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6f460 con 0x563e6ad3d5f0 2024-09-17T05:32:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6f460 con 0x563e6ad3d5f0 2024-09-17T05:32:52.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.755+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad3d5f0 2024-09-17T05:32:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad3d5f0 2024-09-17T05:32:52.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v118'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v118'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.759+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.763+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v118'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.763+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.763+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.763+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.767+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.767+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.767+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v118'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.767+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v118'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.773 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:52.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v118'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e450 con 0x563e6ad5f970 2024-09-17T05:32:52.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.771+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad71260 con 0x563e6ad5f970 2024-09-17T05:32:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.775+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad5f970 2024-09-17T05:32:52.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.775+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.775+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v118'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.775+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.779+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.779+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.782 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.779+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v118'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.779+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v118'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v118'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.783+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v118'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.787+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad64d60 2024-09-17T05:32:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c001e040 con 0x563e6ad64d60 2024-09-17T05:32:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v118'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v118'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.791+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v118'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.796 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.795+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v118'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v118'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v118'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.803 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.799+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.803+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.803+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.803+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.803+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v118'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v118'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v118'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.812 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.807+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6e900 con 0x563e6ad6b0d0 2024-09-17T05:32:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.811+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x563e6ad6ecf0 con 0x563e6ad6b0d0 2024-09-17T05:32:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1c5e00640 1 -- 172.21.15.43:0/2728890384 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b40c5070 con 0x563e6ad6b0d0 2024-09-17T05:32:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563e6ad5b090 msgr2=0x563e6ad5d4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563e6ad5b090 0x563e6ad5d4b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563e6ad67bd0 msgr2=0x563e6ad69ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563e6ad67bd0 0x563e6ad69ff0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563e6ad6b0d0 msgr2=0x563e6ad6d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563e6ad6b0d0 0x563e6ad6d4f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563e6ad5f970 msgr2=0x563e6ad61d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 reap_dead start 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563e6ad5f970 0x563e6ad61d90 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563e6ad3d5f0 msgr2=0x563e6ad587b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563e6ad3d5f0 0x563e6ad587b0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563e6ad64d60 msgr2=0x563e6ad67180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563e6ad64d60 0x563e6ad67180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a8071be0 msgr2=0x7fa1a80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa1a8071be0 0x7fa1a80740f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c8b0 tx=0x7fa1b8004890 comp rx=0 tx=0).stop 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 msgr2=0x563e6ac01cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 --2- 172.21.15.43:0/2728890384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563e6ac26bd0 0x563e6ac01cf0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa1b401b5c0 tx=0x7fa1b40a4f20 comp rx=0 tx=0).stop 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1c7200640 1 -- 172.21.15.43:0/2728890384 reap_dead start 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 shutdown_connections 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 >> 172.21.15.43:0/2728890384 conn(0x563e6abcbd80 msgr2=0x563e6acb2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 shutdown_connections 2024-09-17T05:32:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.815+0000 7fa1cd593080 1 -- 172.21.15.43:0/2728890384 wait complete. 2024-09-17T05:32:52.822 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-09-17T05:32:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 Processor -- start 2024-09-17T05:32:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 -- start start 2024-09-17T05:32:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68b0c5b0 0x556f68bc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 0x556f68bc15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f689a3a00 con 0x556f68a5a9c0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556f68a01f40 con 0x556f68a9f7a0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556f68a9bf00 con 0x556f68b0c5b0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1f6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36604/0 (socket says 172.21.15.43:36604) 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.919+0000 7fe1f6400640 1 -- 172.21.15.43:0/1179067084 learned_addr learned my addr 172.21.15.43:0/1179067084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6e00640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 0x556f68bc15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f5a00640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68b0c5b0 0x556f68bc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 -- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68b0c5b0 msgr2=0x556f68bc1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68b0c5b0 0x556f68bc1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 -- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 msgr2=0x556f68bc15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 0x556f68bc15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 -- 172.21.15.43:0/1179067084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6897cc10 con 0x556f68a5a9c0 2024-09-17T05:32:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6e00640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 0x556f68bc15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1f6400640 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8018da0 tx=0x7fe1e8017f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783974162704d592 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1efe00640 1 -- 172.21.15.43:0/1179067084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80ba070 con 0x556f68a5a9c0 2024-09-17T05:32:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1efe00640 1 -- 172.21.15.43:0/1179067084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e80ab980 con 0x556f68a5a9c0 2024-09-17T05:32:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1efe00640 1 -- 172.21.15.43:0/1179067084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80abd90 con 0x556f68a5a9c0 2024-09-17T05:32:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- 172.21.15.43:0/1179067084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 msgr2=0x556f68a9f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8018da0 tx=0x7fe1e8017f10 comp rx=0 tx=0).stop 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- 172.21.15.43:0/1179067084 shutdown_connections 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a9f7a0 0x556f68bc15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68b0c5b0 0x556f68bc1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/1179067084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68a5a9c0 0x556f68a9f260 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- 172.21.15.43:0/1179067084 >> 172.21.15.43:0/1179067084 conn(0x556f68a41d80 msgr2=0x556f68a97df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- 172.21.15.43:0/1179067084 shutdown_connections 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- 172.21.15.43:0/1179067084 wait complete. 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 Processor -- start 2024-09-17T05:32:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.923+0000 7fe1fd29b080 1 -- start start 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a5a9c0 0x556f68b2b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68a9f7a0 0x556f68b101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 0x556f68b06950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556f68b0b2e0 con 0x556f68b0c5b0 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556f68b2c750 con 0x556f68a5a9c0 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556f68b0b580 con 0x556f68a9f7a0 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 0x556f68b06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 0x556f68b06950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36610/0 (socket says 172.21.15.43:36610) 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 -- 172.21.15.43:0/728695084 learned_addr learned my addr 172.21.15.43:0/728695084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a5a9c0 0x556f68b2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f5a00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68a9f7a0 0x556f68b101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 -- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68a9f7a0 msgr2=0x556f68b101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556f68a9f7a0 0x556f68b101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 -- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a5a9c0 msgr2=0x556f68b2b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556f68a5a9c0 0x556f68b2b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 -- 172.21.15.43:0/728695084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f68af8bd0 con 0x556f68b0c5b0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1f6e00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 0x556f68b06950 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000f680 tx=0x7fe1f000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1eea00640 1 -- 172.21.15.43:0/728695084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000bab0 con 0x556f68b0c5b0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1eea00640 1 -- 172.21.15.43:0/728695084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1f0014070 con 0x556f68b0c5b0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1eea00640 1 -- 172.21.15.43:0/728695084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0019470 con 0x556f68b0c5b0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f68af8e70 con 0x556f68b0c5b0 2024-09-17T05:32:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.927+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f68af93b0 con 0x556f68b0c5b0 2024-09-17T05:32:52.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1eea00640 1 -- 172.21.15.43:0/728695084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe1f0019610 con 0x556f68b0c5b0 2024-09-17T05:32:52.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1eea00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1d8071be0 0x7fe1d80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1eea00640 1 -- 172.21.15.43:0/728695084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7fe1f0094500 con 0x556f68b0c5b0 2024-09-17T05:32:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556f68bc8860 0x556f68bc8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556f68bc91b0 con 0x556f68bc8860 2024-09-17T05:32:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f6400640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1d8071be0 0x7fe1d80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f5a00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556f68bc8860 0x556f68bc8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f5a00640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556f68bc8860 0x556f68bc8c70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f6400640 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1d8071be0 0x7fe1d80740f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x556f68b31bb0 tx=0x7fe1e8004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f5a00640 1 -- 172.21.15.43:0/728695084 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v118'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe1e40106d0 con 0x556f68bc8860 2024-09-17T05:32:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556f68bc8860 msgr2=0x556f68bc8c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556f68bc8860 0x556f68bc8c70 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1d8071be0 msgr2=0x7fe1d80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe1d8071be0 0x7fe1d80740f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x556f68b31bb0 tx=0x7fe1e8004890 comp rx=0 tx=0).stop 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 msgr2=0x556f68b06950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1fd29b080 1 --2- 172.21.15.43:0/728695084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556f68b0c5b0 0x556f68b06950 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000f680 tx=0x7fe1f000fb40 comp rx=0 tx=0).stop 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.931+0000 7fe1f6e00640 1 -- 172.21.15.43:0/728695084 reap_dead start 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.935+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 shutdown_connections 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.935+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 >> 172.21.15.43:0/728695084 conn(0x556f68a41d80 msgr2=0x556f68a9c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.935+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 shutdown_connections 2024-09-17T05:32:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:52.935+0000 7fe1fd29b080 1 -- 172.21.15.43:0/728695084 wait complete. 2024-09-17T05:32:52.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 -- 172.21.15.43:0/159642535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c98070620 msgr2=0x7f7c980b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 --2- 172.21.15.43:0/159642535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c98070620 0x7f7c980b30d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009900 tx=0x7f7c8c01c850 comp rx=0 tx=0).stop 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 -- 172.21.15.43:0/159642535 shutdown_connections 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 --2- 172.21.15.43:0/159642535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c9806d570 0x7f7c980cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 --2- 172.21.15.43:0/159642535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c98070620 0x7f7c980b30d0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 -- 172.21.15.43:0/159642535 >> 172.21.15.43:0/159642535 conn(0x7f7c980b23d0 msgr2=0x7f7c980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 -- 172.21.15.43:0/159642535 shutdown_connections 2024-09-17T05:32:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.103+0000 7f7ca0600640 1 -- 172.21.15.43:0/159642535 wait complete. 2024-09-17T05:32:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 Processor -- start 2024-09-17T05:32:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 -- start start 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c980d0910 0x7f7c9806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c980c54a0 0x7f7c9806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980ce7f0 con 0x7f7c980c54a0 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980ce670 con 0x7f7c980d0910 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7ca0600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7c980ce970 con 0x7f7c98070620 2024-09-17T05:32:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c980c54a0 0x7f7c9806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:44450/0 (socket says 172.21.15.43:44450) 2024-09-17T05:32:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 -- 172.21.15.43:0/1108600558 learned_addr learned my addr 172.21.15.43:0/1108600558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9d200640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c980d0910 0x7f7c9806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 -- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c980d0910 msgr2=0x7f7c9806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c980d0910 0x7f7c9806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 -- 172.21.15.43:0/1108600558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c980c54a0 msgr2=0x7f7c9806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c980c54a0 0x7f7c9806d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 -- 172.21.15.43:0/1108600558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c980b07e0 con 0x7f7c98070620 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c9dc00640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009850 tx=0x7f7c8c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.107+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0b2040 con 0x7f7c98070620 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c8c004310 con 0x7f7c98070620 2024-09-17T05:32:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c005280 con 0x7f7c98070620 2024-09-17T05:32:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c980c9520 con 0x7f7c98070620 2024-09-17T05:32:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c980c9ae0 con 0x7f7c98070620 2024-09-17T05:32:53.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c980cb860 con 0x7f7c98070620 2024-09-17T05:32:53.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c8c005420 con 0x7f7c98070620 2024-09-17T05:32:53.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c95600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 0x7f7c78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 8203+0+0 (secure 0 0 0) 0x7f7c8c12e2d0 con 0x7f7c98070620 2024-09-17T05:32:53.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.111+0000 7f7c9d200640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 0x7f7c78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.115+0000 7f7c9d200640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 0x7f7c78075710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cd210 tx=0x7f7c84009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.119+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c8c0ba020 con 0x7f7c98070620 2024-09-17T05:32:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.327+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7c9809e190 con 0x7f7c98070620 2024-09-17T05:32:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7c95600640 1 -- 172.21.15.43:0/1108600558 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v262) ==== 162+0+0 (secure 0 0 0) 0x7f7c8c0164d0 con 0x7f7c98070620 2024-09-17T05:32:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 msgr2=0x7f7c78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 0x7f7c78075710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cd210 tx=0x7f7c84009040 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 msgr2=0x7f7c98159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009850 tx=0x7f7c8c005de0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 shutdown_connections 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7c78073200 0x7f7c78075710 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c980d0910 0x7f7c9806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c980c54a0 0x7f7c9806d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 --2- 172.21.15.43:0/1108600558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7c98070620 0x7f7c98159160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 >> 172.21.15.43:0/1108600558 conn(0x7f7c980b23d0 msgr2=0x7f7c980b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 shutdown_connections 2024-09-17T05:32:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.535+0000 7f7ca0600640 1 -- 172.21.15.43:0/1108600558 wait complete. 2024-09-17T05:32:53.542 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:32:53.539+0000 7f3c74000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-17T05:32:53.543 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:53.539+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:53.543 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:53.539+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:53.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 -- 172.21.15.43:0/2829318468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15fe10 msgr2=0x7f1f4c0b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2829318468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15fe10 0x7f1f4c0b4b80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009900 tx=0x7f1f3c01c780 comp rx=0 tx=0).stop 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 -- 172.21.15.43:0/2829318468 shutdown_connections 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2829318468 >> v2:172.21.15.62:3300/0 conn(0x7f1f4c0737c0 0x7f1f4c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2829318468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15fe10 0x7f1f4c0b4b80 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 -- 172.21.15.43:0/2829318468 >> 172.21.15.43:0/2829318468 conn(0x7f1f4c0b23d0 msgr2=0x7f1f4c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 -- 172.21.15.43:0/2829318468 shutdown_connections 2024-09-17T05:32:53.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.719+0000 7f1f50c00640 1 -- 172.21.15.43:0/2829318468 wait complete. 2024-09-17T05:32:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 Processor -- start 2024-09-17T05:32:53.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- start start 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f4c15fe10 0x7f1f4c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 0x7f1f4c098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c16a050 con 0x7f1f4c06fc90 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c169ed0 con 0x7f1f4c15bf40 2024-09-17T05:32:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c16a1d0 con 0x7f1f4c15fe10 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34568/0 (socket says 172.21.15.43:34568) 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 -- 172.21.15.43:0/2826648014 learned_addr learned my addr 172.21.15.43:0/2826648014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49400640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f4c15fe10 0x7f1f4c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f4a800640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 0x7f1f4c098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 -- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f4c15fe10 msgr2=0x7f1f4c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f4c15fe10 0x7f1f4c15f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 -- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 msgr2=0x7f1f4c098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 0x7f1f4c098ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 -- 172.21.15.43:0/2826648014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c152790 con 0x7f1f4c06fc90 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f4a800640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 0x7f1f4c098ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f49e00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0b0040 tx=0x7f1f3c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0b2040 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f3c004320 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c005280 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c073e20 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.723+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c152d10 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.727+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c0700a0 con 0x7f1f4c06fc90 2024-09-17T05:32:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.727+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f3c0bd060 con 0x7f1f4c06fc90 2024-09-17T05:32:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.727+0000 7f1f42000640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 0x7f1f2c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.727+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7f1f3c12e490 con 0x7f1f4c06fc90 2024-09-17T05:32:53.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.727+0000 7f1f49400640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 0x7f1f2c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.731+0000 7f1f49400640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 0x7f1f2c075740 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c076180 tx=0x7f1f38009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.731+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f3c016490 con 0x7f1f4c06fc90 2024-09-17T05:32:53.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.939+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f4c0761b0 con 0x7f1f4c06fc90 2024-09-17T05:32:53.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.939+0000 7f1f42000640 1 -- 172.21.15.43:0/2826648014 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 264 v264) ==== 95+0+3557 (secure 0 0 0) 0x7f1f3c0f8a70 con 0x7f1f4c06fc90 2024-09-17T05:32:53.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:53.943 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":264,"btime":"2024-09-17T05:32:53:544063+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:53.543976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:53.944 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 264 2024-09-17T05:32:53.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 msgr2=0x7f1f2c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 0x7f1f2c075740 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c076180 tx=0x7f1f38009060 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 msgr2=0x7f1f4c15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0b0040 tx=0x7f1f3c005de0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 shutdown_connections 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1f2c073230 0x7f1f2c075740 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f4c15bf40 0x7f1f4c098ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1f4c15fe10 0x7f1f4c15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 --2- 172.21.15.43:0/2826648014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c06fc90 0x7f1f4c15eae0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 >> 172.21.15.43:0/2826648014 conn(0x7f1f4c0b23d0 msgr2=0x7f1f4c0ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 shutdown_connections 2024-09-17T05:32:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:53.943+0000 7f1f50c00640 1 -- 172.21.15.43:0/2826648014 wait complete. 2024-09-17T05:32:53.966 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:53.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- 172.21.15.43:0/3193156336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 msgr2=0x7f0a8c160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 --2- 172.21.15.43:0/3193156336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c160a50 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009840 tx=0x7f0a8001c700 comp rx=0 tx=0).stop 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- 172.21.15.43:0/3193156336 shutdown_connections 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 --2- 172.21.15.43:0/3193156336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c160a50 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- 172.21.15.43:0/3193156336 >> 172.21.15.43:0/3193156336 conn(0x7f0a8c0691b0 msgr2=0x7f0a8c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- 172.21.15.43:0/3193156336 shutdown_connections 2024-09-17T05:32:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- 172.21.15.43:0/3193156336 wait complete. 2024-09-17T05:32:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 Processor -- start 2024-09-17T05:32:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.123+0000 7f0a92000640 1 -- start start 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0b5070 0x7f0a8c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0b5480 0x7f0a8c166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c16abc0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c16aa40 con 0x7f0a8c0b5480 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c16ad40 con 0x7f0a8c0b5070 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34586/0 (socket says 172.21.15.43:34586) 2024-09-17T05:32:54.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 -- 172.21.15.43:0/128785644 learned_addr learned my addr 172.21.15.43:0/128785644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a90c00640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0b5070 0x7f0a8c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a8be00640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0b5480 0x7f0a8c166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 -- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0b5070 msgr2=0x7f0a8c15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0b5070 0x7f0a8c15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 -- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0b5480 msgr2=0x7f0a8c166750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0b5480 0x7f0a8c166750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 -- 172.21.15.43:0/128785644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c14b2e0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a91600640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c740 tx=0x7f0a8400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84003b80 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8400ce50 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c1fc400 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400a850 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1fc9c0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.127+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c0cc010 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.131+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a8400a9f0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.131+0000 7f0a88c00640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 0x7f0a6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.131+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7f0a840949f0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.131+0000 7f0a90c00640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 0x7f0a6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.131+0000 7f0a90c00640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 0x7f0a6c075710 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0a800af000 tx=0x7f0a8001af40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.135+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0a8c14b2e0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.249 INFO:tasks.ceph.mds.d.smithi062.stderr: -47> 2024-09-17T05:32:49.496+0000 7fdffd600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-17T05:32:54.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.347+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a8c160a50 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.347+0000 7f0a88c00640 1 -- 172.21.15.43:0/128785644 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 264 v264) ==== 95+0+3557 (secure 0 0 0) 0x7f0a8405e8f0 con 0x7f0a8c15c4e0 2024-09-17T05:32:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:54.350 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":264,"btime":"2024-09-17T05:32:53:544063+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:53.543976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 264 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 msgr2=0x7f0a6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 0x7f0a6c075710 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0a800af000 tx=0x7f0a8001af40 comp rx=0 tx=0).stop 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 msgr2=0x7f0a8c166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c740 tx=0x7f0a8400cc00 comp rx=0 tx=0).stop 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 shutdown_connections 2024-09-17T05:32:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0a6c073200 0x7f0a6c075710 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0b5480 0x7f0a8c166750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8c15c4e0 0x7f0a8c166c90 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 --2- 172.21.15.43:0/128785644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0a8c0b5070 0x7f0a8c15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 >> 172.21.15.43:0/128785644 conn(0x7f0a8c0691b0 msgr2=0x7f0a8c0b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 shutdown_connections 2024-09-17T05:32:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.351+0000 7f0a92000640 1 -- 172.21.15.43:0/128785644 wait complete. 2024-09-17T05:32:54.372 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:54.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:54.375 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:32:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 -- 172.21.15.43:0/1415843397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc668162ff0 msgr2=0x7fc668163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1415843397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc668162ff0 0x7fc668163400 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc668073fd0 tx=0x7fc66001cd80 comp rx=0 tx=0).stop 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 -- 172.21.15.43:0/1415843397 shutdown_connections 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1415843397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc668162ff0 0x7fc668163400 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1415843397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc66806e640 0x7fc6680b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1415843397 >> v2:172.21.15.62:3300/0 conn(0x7fc66806e230 0x7fc6680b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 -- 172.21.15.43:0/1415843397 >> 172.21.15.43:0/1415843397 conn(0x7fc6680cee60 msgr2=0x7fc6680cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 -- 172.21.15.43:0/1415843397 shutdown_connections 2024-09-17T05:32:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.531+0000 7fc66ce00640 1 -- 172.21.15.43:0/1415843397 wait complete. 2024-09-17T05:32:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 Processor -- start 2024-09-17T05:32:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 -- start start 2024-09-17T05:32:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc66806e230 0x7fc6681599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc668162ff0 0x7fc6681fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc6680d14f0 con 0x7fc66806e640 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc6680d1370 con 0x7fc66806e230 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc66ce00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc6680d1670 con 0x7fc668162ff0 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34608/0 (socket says 172.21.15.43:34608) 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 -- 172.21.15.43:0/1362070307 learned_addr learned my addr 172.21.15.43:0/1362070307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc665e00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc66806e230 0x7fc6681599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc666800640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc668162ff0 0x7fc6681fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 -- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc668162ff0 msgr2=0x7fc6681fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc668162ff0 0x7fc6681fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 -- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc66806e230 msgr2=0x7fc6681599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc66806e230 0x7fc6681599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 -- 172.21.15.43:0/1362070307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc668159f20 con 0x7fc66806e640 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.535+0000 7fc65fe00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d750 tx=0x7fc65000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650017070 con 0x7fc66806e640 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc650009d20 con 0x7fc66806e640 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500106c0 con 0x7fc66806e640 2024-09-17T05:32:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6680c5f10 con 0x7fc66806e640 2024-09-17T05:32:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6680c8cf0 con 0x7fc66806e640 2024-09-17T05:32:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6680ca400 con 0x7fc66806e640 2024-09-17T05:32:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc65001a070 con 0x7fc66806e640 2024-09-17T05:32:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 0x7fc644075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7fc650094c50 con 0x7fc66806e640 2024-09-17T05:32:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.539+0000 7fc665e00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 0x7fc644075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.543+0000 7fc665e00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 0x7fc644075710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc658004610 tx=0x7fc65800b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.543+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc65009a020 con 0x7fc66806e640 2024-09-17T05:32:54.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.754+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc668163400 con 0x7fc66806e640 2024-09-17T05:32:54.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc65e000640 1 -- 172.21.15.43:0/1362070307 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+4380 (secure 0 0 0) 0x7fc65000de60 con 0x7fc66806e640 2024-09-17T05:32:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:54.760 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":265,"btime":"2024-09-17T05:32:54:547865+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8045,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/647815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":647815137},{"type":"v1","addr":"172.21.15.62:6835","nonce":647815137}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":265}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:53.543976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:54.760 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 265 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 msgr2=0x7fc644075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 0x7fc644075710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc658004610 tx=0x7fc65800b260 comp rx=0 tx=0).stop 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 msgr2=0x7fc6681fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d750 tx=0x7fc65000dc10 comp rx=0 tx=0).stop 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 shutdown_connections 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc644073200 0x7fc644075710 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc668162ff0 0x7fc6681fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc66806e640 0x7fc6681fc690 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 --2- 172.21.15.43:0/1362070307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc66806e230 0x7fc6681599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 >> 172.21.15.43:0/1362070307 conn(0x7fc6680cee60 msgr2=0x7fc6680b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 shutdown_connections 2024-09-17T05:32:54.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:54.758+0000 7fc66ce00640 1 -- 172.21.15.43:0/1362070307 wait complete. 2024-09-17T05:32:54.782 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-09-17T05:32:54.782 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-09-17T05:32:54.782 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:54.782 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:54.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:54.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:54.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:54.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:54.842 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:54.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:54.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:55.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- 172.21.15.43:0/3112050994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 msgr2=0x7fc8f815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 --2- 172.21.15.43:0/3112050994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f815f530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009870 tx=0x7fc8e801c830 comp rx=0 tx=0).stop 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- 172.21.15.43:0/3112050994 shutdown_connections 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 --2- 172.21.15.43:0/3112050994 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f815f530 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- 172.21.15.43:0/3112050994 >> 172.21.15.43:0/3112050994 conn(0x7fc8f80b2e80 msgr2=0x7fc8f80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- 172.21.15.43:0/3112050994 shutdown_connections 2024-09-17T05:32:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- 172.21.15.43:0/3112050994 wait complete. 2024-09-17T05:32:55.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 Processor -- start 2024-09-17T05:32:55.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.046+0000 7fc8fce00640 1 -- start start 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f80c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 0x7fc8f81fa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f816a170 con 0x7fc8f80b43c0 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8169ff0 con 0x7fc8f815fa70 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f816a2f0 con 0x7fc8f80b5fe0 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34620/0 (socket says 172.21.15.43:34620) 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 -- 172.21.15.43:0/645176861 learned_addr learned my addr 172.21.15.43:0/645176861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8efe00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f80c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f6800640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 0x7fc8f81fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 -- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 msgr2=0x7fc8f80c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f80c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 -- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 msgr2=0x7fc8f81fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 0x7fc8f81fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 -- 172.21.15.43:0/645176861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f814fa00 con 0x7fc8f80b43c0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f6800640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 0x7fc8f81fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8f5e00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc8e801a460 tx=0x7fc8e80a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80bc070 con 0x7fc8f80b43c0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8e8004470 con 0x7fc8f80b43c0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80b2040 con 0x7fc8f80b43c0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f814fce0 con 0x7fc8f80b43c0 2024-09-17T05:32:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f8157290 con 0x7fc8f80b43c0 2024-09-17T05:32:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.050+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f81574e0 con 0x7fc8f80b43c0 2024-09-17T05:32:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.054+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8e80052b0 con 0x7fc8f80b43c0 2024-09-17T05:32:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.054+0000 7fc8ee000640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 0x7fc8d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.054+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7fc8e812e620 con 0x7fc8f80b43c0 2024-09-17T05:32:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.054+0000 7fc8efe00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 0x7fc8d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.054+0000 7fc8efe00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 0x7fc8d4075740 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0007fd0 tx=0x7fc8e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.058+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc8e8016450 con 0x7fc8f80b43c0 2024-09-17T05:32:55.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.266+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8f815f530 con 0x7fc8f80b43c0 2024-09-17T05:32:55.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.554+0000 7fc8ee000640 1 -- 172.21.15.43:0/645176861 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v266) ==== 161+0+0 (secure 0 0 0) 0x7fc8e80055a0 con 0x7fc8f80b43c0 2024-09-17T05:32:55.559 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 msgr2=0x7fc8d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 0x7fc8d4075740 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0007fd0 tx=0x7fc8e0009040 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 msgr2=0x7fc8f814e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc8e801a460 tx=0x7fc8e80a5790 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 shutdown_connections 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc8d4073230 0x7fc8d4075740 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8f815fa70 0x7fc8f81fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc8f80b5fe0 0x7fc8f80c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 --2- 172.21.15.43:0/645176861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f80b43c0 0x7fc8f814e4b0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 >> 172.21.15.43:0/645176861 conn(0x7fc8f80b2e80 msgr2=0x7fc8f815fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:55.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 shutdown_connections 2024-09-17T05:32:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.558+0000 7fc8fce00640 1 -- 172.21.15.43:0/645176861 wait complete. 2024-09-17T05:32:55.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 -- 172.21.15.43:0/592047241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 msgr2=0x7f69400ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 --2- 172.21.15.43:0/592047241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69400ad0b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6938009bc0 tx=0x7f693801cae0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 -- 172.21.15.43:0/592047241 shutdown_connections 2024-09-17T05:32:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 --2- 172.21.15.43:0/592047241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69400ad0b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 --2- 172.21.15.43:0/592047241 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f69400b3af0 0x7f694015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 -- 172.21.15.43:0/592047241 >> 172.21.15.43:0/592047241 conn(0x7f69400b23d0 msgr2=0x7f69400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.738+0000 7f6944e00640 1 -- 172.21.15.43:0/592047241 shutdown_connections 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 -- 172.21.15.43:0/592047241 wait complete. 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 Processor -- start 2024-09-17T05:32:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 -- start start 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69401fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 0x7f694009dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6940168c80 con 0x7f694009d6d0 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6940168b00 con 0x7f694006f110 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.742+0000 7f6944e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6940168e00 con 0x7f6940155d70 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f693de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69401fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f693e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 0x7f694009dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:59574/0 (socket says 172.21.15.43:59574) 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f693de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69401fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:43310/0 (socket says 172.21.15.43:43310) 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 -- 172.21.15.43:0/1736161822 learned_addr learned my addr 172.21.15.43:0/1736161822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 -- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 msgr2=0x7f69401fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69401fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 -- 172.21.15.43:0/1736161822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 msgr2=0x7f694009dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 0x7f694009dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 -- 172.21.15.43:0/1736161822 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694009e020 con 0x7f6940155d70 2024-09-17T05:32:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f693e800640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 0x7f694009dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6937e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f692800e7d0 tx=0x7f692800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692800ccc0 con 0x7f6940155d70 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6928010430 con 0x7f6940155d70 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69400cc560 con 0x7f6940155d70 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928010860 con 0x7f6940155d70 2024-09-17T05:32:55.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69400ccb20 con 0x7f6940155d70 2024-09-17T05:32:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.746+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69400ad0b0 con 0x7f6940155d70 2024-09-17T05:32:55.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.750+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6928010a00 con 0x7f6940155d70 2024-09-17T05:32:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.750+0000 7f6936000640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 0x7f691c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.750+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7f6928095e70 con 0x7f6940155d70 2024-09-17T05:32:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.750+0000 7f693de00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 0x7f691c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:55.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.750+0000 7f693de00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 0x7f691c075710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6930007fd0 tx=0x7f6930009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:55.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.754+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f694009e020 con 0x7f6940155d70 2024-09-17T05:32:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:55.990+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f694015bcc0 con 0x7f6940155d70 2024-09-17T05:32:56.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.566+0000 7f6936000640 1 -- 172.21.15.43:0/1736161822 <== mon.2 v2:172.21.15.62:3301/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v267) ==== 101+0+0 (secure 0 0 0) 0x7f692805fdf0 con 0x7f6940155d70 2024-09-17T05:32:56.570 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:32:56.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 msgr2=0x7f691c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 0x7f691c075710 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6930007fd0 tx=0x7f6930009040 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 msgr2=0x7f694009b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f692800e7d0 tx=0x7f692800ec90 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 shutdown_connections 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f691c073200 0x7f691c075710 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694009d6d0 0x7f694009dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6940155d70 0x7f694009b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 --2- 172.21.15.43:0/1736161822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694006f110 0x7f69401fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 >> 172.21.15.43:0/1736161822 conn(0x7f69400b23d0 msgr2=0x7f69400afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 shutdown_connections 2024-09-17T05:32:56.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.570+0000 7f6944e00640 1 -- 172.21.15.43:0/1736161822 wait complete. 2024-09-17T05:32:56.595 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:32:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 Processor -- start 2024-09-17T05:32:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 -- start start 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf00da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 0x556faf097b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556faef13a00 con 0x556faeff4cb0 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556faef71f40 con 0x556faf07be40 2024-09-17T05:32:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa348a0f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556faf07cdd0 con 0x556faefc5d70 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34642/0 (socket says 172.21.15.43:34642) 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 -- 172.21.15.43:0/3497108352 learned_addr learned my addr 172.21.15.43:0/3497108352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341c00640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf00da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa342600640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 0x556faf097b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 -- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 msgr2=0x556faf00da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf00da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 -- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 msgr2=0x556faf097b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 0x556faf097b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 -- 172.21.15.43:0/3497108352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556faeeecc10 con 0x556faeff4cb0 2024-09-17T05:32:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa342600640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 0x556faf097b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa341200640 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa33001c180 tx=0x7fa33001a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9826f34469c0c6ea server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa33b400640 1 -- 172.21.15.43:0/3497108352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300a7ad0 con 0x556faeff4cb0 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa33b400640 1 -- 172.21.15.43:0/3497108352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3300a7c70 con 0x556faeff4cb0 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.654+0000 7fa33b400640 1 -- 172.21.15.43:0/3497108352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300b06d0 con 0x556faeff4cb0 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- 172.21.15.43:0/3497108352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 msgr2=0x556faf097620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa33001c180 tx=0x7fa33001a870 comp rx=0 tx=0).stop 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- 172.21.15.43:0/3497108352 shutdown_connections 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faf07be40 0x556faf097b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faeff4cb0 0x556faf097620 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- 172.21.15.43:0/3497108352 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf00da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- 172.21.15.43:0/3497108352 >> 172.21.15.43:0/3497108352 conn(0x556faefb1d80 msgr2=0x556faf09aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- 172.21.15.43:0/3497108352 shutdown_connections 2024-09-17T05:32:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- 172.21.15.43:0/3497108352 wait complete. 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 Processor -- start 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- start start 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf07b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faeff4cb0 0x556faf1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 0x556faf0801c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556faf081f00 con 0x556faf07be40 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556faf0980a0 con 0x556faeff4cb0 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa348a0f080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556faf0821a0 con 0x556faefc5d70 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa342600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 0x556faf0801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa342600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 0x556faf0801c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34652/0 (socket says 172.21.15.43:34652) 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 learned_addr learned my addr 172.21.15.43:0/1925633959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf07b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.658+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faeff4cb0 0x556faf1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 msgr2=0x556faf07b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556faefc5d70 0x556faf07b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faeff4cb0 msgr2=0x556faf1356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556faeff4cb0 0x556faf1356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556faf07b860 con 0x556faf07be40 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa342600640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 0x556faf0801c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00a200 tx=0x7fa33c00a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa33a000640 1 -- 172.21.15.43:0/1925633959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c015030 con 0x556faf07be40 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa33a000640 1 -- 172.21.15.43:0/1925633959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa33c011630 con 0x556faf07be40 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa33a000640 1 -- 172.21.15.43:0/1925633959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c011840 con 0x556faf07be40 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556faf136100 con 0x556faf07be40 2024-09-17T05:32:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556faf07e3d0 con 0x556faf07be40 2024-09-17T05:32:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.662+0000 7fa33a000640 1 -- 172.21.15.43:0/1925633959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa33c018070 con 0x556faf07be40 2024-09-17T05:32:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa33a000640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa324071be0 0x7fa3240740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa33a000640 1 -- 172.21.15.43:0/1925633959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7fa33c0935e0 con 0x556faf07be40 2024-09-17T05:32:56.668 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:32:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa324071be0 0x7fa3240740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x556faf135be0 0x556faf07f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf07f560 con 0x556faf135be0 2024-09-17T05:32:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x556faf135be0 0x556faf07f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa324071be0 0x7fa3240740f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa334004fd0 tx=0x7fa33400e040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x556faf135be0 0x556faf07f020 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.666+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v119'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.670+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf07f900 con 0x556faf135be0 2024-09-17T05:32:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.670+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v119'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.670+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13e680 con 0x556faf135be0 2024-09-17T05:32:56.674 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:32:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.670+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v119'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.670+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13fa30 con 0x556faf135be0 2024-09-17T05:32:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13fdd0 con 0x556faf135be0 2024-09-17T05:32:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.674+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.678+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v119'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.678+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.678+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v119'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.678+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.684 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:32:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v119'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf140170 con 0x556faf135be0 2024-09-17T05:32:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf140170 con 0x556faf135be0 2024-09-17T05:32:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.682+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.686+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.686+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.686+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.686+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.693 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:32:56.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf140170 con 0x556faf135be0 2024-09-17T05:32:56.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.690+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf140170 con 0x556faf135be0 2024-09-17T05:32:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.694+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.694+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13ea20 con 0x556faf135be0 2024-09-17T05:32:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.694+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.694+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v119'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v119'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.698+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.704 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:32:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.702+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v119'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.702+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.702+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.702+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.702+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.706+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.706+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.706+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.706+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v119'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.706+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.710+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v119'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.710+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.713 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:32:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.710+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v119'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.710+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf13edf0 con 0x556faf135be0 2024-09-17T05:32:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x556faf13f300 0x556faf142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1426c0 con 0x556faf13f300 2024-09-17T05:32:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.714+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x556faf13f300 0x556faf142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.718+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x556faf13f300 0x556faf142180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.718+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'658 uv641 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.718+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf142a60 con 0x556faf13f300 2024-09-17T05:32:56.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.718+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'659 uv659 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.718+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf142e00 con 0x556faf13f300 2024-09-17T05:32:56.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.722+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.724 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:32:56.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.722+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1431a0 con 0x556faf13f300 2024-09-17T05:32:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.722+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.722+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1435c0 con 0x556faf13f300 2024-09-17T05:32:56.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1439e0 con 0x556faf13f300 2024-09-17T05:32:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v119'663 uv646 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf143dd0 con 0x556faf13f300 2024-09-17T05:32:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v119'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.726+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1441c0 con 0x556faf13f300 2024-09-17T05:32:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v119'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.732 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:32:56.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1445b0 con 0x556faf13f300 2024-09-17T05:32:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1445b0 con 0x556faf13f300 2024-09-17T05:32:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.730+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.734+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v119'668 uv651 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.734+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.734+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v119'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.734+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.739 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:32:56.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v119'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf148470 con 0x556faf13f300 2024-09-17T05:32:56.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v119'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf148470 con 0x556faf13f300 2024-09-17T05:32:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.738+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v119'673 uv657 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v119'674 uv674 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v119'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.747 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:32:56.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.742+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.746+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.746+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1452b0 con 0x556faf13f300 2024-09-17T05:32:56.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.746+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.746+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1442a0 con 0x556faf13f300 2024-09-17T05:32:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf13f300 2024-09-17T05:32:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556faf144970 0x556faf144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1476b0 con 0x556faf144970 2024-09-17T05:32:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556faf144970 0x556faf144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556faf144970 0x556faf144d80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v119'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1476b0 con 0x556faf144970 2024-09-17T05:32:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v119'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.750+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1476b0 con 0x556faf144970 2024-09-17T05:32:56.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v119'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.756 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:32:56.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf148720 con 0x556faf144970 2024-09-17T05:32:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1486e0 con 0x556faf144970 2024-09-17T05:32:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.754+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1486e0 con 0x556faf144970 2024-09-17T05:32:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1476f0 con 0x556faf135be0 2024-09-17T05:32:56.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'126 uv125 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1476d0 con 0x556faf135be0 2024-09-17T05:32:56.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'127 uv127 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.758+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1476d0 con 0x556faf135be0 2024-09-17T05:32:56.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'128 uv128 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.765 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:32:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf147b30 con 0x556faf135be0 2024-09-17T05:32:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x556faf148470 0x556faf14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf147b30 con 0x556faf148470 2024-09-17T05:32:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa342600640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x556faf148470 0x556faf14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.762+0000 7fa342600640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x556faf148470 0x556faf14ab60 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v119'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf14b0a0 con 0x556faf148470 2024-09-17T05:32:56.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v119'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14b440 con 0x556faf148470 2024-09-17T05:32:56.770 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:32:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v119'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.766+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14c890 con 0x556faf148470 2024-09-17T05:32:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14cd10 con 0x556faf148470 2024-09-17T05:32:56.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14b7e0 con 0x556faf148470 2024-09-17T05:32:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556faf14c2a0 0x556faf14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14f690 con 0x556faf14c2a0 2024-09-17T05:32:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.770+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556faf14c2a0 0x556faf14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.774+0000 7fa341c00640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556faf14c2a0 0x556faf14f110 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.774+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v119'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.774+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.774+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v119'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.774+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.779 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:32:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v119'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf150a70 con 0x556faf14c2a0 2024-09-17T05:32:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf150a70 con 0x556faf14c2a0 2024-09-17T05:32:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.778+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v119'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v119'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.782+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.787 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:32:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v119'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf14fa60 con 0x556faf14c2a0 2024-09-17T05:32:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa341c00640 1 -- 172.21.15.43:0/1925633959 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa334008c00 con 0x556faf14c2a0 2024-09-17T05:32:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556faf1500e0 0x556faf152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf152c70 con 0x556faf1500e0 2024-09-17T05:32:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556faf1500e0 0x556faf152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.786+0000 7fa341200640 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556faf1500e0 0x556faf152520 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v119'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf152c70 con 0x556faf1500e0 2024-09-17T05:32:56.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v119'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf1500e0 2024-09-17T05:32:56.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v119'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.795 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:32:56.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.790+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf1500e0 2024-09-17T05:32:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.794+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.794+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf1500e0 2024-09-17T05:32:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.794+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.794+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf135be0 2024-09-17T05:32:56.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v119'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf135be0 2024-09-17T05:32:56.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v119'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf135be0 2024-09-17T05:32:56.802 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:32:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v119'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.798+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1540d0 con 0x556faf135be0 2024-09-17T05:32:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1540d0 con 0x556faf135be0 2024-09-17T05:32:56.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf135be0 2024-09-17T05:32:56.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf135be0 2024-09-17T05:32:56.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.802+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf144970 2024-09-17T05:32:56.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v119'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf144970 2024-09-17T05:32:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v119'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf1530c0 con 0x556faf144970 2024-09-17T05:32:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v119'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.811 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:32:56.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.806+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf0807c0 con 0x556faf144970 2024-09-17T05:32:56.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.810+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.810+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf0807c0 con 0x556faf144970 2024-09-17T05:32:56.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.810+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.810+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf0807c0 con 0x556faf144970 2024-09-17T05:32:56.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v119'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf0807c0 con 0x556faf144970 2024-09-17T05:32:56.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v119'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf0807a0 con 0x556faf144970 2024-09-17T05:32:56.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v119'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.818 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:32:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.814+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf0807a0 con 0x556faf144970 2024-09-17T05:32:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.818+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.818+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf155ed0 con 0x556faf144970 2024-09-17T05:32:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.818+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf144970 2024-09-17T05:32:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.818+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153570 con 0x556faf148470 2024-09-17T05:32:56.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v119'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.826 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:32:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v119'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.822+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.826+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.826+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.826+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v119'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.826+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v119'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.833 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:32:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v119'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.830+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf154920 con 0x556faf148470 2024-09-17T05:32:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf148470 2024-09-17T05:32:56.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa33c002030 con 0x556faf148470 2024-09-17T05:32:56.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v119'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.834+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v119'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v119'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.841 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:32:56.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.838+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.842+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.842+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.842+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v119'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.842+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v119'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v119'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.849 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:32:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.846+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v119'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v119'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.850+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v119'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.856 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:32:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf154920 con 0x556faf1500e0 2024-09-17T05:32:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf154920 con 0x556faf1500e0 2024-09-17T05:32:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.854+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153910 con 0x556faf1500e0 2024-09-17T05:32:56.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556faf153dc0 con 0x556faf1500e0 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa341200640 1 -- 172.21.15.43:0/1925633959 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa3300a7cb0 con 0x556faf1500e0 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x556faf13f300 msgr2=0x556faf142180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x556faf13f300 0x556faf142180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556faf14c2a0 msgr2=0x556faf14f110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x556faf14c2a0 0x556faf14f110 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556faf1500e0 msgr2=0x556faf152520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x556faf1500e0 0x556faf152520 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556faf144970 msgr2=0x556faf144d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 reap_dead start 2024-09-17T05:32:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556faf144970 0x556faf144d80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x556faf135be0 msgr2=0x556faf07f020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x556faf135be0 0x556faf07f020 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x556faf148470 msgr2=0x556faf14ab60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x556faf148470 0x556faf14ab60 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa324071be0 msgr2=0x7fa3240740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa324071be0 0x7fa3240740f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa334004fd0 tx=0x7fa33400e040 comp rx=0 tx=0).stop 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 msgr2=0x556faf0801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 --2- 172.21.15.43:0/1925633959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556faf07be40 0x556faf0801c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00a200 tx=0x7fa33c00a6c0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa342600640 1 -- 172.21.15.43:0/1925633959 reap_dead start 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 shutdown_connections 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 >> 172.21.15.43:0/1925633959 conn(0x556faefb1d80 msgr2=0x556faeff50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 shutdown_connections 2024-09-17T05:32:56.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.858+0000 7fa348a0f080 1 -- 172.21.15.43:0/1925633959 wait complete. 2024-09-17T05:32:56.868 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-09-17T05:32:56.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 Processor -- start 2024-09-17T05:32:56.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 -- start start 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5973c9e70 0x55d5973a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d59729df40 con 0x55d5972f0250 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d5972aad00 con 0x55d5973c9e70 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.970+0000 7fe661b38080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d59723fa00 con 0x55d5973a9010 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34668/0 (socket says 172.21.15.43:34668) 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 -- 172.21.15.43:0/4276502931 learned_addr learned my addr 172.21.15.43:0/4276502931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:56.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65a200640 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65b600640 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5973c9e70 0x55d5973a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 -- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 msgr2=0x55d5973a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 -- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5973c9e70 msgr2=0x55d5973a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5973c9e70 0x55d5973a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 -- 172.21.15.43:0/4276502931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5973c8b50 con 0x55d5972f0250 2024-09-17T05:32:56.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe65ac00640 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b220 tx=0x7fe64c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e02ba735261a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe658e00640 1 -- 172.21.15.43:0/4276502931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00a040 con 0x55d5972f0250 2024-09-17T05:32:56.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe658e00640 1 -- 172.21.15.43:0/4276502931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe64c00f620 con 0x55d5972f0250 2024-09-17T05:32:56.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe658e00640 1 -- 172.21.15.43:0/4276502931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00fa50 con 0x55d5972f0250 2024-09-17T05:32:56.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- 172.21.15.43:0/4276502931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 msgr2=0x55d5973c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b220 tx=0x7fe64c00b6e0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- 172.21.15.43:0/4276502931 shutdown_connections 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5973c9e70 0x55d5973a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 --2- 172.21.15.43:0/4276502931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5972f0250 0x55d5973c6660 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- 172.21.15.43:0/4276502931 >> 172.21.15.43:0/4276502931 conn(0x55d5972ddd80 msgr2=0x55d5972f3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- 172.21.15.43:0/4276502931 shutdown_connections 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- 172.21.15.43:0/4276502931 wait complete. 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 Processor -- start 2024-09-17T05:32:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.974+0000 7fe661b38080 1 -- start start 2024-09-17T05:32:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5972f0250 0x55d5973aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 0x55d59745ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d5973a89c0 con 0x55d5973c9e70 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d5973a8c60 con 0x55d5972f0250 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d5973c5030 con 0x55d5973a9010 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 0x55d59745ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 0x55d59745ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34672/0 (socket says 172.21.15.43:34672) 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 -- 172.21.15.43:0/435607570 learned_addr learned my addr 172.21.15.43:0/435607570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65ac00640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5972f0250 0x55d5973aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65a200640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 -- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 msgr2=0x55d5973aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d5973a9010 0x55d5973aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 -- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5972f0250 msgr2=0x55d5973aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5972f0250 0x55d5973aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 -- 172.21.15.43:0/435607570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5974625f0 con 0x55d5973c9e70 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe65b600640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 0x55d59745ef10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe65400a600 tx=0x7fe6540aa590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe653400640 1 -- 172.21.15.43:0/435607570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540bc070 con 0x55d5973c9e70 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe653400640 1 -- 172.21.15.43:0/435607570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe654004020 con 0x55d5973c9e70 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe653400640 1 -- 172.21.15.43:0/435607570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6540af030 con 0x55d5973c9e70 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d597462890 con 0x55d5973c9e70 2024-09-17T05:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d597462dd0 con 0x55d5973c9e70 2024-09-17T05:32:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.978+0000 7fe653400640 1 -- 172.21.15.43:0/435607570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6540aa840 con 0x55d5973c9e70 2024-09-17T05:32:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe653400640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe63c071be0 0x7fe63c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe653400640 1 -- 172.21.15.43:0/435607570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7fe65412e7c0 con 0x55d5973c9e70 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65ac00640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe63c071be0 0x7fe63c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d5973c3a40 0x55d59745f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55d5973c1e60 con 0x55d5973c3a40 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65a200640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d5973c3a40 0x55d59745f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65ac00640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe63c071be0 0x7fe63c0740f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe64c003040 tx=0x7fe64c00e000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65a200640 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d5973c3a40 0x55d59745f960 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65a200640 1 -- 172.21.15.43:0/435607570 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v119'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe64800f870 con 0x55d5973c3a40 2024-09-17T05:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d5973c3a40 msgr2=0x55d59745f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d5973c3a40 0x55d59745f960 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe63c071be0 msgr2=0x7fe63c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe63c071be0 0x7fe63c0740f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe64c003040 tx=0x7fe64c00e000 comp rx=0 tx=0).stop 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 msgr2=0x55d59745ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 --2- 172.21.15.43:0/435607570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d5973c9e70 0x55d59745ef10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe65400a600 tx=0x7fe6540aa590 comp rx=0 tx=0).stop 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe65b600640 1 -- 172.21.15.43:0/435607570 reap_dead start 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 shutdown_connections 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.982+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 >> 172.21.15.43:0/435607570 conn(0x55d5972ddd80 msgr2=0x55d5972f3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.986+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 shutdown_connections 2024-09-17T05:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:56.986+0000 7fe661b38080 1 -- 172.21.15.43:0/435607570 wait complete. 2024-09-17T05:32:56.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:32:57.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 -- 172.21.15.43:0/3885976989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80b0330 msgr2=0x7effb806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 --2- 172.21.15.43:0/3885976989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80b0330 0x7effb806ce60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7effb80b3e30 tx=0x7effb001ca10 comp rx=0 tx=0).stop 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 -- 172.21.15.43:0/3885976989 shutdown_connections 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 --2- 172.21.15.43:0/3885976989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80b0330 0x7effb806ce60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 --2- 172.21.15.43:0/3885976989 >> v2:172.21.15.62:3300/0 conn(0x7effb80af9e0 0x7effb80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 --2- 172.21.15.43:0/3885976989 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b69a0 0x7effb80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 -- 172.21.15.43:0/3885976989 >> 172.21.15.43:0/3885976989 conn(0x7effb80b6590 msgr2=0x7effb80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 -- 172.21.15.43:0/3885976989 shutdown_connections 2024-09-17T05:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.150+0000 7effbd200640 1 -- 172.21.15.43:0/3885976989 wait complete. 2024-09-17T05:32:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 Processor -- start 2024-09-17T05:32:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 -- start start 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b0330 0x7effb8149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 0x7effb81f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7effb815d180 con 0x7effb80af9e0 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7effb815d000 con 0x7effb80b69a0 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effbd200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7effb815d300 con 0x7effb80b0330 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 0x7effb81f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b0330 0x7effb8149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34692/0 (socket says 172.21.15.43:34692) 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7400640 1 -- 172.21.15.43:0/3278333153 learned_addr learned my addr 172.21.15.43:0/3278333153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 -- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b0330 msgr2=0x7effb8149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b0330 0x7effb8149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 -- 172.21.15.43:0/3278333153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 msgr2=0x7effb814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 -- 172.21.15.43:0/3278333153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb814c9e0 con 0x7effb80b69a0 2024-09-17T05:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7400640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.154+0000 7effb7e00640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 0x7effb81f9d10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7effb8072270 tx=0x7effb0009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00bc070 con 0x7effb80b69a0 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effb0004290 con 0x7effb80b69a0 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00b2040 con 0x7effb80b69a0 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb814a0b0 con 0x7effb80b69a0 2024-09-17T05:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb814ce50 con 0x7effb80b69a0 2024-09-17T05:32:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb814cbb0 con 0x7effb80b69a0 2024-09-17T05:32:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7effb00046e0 con 0x7effb80b69a0 2024-09-17T05:32:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 0x7eff98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8289+0+0 (secure 0 0 0) 0x7effb012ecc0 con 0x7effb80b69a0 2024-09-17T05:32:57.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.158+0000 7effb7400640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 0x7eff98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.162+0000 7effb7400640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 0x7eff98075740 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7effa8005fd0 tx=0x7effa800b410 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:57.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.162+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effb814cbb0 con 0x7effb80b69a0 2024-09-17T05:32:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.370+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7effb806ce60 con 0x7effb80b69a0 2024-09-17T05:32:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.570+0000 7effaf400640 1 -- 172.21.15.43:0/3278333153 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v268) ==== 162+0+0 (secure 0 0 0) 0x7effb0016950 con 0x7effb80b69a0 2024-09-17T05:32:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 msgr2=0x7eff98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 0x7eff98075740 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7effa8005fd0 tx=0x7effa800b410 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 msgr2=0x7effb81f9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 0x7effb81f9d10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7effb8072270 tx=0x7effb0009350 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 shutdown_connections 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7eff98073230 0x7eff98075740 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7effb80b69a0 0x7effb81f9d10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7effb80b0330 0x7effb8149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 --2- 172.21.15.43:0/3278333153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effb80af9e0 0x7effb814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 >> 172.21.15.43:0/3278333153 conn(0x7effb80b6590 msgr2=0x7effb80b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 shutdown_connections 2024-09-17T05:32:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.574+0000 7effbd200640 1 -- 172.21.15.43:0/3278333153 wait complete. 2024-09-17T05:32:57.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:57.599 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:57.595+0000 7f078fa00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-17T05:32:57.599 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:32:57.595+0000 7f078fa00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:32:57.599 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:57.594+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:57.600 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:32:57.594+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 -- 172.21.15.43:0/3303659682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 msgr2=0x7fa5fc0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 --2- 172.21.15.43:0/3303659682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc0cbc70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0d12a0 tx=0x7fa5f401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 -- 172.21.15.43:0/3303659682 shutdown_connections 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 --2- 172.21.15.43:0/3303659682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc0cbc70 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 --2- 172.21.15.43:0/3303659682 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 --2- 172.21.15.43:0/3303659682 >> v2:172.21.15.62:3300/0 conn(0x7fa5fc070620 0x7fa5fc0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 -- 172.21.15.43:0/3303659682 >> 172.21.15.43:0/3303659682 conn(0x7fa5fc0b23d0 msgr2=0x7fa5fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:57.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 -- 172.21.15.43:0/3303659682 shutdown_connections 2024-09-17T05:32:57.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.754+0000 7fa600a00640 1 -- 172.21.15.43:0/3303659682 wait complete. 2024-09-17T05:32:57.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 Processor -- start 2024-09-17T05:32:57.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- start start 2024-09-17T05:32:57.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5fc070620 0x7fa5fc1f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc1f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc0ce490 con 0x7fa5fc0cb860 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc0ce310 con 0x7fa5fc070620 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc0ce610 con 0x7fa5fc06ce60 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f9e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc1f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f9e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:59648/0 (socket says 172.21.15.43:59648) 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f9e00640 1 -- 172.21.15.43:0/2900532853 learned_addr learned my addr 172.21.15.43:0/2900532853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f3e00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5fc070620 0x7fa5fc1f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 -- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 msgr2=0x7fa5fc0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 -- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5fc070620 msgr2=0x7fa5fc1f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5fc070620 0x7fa5fc1f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 -- 172.21.15.43:0/2900532853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc06d270 con 0x7fa5fc0cb860 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5fa800640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc1f9830 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ac000 tx=0x7fa5f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40b2040 con 0x7fa5fc0cb860 2024-09-17T05:32:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5f4004a50 con 0x7fa5fc0cb860 2024-09-17T05:32:57.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4004460 con 0x7fa5fc0cb860 2024-09-17T05:32:57.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc15a540 con 0x7fa5fc0cb860 2024-09-17T05:32:57.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.758+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc15a7e0 con 0x7fa5fc0cb860 2024-09-17T05:32:57.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.762+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc0cebc0 con 0x7fa5fc0cb860 2024-09-17T05:32:57.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.762+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa5f40be060 con 0x7fa5fc0cb860 2024-09-17T05:32:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.762+0000 7fa5f2000640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 0x7fa5d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:57.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.762+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fa5f412ebb0 con 0x7fa5fc0cb860 2024-09-17T05:32:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.762+0000 7fa5f9e00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 0x7fa5d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.766+0000 7fa5f9e00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 0x7fa5d8075740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec004590 tx=0x7fa5ec00b410 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.766+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa5fc06d7b0 con 0x7fa5fc0cb860 2024-09-17T05:32:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5fc0b1800 con 0x7fa5fc0cb860 2024-09-17T05:32:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa5f2000640 1 -- 172.21.15.43:0/2900532853 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 270 v270) ==== 95+0+3557 (secure 0 0 0) 0x7fa5f4016970 con 0x7fa5fc0cb860 2024-09-17T05:32:57.981 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:57.982 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":270,"btime":"2024-09-17T05:32:57:599951+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":270,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:57.599883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 270 2024-09-17T05:32:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 msgr2=0x7fa5d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 0x7fa5d8075740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec004590 tx=0x7fa5ec00b410 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 msgr2=0x7fa5fc1f9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.978+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc1f9830 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ac000 tx=0x7fa5f4004270 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 shutdown_connections 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5d8073230 0x7fa5d8075740 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5fc070620 0x7fa5fc1f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5fc0cb860 0x7fa5fc1f9830 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 --2- 172.21.15.43:0/2900532853 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5fc06ce60 0x7fa5fc0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 >> 172.21.15.43:0/2900532853 conn(0x7fa5fc0b23d0 msgr2=0x7fa5fc0ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 shutdown_connections 2024-09-17T05:32:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:57.982+0000 7fa600a00640 1 -- 172.21.15.43:0/2900532853 wait complete. 2024-09-17T05:32:58.004 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:58.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- 172.21.15.43:0/3011485421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 msgr2=0x7fe9a4004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/3011485421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 0x7fe9a4004150 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001860 tx=0x7fe9b001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9be800640 1 -- 172.21.15.43:0/3011485421 reap_dead start 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- 172.21.15.43:0/3011485421 shutdown_connections 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- 172.21.15.43:0/3011485421 >> 172.21.15.43:0/3011485421 conn(0x7fe9c00b23d0 msgr2=0x7fe9c00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- 172.21.15.43:0/3011485421 shutdown_connections 2024-09-17T05:32:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- 172.21.15.43:0/3011485421 wait complete. 2024-09-17T05:32:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 Processor -- start 2024-09-17T05:32:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 -- start start 2024-09-17T05:32:58.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.162+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 0x7fe9c00c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c01675f0 0x7fe9c0167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0168e80 con 0x7fe9c0161910 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0168d00 con 0x7fe9a4003d00 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c0169000 con 0x7fe9c01675f0 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 0x7fe9c00c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34712/0 (socket says 172.21.15.43:34712) 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 -- 172.21.15.43:0/1044257852 learned_addr learned my addr 172.21.15.43:0/1044257852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9be800640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c01675f0 0x7fe9c0167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 -- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c01675f0 msgr2=0x7fe9c0167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c01675f0 0x7fe9c0167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 -- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 msgr2=0x7fe9c00c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 0x7fe9c00c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 -- 172.21.15.43:0/1044257852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0167f40 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b7e00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8011ae0 tx=0x7fe9a8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a801e070 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9a8004460 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800fc10 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c00c44e0 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c01fc680 con 0x7fe9c0161910 2024-09-17T05:32:58.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c00b4b80 con 0x7fe9c0161910 2024-09-17T05:32:58.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.166+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9a8021070 con 0x7fe9c0161910 2024-09-17T05:32:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.170+0000 7fe9b6000640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 0x7fe99c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.170+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fe9a8095710 con 0x7fe9c0161910 2024-09-17T05:32:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.170+0000 7fe9bde00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 0x7fe99c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.170+0000 7fe9bde00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 0x7fe99c075740 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001830 tx=0x7fe9b0019720 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:58.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.174+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe9c0167f40 con 0x7fe9c0161910 2024-09-17T05:32:58.294 INFO:tasks.ceph.mds.c.smithi043.stderr: -49> 2024-09-17T05:32:53.539+0000 7f3c74000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-17T05:32:58.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.382+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9c00adb60 con 0x7fe9c0161910 2024-09-17T05:32:58.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.382+0000 7fe9b6000640 1 -- 172.21.15.43:0/1044257852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 270 v270) ==== 95+0+3557 (secure 0 0 0) 0x7fe9a805f680 con 0x7fe9c0161910 2024-09-17T05:32:58.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:58.387 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":270,"btime":"2024-09-17T05:32:57:599951+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":270,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:57.599883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 270 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 msgr2=0x7fe99c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 0x7fe99c075740 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001830 tx=0x7fe9b0019720 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 msgr2=0x7fe9c0099890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8011ae0 tx=0x7fe9a8009ea0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 shutdown_connections 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe99c073230 0x7fe99c075740 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe9c01675f0 0x7fe9c0167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c0161910 0x7fe9c0099890 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 --2- 172.21.15.43:0/1044257852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a4003d00 0x7fe9c00c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 >> 172.21.15.43:0/1044257852 conn(0x7fe9c00b23d0 msgr2=0x7fe9c015f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:58.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 shutdown_connections 2024-09-17T05:32:58.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.386+0000 7fe9c4c00640 1 -- 172.21.15.43:0/1044257852 wait complete. 2024-09-17T05:32:58.418 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:32:58.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:58.423 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- 172.21.15.43:0/72632522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b2f60 msgr2=0x7f7bf00b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 --2- 172.21.15.43:0/72632522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b2f60 0x7f7bf00b3370 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009840 tx=0x7f7bdc01c790 comp rx=0 tx=0).stop 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- 172.21.15.43:0/72632522 shutdown_connections 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 --2- 172.21.15.43:0/72632522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b2f60 0x7f7bf00b3370 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- 172.21.15.43:0/72632522 >> 172.21.15.43:0/72632522 conn(0x7f7bf00d0010 msgr2=0x7f7bf00d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- 172.21.15.43:0/72632522 shutdown_connections 2024-09-17T05:32:58.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- 172.21.15.43:0/72632522 wait complete. 2024-09-17T05:32:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 Processor -- start 2024-09-17T05:32:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.578+0000 7f7befe00640 1 -- start start 2024-09-17T05:32:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 0x7f7bf014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01692c0 con 0x7f7bf0162900 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0169140 con 0x7f7bf00b4430 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf0169440 con 0x7f7bf00b2f60 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bee000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7beea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 0x7f7bf014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bee000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:43456/0 (socket says 172.21.15.43:43456) 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7beea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:59676/0 (socket says 172.21.15.43:59676) 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bee000640 1 -- 172.21.15.43:0/927804246 learned_addr learned my addr 172.21.15.43:0/927804246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 -- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 msgr2=0x7f7bf015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 -- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 msgr2=0x7f7bf0157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 -- 172.21.15.43:0/927804246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf009d5d0 con 0x7f7bf0162900 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7beea00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bee000640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7bef400640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 0x7f7bf014eae0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c650 tx=0x7f7be800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8003b80 con 0x7f7bf0162900 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7be8003d20 con 0x7f7bf0162900 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf0160470 con 0x7f7bf0162900 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8005330 con 0x7f7bf0162900 2024-09-17T05:32:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf0160a30 con 0x7f7bf0162900 2024-09-17T05:32:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.582+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf0159040 con 0x7f7bf0162900 2024-09-17T05:32:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.586+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7be80054d0 con 0x7f7bf0162900 2024-09-17T05:32:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.586+0000 7f7be6a00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.586+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f7be80948d0 con 0x7f7bf0162900 2024-09-17T05:32:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.586+0000 7f7beea00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.586+0000 7f7beea00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7be00046c0 tx=0x7f7be0010040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.590+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7be805e7f0 con 0x7f7bf0162900 2024-09-17T05:32:58.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bf00b3370 con 0x7f7bf0162900 2024-09-17T05:32:58.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7be6a00640 1 -- 172.21.15.43:0/927804246 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 271 v271) ==== 95+0+4380 (secure 0 0 0) 0x7f7be8063220 con 0x7f7bf0162900 2024-09-17T05:32:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:32:58.809 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":271,"btime":"2024-09-17T05:32:58:602129+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/351071204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":351071204},{"type":"v1","addr":"172.21.15.43:6835","nonce":351071204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":271}],"filesystems":[{"mdsmap":{"epoch":270,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:32:57.599883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:32:58.809 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 271 2024-09-17T05:32:58.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 msgr2=0x7f7bd0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7be00046c0 tx=0x7f7be0010040 comp rx=0 tx=0).stop 2024-09-17T05:32:58.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 msgr2=0x7f7bf014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:58.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 0x7f7bf014eae0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c650 tx=0x7f7be800cb10 comp rx=0 tx=0).stop 2024-09-17T05:32:58.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 shutdown_connections 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7bd0073230 0x7f7bd0075740 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bf00b4430 0x7f7bf0157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf0162900 0x7f7bf014eae0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 --2- 172.21.15.43:0/927804246 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7bf00b2f60 0x7f7bf015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 >> 172.21.15.43:0/927804246 conn(0x7f7bf00d0010 msgr2=0x7f7bf00d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.806+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 shutdown_connections 2024-09-17T05:32:58.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:58.810+0000 7f7befe00640 1 -- 172.21.15.43:0/927804246 wait complete. 2024-09-17T05:32:58.832 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-09-17T05:32:58.832 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-09-17T05:32:58.832 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:32:58.832 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:58.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:58.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:32:58.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:58.888 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:32:58.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:32:58.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- 172.21.15.43:0/3810905726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91200737c0 msgr2=0x7f91200700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 --2- 172.21.15.43:0/3810905726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91200737c0 0x7f91200700a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f911c009bf0 tx=0x7f911c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- 172.21.15.43:0/3810905726 shutdown_connections 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 --2- 172.21.15.43:0/3810905726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91200737c0 0x7f91200700a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- 172.21.15.43:0/3810905726 >> 172.21.15.43:0/3810905726 conn(0x7f91200b23d0 msgr2=0x7f91200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- 172.21.15.43:0/3810905726 shutdown_connections 2024-09-17T05:32:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- 172.21.15.43:0/3810905726 wait complete. 2024-09-17T05:32:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 Processor -- start 2024-09-17T05:32:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.142+0000 7f9128800640 1 -- start start 2024-09-17T05:32:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912006fc90 0x7f9120098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912015fe10 0x7f91201531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f912016b640 con 0x7f9120099600 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f912016b4c0 con 0x7f912006fc90 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f912016b7c0 con 0x7f912015fe10 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34766/0 (socket says 172.21.15.43:34766) 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 -- 172.21.15.43:0/3087901892 learned_addr learned my addr 172.21.15.43:0/3087901892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9125400640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912015fe10 0x7f91201531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9125e00640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912006fc90 0x7f9120098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 -- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912015fe10 msgr2=0x7f91201531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912015fe10 0x7f91201531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 -- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912006fc90 msgr2=0x7f9120098eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912006fc90 0x7f9120098eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 -- 172.21.15.43:0/3087901892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91200993f0 con 0x7f9120099600 2024-09-17T05:32:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9126800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f911c01c4f0 tx=0x7f911c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c0b2040 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f911c0bb070 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c004570 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9120099b50 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91200b0c80 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.146+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9120099dc0 con 0x7f9120099600 2024-09-17T05:32:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.150+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f911c004710 con 0x7f9120099600 2024-09-17T05:32:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.150+0000 7f911a000640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 0x7f9104075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.150+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f911c12f5e0 con 0x7f9120099600 2024-09-17T05:32:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.150+0000 7f9125e00640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 0x7f9104075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.150+0000 7f9125e00640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 0x7f9104075740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9114007fd0 tx=0x7f9114009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.154+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f911c016790 con 0x7f9120099600 2024-09-17T05:32:59.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.366+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91200700a0 con 0x7f9120099600 2024-09-17T05:32:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.606+0000 7f911a000640 1 -- 172.21.15.43:0/3087901892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v272) ==== 161+0+0 (secure 0 0 0) 0x7f911c0f9490 con 0x7f9120099600 2024-09-17T05:32:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 msgr2=0x7f9104075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 0x7f9104075740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9114007fd0 tx=0x7f9114009040 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 msgr2=0x7f9120153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f911c01c4f0 tx=0x7f911c005ae0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 shutdown_connections 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9104073230 0x7f9104075740 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9120099600 0x7f9120153700 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f912015fe10 0x7f91201531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 --2- 172.21.15.43:0/3087901892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f912006fc90 0x7f9120098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 >> 172.21.15.43:0/3087901892 conn(0x7f91200b23d0 msgr2=0x7f91200ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 shutdown_connections 2024-09-17T05:32:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.610+0000 7f9128800640 1 -- 172.21.15.43:0/3087901892 wait complete. 2024-09-17T05:32:59.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- 172.21.15.43:0/154866908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7415fe10 msgr2=0x7f5a740b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 --2- 172.21.15.43:0/154866908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7415fe10 0x7f5a740b4b80 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009900 tx=0x7f5a6401c780 comp rx=0 tx=0).stop 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- 172.21.15.43:0/154866908 shutdown_connections 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 --2- 172.21.15.43:0/154866908 >> v2:172.21.15.62:3300/0 conn(0x7f5a740737c0 0x7f5a740700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 --2- 172.21.15.43:0/154866908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7415fe10 0x7f5a740b4b80 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- 172.21.15.43:0/154866908 >> 172.21.15.43:0/154866908 conn(0x7f5a740b23d0 msgr2=0x7f5a740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- 172.21.15.43:0/154866908 shutdown_connections 2024-09-17T05:32:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- 172.21.15.43:0/154866908 wait complete. 2024-09-17T05:32:59.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 Processor -- start 2024-09-17T05:32:59.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.790+0000 7f5a79600640 1 -- start start 2024-09-17T05:32:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7415fe10 0x7f5a741fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a74158170 0x7f5a741fc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74169fb0 con 0x7f5a7406fc90 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74169e30 con 0x7f5a7415fe10 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7416a130 con 0x7f5a74158170 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34784/0 (socket says 172.21.15.43:34784) 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 -- 172.21.15.43:0/822607874 learned_addr learned my addr 172.21.15.43:0/822607874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a6be00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7415fe10 0x7f5a741fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a73400640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a74158170 0x7f5a741fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 -- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a74158170 msgr2=0x7f5a741fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a74158170 0x7f5a741fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 -- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7415fe10 msgr2=0x7f5a741fbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7415fe10 0x7f5a741fbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 -- 172.21.15.43:0/822607874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741fcc10 con 0x7f5a7406fc90 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a72a00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009850 tx=0x7f5a64005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640b2040 con 0x7f5a7406fc90 2024-09-17T05:32:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a64004330 con 0x7f5a7406fc90 2024-09-17T05:32:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64005280 con 0x7f5a7406fc90 2024-09-17T05:32:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a74097d20 con 0x7f5a7406fc90 2024-09-17T05:32:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a740982e0 con 0x7f5a7406fc90 2024-09-17T05:32:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.794+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a740700a0 con 0x7f5a7406fc90 2024-09-17T05:32:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.798+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a64005420 con 0x7f5a7406fc90 2024-09-17T05:32:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.798+0000 7f5a6aa00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 0x7f5a50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.798+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f5a640bb070 con 0x7f5a7406fc90 2024-09-17T05:32:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.798+0000 7f5a6be00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 0x7f5a50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.798+0000 7f5a6be00640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 0x7f5a50075710 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5a74159840 tx=0x7f5a5c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:32:59.802+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a64016490 con 0x7f5a7406fc90 2024-09-17T05:33:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.042+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5a74076380 con 0x7f5a7406fc90 2024-09-17T05:33:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.618+0000 7f5a6aa00640 1 -- 172.21.15.43:0/822607874 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v273) ==== 101+0+0 (secure 0 0 0) 0x7f5a640f8980 con 0x7f5a7406fc90 2024-09-17T05:33:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 msgr2=0x7f5a50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 0x7f5a50075710 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5a74159840 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 msgr2=0x7f5a74157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009850 tx=0x7f5a64005de0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 shutdown_connections 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a50073200 0x7f5a50075710 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a7415fe10 0x7f5a741fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a74158170 0x7f5a741fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 --2- 172.21.15.43:0/822607874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7406fc90 0x7f5a74157b70 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 >> 172.21.15.43:0/822607874 conn(0x7f5a740b23d0 msgr2=0x7f5a740adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 shutdown_connections 2024-09-17T05:33:00.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.622+0000 7f5a79600640 1 -- 172.21.15.43:0/822607874 wait complete. 2024-09-17T05:33:00.647 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.702+0000 7f0478e55080 1 Processor -- start 2024-09-17T05:33:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.702+0000 7f0478e55080 1 -- start start 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f3be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f42da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a4f2c4a00 con 0x557a4f3a5cb0 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557a4f322f40 con 0x557a4f432da0 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557a4f431b40 con 0x557a4f374890 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34804/0 (socket says 172.21.15.43:34804) 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 -- 172.21.15.43:0/3621743966 learned_addr learned my addr 172.21.15.43:0/3621743966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0472a00640 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f42da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0472000640 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f3be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 -- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 msgr2=0x557a4f3be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f3be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 -- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 msgr2=0x557a4f42da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f42da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 -- 172.21.15.43:0/3621743966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4f29dc10 con 0x557a4f3a5cb0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0471600640 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f046000b5c0 tx=0x7f046000ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ebe094981d489f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f046be00640 1 -- 172.21.15.43:0/3621743966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046000a950 con 0x557a4f3a5cb0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f046be00640 1 -- 172.21.15.43:0/3621743966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f046000aaf0 con 0x557a4f3a5cb0 2024-09-17T05:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f046be00640 1 -- 172.21.15.43:0/3621743966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460008460 con 0x557a4f3a5cb0 2024-09-17T05:33:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- 172.21.15.43:0/3621743966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 msgr2=0x557a4f3beca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f046000b5c0 tx=0x7f046000ba80 comp rx=0 tx=0).stop 2024-09-17T05:33:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- 172.21.15.43:0/3621743966 shutdown_connections 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f42da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f3beca0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 --2- 172.21.15.43:0/3621743966 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f3be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- 172.21.15.43:0/3621743966 >> 172.21.15.43:0/3621743966 conn(0x557a4f362d80 msgr2=0x557a4f378a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- 172.21.15.43:0/3621743966 shutdown_connections 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.706+0000 7f0478e55080 1 -- 172.21.15.43:0/3621743966 wait complete. 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 Processor -- start 2024-09-17T05:33:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- start start 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f42d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f433690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a4f448980 con 0x557a4f3a5cb0 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557a4f4486e0 con 0x557a4f432da0 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557a4f448c20 con 0x557a4f374890 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34816/0 (socket says 172.21.15.43:34816) 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 learned_addr learned my addr 172.21.15.43:0/2154500838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f42d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0472a00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f433690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 msgr2=0x557a4f42d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557a4f374890 0x557a4f42d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 msgr2=0x557a4f433690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557a4f432da0 0x557a4f433690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4f42e090 con 0x557a4f3a5cb0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0460016a20 tx=0x7f0460016a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f046aa00640 1 -- 172.21.15.43:0/2154500838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460008670 con 0x557a4f3a5cb0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f046aa00640 1 -- 172.21.15.43:0/2154500838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0460008810 con 0x557a4f3a5cb0 2024-09-17T05:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f046aa00640 1 -- 172.21.15.43:0/2154500838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460008aa0 con 0x557a4f3a5cb0 2024-09-17T05:33:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a4f42e330 con 0x557a4f3a5cb0 2024-09-17T05:33:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.710+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4f42e870 con 0x557a4f3a5cb0 2024-09-17T05:33:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f046aa00640 1 -- 172.21.15.43:0/2154500838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0460008c40 con 0x557a4f3a5cb0 2024-09-17T05:33:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f046aa00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 0x7f04540740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f046aa00640 1 -- 172.21.15.43:0/2154500838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f0460095b50 con 0x557a4f3a5cb0 2024-09-17T05:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:00.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 0x7f04540740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 0x557a4f4e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4ea170 con 0x557a4f4340e0 2024-09-17T05:33:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.714+0000 7f0472a00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 0x557a4f4e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.718+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 0x7f04540740f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f04640ab000 tx=0x7f04640ad040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.718+0000 7f0472a00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 0x557a4f4e9c30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.718+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v120'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.718+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4ea510 con 0x557a4f4340e0 2024-09-17T05:33:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.722+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v120'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.722+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4ea8b0 con 0x557a4f4340e0 2024-09-17T05:33:00.726 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.722+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v120'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.722+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3db0 con 0x557a4f4340e0 2024-09-17T05:33:00.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f4150 con 0x557a4f4340e0 2024-09-17T05:33:00.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f2da0 con 0x557a4f4340e0 2024-09-17T05:33:00.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.726+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3190 con 0x557a4f4340e0 2024-09-17T05:33:00.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.730+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v120'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.730+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f3580 con 0x557a4f4340e0 2024-09-17T05:33:00.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.730+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v120'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.730+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3580 con 0x557a4f4340e0 2024-09-17T05:33:00.736 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v120'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.734+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.738+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.738+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.738+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.738+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.744 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:00.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.742+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f3530 con 0x557a4f4340e0 2024-09-17T05:33:00.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f39b0 con 0x557a4f4340e0 2024-09-17T05:33:00.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.746+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v120'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v120'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.755 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:00.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v120'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.750+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.754+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.754+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.754+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.754+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.754+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.758+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.758+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v120'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.758+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.758+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v120'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.758+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.764 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v120'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.762+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4340e0 2024-09-17T05:33:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.766+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.766+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557a4f4f4a00 0x557a4f4f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.766+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f37b0 con 0x557a4f4f4a00 2024-09-17T05:33:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.766+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557a4f4f4a00 0x557a4f4f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.766+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557a4f4f4a00 0x557a4f4f6e20 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'679 uv662 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f7360 con 0x557a4f4f4a00 2024-09-17T05:33:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'680 uv680 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7780 con 0x557a4f4f4a00 2024-09-17T05:33:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.775 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:00.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.770+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f8040 con 0x557a4f4f4a00 2024-09-17T05:33:00.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f83e0 con 0x557a4f4f4a00 2024-09-17T05:33:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f8780 con 0x557a4f4f4a00 2024-09-17T05:33:00.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v120'684 uv667 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.774+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f8b20 con 0x557a4f4f4a00 2024-09-17T05:33:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.778+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v120'685 uv685 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.778+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f8b20 con 0x557a4f4f4a00 2024-09-17T05:33:00.782 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:00.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.778+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v120'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.778+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.782+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.782+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.782+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.782+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v120'689 uv672 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v120'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.790 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v120'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.786+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.790+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v120'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.790+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.790+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.790+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v120'694 uv678 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v120'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.798 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v120'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.794+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4f7c70 con 0x557a4f4f4a00 2024-09-17T05:33:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f4f4a00 2024-09-17T05:33:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557a4f4f94e0 0x557a4f4fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fbe40 con 0x557a4f4f94e0 2024-09-17T05:33:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.798+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557a4f4f94e0 0x557a4f4fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.802+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557a4f4f94e0 0x557a4f4fb900 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.802+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v120'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.802+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4fc220 con 0x557a4f4f94e0 2024-09-17T05:33:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.802+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v120'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.802+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fc700 con 0x557a4f4f94e0 2024-09-17T05:33:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v120'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.808 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fdfb0 con 0x557a4f4f94e0 2024-09-17T05:33:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fe350 con 0x557a4f4f94e0 2024-09-17T05:33:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.806+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fcfa0 con 0x557a4f4f94e0 2024-09-17T05:33:00.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4340e0 2024-09-17T05:33:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'130 uv129 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4340e0 2024-09-17T05:33:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.810+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4340e0 2024-09-17T05:33:00.816 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4340e0 2024-09-17T05:33:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 0x557a4f500b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4fe6f0 2024-09-17T05:33:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0472a00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 0x557a4f500b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.814+0000 7f0472a00640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 0x557a4f500b10 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v120'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4fe6f0 2024-09-17T05:33:00.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v120'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f4fd390 con 0x557a4f4fe6f0 2024-09-17T05:33:00.823 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:00.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v120'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.818+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f5020b0 con 0x557a4f4fe6f0 2024-09-17T05:33:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f5020b0 con 0x557a4f4fe6f0 2024-09-17T05:33:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f5010a0 con 0x557a4f4fe6f0 2024-09-17T05:33:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557a4f5015b0 0x557a4f5039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.822+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504030 con 0x557a4f5015b0 2024-09-17T05:33:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557a4f5015b0 0x557a4f5039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0472000640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557a4f5015b0 0x557a4f5039d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v120'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v120'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.826+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.833 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.830+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v120'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.830+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f505410 con 0x557a4f5015b0 2024-09-17T05:33:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.830+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.830+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f505410 con 0x557a4f5015b0 2024-09-17T05:33:00.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v120'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.834+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v120'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.841 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:00.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v120'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.838+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f504400 con 0x557a4f5015b0 2024-09-17T05:33:00.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0472000640 1 -- 172.21.15.43:0/2154500838 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04640b3040 con 0x557a4f5015b0 2024-09-17T05:33:00.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557a4f504a80 0x557a4f506ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:00.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f5075d0 con 0x557a4f504a80 2024-09-17T05:33:00.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557a4f504a80 0x557a4f506ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:00.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0471600640 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557a4f504a80 0x557a4f506ea0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:00.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v120'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.842+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f5079a0 con 0x557a4f504a80 2024-09-17T05:33:00.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v120'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f504a80 2024-09-17T05:33:00.849 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:00.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v120'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f504a80 2024-09-17T05:33:00.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.846+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f504a80 2024-09-17T05:33:00.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4340e0 2024-09-17T05:33:00.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v120'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4340e0 2024-09-17T05:33:00.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v120'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.850+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4340e0 2024-09-17T05:33:00.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v120'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.856 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:00.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508dd0 con 0x557a4f4340e0 2024-09-17T05:33:00.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508dd0 con 0x557a4f4340e0 2024-09-17T05:33:00.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.854+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4340e0 2024-09-17T05:33:00.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.858+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4340e0 2024-09-17T05:33:00.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.858+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.858+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v120'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.858+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v120'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.865 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:00.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v120'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.862+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.866+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.866+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.866+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v120'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.866+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.866+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v120'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.870+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.870+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v120'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.873 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:00.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.870+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f507dc0 con 0x557a4f4f94e0 2024-09-17T05:33:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.870+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.870+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f50abd0 con 0x557a4f4f94e0 2024-09-17T05:33:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.874+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f4f94e0 2024-09-17T05:33:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.874+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.874+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v120'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.874+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.881 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:00.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v120'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.878+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.882+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v120'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.882+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.882+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v120'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.882+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.887 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:00.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v120'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f509280 con 0x557a4f4fe6f0 2024-09-17T05:33:00.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f509280 con 0x557a4f4fe6f0 2024-09-17T05:33:00.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.886+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f4fe6f0 2024-09-17T05:33:00.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046c003840 con 0x557a4f4fe6f0 2024-09-17T05:33:00.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v120'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v120'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.890+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.896 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:00.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v120'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.894+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.898+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v120'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.898+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.898+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v120'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.898+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.903 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:00.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v120'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.902+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v120'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v120'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.911 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v120'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.906+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f509280 con 0x557a4f504a80 2024-09-17T05:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f509280 con 0x557a4f504a80 2024-09-17T05:33:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f508270 con 0x557a4f504a80 2024-09-17T05:33:00.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.910+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557a4f5084e0 con 0x557a4f504a80 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0471600640 1 -- 172.21.15.43:0/2154500838 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f046005be50 con 0x557a4f504a80 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557a4f4f4a00 msgr2=0x557a4f4f6e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x557a4f4f4a00 0x557a4f4f6e20 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557a4f5015b0 msgr2=0x557a4f5039d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x557a4f5015b0 0x557a4f5039d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557a4f504a80 msgr2=0x557a4f506ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557a4f504a80 0x557a4f506ea0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557a4f4f94e0 msgr2=0x557a4f4fb900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x557a4f4f94e0 0x557a4f4fb900 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 reap_dead start 2024-09-17T05:33:00.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 msgr2=0x557a4f4e9c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 0x557a4f4e9c30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0472a00640 1 -- 172.21.15.43:0/2154500838 reap_dead start 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 msgr2=0x557a4f500b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 0x557a4f500b10 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 msgr2=0x7f04540740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 0x7f04540740f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f04640ab000 tx=0x7f04640ad040 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 msgr2=0x557a4f42bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0460016a20 tx=0x7f0460016a50 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 shutdown_connections 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x557a4f4fe6f0 0x557a4f500b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0454071be0 0x7f04540740f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x557a4f4340e0 0x557a4f4e9c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 --2- 172.21.15.43:0/2154500838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a4f3a5cb0 0x557a4f42bff0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:00.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 >> 172.21.15.43:0/2154500838 conn(0x557a4f362d80 msgr2=0x557a4f378a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:00.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 shutdown_connections 2024-09-17T05:33:00.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:00.914+0000 7f0478e55080 1 -- 172.21.15.43:0/2154500838 wait complete. 2024-09-17T05:33:00.923 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.022+0000 7fc011abf080 1 Processor -- start 2024-09-17T05:33:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.022+0000 7fc011abf080 1 -- start start 2024-09-17T05:33:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac186d70 0x5613ac184a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac23ce40 0x5613ac1cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5613ac0d4a00 con 0x5613ac1b5cb0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5613ac132f40 con 0x5613ac23ce40 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5613ac23d890 con 0x5613ac186d70 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34820/0 (socket says 172.21.15.43:34820) 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 -- 172.21.15.43:0/2793043748 learned_addr learned my addr 172.21.15.43:0/2793043748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00ac00640 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac186d70 0x5613ac184a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00b600640 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac23ce40 0x5613ac1cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 -- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac186d70 msgr2=0x5613ac184a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac186d70 0x5613ac184a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 -- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac23ce40 msgr2=0x5613ac1cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac23ce40 0x5613ac1cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 -- 172.21.15.43:0/2793043748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ac0adc10 con 0x5613ac1b5cb0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc00a200640 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006f70 tx=0x7fbff800b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8111fa2d826060e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc008e00640 1 -- 172.21.15.43:0/2793043748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff800e840 con 0x5613ac1b5cb0 2024-09-17T05:33:01.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc008e00640 1 -- 172.21.15.43:0/2793043748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbff800e9e0 con 0x5613ac1b5cb0 2024-09-17T05:33:01.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc008e00640 1 -- 172.21.15.43:0/2793043748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff800ee30 con 0x5613ac1b5cb0 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- 172.21.15.43:0/2793043748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 msgr2=0x5613ac1cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006f70 tx=0x7fbff800b740 comp rx=0 tx=0).stop 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- 172.21.15.43:0/2793043748 shutdown_connections 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac23ce40 0x5613ac1cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac1b5cb0 0x5613ac1cce90 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 --2- 172.21.15.43:0/2793043748 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac186d70 0x5613ac184a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- 172.21.15.43:0/2793043748 >> 172.21.15.43:0/2793043748 conn(0x5613ac172d80 msgr2=0x5613ac1b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- 172.21.15.43:0/2793043748 shutdown_connections 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.026+0000 7fc011abf080 1 -- 172.21.15.43:0/2793043748 wait complete. 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 Processor -- start 2024-09-17T05:33:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- start start 2024-09-17T05:33:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 0x5613ac23bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac1b5cb0 0x5613ac2e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac23ce40 0x5613ac2e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5613ac1cc890 con 0x5613ac186d70 2024-09-17T05:33:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5613ac2e2be0 con 0x5613ac1b5cb0 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5613ac23c4a0 con 0x5613ac23ce40 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 0x5613ac23bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 0x5613ac23bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34830/0 (socket says 172.21.15.43:34830) 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 -- 172.21.15.43:0/3915133142 learned_addr learned my addr 172.21.15.43:0/3915133142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00a200640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac1b5cb0 0x5613ac2e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00b600640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac23ce40 0x5613ac2e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 -- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac23ce40 msgr2=0x5613ac2e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5613ac23ce40 0x5613ac2e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 -- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac1b5cb0 msgr2=0x5613ac2e48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac1b5cb0 0x5613ac2e48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 -- 172.21.15.43:0/3915133142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ac2e2e10 con 0x5613ac186d70 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00a200640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5613ac1b5cb0 0x5613ac2e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc00ac00640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 0x5613ac23bf60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbffc01b640 tx=0x7fbffc0a78e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc003400640 1 -- 172.21.15.43:0/3915133142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0b0940 con 0x5613ac186d70 2024-09-17T05:33:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc003400640 1 -- 172.21.15.43:0/3915133142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbffc0b0ae0 con 0x5613ac186d70 2024-09-17T05:33:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc003400640 1 -- 172.21.15.43:0/3915133142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0bb3f0 con 0x5613ac186d70 2024-09-17T05:33:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613ac2e30b0 con 0x5613ac186d70 2024-09-17T05:33:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.030+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613ac2e35f0 con 0x5613ac186d70 2024-09-17T05:33:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc003400640 1 -- 172.21.15.43:0/3915133142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbffc0c2060 con 0x5613ac186d70 2024-09-17T05:33:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc003400640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfec071be0 0x7fbfec0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc003400640 1 -- 172.21.15.43:0/3915133142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fbffc130d70 con 0x5613ac186d70 2024-09-17T05:33:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc011abf080 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5613ac2e3fd0 0x5613ac25b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc00a200640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfec071be0 0x7fbfec0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5613ac2ea090 con 0x5613ac2e3fd0 2024-09-17T05:33:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc00b600640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5613ac2e3fd0 0x5613ac25b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc00b600640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5613ac2e3fd0 0x5613ac25b110 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.034+0000 7fc00a200640 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfec071be0 0x7fbfec0740f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006e60 tx=0x7fbff8019040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc00b600640 1 -- 172.21.15.43:0/3915133142 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v120'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc004002040 con 0x5613ac2e3fd0 2024-09-17T05:33:01.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5613ac2e3fd0 msgr2=0x5613ac25b110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5613ac2e3fd0 0x5613ac25b110 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfec071be0 msgr2=0x7fbfec0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbfec071be0 0x7fbfec0740f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbff8006e60 tx=0x7fbff8019040 comp rx=0 tx=0).stop 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 msgr2=0x5613ac23bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 --2- 172.21.15.43:0/3915133142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5613ac186d70 0x5613ac23bf60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbffc01b640 tx=0x7fbffc0a78e0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc00b600640 1 -- 172.21.15.43:0/3915133142 reap_dead start 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 shutdown_connections 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 >> 172.21.15.43:0/3915133142 conn(0x5613ac172d80 msgr2=0x5613ac25c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 shutdown_connections 2024-09-17T05:33:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.038+0000 7fc011abf080 1 -- 172.21.15.43:0/3915133142 wait complete. 2024-09-17T05:33:01.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- 172.21.15.43:0/1946059238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 msgr2=0x7fcb3c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 --2- 172.21.15.43:0/1946059238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c0b3130 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0cc8c0 tx=0x7fcb2c01c800 comp rx=0 tx=0).stop 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- 172.21.15.43:0/1946059238 shutdown_connections 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 --2- 172.21.15.43:0/1946059238 >> v2:172.21.15.62:3300/0 conn(0x7fcb3c0d0650 0x7fcb3c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 --2- 172.21.15.43:0/1946059238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c0b3130 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- 172.21.15.43:0/1946059238 >> 172.21.15.43:0/1946059238 conn(0x7fcb3c0cee60 msgr2=0x7fcb3c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- 172.21.15.43:0/1946059238 shutdown_connections 2024-09-17T05:33:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- 172.21.15.43:0/1946059238 wait complete. 2024-09-17T05:33:01.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 Processor -- start 2024-09-17T05:33:01.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 -- start start 2024-09-17T05:33:01.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.206+0000 7fcb41600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3c0d0240 0x7fcb3c158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3c0d0650 0x7fcb3c06f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c0b2ad0 con 0x7fcb3c09ba60 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c0b2950 con 0x7fcb3c0d0240 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3c169e30 con 0x7fcb3c0d0650 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34850/0 (socket says 172.21.15.43:34850) 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 -- 172.21.15.43:0/4259657576 learned_addr learned my addr 172.21.15.43:0/4259657576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb32000640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3c0d0240 0x7fcb3c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3b400640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3c0d0650 0x7fcb3c06f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 -- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3c0d0650 msgr2=0x7fcb3c06f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3c0d0650 0x7fcb3c06f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 -- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3c0d0240 msgr2=0x7fcb3c158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3c0d0240 0x7fcb3c158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 -- 172.21.15.43:0/4259657576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c157f70 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb3aa00640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c019410 tx=0x7fcb2c0a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0a8c80 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb2c0a8e20 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c004290 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c156990 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c156f50 con 0x7fcb3c09ba60 2024-09-17T05:33:01.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c15aa20 con 0x7fcb3c09ba60 2024-09-17T05:33:01.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.210+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb2c005070 con 0x7fcb3c09ba60 2024-09-17T05:33:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.214+0000 7fcb33400640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 0x7fcb18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.214+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fcb2c12efe0 con 0x7fcb3c09ba60 2024-09-17T05:33:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.214+0000 7fcb32000640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 0x7fcb18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.214+0000 7fcb32000640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 0x7fcb18075710 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c071130 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:01.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.218+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb2c0ac030 con 0x7fcb3c09ba60 2024-09-17T05:33:01.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.430+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcb3c0b3130 con 0x7fcb3c09ba60 2024-09-17T05:33:01.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.618+0000 7fcb33400640 1 -- 172.21.15.43:0/4259657576 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v274) ==== 162+0+0 (secure 0 0 0) 0x7fcb2c0163f0 con 0x7fcb3c09ba60 2024-09-17T05:33:01.622 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 msgr2=0x7fcb18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 0x7fcb18075710 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c071130 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 msgr2=0x7fcb3c166ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c019410 tx=0x7fcb2c0a8590 comp rx=0 tx=0).stop 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 shutdown_connections 2024-09-17T05:33:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb18073200 0x7fcb18075710 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb3c0d0240 0x7fcb3c158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb3c0d0650 0x7fcb3c06f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 --2- 172.21.15.43:0/4259657576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c09ba60 0x7fcb3c166ca0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 >> 172.21.15.43:0/4259657576 conn(0x7fcb3c0cee60 msgr2=0x7fcb3c159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 shutdown_connections 2024-09-17T05:33:01.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:01.622+0000 7fcb41600640 1 -- 172.21.15.43:0/4259657576 wait complete. 2024-09-17T05:33:02.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:02.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 -- 172.21.15.43:0/1491424573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 msgr2=0x7f2514003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:02.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 --2- 172.21.15.43:0/1491424573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f2514003f60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f252c072880 tx=0x7f252001d280 comp rx=0 tx=0).stop 2024-09-17T05:33:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2531a00640 1 -- 172.21.15.43:0/1491424573 reap_dead start 2024-09-17T05:33:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 -- 172.21.15.43:0/1491424573 shutdown_connections 2024-09-17T05:33:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 -- 172.21.15.43:0/1491424573 >> 172.21.15.43:0/1491424573 conn(0x7f252c0b23d0 msgr2=0x7f252c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 -- 172.21.15.43:0/1491424573 shutdown_connections 2024-09-17T05:33:02.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.206+0000 7f2533a00640 1 -- 172.21.15.43:0/1491424573 wait complete. 2024-09-17T05:33:02.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 Processor -- start 2024-09-17T05:33:02.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 -- start start 2024-09-17T05:33:02.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f252c1fc580 0x7f252c1fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f252c0c5180 0x7f252c0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f252c1667f0 con 0x7f2514003af0 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f252c166670 con 0x7f252c1fc580 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2533a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f252c166970 con 0x7f252c0c5180 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2531000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2531000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34856/0 (socket says 172.21.15.43:34856) 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f2531000640 1 -- 172.21.15.43:0/2552038788 learned_addr learned my addr 172.21.15.43:0/2552038788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.210+0000 7f252be00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f252c1fc580 0x7f252c1fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f252c0c5180 0x7f252c0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 -- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f252c0c5180 msgr2=0x7f252c0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f252c0c5180 0x7f252c0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 -- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f252c1fc580 msgr2=0x7f252c1fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f252c1fc580 0x7f252c1fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 -- 172.21.15.43:0/2552038788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c0c5ad0 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2531000640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f25200ae040 tx=0x7f2520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200bb030 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2520004110 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c1f8f70 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520004980 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c1f9530 con 0x7f2514003af0 2024-09-17T05:33:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c0b33e0 con 0x7f2514003af0 2024-09-17T05:33:02.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.214+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25200b5040 con 0x7f2514003af0 2024-09-17T05:33:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.218+0000 7f2528c00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 0x7f250c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.218+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f2520130660 con 0x7f2514003af0 2024-09-17T05:33:02.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.218+0000 7f252be00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 0x7f250c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.218+0000 7f252be00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 0x7f250c075740 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f252c0b3730 tx=0x7f25180091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.222+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f252c0c5ad0 con 0x7f2514003af0 2024-09-17T05:33:02.350 INFO:tasks.ceph.mds.d.smithi062.stderr: -48> 2024-09-17T05:32:57.595+0000 7f078fa00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-17T05:33:02.351 INFO:tasks.ceph.mds.d.smithi062.stderr: -38> 2024-09-17T05:32:57.595+0000 7f078fa00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-17T05:33:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f252c0ca1c0 con 0x7f2514003af0 2024-09-17T05:33:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2528c00640 1 -- 172.21.15.43:0/2552038788 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+4389 (secure 0 0 0) 0x7f2520016fc0 con 0x7f2514003af0 2024-09-17T05:33:02.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:02.436 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":275,"btime":"2024-09-17T05:33:01:625524+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:01.625504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9579":{"gid":9579,"name":"c","rank":0,"incarnation":275,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/351071204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":351071204},{"type":"v1","addr":"172.21.15.43:6835","nonce":351071204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 275 2024-09-17T05:33:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 msgr2=0x7f250c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 0x7f250c075740 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f252c0b3730 tx=0x7f25180091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 msgr2=0x7f252c0c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f25200ae040 tx=0x7f2520004750 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 shutdown_connections 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f250c073230 0x7f250c075740 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f252c1fc580 0x7f252c1fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f252c0c5180 0x7f252c0c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 --2- 172.21.15.43:0/2552038788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2514003af0 0x7f252c0c9c80 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 >> 172.21.15.43:0/2552038788 conn(0x7f252c0b23d0 msgr2=0x7f252c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 shutdown_connections 2024-09-17T05:33:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:02.434+0000 7f2533a00640 1 -- 172.21.15.43:0/2552038788 wait complete. 2024-09-17T05:33:02.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 275, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:01.625504+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9579': {'gid': 9579, 'name': 'c', 'rank': 0, 'incarnation': 275, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/351071204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 351071204}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 351071204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:33:02.458 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9579:up:replay 2024-09-17T05:33:02.458 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:33:02.471 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:33:07.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- 172.21.15.43:0/2581699630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 msgr2=0x7fda1c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 --2- 172.21.15.43:0/2581699630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0cd580 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fda14009bc0 tx=0x7fda1401cba0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- 172.21.15.43:0/2581699630 shutdown_connections 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 --2- 172.21.15.43:0/2581699630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0cd580 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 --2- 172.21.15.43:0/2581699630 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c0d0240 0x7fda1c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- 172.21.15.43:0/2581699630 >> 172.21.15.43:0/2581699630 conn(0x7fda1c0cee60 msgr2=0x7fda1c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- 172.21.15.43:0/2581699630 shutdown_connections 2024-09-17T05:33:08.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- 172.21.15.43:0/2581699630 wait complete. 2024-09-17T05:33:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 Processor -- start 2024-09-17T05:33:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.034+0000 7fda22e00640 1 -- start start 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c09ba60 0x7fda1c14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda1c0d0240 0x7fda1c0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c0b2b80 con 0x7fda1c0d0650 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c0b2a00 con 0x7fda1c0d0240 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c169e30 con 0x7fda1c09ba60 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54084/0 (socket says 172.21.15.43:54084) 2024-09-17T05:33:08.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 -- 172.21.15.43:0/340319379 learned_addr learned my addr 172.21.15.43:0/340319379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda13e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda1c0d0240 0x7fda1c0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda1be00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c09ba60 0x7fda1c14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 -- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c09ba60 msgr2=0x7fda1c14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c09ba60 0x7fda1c14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 -- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda1c0d0240 msgr2=0x7fda1c0b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda1c0d0240 0x7fda1c0b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 -- 172.21.15.43:0/340319379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c14b460 con 0x7fda1c0d0650 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda20e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fda1401a7d0 tx=0x7fda14003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140bb070 con 0x7fda1c0d0650 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda14004d40 con 0x7fda1c0d0650 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c156210 con 0x7fda1c0d0650 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140b2040 con 0x7fda1c0d0650 2024-09-17T05:33:08.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1543e0 con 0x7fda1c0d0650 2024-09-17T05:33:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c0b3130 con 0x7fda1c0d0650 2024-09-17T05:33:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.038+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda140be070 con 0x7fda1c0d0650 2024-09-17T05:33:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.042+0000 7fda18c00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 0x7fd9f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.042+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fda1412e940 con 0x7fda1c0d0650 2024-09-17T05:33:08.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.042+0000 7fda1be00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 0x7fd9f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.042+0000 7fda1be00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 0x7fd9f8075710 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda0c001870 tx=0x7fda0c001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.046+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda14016790 con 0x7fda1c0d0650 2024-09-17T05:33:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.254+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda1c155fa0 con 0x7fda1c0d0650 2024-09-17T05:33:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.254+0000 7fda18c00640 1 -- 172.21.15.43:0/340319379 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 278 v278) ==== 95+0+5222 (secure 0 0 0) 0x7fda140f87f0 con 0x7fda1c0d0650 2024-09-17T05:33:08.259 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:08.259 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":278,"btime":"2024-09-17T05:33:04:651402+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:04.651394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9579":{"gid":9579,"name":"c","rank":0,"incarnation":275,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/351071204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":351071204},{"type":"v1","addr":"172.21.15.43:6835","nonce":351071204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9579,"qdb_cluster":[9579]},"id":6}]} 2024-09-17T05:33:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 278 2024-09-17T05:33:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 msgr2=0x7fd9f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 0x7fd9f8075710 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda0c001870 tx=0x7fda0c001380 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 msgr2=0x7fda1c0b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fda1401a7d0 tx=0x7fda14003f60 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 shutdown_connections 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd9f8073200 0x7fd9f8075710 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda1c0d0240 0x7fda1c0b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda1c0d0650 0x7fda1c0b6cb0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 --2- 172.21.15.43:0/340319379 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fda1c09ba60 0x7fda1c14af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 >> 172.21.15.43:0/340319379 conn(0x7fda1c0cee60 msgr2=0x7fda1c1565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 shutdown_connections 2024-09-17T05:33:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.258+0000 7fda22e00640 1 -- 172.21.15.43:0/340319379 wait complete. 2024-09-17T05:33:08.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 278, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:04.651394+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9579}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9579': {'gid': 9579, 'name': 'c', 'rank': 0, 'incarnation': 275, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/351071204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 351071204}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 351071204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9579, 'qdb_cluster': [9579]} 2024-09-17T05:33:08.282 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:33:08.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 -- 172.21.15.43:0/612052850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64080b3af0 msgr2=0x7f640815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 --2- 172.21.15.43:0/612052850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64080b3af0 0x7f640815c7e0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009840 tx=0x7f63f001c760 comp rx=0 tx=0).stop 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 -- 172.21.15.43:0/612052850 shutdown_connections 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 --2- 172.21.15.43:0/612052850 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f640806f110 0x7f64080ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 --2- 172.21.15.43:0/612052850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64080b3af0 0x7f640815c7e0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 -- 172.21.15.43:0/612052850 >> 172.21.15.43:0/612052850 conn(0x7f64080b23d0 msgr2=0x7f64080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 -- 172.21.15.43:0/612052850 shutdown_connections 2024-09-17T05:33:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.438+0000 7f640de00640 1 -- 172.21.15.43:0/612052850 wait complete. 2024-09-17T05:33:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 Processor -- start 2024-09-17T05:33:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- start start 2024-09-17T05:33:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f640806f110 0x7f640809bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 0x7f64080ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6408168bc0 con 0x7f640809cc90 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6408168a40 con 0x7f640806f110 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6408168d40 con 0x7f6408155d70 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54108/0 (socket says 172.21.15.43:54108) 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 -- 172.21.15.43:0/743381170 learned_addr learned my addr 172.21.15.43:0/743381170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6406a00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f640806f110 0x7f640809bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f63ffe00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 0x7f64080ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 -- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 msgr2=0x7f64080ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 0x7f64080ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 -- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f640806f110 msgr2=0x7f640809bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f640806f110 0x7f640809bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 -- 172.21.15.43:0/743381170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64080cfe00 con 0x7f640809cc90 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f63ffe00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 0x7f64080ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f6407400640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f640000e970 tx=0x7f640000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f63ff400640 1 -- 172.21.15.43:0/743381170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400004d30 con 0x7f640809cc90 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f63ff400640 1 -- 172.21.15.43:0/743381170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f640000cb20 con 0x7f640809cc90 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64081f6540 con 0x7f640809cc90 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f63ff400640 1 -- 172.21.15.43:0/743381170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400005020 con 0x7f640809cc90 2024-09-17T05:33:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.442+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64081f6b00 con 0x7f640809cc90 2024-09-17T05:33:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f63ff400640 1 -- 172.21.15.43:0/743381170 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64000051c0 con 0x7f640809cc90 2024-09-17T05:33:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f63ff400640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 0x7f63e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f63ff400640 1 -- 172.21.15.43:0/743381170 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f64000955f0 con 0x7f640809cc90 2024-09-17T05:33:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f64154bf000 1 Processor -- start 2024-09-17T05:33:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f6406a00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 0x7f63e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f64154bf000 1 -- start start 2024-09-17T05:33:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.446+0000 7f6406a00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 0x7f63e4075710 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f63f8007fd0 tx=0x7f63f8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f6220b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6221040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a2f60f1410 con 0x55a2f60b4ca0 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a2f602eb40 con 0x55a2f60cb950 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a2f601df70 con 0x55a2f5cdb460 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54124/0 (socket says 172.21.15.43:54124) 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 -- 172.21.15.43:0/3463392845 learned_addr learned my addr 172.21.15.43:0/3463392845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6406a00640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f6220b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f63ffe00640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6221040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 -- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 msgr2=0x55a2f6220b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f6220b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 -- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 msgr2=0x55a2f6221040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6221040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 -- 172.21.15.43:0/3463392845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f5fe55b0 con 0x55a2f60b4ca0 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6406a00640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f6220b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f6407400640 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f640000e940 tx=0x7f640005f030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f064cafb12b47d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f63f7400640 1 -- 172.21.15.43:0/3463392845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400084020 con 0x55a2f60b4ca0 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f63f7400640 1 -- 172.21.15.43:0/3463392845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6400068e10 con 0x55a2f60b4ca0 2024-09-17T05:33:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f63f7400640 1 -- 172.21.15.43:0/3463392845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640001f070 con 0x55a2f60b4ca0 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- 172.21.15.43:0/3463392845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 msgr2=0x55a2f6221580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f640000e940 tx=0x7f640005f030 comp rx=0 tx=0).stop 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- 172.21.15.43:0/3463392845 shutdown_connections 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6221040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f6221580 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 --2- 172.21.15.43:0/3463392845 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f6220b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- 172.21.15.43:0/3463392845 >> 172.21.15.43:0/3463392845 conn(0x55a2f61a1d90 msgr2=0x55a2f61d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- 172.21.15.43:0/3463392845 shutdown_connections 2024-09-17T05:33:08.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- 172.21.15.43:0/3463392845 wait complete. 2024-09-17T05:33:08.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 -- start start 2024-09-17T05:33:08.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.450+0000 7f64154bf000 1 Processor -- start 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60b4ca0 0x55a2f620f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 0x55a2f620fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a2f5fb85f0 con 0x55a2f5cdb460 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6021d80 con 0x55a2f60b4ca0 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a2f5fc4520 con 0x55a2f60cb950 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 10 client.0 _ll_register_callbacks cb 0x55a2f5fc8830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54126/0 (socket says 172.21.15.43:54126) 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 -- 172.21.15.43:0/2625383014 learned_addr learned my addr 172.21.15.43:0/2625383014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60b4ca0 0x55a2f620f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6407400640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 0x55a2f620fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 -- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 msgr2=0x55a2f620fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 0x55a2f620fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 -- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60b4ca0 msgr2=0x55a2f620f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60b4ca0 0x55a2f620f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 -- 172.21.15.43:0/2625383014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f5fbc190 con 0x55a2f5cdb460 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6407400640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 0x55a2f620fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f6406a00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f63f8014780 tx=0x7f63f8001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb10d8299196902 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8014e50 con 0x55a2f5cdb460 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2f5fb6110 con 0x55a2f5cdb460 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f63f8013930 con 0x55a2f5cdb460 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=278) ==== 24+0+0 (secure 0 0 0) 0x7f63f8001470 con 0x55a2f5cdb460 2024-09-17T05:33:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 10 client.9660 fetch_fsmap learned FSMap version 278 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=278,monmap=2+}) -- 0x55a2f60a2f10 con 0x55a2f5cdb460 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a2f5fb6110 con 0x55a2f5cdb460 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 278) ==== 3502+0+0 (secure 0 0 0) 0x7f63f8020070 con 0x55a2f5cdb460 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 10 client.9660 fetch_fsmap finished waiting for FSMap version 278 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 10 client.9660 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 20 client.9660 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cc07d0 0x55a2f6210560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 4 client.9660 mds_command: new command op to 9579 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 --> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f6030a90 con 0x55a2f5cc07d0 2024-09-17T05:33:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.454+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cc07d0 0x55a2f6210560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cc07d0 0x55a2f6210560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20db720d4acc43c2 server_cookie=8a899cd5f70b3182 in_seq=0 out_seq=0 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f63ebe00640 10 client.9660 ms_handle_connect on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f63ebe00640 1 -- 172.21.15.43:0/2625383014 <== mds.0 v2:172.21.15.43:6833/351071204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2f6030a90 con 0x55a2f5cc07d0 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f63ebe00640 10 client.9660 handle_command_reply: tid=0 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 client.9660 shutdown 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 msgr2=0x55a2f620f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f63f8014780 tx=0x7f63f8001090 comp rx=0 tx=0).stop 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 shutdown_connections 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cc07d0 0x55a2f6210560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60b4ca0 0x55a2f620f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60cb950 0x55a2f620fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 --2- 172.21.15.43:0/2625383014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f620f0d0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f63ebe00640 0 client.9660 ms_handle_reset on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:08.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 >> 172.21.15.43:0/2625383014 conn(0x55a2f61a1d90 msgr2=0x55a2f61d9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 shutdown_connections 2024-09-17T05:33:08.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 1 -- 172.21.15.43:0/2625383014 wait complete. 2024-09-17T05:33:08.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.458+0000 7f64154bf000 20 client.9660 trim_cache size 0 max 16384 2024-09-17T05:33:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 Processor -- start 2024-09-17T05:33:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 -- start start 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f61de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6237110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6408202880 con 0x55a2f60b4ca0 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6408201880 con 0x55a2f60cb950 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6408200c70 con 0x55a2f5cdb460 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54140/0 (socket says 172.21.15.43:54140) 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 -- 172.21.15.43:0/1862252491 learned_addr learned my addr 172.21.15.43:0/1862252491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f6407400640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6237110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f6406a00640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f61de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 -- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 msgr2=0x55a2f61de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f61de190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 -- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 msgr2=0x55a2f6237110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6237110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 -- 172.21.15.43:0/1862252491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f6079db0 con 0x55a2f60b4ca0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f6406a00640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f61de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f6407400640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6237110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63ffe00640 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f63f001ae10 tx=0x7f63f00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c616ca4eec204384 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63f6a00640 1 -- 172.21.15.43:0/1862252491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b2040 con 0x55a2f60b4ca0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63f6a00640 1 -- 172.21.15.43:0/1862252491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f00a5910 con 0x55a2f60b4ca0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.478+0000 7f63f6a00640 1 -- 172.21.15.43:0/1862252491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00a5cc0 con 0x55a2f60b4ca0 2024-09-17T05:33:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- 172.21.15.43:0/1862252491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 msgr2=0x55a2f61de6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f63f001ae10 tx=0x7f63f00091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- 172.21.15.43:0/1862252491 shutdown_connections 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6237110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f60b4ca0 0x55a2f61de6d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- 172.21.15.43:0/1862252491 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f5cdb460 0x55a2f61de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- 172.21.15.43:0/1862252491 >> 172.21.15.43:0/1862252491 conn(0x55a2f61a1d90 msgr2=0x55a2f61dda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- 172.21.15.43:0/1862252491 shutdown_connections 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- 172.21.15.43:0/1862252491 wait complete. 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- start start 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 Processor -- start 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 0x55a2f6296c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6297150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a2f5fe9ee0 con 0x55a2f5cdb460 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6408201230 con 0x55a2f60cb950 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55a2f5fd71e0 con 0x55a2f60b4ca0 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f64154bf000 10 client.0 _ll_register_callbacks cb 0x7f63f8017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54146/0 (socket says 172.21.15.43:54146) 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 -- 172.21.15.43:0/2440568747 learned_addr learned my addr 172.21.15.43:0/2440568747 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6407400640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6297150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 0x55a2f6296c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 -- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 msgr2=0x55a2f6296c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 0x55a2f6296c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 -- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 msgr2=0x55a2f6297150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6297150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 -- 172.21.15.43:0/2440568747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f5fe4ab0 con 0x55a2f5cdb460 2024-09-17T05:33:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 0x55a2f6296c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f6406a00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f63f800fc40 tx=0x7f63f8013aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e431d711777f801 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.482+0000 7f63ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8038070 con 0x55a2f5cdb460 2024-09-17T05:33:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f63f8021890 con 0x55a2f5cdb460 2024-09-17T05:33:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f00a5910 con 0x55a2f5cdb460 2024-09-17T05:33:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=278) ==== 24+0+0 (secure 0 0 0) 0x7f63f8044050 con 0x55a2f5cdb460 2024-09-17T05:33:08.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 10 client.9666 fetch_fsmap learned FSMap version 278 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=278,monmap=2+}) -- 0x55a2f5fc3070 con 0x55a2f5cdb460 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f00a5910 con 0x55a2f5cdb460 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 278) ==== 3502+0+0 (secure 0 0 0) 0x7f63f8020070 con 0x55a2f5cdb460 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 10 client.9666 fetch_fsmap finished waiting for FSMap version 278 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 10 client.9666 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 20 client.9666 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cae830 0x55a2f6233cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 4 client.9666 mds_command: new command op to 9579 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 --> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] -- command(tid 0: {"prefix": "status"}) -- 0x55a2f602c540 con 0x55a2f5cae830 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cae830 0x55a2f6233cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ffe00640 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cae830 0x55a2f6233cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fb49023f0224f4 server_cookie=bc3097cdc3bd19c6 in_seq=0 out_seq=0 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 10 client.9666 ms_handle_connect on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 1 -- 172.21.15.43:0/2440568747 <== mds.0 v2:172.21.15.43:6833/351071204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55a2f602c540 con 0x55a2f5cae830 2024-09-17T05:33:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 10 client.9666 handle_command_reply: tid=0 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 client.9666 shutdown 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 msgr2=0x55a2f62966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f63f800fc40 tx=0x7f63f8013aa0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 shutdown_connections 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x55a2f5cae830 0x55a2f6233cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f60cb950 0x55a2f6297150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55a2f60b4ca0 0x55a2f6296c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 --2- 172.21.15.43:0/2440568747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a2f5cdb460 0x55a2f62966d0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f63ebe00640 0 client.9666 ms_handle_reset on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 >> 172.21.15.43:0/2440568747 conn(0x55a2f61a1d90 msgr2=0x55a2f61de440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 shutdown_connections 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 1 -- 172.21.15.43:0/2440568747 wait complete. 2024-09-17T05:33:08.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f64154bf000 20 client.9666 trim_cache size 0 max 16384 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9579, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8583069759999997, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 278, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 120, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 120, 2024-09-17T05:33:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.063135595 2024-09-17T05:33:08.497 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:33:08.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 msgr2=0x7f63e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 0x7f63e4075710 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f63f8007fd0 tx=0x7f63f8009040 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 msgr2=0x7f64080aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f640000e970 tx=0x7f640000ee30 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 shutdown_connections 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f63e4073200 0x7f63e4075710 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640809cc90 0x7f64080aeb00 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6408155d70 0x7f64080ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 --2- 172.21.15.43:0/743381170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f640806f110 0x7f640809bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 >> 172.21.15.43:0/743381170 conn(0x7f64080b23d0 msgr2=0x7f640806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 shutdown_connections 2024-09-17T05:33:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.486+0000 7f640de00640 1 -- 172.21.15.43:0/743381170 wait complete. 2024-09-17T05:33:08.510 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:33:08.510 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-09-17T05:33:08.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:33:08.613 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:08.610+0000 7fcfb7400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 -- 172.21.15.43:0/2458308530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d2fd0 msgr2=0x7fc06c0d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 --2- 172.21.15.43:0/2458308530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d2fd0 0x7fc06c0d33e0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc064009bc0 tx=0x7fc06401cac0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 -- 172.21.15.43:0/2458308530 shutdown_connections 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 --2- 172.21.15.43:0/2458308530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d2fd0 0x7fc06c0d33e0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 --2- 172.21.15.43:0/2458308530 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c15f730 0x7fc06c0d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 -- 172.21.15.43:0/2458308530 >> 172.21.15.43:0/2458308530 conn(0x7fc06c06de90 msgr2=0x7fc06c06e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 -- 172.21.15.43:0/2458308530 shutdown_connections 2024-09-17T05:33:08.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.666+0000 7fc06be00640 1 -- 172.21.15.43:0/2458308530 wait complete. 2024-09-17T05:33:08.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 Processor -- start 2024-09-17T05:33:08.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 -- start start 2024-09-17T05:33:08.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c0d2fd0 0x7fc06c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc06c15f730 0x7fc06c1f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c16bed0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c0b3b90 con 0x7fc06c15f730 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c16c050 con 0x7fc06c0d2fd0 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54170/0 (socket says 172.21.15.43:54170) 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06aa00640 1 -- 172.21.15.43:0/2908203677 learned_addr learned my addr 172.21.15.43:0/2908203677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc062000640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c0d2fd0 0x7fc06c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.670+0000 7fc06b400640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc06c15f730 0x7fc06c1f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 -- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c0d2fd0 msgr2=0x7fc06c14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c0d2fd0 0x7fc06c14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 -- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc06c15f730 msgr2=0x7fc06c1f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc06c15f730 0x7fc06c1f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 -- 172.21.15.43:0/2908203677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1f9310 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06aa00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00c4d0 tx=0x7fc05c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c017070 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc05c003600 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c003a50 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c14c930 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c14cef0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06c0d33e0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.674+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc05c01a070 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.678+0000 7fc063400640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 0x7fc048075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:08.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.678+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fc05c0940e0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.678+0000 7fc062000640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 0x7fc048075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:08.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.678+0000 7fc062000640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 0x7fc048075710 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0c9010 tx=0x7fc0540091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:08.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.682+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc05c05df90 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc06c06a1e0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc063400640 1 -- 172.21.15.43:0/2908203677 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v278) ==== 53+0+73 (secure 0 0 0) 0x7fc05c0629c0 con 0x7fc06c0d0ff0 2024-09-17T05:33:08.900 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:33:08.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 msgr2=0x7fc048075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 0x7fc048075710 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0c9010 tx=0x7fc0540091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 msgr2=0x7fc06c14c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:08.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00c4d0 tx=0x7fc05c00c990 comp rx=0 tx=0).stop 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 shutdown_connections 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc048073200 0x7fc048075710 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc06c15f730 0x7fc06c1f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc06c0d2fd0 0x7fc06c14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 --2- 172.21.15.43:0/2908203677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c14c180 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 >> 172.21.15.43:0/2908203677 conn(0x7fc06c06de90 msgr2=0x7fc06c06e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 shutdown_connections 2024-09-17T05:33:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:08.898+0000 7fc06be00640 1 -- 172.21.15.43:0/2908203677 wait complete. 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:33:08.922 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:08.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:08.923 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:33:08.931 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.932 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41855sec preferred_lft 41855sec 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.933 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:33:08.935 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:33:08.935 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:08.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:08.936 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:33:09.001 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:33:09.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:09.003 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:33:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:33:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:33:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:33:09.057 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:33:09.057 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:33:09.057 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:33:09.057 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:33:09.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:33:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:09.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:09.196 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:33:09.206 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:33:09.206 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:33:09.206 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:09.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:09.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:33:09.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:09.267 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:09.319 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:09.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:09.319 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:09.370 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:33:09.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:09.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:09.472 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:09.472 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:09.472 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:09.490 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:09.490 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:09.491 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:09.491 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:09.491 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:33:09.501 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:33:09.505 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:33:09.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:09.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:09.524 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:33:09.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:33:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:09.624 INFO:teuthology.orchestra.run.smithi043.stdout:client8171 2024-09-17T05:33:09.624 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:09.624 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:09.634 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:33:09.634 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:33:09.634 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:33:09.634 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-09-17T05:33:09.635 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:33:09.678 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:33:09.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-17T05:33:09.757 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-17T05:33:09.757 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-17T05:33:09.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 -- 172.21.15.43:0/4133947877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 msgr2=0x7f2f3c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 --2- 172.21.15.43:0/4133947877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c15c7e0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009840 tx=0x7f2f2c01c760 comp rx=0 tx=0).stop 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 -- 172.21.15.43:0/4133947877 shutdown_connections 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 --2- 172.21.15.43:0/4133947877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 0x7f2f3c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 --2- 172.21.15.43:0/4133947877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c15c7e0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 -- 172.21.15.43:0/4133947877 >> 172.21.15.43:0/4133947877 conn(0x7f2f3c0b23d0 msgr2=0x7f2f3c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 -- 172.21.15.43:0/4133947877 shutdown_connections 2024-09-17T05:33:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.954+0000 7f2f41200640 1 -- 172.21.15.43:0/4133947877 wait complete. 2024-09-17T05:33:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 Processor -- start 2024-09-17T05:33:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- start start 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 0x7f2f3c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f3c0b3920 0x7f2f3c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c06d5e0 con 0x7f2f3c155d70 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c06d460 con 0x7f2f3c0b3920 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c06d760 con 0x7f2f3c06f110 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54182/0 (socket says 172.21.15.43:54182) 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 -- 172.21.15.43:0/3110358275 learned_addr learned my addr 172.21.15.43:0/3110358275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f31400640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f3c0b3920 0x7f2f3c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f39e00640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 0x7f2f3c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 -- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 msgr2=0x7f2f3c0740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 0x7f2f3c0740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 -- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f3c0b3920 msgr2=0x7f2f3c074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f3c0b3920 0x7f2f3c074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 -- 172.21.15.43:0/3110358275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c0cfdd0 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f3a800640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400cad0 tx=0x7f2f3400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34016020 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f3400ab40 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c0d00b0 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400ce10 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.958+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c0ce3c0 con 0x7f2f3c155d70 2024-09-17T05:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c0cfb40 con 0x7f2f3c155d70 2024-09-17T05:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2f34019020 con 0x7f2f3c155d70 2024-09-17T05:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f32a00640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 0x7f2f18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f2f34094bd0 con 0x7f2f3c155d70 2024-09-17T05:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f39e00640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 0x7f2f18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.962+0000 7f2f39e00640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 0x7f2f18075710 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0af000 tx=0x7f2f2c01afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:09.966+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2f34020030 con 0x7f2f3c155d70 2024-09-17T05:33:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.178+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f3c0d1150 con 0x7f2f3c155d70 2024-09-17T05:33:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f32a00640 1 -- 172.21.15.43:0/3110358275 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 278 v278) ==== 95+0+5222 (secure 0 0 0) 0x7f2f3c0d1150 con 0x7f2f3c155d70 2024-09-17T05:33:10.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:10.183 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":278,"btime":"2024-09-17T05:33:04:651402+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:04.651394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9579},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9579":{"gid":9579,"name":"c","rank":0,"incarnation":275,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/351071204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":351071204},{"type":"v1","addr":"172.21.15.43:6835","nonce":351071204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9579,"qdb_cluster":[9579]},"id":6}]} 2024-09-17T05:33:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 278 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 msgr2=0x7f2f18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 0x7f2f18075710 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0af000 tx=0x7f2f2c01afa0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 msgr2=0x7f2f3c0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400cad0 tx=0x7f2f3400cbc0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 shutdown_connections 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f18073200 0x7f2f18075710 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f3c0b3920 0x7f2f3c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c155d70 0x7f2f3c0ceb10 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 --2- 172.21.15.43:0/3110358275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f3c06f110 0x7f2f3c0740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 >> 172.21.15.43:0/3110358275 conn(0x7f2f3c0b23d0 msgr2=0x7f2f3c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 shutdown_connections 2024-09-17T05:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.182+0000 7f2f41200640 1 -- 172.21.15.43:0/3110358275 wait complete. 2024-09-17T05:33:10.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.362+0000 7fadb4800640 1 -- 172.21.15.43:0/365138103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94003af0 msgr2=0x7fad94003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.362+0000 7fadb4800640 1 --2- 172.21.15.43:0/365138103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94003af0 0x7fad94003f20 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fada0001770 tx=0x7fada001c840 comp rx=0 tx=0).stop 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 -- 172.21.15.43:0/365138103 shutdown_connections 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 --2- 172.21.15.43:0/365138103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94003af0 0x7fad94003f20 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 -- 172.21.15.43:0/365138103 >> 172.21.15.43:0/365138103 conn(0x7fadac0b23d0 msgr2=0x7fadac0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 -- 172.21.15.43:0/365138103 shutdown_connections 2024-09-17T05:33:10.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 -- 172.21.15.43:0/365138103 wait complete. 2024-09-17T05:33:10.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 Processor -- start 2024-09-17T05:33:10.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.366+0000 7fadb4800640 1 -- start start 2024-09-17T05:33:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad94003af0 0x7fadac1fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 0x7fadac1eac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0ce680 con 0x7fadac091d90 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0ce500 con 0x7fad94003af0 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0ce800 con 0x7fadac1ea850 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54194/0 (socket says 172.21.15.43:54194) 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 -- 172.21.15.43:0/825758399 learned_addr learned my addr 172.21.15.43:0/825758399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1e00640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad94003af0 0x7fadac1fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb2800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 0x7fadac1eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 -- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 msgr2=0x7fadac1eac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 0x7fadac1eac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 -- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad94003af0 msgr2=0x7fadac1fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad94003af0 0x7fadac1fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 -- 172.21.15.43:0/825758399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1fd7f0 con 0x7fadac091d90 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb2800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 0x7fadac1eac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb1400640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00fd70 tx=0x7fad9c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fada6000640 1 -- 172.21.15.43:0/825758399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00fe60 con 0x7fadac091d90 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fada6000640 1 -- 172.21.15.43:0/825758399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c00c8d0 con 0x7fadac091d90 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fada6000640 1 -- 172.21.15.43:0/825758399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00cce0 con 0x7fadac091d90 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac1eb1a0 con 0x7fadac091d90 2024-09-17T05:33:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.370+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac0912b0 con 0x7fadac091d90 2024-09-17T05:33:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fada6000640 1 -- 172.21.15.43:0/825758399 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad9c017420 con 0x7fadac091d90 2024-09-17T05:33:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fada6000640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 0x7fad90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fada6000640 1 -- 172.21.15.43:0/825758399 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fad9c095b00 con 0x7fadac091d90 2024-09-17T05:33:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 Processor -- start 2024-09-17T05:33:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 -- start start 2024-09-17T05:33:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadb1e00640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 0x7fad90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadb1e00640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 0x7fad90075710 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fada00a9000 tx=0x7fada00195e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ee4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ee4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560500d82d00 con 0x560500d9eb60 2024-09-17T05:33:10.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560500cf6c90 con 0x560500d7dd90 2024-09-17T05:33:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560500ce60c0 con 0x560500d941c0 2024-09-17T05:33:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54208/0 (socket says 172.21.15.43:54208) 2024-09-17T05:33:10.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.374+0000 7fadb1e00640 1 -- 172.21.15.43:0/2096388328 learned_addr learned my addr 172.21.15.43:0/2096388328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb2800640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ee4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1400640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ee4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 -- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 msgr2=0x560500ee4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ee4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 -- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 msgr2=0x560500ee4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ee4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 -- 172.21.15.43:0/2096388328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560500cc1290 con 0x560500d9eb60 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb2800640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ee4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fada0098110 tx=0x7fada00b7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00138610036b0c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fad9a600640 1 -- 172.21.15.43:0/2096388328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0003cd0 con 0x560500d9eb60 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fad9a600640 1 -- 172.21.15.43:0/2096388328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada00c4070 con 0x560500d9eb60 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fad9a600640 1 -- 172.21.15.43:0/2096388328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00a8680 con 0x560500d9eb60 2024-09-17T05:33:10.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- 172.21.15.43:0/2096388328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 msgr2=0x560500ee42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fada0098110 tx=0x7fada00b7040 comp rx=0 tx=0).stop 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- 172.21.15.43:0/2096388328 shutdown_connections 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ee4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ee4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 --2- 172.21.15.43:0/2096388328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d9eb60 0x560500ee42e0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- 172.21.15.43:0/2096388328 >> 172.21.15.43:0/2096388328 conn(0x5605009a3760 msgr2=0x560500ec7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- 172.21.15.43:0/2096388328 shutdown_connections 2024-09-17T05:33:10.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- 172.21.15.43:0/2096388328 wait complete. 2024-09-17T05:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 -- start start 2024-09-17T05:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.378+0000 7fadbc040000 1 Processor -- start 2024-09-17T05:33:10.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ea00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500ec68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560500c80940 con 0x560500d941c0 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560500ce9ed0 con 0x560500d7dd90 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5605009ab820 con 0x560500d9eb60 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 10 client.0 _ll_register_callbacks cb 0x560500cc47f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54214/0 (socket says 172.21.15.43:54214) 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 -- 172.21.15.43:0/3726119808 learned_addr learned my addr 172.21.15.43:0/3726119808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1e00640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ea00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb2800640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500ec68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 -- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 msgr2=0x560500ec68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500ec68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 -- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 msgr2=0x560500ea00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ea00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 -- 172.21.15.43:0/3726119808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560500c99550 con 0x560500d941c0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb2800640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500ec68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1400640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0058d0 tx=0x7fad9c060120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42ff35fff6a35f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c05b650 con 0x560500d941c0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fad9c05e7d0 con 0x560500d941c0 2024-09-17T05:33:10.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560500c7e400 con 0x560500d941c0 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=278) ==== 24+0+0 (secure 0 0 0) 0x7fad9c064310 con 0x560500d941c0 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 10 client.9696 fetch_fsmap learned FSMap version 278 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=278,monmap=2+}) -- 0x560500c84480 con 0x560500d941c0 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560500c7e400 con 0x560500d941c0 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 278) ==== 3502+0+0 (secure 0 0 0) 0x7fad9c00b040 con 0x560500d941c0 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 10 client.9696 fetch_fsmap finished waiting for FSMap version 278 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 10 client.9696 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 20 client.9696 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500988ad0 0x560500ef58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 4 client.9696 mds_command: new command op to 9579 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:33:10.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 --> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560500cf8be0 con 0x560500988ad0 2024-09-17T05:33:10.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1e00640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500988ad0 0x560500ef58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fadb1e00640 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500988ad0 0x560500ef58a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381d2352a1806bda server_cookie=41fe40a8ae96347f in_seq=0 out_seq=0 2024-09-17T05:33:10.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.382+0000 7fad8ea00640 10 client.9696 ms_handle_connect on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:10.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fad8ea00640 1 -- 172.21.15.43:0/3726119808 <== mds.0 v2:172.21.15.43:6833/351071204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560500cf8be0 con 0x560500988ad0 2024-09-17T05:33:10.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fad8ea00640 10 client.9696 handle_command_reply: tid=0 2024-09-17T05:33:10.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 client.9696 shutdown 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 msgr2=0x560500ea0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0058d0 tx=0x7fad9c060120 comp rx=0 tx=0).stop 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 shutdown_connections 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500988ad0 0x560500ef58a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500ec68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500ea0610 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 --2- 172.21.15.43:0/3726119808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500ea00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fad8ea00640 0 client.9696 ms_handle_reset on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 >> 172.21.15.43:0/3726119808 conn(0x5605009a3760 msgr2=0x560500e98680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 shutdown_connections 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 1 -- 172.21.15.43:0/3726119808 wait complete. 2024-09-17T05:33:10.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.386+0000 7fadbc040000 20 client.9696 trim_cache size 0 max 16384 2024-09-17T05:33:10.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.398+0000 7fadbc040000 1 Processor -- start 2024-09-17T05:33:10.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.398+0000 7fadbc040000 1 -- start start 2024-09-17T05:33:10.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500e9a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500e9b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560500ce91e0 con 0x560500d941c0 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560500cf5a60 con 0x560500d7dd90 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560500cb2300 con 0x560500d9eb60 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54218/0 (socket says 172.21.15.43:54218) 2024-09-17T05:33:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 -- 172.21.15.43:0/2134322767 learned_addr learned my addr 172.21.15.43:0/2134322767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500e9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb2800640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500e9b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 -- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 msgr2=0x560500e9b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500e9b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 -- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 msgr2=0x560500e9a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500e9a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 -- 172.21.15.43:0/2134322767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560500d6bd10 con 0x560500d941c0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb2800640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500e9b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500e9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadb1400640 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fad9c05f5c0 tx=0x7fad9c00de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db124bc9301575f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fad99c00640 1 -- 172.21.15.43:0/2134322767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c07e070 con 0x560500d941c0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fad99c00640 1 -- 172.21.15.43:0/2134322767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c079e50 con 0x560500d941c0 2024-09-17T05:33:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fad99c00640 1 -- 172.21.15.43:0/2134322767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c064580 con 0x560500d941c0 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- 172.21.15.43:0/2134322767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 msgr2=0x560500e9abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fad9c05f5c0 tx=0x7fad9c00de00 comp rx=0 tx=0).stop 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- 172.21.15.43:0/2134322767 shutdown_connections 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d9eb60 0x560500e9b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d941c0 0x560500e9abd0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 --2- 172.21.15.43:0/2134322767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d7dd90 0x560500e9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- 172.21.15.43:0/2134322767 >> 172.21.15.43:0/2134322767 conn(0x5605009a3760 msgr2=0x560500ec64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- 172.21.15.43:0/2134322767 shutdown_connections 2024-09-17T05:33:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.402+0000 7fadbc040000 1 -- 172.21.15.43:0/2134322767 wait complete. 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- start start 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 Processor -- start 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ef8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d9eb60 0x560500f15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560500cee500 con 0x560500d7dd90 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560500c91300 con 0x560500d9eb60 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560500ca3670 con 0x560500d941c0 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 10 client.0 _ll_register_callbacks cb 0x7fad9c00ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54220/0 (socket says 172.21.15.43:54220) 2024-09-17T05:33:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 -- 172.21.15.43:0/2853959318 learned_addr learned my addr 172.21.15.43:0/2853959318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1400640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ef8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb2800640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d9eb60 0x560500f15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 -- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 msgr2=0x560500ef8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ef8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 -- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d9eb60 msgr2=0x560500f15dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d9eb60 0x560500f15dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 -- 172.21.15.43:0/2853959318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560500c9ce50 con 0x560500d7dd90 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadb1e00640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fada00011b0 tx=0x7fada0001390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62fd903261bf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00d5020 con 0x560500d7dd90 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fada00a8740 con 0x560500d7dd90 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c079e50 con 0x560500d7dd90 2024-09-17T05:33:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=278) ==== 24+0+0 (secure 0 0 0) 0x7fada00a8940 con 0x560500d7dd90 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 10 client.9702 fetch_fsmap learned FSMap version 278 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=278,monmap=2+}) -- 0x560500c8b2d0 con 0x560500d7dd90 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad9c079e50 con 0x560500d7dd90 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 278) ==== 3502+0+0 (secure 0 0 0) 0x7fada00bc030 con 0x560500d7dd90 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 10 client.9702 fetch_fsmap finished waiting for FSMap version 278 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 10 client.9702 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 20 client.9702 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500976b30 0x560500f16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 4 client.9702 mds_command: new command op to 9579 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.406+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 --> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560500cf4690 con 0x560500976b30 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb1400640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500976b30 0x560500f16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb1400640 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500976b30 0x560500f16750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a7ce3215078725 server_cookie=3bd46b767e65746c in_seq=0 out_seq=0 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fad8ea00640 10 client.9702 ms_handle_connect on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fad8ea00640 1 -- 172.21.15.43:0/2853959318 <== mds.0 v2:172.21.15.43:6833/351071204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+215 (crc 0 0 0) 0x560500cf4690 con 0x560500976b30 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fad8ea00640 10 client.9702 handle_command_reply: tid=0 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 client.9702 shutdown 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 msgr2=0x560500ef7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fada00011b0 tx=0x7fada0001390 comp rx=0 tx=0).stop 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 shutdown_connections 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:6833/351071204,v1:172.21.15.43:6835/351071204] conn(0x560500976b30 0x560500f16750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560500d9eb60 0x560500f15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560500d941c0 0x560500ef8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 --2- 172.21.15.43:0/2853959318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560500d7dd90 0x560500ef7ed0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fad8ea00640 0 client.9702 ms_handle_reset on v2:172.21.15.43:6833/351071204 2024-09-17T05:33:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 >> 172.21.15.43:0/2853959318 conn(0x5605009a3760 msgr2=0x560500ef0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 shutdown_connections 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 1 -- 172.21.15.43:0/2853959318 wait complete. 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadbc040000 20 client.9702 trim_cache size 0 max 16384 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "dentry", 2024-09-17T05:33:10.417 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 3154504483, 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511627776, 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: "frag": "*", 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dname": "sixmegs", 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_id": "head", 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir/sixmegs" 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:33:10.418 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 msgr2=0x7fad90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 0x7fad90075710 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fada00a9000 tx=0x7fada00195e0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 msgr2=0x7fadac1581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00fd70 tx=0x7fad9c00d7b0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 shutdown_connections 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad90073200 0x7fad90075710 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fadac1ea850 0x7fadac1eac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadac091d90 0x7fadac1581e0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 --2- 172.21.15.43:0/825758399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad94003af0 0x7fadac1fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 >> 172.21.15.43:0/825758399 conn(0x7fadac0b23d0 msgr2=0x7fadac06f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 shutdown_connections 2024-09-17T05:33:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.410+0000 7fadb4800640 1 -- 172.21.15.43:0/825758399 wait complete. 2024-09-17T05:33:10.433 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-09-17T05:33:10.433 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:10.433 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:10.442 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:10.442 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:10.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.690+0000 7fcdeea00640 1 -- 172.21.15.43:0/3072250086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc005920 msgr2=0x7fcdcc005df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.690+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3072250086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc005920 0x7fcdcc005df0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fcde80afa20 tx=0x7fcdd001d2d0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.690+0000 7fcdeca00640 1 -- 172.21.15.43:0/3072250086 reap_dead start 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 -- 172.21.15.43:0/3072250086 shutdown_connections 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 -- 172.21.15.43:0/3072250086 >> 172.21.15.43:0/3072250086 conn(0x7fcde80b23d0 msgr2=0x7fcde80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 -- 172.21.15.43:0/3072250086 shutdown_connections 2024-09-17T05:33:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 -- 172.21.15.43:0/3072250086 wait complete. 2024-09-17T05:33:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 Processor -- start 2024-09-17T05:33:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 -- start start 2024-09-17T05:33:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdcc005920 0x7fcde80c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.694+0000 7fcdeea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde808c520 0x7fcde8158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80ce660 con 0x7fcde809dcf0 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80ce4e0 con 0x7fcdcc005920 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcde80ce7e0 con 0x7fcde808c520 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54228/0 (socket says 172.21.15.43:54228) 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 -- 172.21.15.43:0/3354619878 learned_addr learned my addr 172.21.15.43:0/3354619878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeca00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde808c520 0x7fcde8158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcde7e00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdcc005920 0x7fcde80c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 -- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde808c520 msgr2=0x7fcde8158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde808c520 0x7fcde8158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 -- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdcc005920 msgr2=0x7fcde80c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdcc005920 0x7fcde80c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 -- 172.21.15.43:0/3354619878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80c7350 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcddf400640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fcdd001d100 tx=0x7fcdd00adeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00b1040 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd00ac400 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0005240 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde80c98a0 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde8158de0 con 0x7fcde809dcf0 2024-09-17T05:33:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde80ced70 con 0x7fcde809dcf0 2024-09-17T05:33:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.698+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcdd00c0020 con 0x7fcde809dcf0 2024-09-17T05:33:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.702+0000 7fcde4c00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 0x7fcdc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.702+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7fcdd012ffe0 con 0x7fcde809dcf0 2024-09-17T05:33:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.702+0000 7fcde7e00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 0x7fcdc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.702+0000 7fcde7e00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 0x7fcdc4075710 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8006fd0 tx=0x7fcdd8006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:10.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.706+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcdd0016fe0 con 0x7fcde809dcf0 2024-09-17T05:33:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcde808cdf0 con 0x7fcde809dcf0 2024-09-17T05:33:10.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcde4c00640 1 -- 172.21.15.43:0/3354619878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v120) ==== 81+0+3770 (secure 0 0 0) 0x7fcdd00fa540 con 0x7fcde809dcf0 2024-09-17T05:33:10.900 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:10.901 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:10.903 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:10.904 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:10.905 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:10.906 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:listed 62 entries 2024-09-17T05:33:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 msgr2=0x7fcdc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 0x7fcdc4075710 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8006fd0 tx=0x7fcdd8006b20 comp rx=0 tx=0).stop 2024-09-17T05:33:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 msgr2=0x7fcde808bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fcdd001d100 tx=0x7fcdd00adeb0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 shutdown_connections 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcdc4073200 0x7fcdc4075710 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcde808c520 0x7fcde8158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde809dcf0 0x7fcde808bfe0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 --2- 172.21.15.43:0/3354619878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdcc005920 0x7fcde80c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 >> 172.21.15.43:0/3354619878 conn(0x7fcde80b23d0 msgr2=0x7fcde8160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 shutdown_connections 2024-09-17T05:33:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:10.898+0000 7fcdeea00640 1 -- 172.21.15.43:0/3354619878 wait complete. 2024-09-17T05:33:10.922 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:33:10.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:10.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:33:10.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:33:10.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:10.947 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:10.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 -- 172.21.15.43:0/3982437786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 msgr2=0x7f00540b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 --2- 172.21.15.43:0/3982437786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f00540b4930 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f00540d3360 tx=0x7f004c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 -- 172.21.15.43:0/3982437786 shutdown_connections 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 --2- 172.21.15.43:0/3982437786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f00540b4930 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 --2- 172.21.15.43:0/3982437786 >> v2:172.21.15.62:3300/0 conn(0x7f00540cccb0 0x7f00540b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 --2- 172.21.15.43:0/3982437786 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005415cea0 0x7f00540d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 -- 172.21.15.43:0/3982437786 >> 172.21.15.43:0/3982437786 conn(0x7f00540691b0 msgr2=0x7f00540695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 -- 172.21.15.43:0/3982437786 shutdown_connections 2024-09-17T05:33:11.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.198+0000 7f0053200640 1 -- 172.21.15.43:0/3982437786 wait complete. 2024-09-17T05:33:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 Processor -- start 2024-09-17T05:33:11.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 -- start start 2024-09-17T05:33:11.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540cccb0 0x7f00541654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005415cea0 0x7f00541fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00541531b0 con 0x7f00540cd0c0 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0054153030 con 0x7f00540cccb0 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f0053200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0054153330 con 0x7f005415cea0 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54248/0 (socket says 172.21.15.43:54248) 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 -- 172.21.15.43:0/3615685790 learned_addr learned my addr 172.21.15.43:0/3615685790 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 -- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005415cea0 msgr2=0x7f00541fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005415cea0 0x7f00541fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 -- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540cccb0 msgr2=0x7f00541654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540cccb0 0x7f00541654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 -- 172.21.15.43:0/3615685790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00541519b0 con 0x7f00540cd0c0 2024-09-17T05:33:11.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.202+0000 7f004be00640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c940 tx=0x7f003c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c007bb0 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f003c007e40 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c00f6b0 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0054151c90 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0054152250 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0054151420 con 0x7f00540cd0c0 2024-09-17T05:33:11.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f003c00f850 con 0x7f00540cd0c0 2024-09-17T05:33:11.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 0x7f0030075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8375+0+0 (secure 0 0 0) 0x7f003c0954a0 con 0x7f00540cd0c0 2024-09-17T05:33:11.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.206+0000 7f0051e00640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 0x7f0030075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.210+0000 7f0051e00640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 0x7f0030075710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f004400a860 tx=0x7f0044005b60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:11.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.210+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f003c014030 con 0x7f00540cd0c0 2024-09-17T05:33:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.426+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0054153810 con 0x7f00540cd0c0 2024-09-17T05:33:11.429 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:11.426+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f004a000640 1 -- 172.21.15.43:0/3615685790 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v279) ==== 161+0+0 (secure 0 0 0) 0x7f003c05f9a0 con 0x7f00540cd0c0 2024-09-17T05:33:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 msgr2=0x7f0030075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 0x7f0030075710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f004400a860 tx=0x7f0044005b60 comp rx=0 tx=0).stop 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 msgr2=0x7f0054165a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c940 tx=0x7f003c00ce00 comp rx=0 tx=0).stop 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 shutdown_connections 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0030073200 0x7f0030075710 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f005415cea0 0x7f00541fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00540cd0c0 0x7f0054165a10 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 --2- 172.21.15.43:0/3615685790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00540cccb0 0x7f00541654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 >> 172.21.15.43:0/3615685790 conn(0x7f00540691b0 msgr2=0x7f005415b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 shutdown_connections 2024-09-17T05:33:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.430+0000 7f0053200640 1 -- 172.21.15.43:0/3615685790 wait complete. 2024-09-17T05:33:11.440 INFO:tasks.ceph.mds.c.smithi043.stderr: -577> 2024-09-17T05:33:08.610+0000 7fcfb7400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:11.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:33:11.562 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:33:11.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 -- 172.21.15.43:0/3671369272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780d0d80 msgr2=0x7f5e780cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3671369272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780d0d80 0x7f5e780cce70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5e780739f0 tx=0x7f5e6001c600 comp rx=0 tx=0).stop 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 -- 172.21.15.43:0/3671369272 shutdown_connections 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3671369272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780d0d80 0x7f5e780cce70 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3671369272 >> v2:172.21.15.62:3300/0 conn(0x7f5e780cd3b0 0x7f5e780b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3671369272 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780b7ce0 0x7f5e780d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 -- 172.21.15.43:0/3671369272 >> 172.21.15.43:0/3671369272 conn(0x7f5e780cee80 msgr2=0x7f5e780cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 -- 172.21.15.43:0/3671369272 shutdown_connections 2024-09-17T05:33:11.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.614+0000 7f5e7d200640 1 -- 172.21.15.43:0/3671369272 wait complete. 2024-09-17T05:33:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 Processor -- start 2024-09-17T05:33:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 -- start start 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e780cd3b0 0x7f5e780b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780d0d80 0x7f5e780b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78160b40 con 0x7f5e780b7ce0 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781609c0 con 0x7f5e780cd3b0 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e7d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5e78160cc0 con 0x7f5e780d0d80 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e75e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e75e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54278/0 (socket says 172.21.15.43:54278) 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.618+0000 7f5e75e00640 1 -- 172.21.15.43:0/3386193948 learned_addr learned my addr 172.21.15.43:0/3386193948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e6fe00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e780cd3b0 0x7f5e780b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e76800640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780d0d80 0x7f5e780b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 -- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780d0d80 msgr2=0x7f5e780b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780d0d80 0x7f5e780b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 -- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e780cd3b0 msgr2=0x7f5e780b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:11.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e780cd3b0 0x7f5e780b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:11.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 -- 172.21.15.43:0/3386193948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7815d450 con 0x7f5e780b7ce0 2024-09-17T05:33:11.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e75e00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800d650 tx=0x7f5e6800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68017070 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e68004310 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68009a20 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e781fc810 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781fcdd0 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e780b1c30 con 0x7f5e780b7ce0 2024-09-17T05:33:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.622+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e68009bc0 con 0x7f5e780b7ce0 2024-09-17T05:33:11.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.626+0000 7f5e6e000640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 0x7f5e54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:11.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.626+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8461+0+0 (secure 0 0 0) 0x7f5e68094c70 con 0x7f5e780b7ce0 2024-09-17T05:33:11.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.626+0000 7f5e6fe00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 0x7f5e54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:11.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.626+0000 7f5e6fe00640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 0x7f5e54075710 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e600af000 tx=0x7f5e6001ae40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.630+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e6805eb60 con 0x7f5e780b7ce0 2024-09-17T05:33:11.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:11.870+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5e7815a0b0 con 0x7f5e780b7ce0 2024-09-17T05:33:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.438+0000 7f5e6e000640 1 -- 172.21.15.43:0/3386193948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v280) ==== 107+0+0 (secure 0 0 0) 0x7f5e68063590 con 0x7f5e780b7ce0 2024-09-17T05:33:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:33:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 msgr2=0x7f5e54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 0x7f5e54075710 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e600af000 tx=0x7f5e6001ae40 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 msgr2=0x7f5e7815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800d650 tx=0x7f5e6800db10 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 shutdown_connections 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5e54073200 0x7f5e54075710 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e780cd3b0 0x7f5e780b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5e780d0d80 0x7f5e780b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 --2- 172.21.15.43:0/3386193948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e780b7ce0 0x7f5e7815f370 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 >> 172.21.15.43:0/3386193948 conn(0x7f5e780cee80 msgr2=0x7f5e780af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 shutdown_connections 2024-09-17T05:33:12.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.442+0000 7f5e7d200640 1 -- 172.21.15.43:0/3386193948 wait complete. 2024-09-17T05:33:12.471 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:12.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.530+0000 7f305247e080 1 Processor -- start 2024-09-17T05:33:12.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.530+0000 7f305247e080 1 -- start start 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.530+0000 7f305247e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d2749c0 0x55d01d2b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d3265b0 0x55d01d2709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d01d1bda00 con 0x55d01d3269c0 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d01d21bf40 con 0x55d01d2749c0 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d01d2b5940 con 0x55d01d3265b0 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54280/0 (socket says 172.21.15.43:54280) 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d3265b0 0x55d01d2709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 -- 172.21.15.43:0/1505996085 learned_addr learned my addr 172.21.15.43:0/1505996085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304b600640 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d2749c0 0x55d01d2b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 -- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d3265b0 msgr2=0x55d01d2709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d3265b0 0x55d01d2709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 -- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d2749c0 msgr2=0x55d01d2b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d2749c0 0x55d01d2b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 -- 172.21.15.43:0/1505996085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01d196c10 con 0x55d01d3269c0 2024-09-17T05:33:12.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f304c000640 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3044019b60 tx=0x7f3044018250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd69c557e092e983 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f3049800640 1 -- 172.21.15.43:0/1505996085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440bb020 con 0x55d01d3269c0 2024-09-17T05:33:12.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f3049800640 1 -- 172.21.15.43:0/1505996085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30440b1040 con 0x55d01d3269c0 2024-09-17T05:33:12.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f3049800640 1 -- 172.21.15.43:0/1505996085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440b0070 con 0x55d01d3269c0 2024-09-17T05:33:12.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- 172.21.15.43:0/1505996085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 msgr2=0x55d01d3250d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3044019b60 tx=0x7f3044018250 comp rx=0 tx=0).stop 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- 172.21.15.43:0/1505996085 shutdown_connections 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d3250d0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d3265b0 0x55d01d2709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 --2- 172.21.15.43:0/1505996085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d2749c0 0x55d01d2b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- 172.21.15.43:0/1505996085 >> 172.21.15.43:0/1505996085 conn(0x55d01d25bd80 msgr2=0x55d01d2b7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- 172.21.15.43:0/1505996085 shutdown_connections 2024-09-17T05:33:12.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- 172.21.15.43:0/1505996085 wait complete. 2024-09-17T05:33:12.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 Processor -- start 2024-09-17T05:33:12.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.534+0000 7f305247e080 1 -- start start 2024-09-17T05:33:12.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d2749c0 0x55d01d34c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d3265b0 0x55d01d345700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d01d343f80 con 0x55d01d3269c0 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d01d34dd20 con 0x55d01d3265b0 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d01d344220 con 0x55d01d2749c0 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54286/0 (socket says 172.21.15.43:54286) 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 learned_addr learned my addr 172.21.15.43:0/3167174231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:12.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d2749c0 0x55d01d34c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d3265b0 0x55d01d345700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d2749c0 msgr2=0x55d01d34c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d01d2749c0 0x55d01d34c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d3265b0 msgr2=0x55d01d345700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d01d3265b0 0x55d01d345700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01d34c7c0 con 0x55d01d3269c0 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f304c000640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3044019b60 tx=0x7f30440abd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f3043e00640 1 -- 172.21.15.43:0/3167174231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440c8070 con 0x55d01d3269c0 2024-09-17T05:33:12.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f3043e00640 1 -- 172.21.15.43:0/3167174231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30440b7570 con 0x55d01d3269c0 2024-09-17T05:33:12.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f3043e00640 1 -- 172.21.15.43:0/3167174231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440bb030 con 0x55d01d3269c0 2024-09-17T05:33:12.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d01d34ca30 con 0x55d01d3269c0 2024-09-17T05:33:12.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.538+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01d34cf70 con 0x55d01d3269c0 2024-09-17T05:33:12.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f3043e00640 1 -- 172.21.15.43:0/3167174231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30440cb070 con 0x55d01d3269c0 2024-09-17T05:33:12.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f3043e00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 0x7f302c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f3043e00640 1 -- 172.21.15.43:0/3167174231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8461+0+0 (secure 0 0 0) 0x7f304412f2c0 con 0x55d01d3269c0 2024-09-17T05:33:12.547 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:12.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 0x7f302c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d01d31a020 0x55d01d31a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d31a970 con 0x55d01d31a020 2024-09-17T05:33:12.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d01d31a020 0x55d01d31a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 0x7f302c0740f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f303c011490 tx=0x7f303c001740 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.542+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d01d31a020 0x55d01d31a430 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.550+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v121'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.550+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d31ad60 con 0x55d01d31a020 2024-09-17T05:33:12.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.550+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v121'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.550+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3eb280 con 0x55d01d31a020 2024-09-17T05:33:12.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v121'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.556 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec630 con 0x55d01d31a020 2024-09-17T05:33:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec9d0 con 0x55d01d31a020 2024-09-17T05:33:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.554+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3eb620 con 0x55d01d31a020 2024-09-17T05:33:12.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3eba10 con 0x55d01d31a020 2024-09-17T05:33:12.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v121'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3ebe00 con 0x55d01d31a020 2024-09-17T05:33:12.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v121'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.558+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebe00 con 0x55d01d31a020 2024-09-17T05:33:12.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.562+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v121'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.564 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:12.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.562+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.562+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.562+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.562+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.566+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.566+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.566+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.566+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.566+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.573 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:12.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.570+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.574+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.574+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ebdb0 con 0x55d01d31a020 2024-09-17T05:33:12.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.574+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.574+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec190 con 0x55d01d31a020 2024-09-17T05:33:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.574+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.578+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.578+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v121'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.578+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.578+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v121'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.578+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v121'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.584 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.582+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.586+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.586+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.586+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v121'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.586+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v121'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v121'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.593 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.590+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d31a020 2024-09-17T05:33:12.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d01d3ed280 0x55d01d3ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3ec030 con 0x55d01d3ed280 2024-09-17T05:33:12.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d01d3ed280 0x55d01d3ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.594+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d01d3ed280 0x55d01d3ef6a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'704 uv683 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3efbe0 con 0x55d01d3ed280 2024-09-17T05:33:12.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'705 uv705 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3eff80 con 0x55d01d3ed280 2024-09-17T05:33:12.603 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:12.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.598+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f07d0 con 0x55d01d3ed280 2024-09-17T05:33:12.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.602+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.602+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f0b70 con 0x55d01d3ed280 2024-09-17T05:33:12.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.602+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.602+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f0f10 con 0x55d01d3ed280 2024-09-17T05:33:12.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v121'709 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f12b0 con 0x55d01d3ed280 2024-09-17T05:33:12.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v121'710 uv710 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.611 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:12.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v121'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.606+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.610+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.610+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.610+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.610+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v121'714 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v121'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.618 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:12.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v121'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.614+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f5820 con 0x55d01d3ed280 2024-09-17T05:33:12.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.618+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v121'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.618+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f5820 con 0x55d01d3ed280 2024-09-17T05:33:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.618+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.618+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v121'719 uv699 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v121'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.626 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v121'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.622+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f2660 con 0x55d01d3ed280 2024-09-17T05:33:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f2660 con 0x55d01d3ed280 2024-09-17T05:33:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f1650 con 0x55d01d3ed280 2024-09-17T05:33:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3ed280 2024-09-17T05:33:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d01d3f1f50 0x55d01d3f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f48b0 con 0x55d01d3f1f50 2024-09-17T05:33:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.626+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d01d3f1f50 0x55d01d3f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.630+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d01d3f1f50 0x55d01d3f4370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.630+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v121'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.630+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f48b0 con 0x55d01d3f1f50 2024-09-17T05:33:12.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.630+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v121'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.630+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f4930 con 0x55d01d3f1f50 2024-09-17T05:33:12.635 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v121'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f5d60 con 0x55d01d3f1f50 2024-09-17T05:33:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f5d60 con 0x55d01d3f1f50 2024-09-17T05:33:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.634+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f4d50 con 0x55d01d3f1f50 2024-09-17T05:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.638+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.638+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f4d50 con 0x55d01d31a020 2024-09-17T05:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.638+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'136 uv135 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.638+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f4d50 con 0x55d01d31a020 2024-09-17T05:33:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'137 uv137 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f4d50 con 0x55d01d31a020 2024-09-17T05:33:12.645 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'138 uv138 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f4d50 con 0x55d01d31a020 2024-09-17T05:33:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'139 uv139 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 0x55d01d3f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f7c90 con 0x55d01d3f5260 2024-09-17T05:33:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.642+0000 7f304c000640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 0x55d01d3f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.646+0000 7f304c000640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 0x55d01d3f7680 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v121'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3f8160 con 0x55d01d3f5260 2024-09-17T05:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v121'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f8640 con 0x55d01d3f5260 2024-09-17T05:33:12.654 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v121'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.650+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f9ef0 con 0x55d01d3f5260 2024-09-17T05:33:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fa290 con 0x55d01d3f5260 2024-09-17T05:33:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3f8ee0 con 0x55d01d3f5260 2024-09-17T05:33:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d01d3f97e0 0x55d01d3fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcb80 con 0x55d01d3f97e0 2024-09-17T05:33:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.654+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d01d3f97e0 0x55d01d3fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.658+0000 7f304b600640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d01d3f97e0 0x55d01d3fc640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.658+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v121'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.658+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3fcb80 con 0x55d01d3f97e0 2024-09-17T05:33:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.658+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v121'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.658+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcbc0 con 0x55d01d3f97e0 2024-09-17T05:33:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v121'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fdff0 con 0x55d01d3f97e0 2024-09-17T05:33:12.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fdff0 con 0x55d01d3f97e0 2024-09-17T05:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.662+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.666+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.666+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.666+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v121'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.666+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v121'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.673 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:12.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v121'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.670+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d3fcfe0 con 0x55d01d3f97e0 2024-09-17T05:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f304b600640 1 -- 172.21.15.43:0/3167174231 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303c00d040 con 0x55d01d3f97e0 2024-09-17T05:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d01d3fd4f0 0x55d01d3ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400040 con 0x55d01d3fd4f0 2024-09-17T05:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d01d3fd4f0 0x55d01d3ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f304ac00640 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d01d3fd4f0 0x55d01d3ff910 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v121'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.674+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400040 con 0x55d01d3fd4f0 2024-09-17T05:33:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v121'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3fd4f0 2024-09-17T05:33:12.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v121'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.681 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:12.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3fd4f0 2024-09-17T05:33:12.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.678+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3fd4f0 2024-09-17T05:33:12.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d31a020 2024-09-17T05:33:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v121'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d31a020 2024-09-17T05:33:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v121'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.682+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d31a020 2024-09-17T05:33:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v121'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.688 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d4014a0 con 0x55d01d31a020 2024-09-17T05:33:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d4014a0 con 0x55d01d31a020 2024-09-17T05:33:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.686+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d31a020 2024-09-17T05:33:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.690+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d31a020 2024-09-17T05:33:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.690+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.690+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v121'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.690+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v121'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.697 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v121'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.694+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.698+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.698+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.698+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v121'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.698+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v121'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.705 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:12.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v121'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400490 con 0x55d01d3f1f50 2024-09-17T05:33:12.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.702+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d4032a0 con 0x55d01d3f1f50 2024-09-17T05:33:12.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3f1f50 2024-09-17T05:33:12.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400940 con 0x55d01d3f5260 2024-09-17T05:33:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v121'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400d30 con 0x55d01d3f5260 2024-09-17T05:33:12.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.706+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400d30 con 0x55d01d3f5260 2024-09-17T05:33:12.713 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:12.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.710+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v121'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.710+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.710+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.710+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.714+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v121'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.714+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.714+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v121'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.714+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v121'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.719 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d401cf0 con 0x55d01d3f5260 2024-09-17T05:33:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.718+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3f5260 2024-09-17T05:33:12.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30440ca030 con 0x55d01d3f5260 2024-09-17T05:33:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v121'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v121'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.722+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v121'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.728 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:12.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.726+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.730+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v121'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.730+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.730+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v121'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.730+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.735 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:12.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.730+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v121'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.734+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.734+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.734+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.734+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.734+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v121'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v121'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.742 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v121'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.738+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d401cf0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d401cf0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d400ce0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.742+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d01d401190 con 0x55d01d3fd4f0 2024-09-17T05:33:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f304ac00640 1 -- 172.21.15.43:0/3167174231 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f303800b8f0 con 0x55d01d3fd4f0 2024-09-17T05:33:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d01d3ed280 msgr2=0x55d01d3ef6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d01d3ed280 0x55d01d3ef6a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d01d3f97e0 msgr2=0x55d01d3fc640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d01d3f97e0 0x55d01d3fc640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d01d3fd4f0 msgr2=0x55d01d3ff910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d01d3fd4f0 0x55d01d3ff910 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d01d3f1f50 msgr2=0x55d01d3f4370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d01d3f1f50 0x55d01d3f4370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 reap_dead start 2024-09-17T05:33:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d01d31a020 msgr2=0x55d01d31a430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d01d31a020 0x55d01d31a430 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f304c000640 1 -- 172.21.15.43:0/3167174231 reap_dead start 2024-09-17T05:33:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 msgr2=0x55d01d3f7680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 0x55d01d3f7680 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 msgr2=0x7f302c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 0x7f302c0740f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f303c011490 tx=0x7f303c001740 comp rx=0 tx=0).stop 2024-09-17T05:33:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 msgr2=0x55d01d345c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3044019b60 tx=0x7f30440abd90 comp rx=0 tx=0).stop 2024-09-17T05:33:12.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 shutdown_connections 2024-09-17T05:33:12.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d01d3f5260 0x55d01d3f7680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f302c071be0 0x7f302c0740f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 --2- 172.21.15.43:0/3167174231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d01d3269c0 0x55d01d345c40 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 >> 172.21.15.43:0/3167174231 conn(0x55d01d25bd80 msgr2=0x55d01d2b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:12.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 shutdown_connections 2024-09-17T05:33:12.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.746+0000 7f305247e080 1 -- 172.21.15.43:0/3167174231 wait complete. 2024-09-17T05:33:12.756 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 Processor -- start 2024-09-17T05:33:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- start start 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b41487fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb5b0 0x555b415aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b41392a00 con 0x555b414fb9c0 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555b413f0f40 con 0x555b414fb5b0 2024-09-17T05:33:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555b415252d0 con 0x555b414499c0 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c13600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb5b0 0x555b415aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54306/0 (socket says 172.21.15.43:54306) 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b41487fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 -- 172.21.15.43:0/3970820264 learned_addr learned my addr 172.21.15.43:0/3970820264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:12.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 -- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 msgr2=0x555b41487fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b41487fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 -- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb5b0 msgr2=0x555b415aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb5b0 0x555b415aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 -- 172.21.15.43:0/3970820264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4136bc10 con 0x555b414fb9c0 2024-09-17T05:33:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c14a00640 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c001600 tx=0x7f6c0c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab550d040c418c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c12200640 1 -- 172.21.15.43:0/3970820264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c017070 con 0x555b414fb9c0 2024-09-17T05:33:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c12200640 1 -- 172.21.15.43:0/3970820264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c0c0027b0 con 0x555b414fb9c0 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c12200640 1 -- 172.21.15.43:0/3970820264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c002c00 con 0x555b414fb9c0 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- 172.21.15.43:0/3970820264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 msgr2=0x555b415aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c001600 tx=0x7f6c0c006f90 comp rx=0 tx=0).stop 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- 172.21.15.43:0/3970820264 shutdown_connections 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb5b0 0x555b415aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb9c0 0x555b415aefc0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/3970820264 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b41487fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- 172.21.15.43:0/3970820264 >> 172.21.15.43:0/3970820264 conn(0x555b41430d80 msgr2=0x555b4151b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- 172.21.15.43:0/3970820264 shutdown_connections 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.858+0000 7f6c1ad78080 1 -- 172.21.15.43:0/3970820264 wait complete. 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 Processor -- start 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 -- start start 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b415177b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb5b0 0x555b415aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb9c0 0x555b415aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555b4148e280 con 0x555b414fb5b0 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555b4151aa80 con 0x555b414fb9c0 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555b4151ad20 con 0x555b414499c0 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b415177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c13600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb5b0 0x555b415aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b415177b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:54636/0 (socket says 172.21.15.43:54636) 2024-09-17T05:33:12.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 -- 172.21.15.43:0/4012791694 learned_addr learned my addr 172.21.15.43:0/4012791694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14a00640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb9c0 0x555b415aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 -- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb9c0 msgr2=0x555b415aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b414fb9c0 0x555b415aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 -- 172.21.15.43:0/4012791694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb5b0 msgr2=0x555b415aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb5b0 0x555b415aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 -- 172.21.15.43:0/4012791694 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4151afc0 con 0x555b414499c0 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c13600640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555b414fb5b0 0x555b415aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c14000640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b415177b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400a6b0 tx=0x7f6c0400ab70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c10e00640 1 -- 172.21.15.43:0/4012791694 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401c070 con 0x555b414499c0 2024-09-17T05:33:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c10e00640 1 -- 172.21.15.43:0/4012791694 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c04008db0 con 0x555b414499c0 2024-09-17T05:33:12.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.862+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b4151a4d0 con 0x555b414499c0 2024-09-17T05:33:12.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c10e00640 1 -- 172.21.15.43:0/4012791694 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040176e0 con 0x555b414499c0 2024-09-17T05:33:12.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x555b414f15a0 con 0x555b414499c0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c10e00640 1 -- 172.21.15.43:0/4012791694 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c0401f070 con 0x555b414499c0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c10e00640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bf4071be0 0x7f6bf40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c10e00640 1 -- 172.21.15.43:0/4012791694 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(121..121 src has 1..121) ==== 8461+0+0 (secure 0 0 0) 0x7f6c04093dc0 con 0x555b414499c0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c13600640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bf4071be0 0x7f6bf40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555b414f4690 0x555b415bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.866+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x555b41517cf0 con 0x555b414f4690 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c14a00640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555b414f4690 0x555b415bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c13600640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bf4071be0 0x7f6bf40740f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6c000ab000 tx=0x7f6c00019990 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c14a00640 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555b414f4690 0x555b415bdf60 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c14a00640 1 -- 172.21.15.43:0/4012791694 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v121'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6c0c00f070 con 0x555b414f4690 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555b414f4690 msgr2=0x555b415bdf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x555b414f4690 0x555b415bdf60 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:12.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bf4071be0 msgr2=0x7f6bf40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6bf4071be0 0x7f6bf40740f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6c000ab000 tx=0x7f6c00019990 comp rx=0 tx=0).stop 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 msgr2=0x555b415177b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 --2- 172.21.15.43:0/4012791694 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555b414499c0 0x555b415177b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400a6b0 tx=0x7f6c0400ab70 comp rx=0 tx=0).stop 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c14a00640 1 -- 172.21.15.43:0/4012791694 reap_dead start 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 shutdown_connections 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 >> 172.21.15.43:0/4012791694 conn(0x555b41430d80 msgr2=0x555b41487fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 shutdown_connections 2024-09-17T05:33:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:12.870+0000 7f6c1ad78080 1 -- 172.21.15.43:0/4012791694 wait complete. 2024-09-17T05:33:12.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- 172.21.15.43:0/3028665631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c155d70 msgr2=0x7f023c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 --2- 172.21.15.43:0/3028665631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c155d70 0x7f023c15c7e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f022c009840 tx=0x7f022c01c760 comp rx=0 tx=0).stop 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- 172.21.15.43:0/3028665631 shutdown_connections 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 --2- 172.21.15.43:0/3028665631 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c0b3af0 0x7f023c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 --2- 172.21.15.43:0/3028665631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c155d70 0x7f023c15c7e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- 172.21.15.43:0/3028665631 >> 172.21.15.43:0/3028665631 conn(0x7f023c0b23d0 msgr2=0x7f023c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- 172.21.15.43:0/3028665631 shutdown_connections 2024-09-17T05:33:13.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- 172.21.15.43:0/3028665631 wait complete. 2024-09-17T05:33:13.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 Processor -- start 2024-09-17T05:33:13.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.038+0000 7f0242200640 1 -- start start 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c06f110 0x7f023c1fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c155d70 0x7f023c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f023c168c60 con 0x7f023c1556b0 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f023c168ae0 con 0x7f023c06f110 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f023c168de0 con 0x7f023c155d70 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54318/0 (socket says 172.21.15.43:54318) 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 -- 172.21.15.43:0/3004644628 learned_addr learned my addr 172.21.15.43:0/3004644628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023b400640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c06f110 0x7f023c1fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0232a00640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c155d70 0x7f023c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 -- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c155d70 msgr2=0x7f023c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c155d70 0x7f023c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 -- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c06f110 msgr2=0x7f023c1fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c06f110 0x7f023c1fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 -- 172.21.15.43:0/3004644628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c0c9d70 con 0x7f023c1556b0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f023be00640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f023400ea80 tx=0x7f023400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234004b30 con 0x7f023c1556b0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0234004cf0 con 0x7f023c1556b0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234005280 con 0x7f023c1556b0 2024-09-17T05:33:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c0ca050 con 0x7f023c1556b0 2024-09-17T05:33:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c073090 con 0x7f023c1556b0 2024-09-17T05:33:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.042+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c0ad0b0 con 0x7f023c1556b0 2024-09-17T05:33:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.046+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0234005420 con 0x7f023c1556b0 2024-09-17T05:33:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.046+0000 7f0233e00640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 0x7f0218075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.046+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8461+0+0 (secure 0 0 0) 0x7f0234095970 con 0x7f023c1556b0 2024-09-17T05:33:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.046+0000 7f023b400640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 0x7f0218075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.046+0000 7f023b400640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 0x7f0218075740 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f022c0af000 tx=0x7f022c01afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:13.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.050+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f023405feb0 con 0x7f023c1556b0 2024-09-17T05:33:13.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.258+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f023c0af970 con 0x7f023c1556b0 2024-09-17T05:33:13.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0233e00640 1 -- 172.21.15.43:0/3004644628 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v281) ==== 162+0+0 (secure 0 0 0) 0x7f023405f7b0 con 0x7f023c1556b0 2024-09-17T05:33:13.444 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 msgr2=0x7f0218075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 0x7f0218075740 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f022c0af000 tx=0x7f022c01afa0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 msgr2=0x7f023c15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f023400ea80 tx=0x7f023400ef40 comp rx=0 tx=0).stop 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 shutdown_connections 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0218073230 0x7f0218075740 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f023c1556b0 0x7f023c15ac50 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f023c155d70 0x7f023c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 --2- 172.21.15.43:0/3004644628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f023c06f110 0x7f023c1fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 >> 172.21.15.43:0/3004644628 conn(0x7f023c0b23d0 msgr2=0x7f023c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:13.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 shutdown_connections 2024-09-17T05:33:13.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.442+0000 7f0242200640 1 -- 172.21.15.43:0/3004644628 wait complete. 2024-09-17T05:33:13.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- 172.21.15.43:0/3657662424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c00b5610 msgr2=0x7ff7c00b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 --2- 172.21.15.43:0/3657662424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c00b5610 0x7ff7c00b5a20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0073390 tx=0x7ff7ac01c6e0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- 172.21.15.43:0/3657662424 shutdown_connections 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 --2- 172.21.15.43:0/3657662424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c00b5610 0x7ff7c00b5a20 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 --2- 172.21.15.43:0/3657662424 >> v2:172.21.15.62:3300/0 conn(0x7ff7c015f470 0x7ff7c00d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- 172.21.15.43:0/3657662424 >> 172.21.15.43:0/3657662424 conn(0x7ff7c00b2810 msgr2=0x7ff7c00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- 172.21.15.43:0/3657662424 shutdown_connections 2024-09-17T05:33:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- 172.21.15.43:0/3657662424 wait complete. 2024-09-17T05:33:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 Processor -- start 2024-09-17T05:33:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.626+0000 7ff7c8000640 1 -- start start 2024-09-17T05:33:13.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 0x7ff7c01f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 0x7ff7c01538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0169480 con 0x7ff7c015f470 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0169300 con 0x7ff7c00b5610 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0169600 con 0x7ff7c00b3b10 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54336/0 (socket says 172.21.15.43:54336) 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 -- 172.21.15.43:0/2791562034 learned_addr learned my addr 172.21.15.43:0/2791562034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c4c00640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 0x7ff7c01538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c5600640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 0x7ff7c01f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 -- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 msgr2=0x7ff7c01f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 0x7ff7c01f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 -- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 msgr2=0x7ff7c01538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 0x7ff7c01538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 -- 172.21.15.43:0/2791562034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01f9730 con 0x7ff7c015f470 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c5600640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 0x7ff7c01f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c4c00640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 0x7ff7c01538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:13.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c6000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c730 tx=0x7ff7b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8003b80 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b8003d40 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80052d0 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01f9a10 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c0153e10 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.630+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c00b5a20 con 0x7ff7c015f470 2024-09-17T05:33:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.634+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7b8005680 con 0x7ff7c015f470 2024-09-17T05:33:13.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.634+0000 7ff7bd600640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:13.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.634+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8461+0+0 (secure 0 0 0) 0x7ff7b8094df0 con 0x7ff7c015f470 2024-09-17T05:33:13.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.634+0000 7ff7c5600640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:13.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.634+0000 7ff7c5600640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 0x7ff7a0075710 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0069610 tx=0x7ff7b40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:13.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.638+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff7c00b5f60 con 0x7ff7c015f470 2024-09-17T05:33:13.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.846+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c015ba00 con 0x7ff7c015f470 2024-09-17T05:33:13.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.846+0000 7ff7bd600640 1 -- 172.21.15.43:0/2791562034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5218 (secure 0 0 0) 0x7ff7b805f330 con 0x7ff7c015f470 2024-09-17T05:33:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:13.850 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":282,"btime":"2024-09-17T05:33:13:447185+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:13.447165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9723":{"gid":9723,"name":"c","rank":0,"incarnation":282,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/3762350799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3762350799},{"type":"v1","addr":"172.21.15.43:6835","nonce":3762350799}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 282 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 msgr2=0x7ff7a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 0x7ff7a0075710 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0069610 tx=0x7ff7b40091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 msgr2=0x7ff7c015e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c730 tx=0x7ff7b800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 shutdown_connections 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7c00b5610 0x7ff7c01538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7c015f470 0x7ff7c015e7e0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 --2- 172.21.15.43:0/2791562034 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff7c00b3b10 0x7ff7c01f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 >> 172.21.15.43:0/2791562034 conn(0x7ff7c00b2810 msgr2=0x7ff7c01522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 shutdown_connections 2024-09-17T05:33:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:13.850+0000 7ff7c8000640 1 -- 172.21.15.43:0/2791562034 wait complete. 2024-09-17T05:33:13.873 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:33:16.472 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:16.470+0000 7f1b14a00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-17T05:33:16.473 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:16.470+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:16.473 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:16.470+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:33:17.451 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:17.450+0000 7f1b19a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:18.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 -- 172.21.15.43:0/1621954032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421415fe10 msgr2=0x7f42140b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 --2- 172.21.15.43:0/1621954032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421415fe10 0x7f42140b4b80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4204009930 tx=0x7f420401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 -- 172.21.15.43:0/1621954032 shutdown_connections 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 --2- 172.21.15.43:0/1621954032 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f42140737c0 0x7f42140700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 --2- 172.21.15.43:0/1621954032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421415fe10 0x7f42140b4b80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 -- 172.21.15.43:0/1621954032 >> 172.21.15.43:0/1621954032 conn(0x7f42140b23d0 msgr2=0x7f42140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 -- 172.21.15.43:0/1621954032 shutdown_connections 2024-09-17T05:33:19.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.042+0000 7f4219400640 1 -- 172.21.15.43:0/1621954032 wait complete. 2024-09-17T05:33:19.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 Processor -- start 2024-09-17T05:33:19.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- start start 2024-09-17T05:33:19.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421406fc90 0x7f4214157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 0x7f4214099690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f421416a0a0 con 0x7f421415e0c0 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4214169f20 con 0x7f421406fc90 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f421416a220 con 0x7f421415fe10 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53452/0 (socket says 172.21.15.43:53452) 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 -- 172.21.15.43:0/3534019313 learned_addr learned my addr 172.21.15.43:0/3534019313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f420be00640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 0x7f4214099690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4212800640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421406fc90 0x7f4214157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 -- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 msgr2=0x7f4214099690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 0x7f4214099690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 -- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421406fc90 msgr2=0x7f4214157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421406fc90 0x7f4214157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 -- 172.21.15.43:0/3534019313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421415ea10 con 0x7f421415e0c0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f420be00640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 0x7f4214099690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4213200640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ec30 tx=0x7f420c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c003b80 con 0x7f421415e0c0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f420c003e10 con 0x7f421415e0c0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421415c170 con 0x7f421415e0c0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c004280 con 0x7f421415e0c0 2024-09-17T05:33:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.046+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421415c730 con 0x7f421415e0c0 2024-09-17T05:33:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42141583f0 con 0x7f421415e0c0 2024-09-17T05:33:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f420c004420 con 0x7f421415e0c0 2024-09-17T05:33:19.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f420aa00640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 0x7f41f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f420c095bf0 con 0x7f421415e0c0 2024-09-17T05:33:19.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f4212800640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 0x7f41f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.050+0000 7f4212800640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 0x7f41f0075710 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42040af000 tx=0x7f420401b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.054+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f42141583f0 con 0x7f421415e0c0 2024-09-17T05:33:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42140700a0 con 0x7f421415e0c0 2024-09-17T05:33:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f420aa00640 1 -- 172.21.15.43:0/3534019313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 286 v286) ==== 95+0+4383 (secure 0 0 0) 0x7f421415ea10 con 0x7f421415e0c0 2024-09-17T05:33:19.269 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:19.269 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":286,"btime":"2024-09-17T05:33:16:473850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":286,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:16.473847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 286 2024-09-17T05:33:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 msgr2=0x7f41f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 0x7f41f0075710 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42040af000 tx=0x7f420401b000 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 msgr2=0x7f421415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.266+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ec30 tx=0x7f420c00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 shutdown_connections 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f41f0073200 0x7f41f0075710 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421415e0c0 0x7f421415e4d0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f421415fe10 0x7f4214099690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 --2- 172.21.15.43:0/3534019313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f421406fc90 0x7f4214157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 >> 172.21.15.43:0/3534019313 conn(0x7f42140b23d0 msgr2=0x7f42140ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 shutdown_connections 2024-09-17T05:33:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.270+0000 7f4219400640 1 -- 172.21.15.43:0/3534019313 wait complete. 2024-09-17T05:33:19.292 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:33:19.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 -- 172.21.15.43:0/1361862936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4481603e0 msgr2=0x7fd4481607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 --2- 172.21.15.43:0/1361862936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4481603e0 0x7fd4481607f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fd4480752d0 tx=0x7fd44001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 -- 172.21.15.43:0/1361862936 shutdown_connections 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 --2- 172.21.15.43:0/1361862936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4481603e0 0x7fd4481607f0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 --2- 172.21.15.43:0/1361862936 >> v2:172.21.15.62:3300/0 conn(0x7fd4480ce3c0 0x7fd4480cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 --2- 172.21.15.43:0/1361862936 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4480cdfb0 0x7fd4480cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 -- 172.21.15.43:0/1361862936 >> 172.21.15.43:0/1361862936 conn(0x7fd4480b2810 msgr2=0x7fd4480b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 -- 172.21.15.43:0/1361862936 shutdown_connections 2024-09-17T05:33:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.450+0000 7fd44e600640 1 -- 172.21.15.43:0/1361862936 wait complete. 2024-09-17T05:33:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 Processor -- start 2024-09-17T05:33:19.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- start start 2024-09-17T05:33:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4480cdfb0 0x7fd448154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4481603e0 0x7fd44814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd448074640 con 0x7fd4480ce3c0 2024-09-17T05:33:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4480744c0 con 0x7fd4480cdfb0 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd4480747c0 con 0x7fd4481603e0 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd447400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4480cdfb0 0x7fd448154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53472/0 (socket says 172.21.15.43:53472) 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 -- 172.21.15.43:0/1477478412 learned_addr learned my addr 172.21.15.43:0/1477478412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd447e00640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4481603e0 0x7fd44814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 -- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4481603e0 msgr2=0x7fd44814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4481603e0 0x7fd44814d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 -- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4480cdfb0 msgr2=0x7fd448154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4480cdfb0 0x7fd448154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 -- 172.21.15.43:0/1477478412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4481f90a0 con 0x7fd4480ce3c0 2024-09-17T05:33:19.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43fe00640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e7a0 tx=0x7fd43000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43000cc60 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43000ee50 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430010670 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4481f9380 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.454+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4481f9940 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4481607f0 con 0x7fd4480ce3c0 2024-09-17T05:33:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd430010810 con 0x7fd4480ce3c0 2024-09-17T05:33:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd43f400640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 0x7fd424075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7fd4300959a0 con 0x7fd4480ce3c0 2024-09-17T05:33:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd447400640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 0x7fd424075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.458+0000 7fd447400640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 0x7fd424075740 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd438009ac0 tx=0x7fd4380091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.462+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd43005f8b0 con 0x7fd4480ce3c0 2024-09-17T05:33:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.674+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4481f8e20 con 0x7fd4480ce3c0 2024-09-17T05:33:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.674+0000 7fd43f400640 1 -- 172.21.15.43:0/1477478412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 286 v286) ==== 95+0+4383 (secure 0 0 0) 0x7fd4481f8e20 con 0x7fd4480ce3c0 2024-09-17T05:33:19.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:19.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":286,"btime":"2024-09-17T05:33:16:473850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":286,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:16.473847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:19.678 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 286 2024-09-17T05:33:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 msgr2=0x7fd424075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 0x7fd424075740 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd438009ac0 tx=0x7fd4380091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 msgr2=0x7fd44814d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e7a0 tx=0x7fd43000ec60 comp rx=0 tx=0).stop 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 shutdown_connections 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd424073230 0x7fd424075740 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd4481603e0 0x7fd44814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4480ce3c0 0x7fd44814d090 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 --2- 172.21.15.43:0/1477478412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4480cdfb0 0x7fd448154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 >> 172.21.15.43:0/1477478412 conn(0x7fd4480b2810 msgr2=0x7fd4481536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 shutdown_connections 2024-09-17T05:33:19.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.678+0000 7fd44e600640 1 -- 172.21.15.43:0/1477478412 wait complete. 2024-09-17T05:33:19.700 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:33:19.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.854+0000 7feb7a400640 1 -- 172.21.15.43:0/953097751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb740737c0 msgr2=0x7feb740700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.854+0000 7feb7a400640 1 --2- 172.21.15.43:0/953097751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb740737c0 0x7feb740700a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009bf0 tx=0x7feb6c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.854+0000 7feb7a400640 1 -- 172.21.15.43:0/953097751 shutdown_connections 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.854+0000 7feb7a400640 1 --2- 172.21.15.43:0/953097751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb740737c0 0x7feb740700a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- 172.21.15.43:0/953097751 >> 172.21.15.43:0/953097751 conn(0x7feb740b23d0 msgr2=0x7feb740d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- 172.21.15.43:0/953097751 shutdown_connections 2024-09-17T05:33:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- 172.21.15.43:0/953097751 wait complete. 2024-09-17T05:33:19.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 Processor -- start 2024-09-17T05:33:19.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- start start 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7406fc90 0x7feb740997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 0x7feb74157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feb7416b640 con 0x7feb74155950 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb7416b4c0 con 0x7feb7415fe10 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.858+0000 7feb7a400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7feb7416b7c0 con 0x7feb7406fc90 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53476/0 (socket says 172.21.15.43:53476) 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 -- 172.21.15.43:0/2063518297 learned_addr learned my addr 172.21.15.43:0/2063518297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb72a00640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 0x7feb74157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7406fc90 0x7feb740997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 -- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7406fc90 msgr2=0x7feb740997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7406fc90 0x7feb740997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 -- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 msgr2=0x7feb74157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 0x7feb74157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 -- 172.21.15.43:0/2063518297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb740cc150 con 0x7feb74155950 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb72a00640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 0x7feb74157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb73e00640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b0040 tx=0x7feb6c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0b2040 con 0x7feb74155950 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb6c004040 con 0x7feb74155950 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb740cc650 con 0x7feb74155950 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004430 con 0x7feb74155950 2024-09-17T05:33:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb74155150 con 0x7feb74155950 2024-09-17T05:33:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb740cc3c0 con 0x7feb74155950 2024-09-17T05:33:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.862+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb6c0bd030 con 0x7feb74155950 2024-09-17T05:33:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.866+0000 7feb6b400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 0x7feb54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.866+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7feb6c12ece0 con 0x7feb74155950 2024-09-17T05:33:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.866+0000 7feb73400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 0x7feb54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.866+0000 7feb73400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 0x7feb54075710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7feb64007fd0 tx=0x7feb64009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:19.870+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb6c0c4060 con 0x7feb74155950 2024-09-17T05:33:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb740700a0 con 0x7feb74155950 2024-09-17T05:33:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb6b400640 1 -- 172.21.15.43:0/2063518297 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 286 v286) ==== 95+0+4383 (secure 0 0 0) 0x7feb6c016790 con 0x7feb74155950 2024-09-17T05:33:20.081 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:20.082 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":286,"btime":"2024-09-17T05:33:16:473850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":286,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:16.473847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 286 2024-09-17T05:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 msgr2=0x7feb54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 0x7feb54075710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7feb64007fd0 tx=0x7feb64009040 comp rx=0 tx=0).stop 2024-09-17T05:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 msgr2=0x7feb74155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.078+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b0040 tx=0x7feb6c005ae0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 shutdown_connections 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7feb54073200 0x7feb54075710 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb7415fe10 0x7feb74157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb74155950 0x7feb74155d60 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 --2- 172.21.15.43:0/2063518297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feb7406fc90 0x7feb740997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 >> 172.21.15.43:0/2063518297 conn(0x7feb740b23d0 msgr2=0x7feb740ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 shutdown_connections 2024-09-17T05:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.082+0000 7feb7a400640 1 -- 172.21.15.43:0/2063518297 wait complete. 2024-09-17T05:33:20.104 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-09-17T05:33:20.104 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-09-17T05:33:20.104 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:20.104 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:20.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:33:20.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:33:20.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:20.113 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:20.163 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:33:20.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:33:20.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.362+0000 7fe669c00640 1 -- 172.21.15.43:0/493914585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe648003d00 msgr2=0x7fe648004150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.362+0000 7fe669c00640 1 --2- 172.21.15.43:0/493914585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe648003d00 0x7fe648004150 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fe654001830 tx=0x7fe65401c8a0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.362+0000 7fe663400640 1 -- 172.21.15.43:0/493914585 reap_dead start 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- 172.21.15.43:0/493914585 shutdown_connections 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- 172.21.15.43:0/493914585 >> 172.21.15.43:0/493914585 conn(0x7fe6640b2810 msgr2=0x7fe6640b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- 172.21.15.43:0/493914585 shutdown_connections 2024-09-17T05:33:20.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- 172.21.15.43:0/493914585 wait complete. 2024-09-17T05:33:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 Processor -- start 2024-09-17T05:33:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- start start 2024-09-17T05:33:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 0x7fe66415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66413f200 0x7fe66413f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe664073de0 con 0x7fe6641fc200 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe664073c60 con 0x7fe648003d00 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.366+0000 7fe669c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe664073f60 con 0x7fe66413f200 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53488/0 (socket says 172.21.15.43:53488) 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 -- 172.21.15.43:0/1196742424 learned_addr learned my addr 172.21.15.43:0/1196742424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe662a00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 0x7fe66415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe662000640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66413f200 0x7fe66413f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 -- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66413f200 msgr2=0x7fe66413f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66413f200 0x7fe66413f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 -- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 msgr2=0x7fe66415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 0x7fe66415b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 -- 172.21.15.43:0/1196742424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66413fb50 con 0x7fe6641fc200 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe662a00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 0x7fe66415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe663400640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe6640736e0 tx=0x7fe65c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c010950 con 0x7fe6641fc200 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c017070 con 0x7fe6641fc200 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c011460 con 0x7fe6641fc200 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641fcb50 con 0x7fe6641fc200 2024-09-17T05:33:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe664159fd0 con 0x7fe6641fc200 2024-09-17T05:33:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe664159d40 con 0x7fe6641fc200 2024-09-17T05:33:20.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.370+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe65c011600 con 0x7fe6641fc200 2024-09-17T05:33:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.374+0000 7fe65aa00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 0x7fe644075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.374+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7fe65c0950c0 con 0x7fe6641fc200 2024-09-17T05:33:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.374+0000 7fe662a00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 0x7fe644075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.374+0000 7fe662a00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 0x7fe644075710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe664069610 tx=0x7fe6540196e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.378+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe6641fcb50 con 0x7fe6641fc200 2024-09-17T05:33:20.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:20.586+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6640743f0 con 0x7fe6641fc200 2024-09-17T05:33:21.228 INFO:tasks.ceph.mds.c.smithi043.stderr: -53> 2024-09-17T05:33:16.470+0000 7f1b14a00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-17T05:33:21.229 INFO:tasks.ceph.mds.c.smithi043.stderr: -33> 2024-09-17T05:33:17.450+0000 7f1b19a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:21.351 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:33:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe65aa00640 1 -- 172.21.15.43:0/1196742424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v287) ==== 161+0+0 (secure 0 0 0) 0x7fe65c05f630 con 0x7fe6641fc200 2024-09-17T05:33:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 msgr2=0x7fe644075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 0x7fe644075710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe664069610 tx=0x7fe6540196e0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 msgr2=0x7fe6641fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe6640736e0 tx=0x7fe65c00e970 comp rx=0 tx=0).stop 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 shutdown_connections 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe644073200 0x7fe644075710 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6641fc200 0x7fe6641fc610 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe66413f200 0x7fe66413f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 --2- 172.21.15.43:0/1196742424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe648003d00 0x7fe66415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 >> 172.21.15.43:0/1196742424 conn(0x7fe6640b2810 msgr2=0x7fe6640700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.474+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 shutdown_connections 2024-09-17T05:33:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.478+0000 7fe669c00640 1 -- 172.21.15.43:0/1196742424 wait complete. 2024-09-17T05:33:21.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 -- 172.21.15.43:0/3295946780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0b3af0 msgr2=0x7f38dc0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 --2- 172.21.15.43:0/3295946780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0b3af0 0x7f38dc0ad0b0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009c80 tx=0x7f38d401cce0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 -- 172.21.15.43:0/3295946780 shutdown_connections 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 --2- 172.21.15.43:0/3295946780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0b3af0 0x7f38dc0ad0b0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 --2- 172.21.15.43:0/3295946780 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc155d70 0x7f38dc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 -- 172.21.15.43:0/3295946780 >> 172.21.15.43:0/3295946780 conn(0x7f38dc0b23d0 msgr2=0x7f38dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 -- 172.21.15.43:0/3295946780 shutdown_connections 2024-09-17T05:33:21.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.658+0000 7f38e3800640 1 -- 172.21.15.43:0/3295946780 wait complete. 2024-09-17T05:33:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 Processor -- start 2024-09-17T05:33:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- start start 2024-09-17T05:33:21.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc06f110 0x7f38dc162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38dc1627a0 0x7f38dc09bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc168c80 con 0x7f38dc155d70 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc168b00 con 0x7f38dc1627a0 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc168e00 con 0x7f38dc06f110 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53526/0 (socket says 172.21.15.43:53526) 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 -- 172.21.15.43:0/2134339481 learned_addr learned my addr 172.21.15.43:0/2134339481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e0e00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc06f110 0x7f38dc162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e1800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38dc1627a0 0x7f38dc09bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 -- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc06f110 msgr2=0x7f38dc162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc06f110 0x7f38dc162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 -- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38dc1627a0 msgr2=0x7f38dc09bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38dc1627a0 0x7f38dc09bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 -- 172.21.15.43:0/2134339481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc160ab0 con 0x7f38dc155d70 2024-09-17T05:33:21.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38dbe00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c3f0 tx=0x7f38c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8017070 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38c80053c0 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003860 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc160d90 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.662+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc161350 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.666+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc0ad0b0 con 0x7f38dc155d70 2024-09-17T05:33:21.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.666+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f38c801a070 con 0x7f38dc155d70 2024-09-17T05:33:21.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.666+0000 7f38d8c00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 0x7f38bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.666+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f38c8094130 con 0x7f38dc155d70 2024-09-17T05:33:21.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.666+0000 7f38e0e00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 0x7f38bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.670+0000 7f38e0e00640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 0x7f38bc075710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0715c0 tx=0x7f38d000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:21.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.670+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f38c805df30 con 0x7f38dc155d70 2024-09-17T05:33:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:21.914+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f38dc0ca2f0 con 0x7f38dc155d70 2024-09-17T05:33:22.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:22.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38d8c00640 1 -- 172.21.15.43:0/2134339481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v288) ==== 101+0+0 (secure 0 0 0) 0x7f38c8062960 con 0x7f38dc155d70 2024-09-17T05:33:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:33:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 msgr2=0x7f38bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 0x7f38bc075710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0715c0 tx=0x7f38d000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 msgr2=0x7f38dc09b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f38c800c3f0 tx=0x7f38c800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 shutdown_connections 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f38bc073200 0x7f38bc075710 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38dc1627a0 0x7f38dc09bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc155d70 0x7f38dc09b9e0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 --2- 172.21.15.43:0/2134339481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f38dc06f110 0x7f38dc162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.486+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 >> 172.21.15.43:0/2134339481 conn(0x7f38dc0b23d0 msgr2=0x7f38dc15db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.490+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 shutdown_connections 2024-09-17T05:33:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.490+0000 7f38e3800640 1 -- 172.21.15.43:0/2134339481 wait complete. 2024-09-17T05:33:22.513 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:22.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 Processor -- start 2024-09-17T05:33:22.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 -- start start 2024-09-17T05:33:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e850d70 0x562f6e922180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e909100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f6e79ea00 con 0x562f6e87fcb0 2024-09-17T05:33:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f6e7fcf40 con 0x562f6e906e40 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec42447080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562f6e907890 con 0x562f6e850d70 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53534/0 (socket says 172.21.15.43:53534) 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 -- 172.21.15.43:0/2109264834 learned_addr learned my addr 172.21.15.43:0/2109264834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3b400640 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e909100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e850d70 0x562f6e922180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 -- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e850d70 msgr2=0x562f6e922180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e850d70 0x562f6e922180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 -- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 msgr2=0x562f6e909100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e909100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 -- 172.21.15.43:0/2109264834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6e777c10 con 0x562f6e87fcb0 2024-09-17T05:33:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec3a000640 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fec2800a470 tx=0x7fec2800a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72240be5f083dfcf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec38c00640 1 -- 172.21.15.43:0/2109264834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2800b930 con 0x562f6e87fcb0 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec38c00640 1 -- 172.21.15.43:0/2109264834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec2800f030 con 0x562f6e87fcb0 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.570+0000 7fec38c00640 1 -- 172.21.15.43:0/2109264834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28014420 con 0x562f6e87fcb0 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- 172.21.15.43:0/2109264834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 msgr2=0x562f6e908bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fec2800a470 tx=0x7fec2800a930 comp rx=0 tx=0).stop 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- 172.21.15.43:0/2109264834 shutdown_connections 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e909100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e87fcb0 0x562f6e908bc0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- 172.21.15.43:0/2109264834 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e850d70 0x562f6e922180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- 172.21.15.43:0/2109264834 >> 172.21.15.43:0/2109264834 conn(0x562f6e83cd80 msgr2=0x562f6e898ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- 172.21.15.43:0/2109264834 shutdown_connections 2024-09-17T05:33:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- 172.21.15.43:0/2109264834 wait complete. 2024-09-17T05:33:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 Processor -- start 2024-09-17T05:33:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- start start 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e87fcb0 0x562f6e84ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e925c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562f6e892080 con 0x562f6e850d70 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562f6e892320 con 0x562f6e906e40 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec42447080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x562f6e8925c0 con 0x562f6e87fcb0 2024-09-17T05:33:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53546/0 (socket says 172.21.15.43:53546) 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 learned_addr learned my addr 172.21.15.43:0/2003967443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e87fcb0 0x562f6e84ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e925c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e87fcb0 msgr2=0x562f6e84ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x562f6e87fcb0 0x562f6e84ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 msgr2=0x562f6e925c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e925c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6e920990 con 0x562f6e850d70 2024-09-17T05:33:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562f6e906e40 0x562f6e925c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.574+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fec2c01c160 tx=0x7fec2c0a7840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 -- 172.21.15.43:0/2003967443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0b09c0 con 0x562f6e850d70 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f6e920c30 con 0x562f6e850d70 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 -- 172.21.15.43:0/2003967443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec2c0b0b60 con 0x562f6e850d70 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 -- 172.21.15.43:0/2003967443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0bc400 con 0x562f6e850d70 2024-09-17T05:33:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6e921170 con 0x562f6e850d70 2024-09-17T05:33:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 -- 172.21.15.43:0/2003967443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fec2c0bb3a0 con 0x562f6e850d70 2024-09-17T05:33:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 0x7fec1c074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec33400640 1 -- 172.21.15.43:0/2003967443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7fec2c131580 con 0x562f6e850d70 2024-09-17T05:33:22.584 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.578+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 0x7fec1c074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562f6e84efb0 0x562f6e91bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9095f0 con 0x562f6e84efb0 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562f6e84efb0 0x562f6e91bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 0x7fec1c074120 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fec28010040 tx=0x7fec28016040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562f6e84efb0 0x562f6e91bb90 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v122'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.582+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e91c0d0 con 0x562f6e84efb0 2024-09-17T05:33:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.586+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v122'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.586+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e91c4a0 con 0x562f6e84efb0 2024-09-17T05:33:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.586+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v122'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.591 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:22.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.586+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e91c840 con 0x562f6e84efb0 2024-09-17T05:33:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c91b0 con 0x562f6e84efb0 2024-09-17T05:33:22.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c81a0 con 0x562f6e84efb0 2024-09-17T05:33:22.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.590+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8540 con 0x562f6e84efb0 2024-09-17T05:33:22.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.594+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v122'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.594+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9c8930 con 0x562f6e84efb0 2024-09-17T05:33:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.594+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v122'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.594+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8930 con 0x562f6e84efb0 2024-09-17T05:33:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.598+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v122'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.600 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.598+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.598+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.598+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.602+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.610 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:22.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.606+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.610+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.610+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c88e0 con 0x562f6e84efb0 2024-09-17T05:33:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.610+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.610+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8cb0 con 0x562f6e84efb0 2024-09-17T05:33:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.614+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.614+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8cb0 con 0x562f6e84efb0 2024-09-17T05:33:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.614+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v122'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.614+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.614+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v122'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.618+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.618+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v122'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.621 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.618+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.618+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.618+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.622+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.622+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.622+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.622+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.622+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v122'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.626+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.626+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v122'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.626+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.626+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v122'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.631 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:22.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.626+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.630+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.630+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.630+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.630+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e84efb0 2024-09-17T05:33:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562f6e9c9a60 0x562f6e9cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9c8b60 con 0x562f6e9c9a60 2024-09-17T05:33:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562f6e9c9a60 0x562f6e9cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562f6e9c9a60 0x562f6e9cbe80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'726 uv708 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.634+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9cc3c0 con 0x562f6e9c9a60 2024-09-17T05:33:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'727 uv727 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9cc760 con 0x562f6e9c9a60 2024-09-17T05:33:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.641 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccfb0 con 0x562f6e9c9a60 2024-09-17T05:33:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.638+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9cd350 con 0x562f6e9c9a60 2024-09-17T05:33:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.642+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.642+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9cd6f0 con 0x562f6e9c9a60 2024-09-17T05:33:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.642+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v122'731 uv713 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.642+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9cda90 con 0x562f6e9c9a60 2024-09-17T05:33:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v122'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9cda90 con 0x562f6e9c9a60 2024-09-17T05:33:22.649 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:22.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v122'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.646+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.650+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.650+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.650+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v122'736 uv718 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.650+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v122'737 uv737 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.657 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:22.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v122'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v122'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.654+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.658+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.658+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.658+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v122'741 uv724 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.658+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v122'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.665 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v122'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.662+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9ccbe0 con 0x562f6e9c9a60 2024-09-17T05:33:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9c9a60 2024-09-17T05:33:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562f6e9ce450 0x562f6e9d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d0db0 con 0x562f6e9ce450 2024-09-17T05:33:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562f6e9ce450 0x562f6e9d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.666+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562f6e9ce450 0x562f6e9d0870 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v122'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1150 con 0x562f6e9ce450 2024-09-17T05:33:22.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v122'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1630 con 0x562f6e9ce450 2024-09-17T05:33:22.675 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:22.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v122'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.670+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d2ee0 con 0x562f6e9ce450 2024-09-17T05:33:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d3280 con 0x562f6e9ce450 2024-09-17T05:33:22.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e9ce450 2024-09-17T05:33:22.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.674+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e84efb0 2024-09-17T05:33:22.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.678+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'140 uv139 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.678+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e84efb0 2024-09-17T05:33:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.678+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'141 uv141 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.678+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e84efb0 2024-09-17T05:33:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'142 uv142 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.684 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e84efb0 2024-09-17T05:33:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'143 uv143 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 0x562f6e9d5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e9d2820 2024-09-17T05:33:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 0x562f6e9d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.682+0000 7fec3b400640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 0x562f6e9d5290 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v122'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e9d2820 2024-09-17T05:33:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v122'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ed0 con 0x562f6e9d2820 2024-09-17T05:33:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v122'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.690 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.686+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d67e0 con 0x562f6e9d2820 2024-09-17T05:33:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d67e0 con 0x562f6e9d2820 2024-09-17T05:33:22.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d57d0 con 0x562f6e9d2820 2024-09-17T05:33:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562f6e9d5ce0 0x562f6e9d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d86d0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562f6e9d5ce0 0x562f6e9d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.690+0000 7fec3aa00640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562f6e9d5ce0 0x562f6e9d8100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.694+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v122'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.694+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.694+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v122'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.694+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.699 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:22.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v122'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d9ab0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d9ab0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.698+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v122'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v122'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.702+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.707 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v122'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d8aa0 con 0x562f6e9d5ce0 2024-09-17T05:33:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec3aa00640 1 -- 172.21.15.43:0/2003967443 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2c0c4070 con 0x562f6e9d5ce0 2024-09-17T05:33:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f6e9d9120 0x562f6e9db540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f6e9d9120 0x562f6e9db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.706+0000 7fec3a000640 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f6e9d9120 0x562f6e9db540 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.710+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v122'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.710+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9d9120 2024-09-17T05:33:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.710+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v122'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.710+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9d9120 2024-09-17T05:33:22.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v122'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.715 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9d9120 2024-09-17T05:33:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9d9120 2024-09-17T05:33:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.714+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e84efb0 2024-09-17T05:33:22.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v122'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e84efb0 2024-09-17T05:33:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v122'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e84efb0 2024-09-17T05:33:22.722 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v122'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.718+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dcfa0 con 0x562f6e84efb0 2024-09-17T05:33:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dcfa0 con 0x562f6e84efb0 2024-09-17T05:33:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e84efb0 2024-09-17T05:33:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e84efb0 2024-09-17T05:33:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.722+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9ce450 2024-09-17T05:33:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v122'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9ce450 2024-09-17T05:33:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v122'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dbf90 con 0x562f6e9ce450 2024-09-17T05:33:22.731 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:22.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v122'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.726+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc350 con 0x562f6e9ce450 2024-09-17T05:33:22.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.730+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.730+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc350 con 0x562f6e9ce450 2024-09-17T05:33:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.730+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.730+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc350 con 0x562f6e9ce450 2024-09-17T05:33:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.734+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v122'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.734+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9dc350 con 0x562f6e9ce450 2024-09-17T05:33:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.734+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v122'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.734+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc030 con 0x562f6e9ce450 2024-09-17T05:33:22.739 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.734+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v122'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.738+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc030 con 0x562f6e9ce450 2024-09-17T05:33:22.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.738+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.738+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1b30 con 0x562f6e9ce450 2024-09-17T05:33:22.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.738+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9ce450 2024-09-17T05:33:22.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.738+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1af0 con 0x562f6e9d2820 2024-09-17T05:33:22.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.742+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v122'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.742+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1af0 con 0x562f6e9d2820 2024-09-17T05:33:22.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.742+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.742+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1af0 con 0x562f6e9d2820 2024-09-17T05:33:22.748 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:22.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.746+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v122'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.746+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1af0 con 0x562f6e9d2820 2024-09-17T05:33:22.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.746+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.746+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v122'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v122'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.754 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:22.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v122'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.750+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d2820 2024-09-17T05:33:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec34002040 con 0x562f6e9d2820 2024-09-17T05:33:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.754+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v122'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v122'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.762 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v122'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.758+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.762+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.762+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.762+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.762+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v122'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v122'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.770 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:22.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v122'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.766+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v122'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.770+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v122'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v122'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.777 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:22.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.774+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9d1ad0 con 0x562f6e9d9120 2024-09-17T05:33:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x562f6e9dc8a0 con 0x562f6e9d9120 2024-09-17T05:33:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec3a000640 1 -- 172.21.15.43:0/2003967443 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec2800bbc0 con 0x562f6e9d9120 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562f6e9c9a60 msgr2=0x562f6e9cbe80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x562f6e9c9a60 0x562f6e9cbe80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562f6e9d5ce0 msgr2=0x562f6e9d8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x562f6e9d5ce0 0x562f6e9d8100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f6e9d9120 msgr2=0x562f6e9db540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x562f6e9d9120 0x562f6e9db540 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562f6e9ce450 msgr2=0x562f6e9d0870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x562f6e9ce450 0x562f6e9d0870 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 reap_dead start 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562f6e84efb0 msgr2=0x562f6e91bb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x562f6e84efb0 0x562f6e91bb90 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec3b400640 1 -- 172.21.15.43:0/2003967443 reap_dead start 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 msgr2=0x562f6e9d5290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 0x562f6e9d5290 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 msgr2=0x7fec1c074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 0x7fec1c074120 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fec28010040 tx=0x7fec28016040 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 msgr2=0x562f6e898060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fec2c01c160 tx=0x7fec2c0a7840 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 shutdown_connections 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x562f6e9d2820 0x562f6e9d5290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fec1c071c10 0x7fec1c074120 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 --2- 172.21.15.43:0/2003967443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562f6e850d70 0x562f6e898060 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 >> 172.21.15.43:0/2003967443 conn(0x562f6e83cd80 msgr2=0x562f6e925310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 shutdown_connections 2024-09-17T05:33:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.778+0000 7fec42447080 1 -- 172.21.15.43:0/2003967443 wait complete. 2024-09-17T05:33:22.788 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:22.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 Processor -- start 2024-09-17T05:33:22.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- start start 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 0x557b776ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b77821700 0x557b77821b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b77664f40 con 0x557b776ff560 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557b77671d00 con 0x557b77821700 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557b77606a00 con 0x557b7776e5a0 2024-09-17T05:33:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ace800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 0x557b776ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ace800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53566/0 (socket says 172.21.15.43:53566) 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ace800640 1 -- 172.21.15.43:0/1266380826 learned_addr learned my addr 172.21.15.43:0/1266380826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acfc00640 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b77821700 0x557b77821b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 -- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b77821700 msgr2=0x557b77821b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b77821700 0x557b77821b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 -- 172.21.15.43:0/1266380826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 msgr2=0x557b778211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 -- 172.21.15.43:0/1266380826 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7776eae0 con 0x557b7776e5a0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ace800640 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acf200640 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 0x557b776ff020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004c20 tx=0x7f9ac00107b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7e91d59f3874e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acd400640 1 -- 172.21.15.43:0/1266380826 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0011650 con 0x557b7776e5a0 2024-09-17T05:33:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acd400640 1 -- 172.21.15.43:0/1266380826 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ac00117f0 con 0x557b7776e5a0 2024-09-17T05:33:22.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9acd400640 1 -- 172.21.15.43:0/1266380826 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0011c20 con 0x557b7776e5a0 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 msgr2=0x557b776ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 0x557b776ff020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004c20 tx=0x7f9ac00107b0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/1266380826 shutdown_connections 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b77821700 0x557b77821b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b776ff560 0x557b778211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/1266380826 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b7776e5a0 0x557b776ff020 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/1266380826 >> 172.21.15.43:0/1266380826 conn(0x557b776a4d80 msgr2=0x557b77701de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/1266380826 shutdown_connections 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.890+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/1266380826 wait complete. 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 Processor -- start 2024-09-17T05:33:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 -- start start 2024-09-17T05:33:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 0x557b77767f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b7776e5a0 0x557b7778e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b77821700 0x557b7778ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557b77822050 con 0x557b77821700 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557b7778f630 con 0x557b7776e5a0 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ad5f7a080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x557b77815890 con 0x557b776ff560 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 0x557b77767f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b77821700 0x557b7778ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 0x557b77767f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:52356/0 (socket says 172.21.15.43:52356) 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ace800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b7776e5a0 0x557b7778e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 -- 172.21.15.43:0/826471601 learned_addr learned my addr 172.21.15.43:0/826471601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 -- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b7776e5a0 msgr2=0x557b7778e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557b7776e5a0 0x557b7778e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 -- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b77821700 msgr2=0x557b7778ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557b77821700 0x557b7778ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 -- 172.21.15.43:0/826471601 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b77815350 con 0x557b776ff560 2024-09-17T05:33:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9acf200640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 0x557b77767f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012e60 tx=0x7f9ac0012e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ac7e00640 1 -- 172.21.15.43:0/826471601 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000e480 con 0x557b776ff560 2024-09-17T05:33:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ac7e00640 1 -- 172.21.15.43:0/826471601 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ac000e620 con 0x557b776ff560 2024-09-17T05:33:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.894+0000 7f9ac7e00640 1 -- 172.21.15.43:0/826471601 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000ea70 con 0x557b776ff560 2024-09-17T05:33:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x557b778155f0 con 0x557b776ff560 2024-09-17T05:33:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x557b7776c010 con 0x557b776ff560 2024-09-17T05:33:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ac7e00640 1 -- 172.21.15.43:0/826471601 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ac000ec10 con 0x557b776ff560 2024-09-17T05:33:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ac7e00640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9ab0071c10 0x7f9ab0074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ac7e00640 1 -- 172.21.15.43:0/826471601 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f9ac0097ef0 con 0x557b776ff560 2024-09-17T05:33:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ace800640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9ab0071c10 0x7f9ab0074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.898+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557b7776d4c0 0x557b77768b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x557b77769080 con 0x557b7776d4c0 2024-09-17T05:33:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9acfc00640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557b7776d4c0 0x557b77768b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ace800640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9ab0071c10 0x7f9ab0074120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9abc004fd0 tx=0x7f9abc004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9acfc00640 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557b7776d4c0 0x557b77768b40 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9acfc00640 1 -- 172.21.15.43:0/826471601 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v122'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9ac8001880 con 0x557b7776d4c0 2024-09-17T05:33:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557b7776d4c0 msgr2=0x557b77768b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x557b7776d4c0 0x557b77768b40 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9ab0071c10 msgr2=0x7f9ab0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9ab0071c10 0x7f9ab0074120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9abc004fd0 tx=0x7f9abc004890 comp rx=0 tx=0).stop 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 msgr2=0x557b77767f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 --2- 172.21.15.43:0/826471601 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x557b776ff560 0x557b77767f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012e60 tx=0x7f9ac0012e90 comp rx=0 tx=0).stop 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9acfc00640 1 -- 172.21.15.43:0/826471601 reap_dead start 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 shutdown_connections 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 >> 172.21.15.43:0/826471601 conn(0x557b776a4d80 msgr2=0x557b77701b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 shutdown_connections 2024-09-17T05:33:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:22.902+0000 7f9ad5f7a080 1 -- 172.21.15.43:0/826471601 wait complete. 2024-09-17T05:33:22.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 -- 172.21.15.43:0/293198117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a80155d70 msgr2=0x7f5a8015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 --2- 172.21.15.43:0/293198117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a80155d70 0x7f5a8015c7e0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009840 tx=0x7f5a7401c790 comp rx=0 tx=0).stop 2024-09-17T05:33:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 -- 172.21.15.43:0/293198117 shutdown_connections 2024-09-17T05:33:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 --2- 172.21.15.43:0/293198117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a80155d70 0x7f5a8015c7e0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 -- 172.21.15.43:0/293198117 >> 172.21.15.43:0/293198117 conn(0x7f5a800b23d0 msgr2=0x7f5a800d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 -- 172.21.15.43:0/293198117 shutdown_connections 2024-09-17T05:33:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 -- 172.21.15.43:0/293198117 wait complete. 2024-09-17T05:33:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.070+0000 7f5a88800640 1 Processor -- start 2024-09-17T05:33:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- start start 2024-09-17T05:33:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 0x7f5a800c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 0x7f5a801fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80169e10 con 0x7f5a8006f110 2024-09-17T05:33:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80169c90 con 0x7f5a8009d960 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5a80169f90 con 0x7f5a80155d70 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53574/0 (socket says 172.21.15.43:53574) 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 -- 172.21.15.43:0/4183513674 learned_addr learned my addr 172.21.15.43:0/4183513674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a86800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 0x7f5a801fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85400640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 0x7f5a800c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 -- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 msgr2=0x7f5a800c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 0x7f5a800c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 -- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 msgr2=0x7f5a801fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 0x7f5a801fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 -- 172.21.15.43:0/4183513674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8009d220 con 0x7f5a8006f110 2024-09-17T05:33:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85400640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 0x7f5a800c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a86800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 0x7f5a801fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a85e00640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009790 tx=0x7f5a740a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740bb070 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a74004470 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8009d410 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740b2040 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a800cb8e0 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.074+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8015c7e0 con 0x7f5a8006f110 2024-09-17T05:33:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.078+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a74005290 con 0x7f5a8006f110 2024-09-17T05:33:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.078+0000 7f5a7a000640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 0x7f5a64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.078+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f5a7412e890 con 0x7f5a8006f110 2024-09-17T05:33:23.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.078+0000 7f5a85400640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 0x7f5a64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.078+0000 7f5a85400640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 0x7f5a64075710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5a700046c0 tx=0x7f5a70009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.082+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a74016410 con 0x7f5a8006f110 2024-09-17T05:33:23.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.294+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5a800008e0 con 0x7f5a8006f110 2024-09-17T05:33:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.486+0000 7f5a7a000640 1 -- 172.21.15.43:0/4183513674 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v289) ==== 162+0+0 (secure 0 0 0) 0x7f5a740f8d80 con 0x7f5a8006f110 2024-09-17T05:33:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 msgr2=0x7f5a64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 0x7f5a64075710 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5a700046c0 tx=0x7f5a70009040 comp rx=0 tx=0).stop 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 msgr2=0x7f5a8009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009790 tx=0x7f5a740a55e0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 shutdown_connections 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5a64073200 0x7f5a64075710 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a8009d960 0x7f5a801fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a80155d70 0x7f5a800c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 --2- 172.21.15.43:0/4183513674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8006f110 0x7f5a8009cce0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 >> 172.21.15.43:0/4183513674 conn(0x7f5a800b23d0 msgr2=0x7f5a8015b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 shutdown_connections 2024-09-17T05:33:23.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:23.490+0000 7f5a88800640 1 -- 172.21.15.43:0/4183513674 wait complete. 2024-09-17T05:33:23.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 -- 172.21.15.43:0/821293275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da8003af0 msgr2=0x7f2da8003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 --2- 172.21.15.43:0/821293275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2da8003af0 0x7f2da8003f60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c470 tx=0x7f2db401cfa0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dc3e00640 1 -- 172.21.15.43:0/821293275 reap_dead start 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 -- 172.21.15.43:0/821293275 shutdown_connections 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 -- 172.21.15.43:0/821293275 >> 172.21.15.43:0/821293275 conn(0x7f2dc40b23d0 msgr2=0x7f2dc40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 -- 172.21.15.43:0/821293275 shutdown_connections 2024-09-17T05:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.078+0000 7f2dca200640 1 -- 172.21.15.43:0/821293275 wait complete. 2024-09-17T05:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 Processor -- start 2024-09-17T05:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 -- start start 2024-09-17T05:33:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 0x7f2dc41fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 0x7f2dc41fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4167ec0 con 0x7f2dc4159610 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4167d40 con 0x7f2da8003af0 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dca200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc4168040 con 0x7f2dc41f9b50 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59448/0 (socket says 172.21.15.43:59448) 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 -- 172.21.15.43:0/2494105783 learned_addr learned my addr 172.21.15.43:0/2494105783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3400640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 0x7f2dc41fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dbbe00640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 0x7f2dc41fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 -- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 msgr2=0x7f2dc41fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 0x7f2dc41fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 -- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 msgr2=0x7f2dc41fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 0x7f2dc41fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3e00640 1 -- 172.21.15.43:0/2494105783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc4158100 con 0x7f2dc4159610 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dc3400640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 0x7f2dc41fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.082+0000 7f2dbbe00640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 0x7f2dc41fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dc3e00640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40d0a30 tx=0x7f2dbc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc004b30 con 0x7f2dc4159610 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dbc004cd0 con 0x7f2dc4159610 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc003be0 con 0x7f2dc4159610 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc4158570 con 0x7f2dc4159610 2024-09-17T05:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc4158b30 con 0x7f2dc4159610 2024-09-17T05:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc40700a0 con 0x7f2dc4159610 2024-09-17T05:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2dbc004030 con 0x7f2dc4159610 2024-09-17T05:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 0x7f2da0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f2dbc095310 con 0x7f2dc4159610 2024-09-17T05:33:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.086+0000 7f2dc3400640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 0x7f2da0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.090+0000 7f2dc3400640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 0x7f2da0075710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2db40ac000 tx=0x7f2db4019de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.090+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2dbc05f7c0 con 0x7f2dc4159610 2024-09-17T05:33:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.302+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dc40ced60 con 0x7f2dc4159610 2024-09-17T05:33:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.302+0000 7f2dbb400640 1 -- 172.21.15.43:0/2494105783 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 290 v290) ==== 95+0+5218 (secure 0 0 0) 0x7f2dbc05f0c0 con 0x7f2dc4159610 2024-09-17T05:33:24.307 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:24.307 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":290,"btime":"2024-09-17T05:33:23:493654+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":290,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:23.493634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":290,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/1507988053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1507988053},{"type":"v1","addr":"172.21.15.43:6835","nonce":1507988053}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 290 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 msgr2=0x7f2da0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 0x7f2da0075710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2db40ac000 tx=0x7f2db4019de0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 msgr2=0x7f2dc4159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40d0a30 tx=0x7f2dbc00f3c0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 shutdown_connections 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2da0073200 0x7f2da0075710 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dc4159610 0x7f2dc4159a20 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2dc41f9b50 0x7f2dc41fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 --2- 172.21.15.43:0/2494105783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da8003af0 0x7f2dc41fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 >> 172.21.15.43:0/2494105783 conn(0x7f2dc40b23d0 msgr2=0x7f2dc40748a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 shutdown_connections 2024-09-17T05:33:24.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:24.306+0000 7f2dca200640 1 -- 172.21.15.43:0/2494105783 wait complete. 2024-09-17T05:33:24.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 290, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:23.493634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9786}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9786': {'gid': 9786, 'name': 'c', 'rank': 0, 'incarnation': 290, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/1507988053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 1507988053}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1507988053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:33:24.329 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9786:up:replay 2024-09-17T05:33:24.329 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:33:27.486 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:27.482+0000 7f6d91000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:29.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:29.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 -- 172.21.15.43:0/2007552930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15bc0099c0 msgr2=0x7f15bc00bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 --2- 172.21.15.43:0/2007552930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15bc0099c0 0x7f15bc00bde0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009e90 tx=0x7f15d001d5f0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 -- 172.21.15.43:0/2007552930 shutdown_connections 2024-09-17T05:33:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 --2- 172.21.15.43:0/2007552930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15bc0099c0 0x7f15bc00bde0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 -- 172.21.15.43:0/2007552930 >> 172.21.15.43:0/2007552930 conn(0x7f15d80b23d0 msgr2=0x7f15d80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 -- 172.21.15.43:0/2007552930 shutdown_connections 2024-09-17T05:33:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.902+0000 7f15dde00640 1 -- 172.21.15.43:0/2007552930 wait complete. 2024-09-17T05:33:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 Processor -- start 2024-09-17T05:33:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- start start 2024-09-17T05:33:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 0x7f15d80bde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d815a8a0 0x7f15d8153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b4f70 con 0x7f15d80be390 2024-09-17T05:33:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b4df0 con 0x7f15d815a8a0 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80b50f0 con 0x7f15bc0099c0 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 0x7f15d80bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 0x7f15d80bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:43956/0 (socket says 172.21.15.43:43956) 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59458/0 (socket says 172.21.15.43:59458) 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d6a00640 1 -- 172.21.15.43:0/878903778 learned_addr learned my addr 172.21.15.43:0/878903778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15cfe00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d815a8a0 0x7f15d8153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 -- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 msgr2=0x7f15d80bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 0x7f15d80bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 -- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d815a8a0 msgr2=0x7f15d8153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d815a8a0 0x7f15d8153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 -- 172.21.15.43:0/878903778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d8159de0 con 0x7f15d80be390 2024-09-17T05:33:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15d7400640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b3640 tx=0x7f15d0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00af040 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d00aa9e0 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00aadf0 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d80beb80 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.906+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d815a380 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.910+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d80be8f0 con 0x7f15d80be390 2024-09-17T05:33:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.910+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f15d00bd060 con 0x7f15d80be390 2024-09-17T05:33:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.910+0000 7f15cf400640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 0x7f15b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.910+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f15d012f270 con 0x7f15d80be390 2024-09-17T05:33:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.910+0000 7f15d6a00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 0x7f15b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.914+0000 7f15d6a00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 0x7f15b4075710 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004610 tx=0x7f15c8006e20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:29.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:29.914+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15d0017300 con 0x7f15d80be390 2024-09-17T05:33:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.126+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d80733c0 con 0x7f15d80be390 2024-09-17T05:33:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15cf400640 1 -- 172.21.15.43:0/878903778 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) ==== 95+0+5225 (secure 0 0 0) 0x7f15d0100e20 con 0x7f15d80be390 2024-09-17T05:33:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:30.132 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":293,"btime":"2024-09-17T05:33:26:515292+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:26.515281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":290,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/1507988053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1507988053},{"type":"v1","addr":"172.21.15.43:6835","nonce":1507988053}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9786,"qdb_cluster":[9786]},"id":6}]} 2024-09-17T05:33:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 293 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 msgr2=0x7f15b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 0x7f15b4075710 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f15c8004610 tx=0x7f15c8006e20 comp rx=0 tx=0).stop 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 msgr2=0x7f15d8153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b3640 tx=0x7f15d0009a90 comp rx=0 tx=0).stop 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 shutdown_connections 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f15b4073200 0x7f15b4075710 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d815a8a0 0x7f15d8153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80be390 0x7f15d8153a70 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 --2- 172.21.15.43:0/878903778 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f15bc0099c0 0x7f15d80bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 >> 172.21.15.43:0/878903778 conn(0x7f15d80b23d0 msgr2=0x7f15d80cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 shutdown_connections 2024-09-17T05:33:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.130+0000 7f15dde00640 1 -- 172.21.15.43:0/878903778 wait complete. 2024-09-17T05:33:30.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 293, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:26.515281+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9786}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9786': {'gid': 9786, 'name': 'c', 'rank': 0, 'incarnation': 290, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/1507988053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 1507988053}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1507988053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9786, 'qdb_cluster': [9786]} 2024-09-17T05:33:30.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:33:30.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:33:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 -- 172.21.15.43:0/737212137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80099c0 msgr2=0x7f65b800bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 --2- 172.21.15.43:0/737212137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80099c0 0x7f65b800bde0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f65d415bc80 tx=0x7f65cc01f390 comp rx=0 tx=0).stop 2024-09-17T05:33:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 -- 172.21.15.43:0/737212137 shutdown_connections 2024-09-17T05:33:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 --2- 172.21.15.43:0/737212137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80099c0 0x7f65b800bde0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 -- 172.21.15.43:0/737212137 >> 172.21.15.43:0/737212137 conn(0x7f65d40b23d0 msgr2=0x7f65d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 -- 172.21.15.43:0/737212137 shutdown_connections 2024-09-17T05:33:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.310+0000 7f65d9400640 1 -- 172.21.15.43:0/737212137 wait complete. 2024-09-17T05:33:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 Processor -- start 2024-09-17T05:33:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 -- start start 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65d40740e0 0x7f65d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d40cfc90 0x7f65d40d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65d415cb70 con 0x7f65d40cfc90 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65d415c9f0 con 0x7f65d40740e0 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d9400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f65d40b33e0 con 0x7f65b80099c0 2024-09-17T05:33:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d40cfc90 0x7f65d40d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65d40740e0 0x7f65d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:43970/0 (socket says 172.21.15.43:43970) 2024-09-17T05:33:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.314+0000 7f65d2800640 1 -- 172.21.15.43:0/2082831814 learned_addr learned my addr 172.21.15.43:0/2082831814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 -- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65d40740e0 msgr2=0x7f65d40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65d40740e0 0x7f65d40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 -- 172.21.15.43:0/2082831814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d40cfc90 msgr2=0x7f65d40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d40cfc90 0x7f65d40d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 -- 172.21.15.43:0/2082831814 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d4074a30 con 0x7f65b80099c0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d2800640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006370 tx=0x7f65c40039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65caa00640 1 -- 172.21.15.43:0/2082831814 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4015070 con 0x7f65b80099c0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65caa00640 1 -- 172.21.15.43:0/2082831814 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c4011670 con 0x7f65b80099c0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65caa00640 1 -- 172.21.15.43:0/2082831814 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4011ac0 con 0x7f65b80099c0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d40c9ea0 con 0x7f65b80099c0 2024-09-17T05:33:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d40cf1b0 con 0x7f65b80099c0 2024-09-17T05:33:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.318+0000 7f65caa00640 1 -- 172.21.15.43:0/2082831814 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f65c4018070 con 0x7f65b80099c0 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65caa00640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 0x7f65b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65caa00640 1 -- 172.21.15.43:0/2082831814 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f65c40957c0 con 0x7f65b80099c0 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 Processor -- start 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 -- start start 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65cbe00640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 0x7f65b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65cbe00640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 0x7f65b0075710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f65d40c6cc0 tx=0x7f65bc0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d1ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556711bf3410 con 0x556711bb6ca0 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556711b30b40 con 0x556711bcd950 2024-09-17T05:33:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556711b1ff70 con 0x5567117dd460 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d1ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59480/0 (socket says 172.21.15.43:59480) 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 -- 172.21.15.43:0/1609230884 learned_addr learned my addr 172.21.15.43:0/1609230884 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d2800640 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 -- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 msgr2=0x556711d1e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 -- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 msgr2=0x556711d1ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d1ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 -- 172.21.15.43:0/1609230884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556711ae75b0 con 0x556711bb6ca0 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65d3200640 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f65cc01dc60 tx=0x7f65b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c584bbf604909234 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65c3000640 1 -- 172.21.15.43:0/1609230884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0c1070 con 0x556711bb6ca0 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65c3000640 1 -- 172.21.15.43:0/1609230884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65cc003980 con 0x556711bb6ca0 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.322+0000 7f65c3000640 1 -- 172.21.15.43:0/1609230884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc003d90 con 0x556711bb6ca0 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- 172.21.15.43:0/1609230884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 msgr2=0x556711d1f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f65cc01dc60 tx=0x7f65b8005840 comp rx=0 tx=0).stop 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- 172.21.15.43:0/1609230884 shutdown_connections 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d1ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1f120 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1609230884 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- 172.21.15.43:0/1609230884 >> 172.21.15.43:0/1609230884 conn(0x556711ca3d90 msgr2=0x556711cd3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- 172.21.15.43:0/1609230884 shutdown_connections 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- 172.21.15.43:0/1609230884 wait complete. 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- start start 2024-09-17T05:33:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 Processor -- start 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d25920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bb6ca0 0x556711d25e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556711aba5f0 con 0x556711bcd950 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556711b23d80 con 0x556711bb6ca0 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556711ac6520 con 0x5567117dd460 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65e09d1000 10 client.0 _ll_register_callbacks cb 0x556711aca830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59494/0 (socket says 172.21.15.43:59494) 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65d3200640 1 -- 172.21.15.43:0/123235948 learned_addr learned my addr 172.21.15.43:0/123235948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65d2800640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d25920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.326+0000 7f65cbe00640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bb6ca0 0x556711d25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 -- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 msgr2=0x556711d25920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d25920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 -- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bb6ca0 msgr2=0x556711d25e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bb6ca0 0x556711d25e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 -- 172.21.15.43:0/123235948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556711abe190 con 0x556711bcd950 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d3200640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0bf040 tx=0x7f65cc003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee83709c34a683f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0cc060 con 0x556711bcd950 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556711ab8110 con 0x556711bcd950 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f65cc006170 con 0x556711bcd950 2024-09-17T05:33:30.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=293) ==== 24+0+0 (secure 0 0 0) 0x7f65cc0c1040 con 0x556711bcd950 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 10 client.9843 fetch_fsmap learned FSMap version 293 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=293,monmap=2+}) -- 0x556711ba4f10 con 0x556711bcd950 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556711ab8110 con 0x556711bcd950 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 293) ==== 3502+0+0 (secure 0 0 0) 0x7f65cc006370 con 0x556711bcd950 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 10 client.9843 fetch_fsmap finished waiting for FSMap version 293 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 10 client.9843 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 20 client.9843 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117c27d0 0x556711d2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 4 client.9843 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 --> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556711b32a90 con 0x5567117c27d0 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d2800640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117c27d0 0x556711d2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65d2800640 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117c27d0 0x556711d2fd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bef92892b3f23be server_cookie=6d3f374b6d9356a1 in_seq=0 out_seq=0 2024-09-17T05:33:30.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 10 client.9843 ms_handle_connect on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 1 -- 172.21.15.43:0/123235948 <== mds.0 v2:172.21.15.43:6833/1507988053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556711b32a90 con 0x5567117c27d0 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 10 client.9843 handle_command_reply: tid=0 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 client.9843 shutdown 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 msgr2=0x556711d263a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0bf040 tx=0x7f65cc003d10 comp rx=0 tx=0).stop 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 shutdown_connections 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117c27d0 0x556711d2fd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bb6ca0 0x556711d25e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bcd950 0x556711d263a0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 --2- 172.21.15.43:0/123235948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d25920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65b7400640 0 client.9843 ms_handle_reset on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 >> 172.21.15.43:0/123235948 conn(0x556711ca3d90 msgr2=0x556711cda440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 shutdown_connections 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 1 -- 172.21.15.43:0/123235948 wait complete. 2024-09-17T05:33:30.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.330+0000 7f65e09d1000 20 client.9843 trim_cache size 0 max 16384 2024-09-17T05:33:30.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.350+0000 7f65e09d1000 1 Processor -- start 2024-09-17T05:33:30.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.350+0000 7f65e09d1000 1 -- start start 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711cd2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4204180 con 0x556711bb6ca0 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65d42031c0 con 0x556711bcd950 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f65d4202520 con 0x5567117dd460 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59504/0 (socket says 172.21.15.43:59504) 2024-09-17T05:33:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 -- 172.21.15.43:0/792907490 learned_addr learned my addr 172.21.15.43:0/792907490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65d2800640 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65d3200640 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711cd2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 -- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 msgr2=0x556711d1d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 -- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 msgr2=0x556711cd2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711cd2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 -- 172.21.15.43:0/792907490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556711b7bdb0 con 0x556711bb6ca0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65cbe00640 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0041f0 tx=0x7f65bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb12e0ecbc911897 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65c2600640 1 -- 172.21.15.43:0/792907490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc031070 con 0x556711bb6ca0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65c2600640 1 -- 172.21.15.43:0/792907490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65bc0043a0 con 0x556711bb6ca0 2024-09-17T05:33:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65c2600640 1 -- 172.21.15.43:0/792907490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc003c80 con 0x556711bb6ca0 2024-09-17T05:33:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- 172.21.15.43:0/792907490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 msgr2=0x556711d1db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0041f0 tx=0x7f65bc01a000 comp rx=0 tx=0).stop 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- 172.21.15.43:0/792907490 shutdown_connections 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711cd2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711d1db70 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 --2- 172.21.15.43:0/792907490 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711d1d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- 172.21.15.43:0/792907490 >> 172.21.15.43:0/792907490 conn(0x556711ca3d90 msgr2=0x556711ce00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- 172.21.15.43:0/792907490 shutdown_connections 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- 172.21.15.43:0/792907490 wait complete. 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 -- start start 2024-09-17T05:33:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.354+0000 7f65e09d1000 1 Processor -- start 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711cd5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556711ac6520 con 0x556711bb6ca0 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4202b70 con 0x556711bcd950 2024-09-17T05:33:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556711aba5f0 con 0x5567117dd460 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 10 client.0 _ll_register_callbacks cb 0x7f65cc0bf070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711cd5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59510/0 (socket says 172.21.15.43:59510) 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 -- 172.21.15.43:0/1876536875 learned_addr learned my addr 172.21.15.43:0/1876536875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65d2800640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711cd5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:44018/0 (socket says 172.21.15.43:44018) 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65d3200640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 -- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 msgr2=0x556711cd5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711cd5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 -- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 msgr2=0x556711d50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d50220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 -- 172.21.15.43:0/1876536875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556711ae6ab0 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65cbe00640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x556711bdc320 tx=0x7f65bc003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b338159c814819c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc031050 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f65bc02c670 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65bc0043a0 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=293) ==== 24+0+0 (secure 0 0 0) 0x7f65bc043050 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 10 client.9849 fetch_fsmap learned FSMap version 293 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=293,monmap=2+}) -- 0x556711ac5070 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65bc0043a0 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 293) ==== 3502+0+0 (secure 0 0 0) 0x7f65bc03e030 con 0x556711bb6ca0 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 10 client.9849 fetch_fsmap finished waiting for FSMap version 293 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 10 client.9849 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 20 client.9849 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117b0830 0x556711d50c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 4 client.9849 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 --> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] -- command(tid 0: {"prefix": "status"}) -- 0x556711b2e540 con 0x5567117b0830 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65d2800640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117b0830 0x556711d50c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65d2800640 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117b0830 0x556711d50c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62df8e00fa74866 server_cookie=6f0c767cfa3df149 in_seq=0 out_seq=0 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.358+0000 7f65b7400640 10 client.9849 ms_handle_connect on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65b7400640 1 -- 172.21.15.43:0/1876536875 <== mds.0 v2:172.21.15.43:6833/1507988053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x556711b2e540 con 0x5567117b0830 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65b7400640 10 client.9849 handle_command_reply: tid=0 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 client.9849 shutdown 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 msgr2=0x556711cd6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x556711bdc320 tx=0x7f65bc003ee0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 shutdown_connections 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x5567117b0830 0x556711d50c10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556711bcd950 0x556711d50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556711bb6ca0 0x556711cd6210 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65b7400640 0 client.9849 ms_handle_reset on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 --2- 172.21.15.43:0/1876536875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5567117dd460 0x556711cd5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 >> 172.21.15.43:0/1876536875 conn(0x556711ca3d90 msgr2=0x556711d2f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 shutdown_connections 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 1 -- 172.21.15.43:0/1876536875 wait complete. 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65e09d1000 20 client.9849 trim_cache size 0 max 16384 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:33:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9786, 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8682235279999997, 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 293, 2024-09-17T05:33:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 122, 2024-09-17T05:33:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 122, 2024-09-17T05:33:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 9.0097406079999995 2024-09-17T05:33:30.371 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:33:30.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 msgr2=0x7f65b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 0x7f65b0075710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f65d40c6cc0 tx=0x7f65bc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 msgr2=0x7f65d40cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006370 tx=0x7f65c40039d0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 shutdown_connections 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f65b0073200 0x7f65b0075710 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65d40740e0 0x7f65d40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d40cfc90 0x7f65d40d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 --2- 172.21.15.43:0/2082831814 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f65b80099c0 0x7f65d40cc410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 >> 172.21.15.43:0/2082831814 conn(0x7f65d40b23d0 msgr2=0x7f65d40ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 shutdown_connections 2024-09-17T05:33:30.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.362+0000 7f65d9400640 1 -- 172.21.15.43:0/2082831814 wait complete. 2024-09-17T05:33:30.384 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:33:30.384 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-09-17T05:33:30.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:33:30.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- 172.21.15.43:0/2716291849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770155d70 msgr2=0x7f477015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 --2- 172.21.15.43:0/2716291849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770155d70 0x7f477015c7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4760009870 tx=0x7f476001c7d0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- 172.21.15.43:0/2716291849 shutdown_connections 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 --2- 172.21.15.43:0/2716291849 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770155d70 0x7f477015c7e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- 172.21.15.43:0/2716291849 >> 172.21.15.43:0/2716291849 conn(0x7f47700b23d0 msgr2=0x7f47700d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- 172.21.15.43:0/2716291849 shutdown_connections 2024-09-17T05:33:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- 172.21.15.43:0/2716291849 wait complete. 2024-09-17T05:33:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 Processor -- start 2024-09-17T05:33:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.542+0000 7f4774c00640 1 -- start start 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4770155d70 0x7f477009c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770154b00 0x7f477009cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4770169df0 con 0x7f477006f110 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4770169c70 con 0x7f4770155d70 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4770169f70 con 0x7f4770154b00 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59520/0 (socket says 172.21.15.43:59520) 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 -- 172.21.15.43:0/2577666892 learned_addr learned my addr 172.21.15.43:0/2577666892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476e800640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770154b00 0x7f477009cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4767e00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4770155d70 0x7f477009c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 -- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770154b00 msgr2=0x7f477009cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770154b00 0x7f477009cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 -- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4770155d70 msgr2=0x7f477009c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4770155d70 0x7f477009c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 -- 172.21.15.43:0/2577666892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477009d8e0 con 0x7f477006f110 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f476de00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f476001ccb0 tx=0x7f4760005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600bb070 con 0x7f477006f110 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4760004330 con 0x7f477006f110 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600b2040 con 0x7f477006f110 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477009d490 con 0x7f477006f110 2024-09-17T05:33:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47700c2350 con 0x7f477006f110 2024-09-17T05:33:30.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.546+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47700c0ba0 con 0x7f477006f110 2024-09-17T05:33:30.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.550+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4760005250 con 0x7f477006f110 2024-09-17T05:33:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.550+0000 7f4766000640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 0x7f474c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.550+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f476012e6a0 con 0x7f477006f110 2024-09-17T05:33:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.550+0000 7f4767e00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 0x7f474c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:30.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.550+0000 7f4767e00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 0x7f474c075710 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f47700c3230 tx=0x7f4758007cd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.554+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4760016450 con 0x7f477006f110 2024-09-17T05:33:30.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.766+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f477015c7e0 con 0x7f477006f110 2024-09-17T05:33:30.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4766000640 1 -- 172.21.15.43:0/2577666892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v293) ==== 53+0+73 (secure 0 0 0) 0x7f47600f84a0 con 0x7f477006f110 2024-09-17T05:33:30.772 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 msgr2=0x7f474c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 0x7f474c075710 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f47700c3230 tx=0x7f4758007cd0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 msgr2=0x7f47701545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f476001ccb0 tx=0x7f4760005d30 comp rx=0 tx=0).stop 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 shutdown_connections 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f474c073200 0x7f474c075710 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4770155d70 0x7f477009c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4770154b00 0x7f477009cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 --2- 172.21.15.43:0/2577666892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477006f110 0x7f47701545c0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 >> 172.21.15.43:0/2577666892 conn(0x7f47700b23d0 msgr2=0x7f47700b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 shutdown_connections 2024-09-17T05:33:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:30.770+0000 7f4774c00640 1 -- 172.21.15.43:0/2577666892 wait complete. 2024-09-17T05:33:30.794 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:33:30.794 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:33:30.794 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:33:30.794 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:33:30.795 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:30.795 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:33:30.795 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:33:30.795 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:33:30.795 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:30.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:30.795 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:33:30.805 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:33:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:33:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:33:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41833sec preferred_lft 41833sec 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.806 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:33:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:30.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:30.809 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:33:30.865 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:33:30.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:30.867 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:33:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:33:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:33:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:33:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:33:30.921 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:33:30.922 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:33:30.922 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:33:31.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:31.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:31.060 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:33:31.115 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:33:31.116 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:33:31.116 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:31.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:31.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:33:31.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:31.179 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:31.227 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:31.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:31.227 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:31.278 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:33:31.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:31.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:31.380 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:31.380 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:31.380 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:31.398 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:31.398 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:31.399 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:31.399 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:31.399 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:33:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:33:31.414 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:33:31.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:31.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:31.437 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:33:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:33:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:client9858 2024-09-17T05:33:31.537 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:31.537 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:31.587 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T05:33:31.588 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-09-17T05:33:31.589 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T05:33:31.589 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-09-17T05:33:31.589 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:33:31.634 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:33:31.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-17T05:33:31.712 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-17T05:33:31.712 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-17T05:33:31.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- 172.21.15.43:0/1947340877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c155d70 msgr2=0x7f607c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 --2- 172.21.15.43:0/1947340877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c155d70 0x7f607c15c7e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f606c009900 tx=0x7f606c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- 172.21.15.43:0/1947340877 shutdown_connections 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 --2- 172.21.15.43:0/1947340877 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c0b3af0 0x7f607c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 --2- 172.21.15.43:0/1947340877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c155d70 0x7f607c15c7e0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- 172.21.15.43:0/1947340877 >> 172.21.15.43:0/1947340877 conn(0x7f607c0b23d0 msgr2=0x7f607c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- 172.21.15.43:0/1947340877 shutdown_connections 2024-09-17T05:33:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- 172.21.15.43:0/1947340877 wait complete. 2024-09-17T05:33:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 Processor -- start 2024-09-17T05:33:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.910+0000 7f6083000640 1 -- start start 2024-09-17T05:33:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c06f110 0x7f607c1f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 0x7f607c09caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f607c168bc0 con 0x7f607c0c7c50 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f607c168a40 con 0x7f607c06f110 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f607c168d40 con 0x7f607c155d70 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59528/0 (socket says 172.21.15.43:59528) 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 -- 172.21.15.43:0/2744340070 learned_addr learned my addr 172.21.15.43:0/2744340070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f607be00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c06f110 0x7f607c1f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6073e00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 0x7f607c09caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 -- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 msgr2=0x7f607c09caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 0x7f607c09caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 -- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c06f110 msgr2=0x7f607c1f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c06f110 0x7f607c1f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 -- 172.21.15.43:0/2744340070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c0c6990 con 0x7f607c0c7c50 2024-09-17T05:33:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6073e00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 0x7f607c09caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6081000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f607400e970 tx=0x7f607400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074004d30 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f607400cb20 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c165a40 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074005020 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1f9250 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.914+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c0ad0b0 con 0x7f607c0c7c50 2024-09-17T05:33:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.918+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60740051c0 con 0x7f607c0c7c50 2024-09-17T05:33:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.918+0000 7f6078c00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 0x7f6058075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.918+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f6074095560 con 0x7f607c0c7c50 2024-09-17T05:33:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.918+0000 7f607be00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 0x7f6058075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.918+0000 7f607be00640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 0x7f6058075710 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f606c0af000 tx=0x7f606c01b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:31.922+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6074016030 con 0x7f607c0c7c50 2024-09-17T05:33:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.134+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f607c1fc3c0 con 0x7f607c0c7c50 2024-09-17T05:33:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.134+0000 7f6078c00640 1 -- 172.21.15.43:0/2744340070 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) ==== 95+0+5225 (secure 0 0 0) 0x7f607405f480 con 0x7f607c0c7c50 2024-09-17T05:33:32.138 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:32.139 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":293,"btime":"2024-09-17T05:33:26:515292+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:26.515281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9786},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9786":{"gid":9786,"name":"c","rank":0,"incarnation":290,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/1507988053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1507988053},{"type":"v1","addr":"172.21.15.43:6835","nonce":1507988053}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9786,"qdb_cluster":[9786]},"id":6}]} 2024-09-17T05:33:32.139 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 293 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 msgr2=0x7f6058075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 0x7f6058075710 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f606c0af000 tx=0x7f606c01b060 comp rx=0 tx=0).stop 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 msgr2=0x7f607c1fc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f607400e970 tx=0x7f607400ee30 comp rx=0 tx=0).stop 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 shutdown_connections 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6058073200 0x7f6058075710 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0c7c50 0x7f607c1fc7a0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f607c155d70 0x7f607c09caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 --2- 172.21.15.43:0/2744340070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c06f110 0x7f607c1f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 >> 172.21.15.43:0/2744340070 conn(0x7f607c0b23d0 msgr2=0x7f607c09d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 shutdown_connections 2024-09-17T05:33:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.138+0000 7f6083000640 1 -- 172.21.15.43:0/2744340070 wait complete. 2024-09-17T05:33:32.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T05:33:32.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 -- 172.21.15.43:0/1310233475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c06f110 msgr2=0x7f4c7c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1310233475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c06f110 0x7f4c7c0b33e0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c072b60 tx=0x7f4c6801c740 comp rx=0 tx=0).stop 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 -- 172.21.15.43:0/1310233475 shutdown_connections 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1310233475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c06f110 0x7f4c7c0b33e0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1310233475 >> v2:172.21.15.62:3300/0 conn(0x7f4c7c155d70 0x7f4c7c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 -- 172.21.15.43:0/1310233475 >> 172.21.15.43:0/1310233475 conn(0x7f4c7c0b23d0 msgr2=0x7f4c7c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 -- 172.21.15.43:0/1310233475 shutdown_connections 2024-09-17T05:33:32.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.318+0000 7f4c83c00640 1 -- 172.21.15.43:0/1310233475 wait complete. 2024-09-17T05:33:32.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 Processor -- start 2024-09-17T05:33:32.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- start start 2024-09-17T05:33:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 0x7f4c7c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 0x7f4c7c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c168a60 con 0x7f4c7c0b3920 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1688e0 con 0x7f4c7c155d70 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c168be0 con 0x7f4c7c06f110 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59554/0 (socket says 172.21.15.43:59554) 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 -- 172.21.15.43:0/1109996436 learned_addr learned my addr 172.21.15.43:0/1109996436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c81200640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 0x7f4c7c1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 -- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 msgr2=0x7f4c7c1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 0x7f4c7c1553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 -- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 msgr2=0x7f4c7c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c81c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 0x7f4c7c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 0x7f4c7c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 -- 172.21.15.43:0/1109996436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1f9750 con 0x7f4c7c0b3920 2024-09-17T05:33:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c81200640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 0x7f4c7c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c81c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 0x7f4c7c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c7be00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000c10 tx=0x7f4c680a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c78c00640 1 -- 172.21.15.43:0/1109996436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680bb070 con 0x7f4c7c0b3920 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c78c00640 1 -- 172.21.15.43:0/1109996436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c68005030 con 0x7f4c7c0b3920 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c78c00640 1 -- 172.21.15.43:0/1109996436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680b2040 con 0x7f4c7c0b3920 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c0cc240 con 0x7f4c7c0b3920 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.322+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c0cc800 con 0x7f4c7c0b3920 2024-09-17T05:33:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c78c00640 1 -- 172.21.15.43:0/1109996436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4c680be070 con 0x7f4c7c0b3920 2024-09-17T05:33:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c78c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 0x7f4c5c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c78c00640 1 -- 172.21.15.43:0/1109996436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f4c6812e840 con 0x7f4c7c0b3920 2024-09-17T05:33:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c81200640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 0x7f4c5c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c8b318000 1 Processor -- start 2024-09-17T05:33:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c8b318000 1 -- start start 2024-09-17T05:33:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.326+0000 7f4c81200640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 0x7f4c5c075740 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005fd0 tx=0x7f4c70005de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191378b60 0x5611914be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56119135cd00 con 0x561191357d90 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5611912d0c90 con 0x561191378b60 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5611912c00c0 con 0x56119136e1c0 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59566/0 (socket says 172.21.15.43:59566) 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 -- 172.21.15.43:0/3224452926 learned_addr learned my addr 172.21.15.43:0/3224452926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81200640 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191378b60 0x5611914be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c7be00640 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 -- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 msgr2=0x5611914be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 -- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191378b60 msgr2=0x5611914be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191378b60 0x5611914be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 -- 172.21.15.43:0/3224452926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119129b290 con 0x561191357d90 2024-09-17T05:33:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c81c00640 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4c74001cf0 tx=0x7f4c740021b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef46c7d8e2bd974d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c6d600640 1 -- 172.21.15.43:0/3224452926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74018070 con 0x561191357d90 2024-09-17T05:33:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c6d600640 1 -- 172.21.15.43:0/3224452926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c74006b40 con 0x561191357d90 2024-09-17T05:33:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c6d600640 1 -- 172.21.15.43:0/3224452926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400d910 con 0x561191357d90 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- 172.21.15.43:0/3224452926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 msgr2=0x5611914bed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4c74001cf0 tx=0x7f4c740021b0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- 172.21.15.43:0/3224452926 shutdown_connections 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191357d90 0x5611914bed60 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3224452926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191378b60 0x5611914be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- 172.21.15.43:0/3224452926 >> 172.21.15.43:0/3224452926 conn(0x561190f7d760 msgr2=0x5611914a1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- 172.21.15.43:0/3224452926 shutdown_connections 2024-09-17T05:33:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.330+0000 7f4c8b318000 1 -- 172.21.15.43:0/3224452926 wait complete. 2024-09-17T05:33:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- start start 2024-09-17T05:33:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 Processor -- start 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x56119147a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x5611914a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56119125a940 con 0x56119136e1c0 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5611912c3ed0 con 0x561191357d90 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x561190f85820 con 0x561191378b60 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 10 client.0 _ll_register_callbacks cb 0x56119129e7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59580/0 (socket says 172.21.15.43:59580) 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 -- 172.21.15.43:0/3835688554 learned_addr learned my addr 172.21.15.43:0/3835688554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c81c00640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x5611914a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c81200640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x56119147a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 -- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 msgr2=0x5611914a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x5611914a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 -- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 msgr2=0x56119147a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x56119147a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 -- 172.21.15.43:0/3835688554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561191273550 con 0x56119136e1c0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c81200640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x56119147a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c7be00640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4c680f8640 tx=0x7f4c680044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804d1d058c26833d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680f8db0 con 0x56119136e1c0 2024-09-17T05:33:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561191258400 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f4c680053e0 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=293) ==== 24+0+0 (secure 0 0 0) 0x7f4c680f86b0 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 10 client.9879 fetch_fsmap learned FSMap version 293 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=293,monmap=2+}) -- 0x56119125e480 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561191258400 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 293) ==== 3502+0+0 (secure 0 0 0) 0x7f4c680b5030 con 0x56119136e1c0 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 10 client.9879 fetch_fsmap finished waiting for FSMap version 293 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 10 client.9879 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 20 client.9879 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f62ad0 0x5611914cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 4 client.9879 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:33:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.334+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 --> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611912d2be0 con 0x561190f62ad0 2024-09-17T05:33:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c81200640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f62ad0 0x5611914cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c81200640 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f62ad0 0x5611914cf8a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d0713d26349e2f server_cookie=91d5e21c1a8a113a in_seq=0 out_seq=0 2024-09-17T05:33:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c62000640 10 client.9879 ms_handle_connect on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c62000640 1 -- 172.21.15.43:0/3835688554 <== mds.0 v2:172.21.15.43:6833/1507988053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611912d2be0 con 0x561190f62ad0 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c62000640 10 client.9879 handle_command_reply: tid=0 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 client.9879 shutdown 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 msgr2=0x56119147a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4c680f8640 tx=0x7f4c680044f0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 shutdown_connections 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f62ad0 0x5611914cf8a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x5611914a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x56119147a610 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 --2- 172.21.15.43:0/3835688554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x56119147a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c62000640 0 client.9879 ms_handle_reset on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 >> 172.21.15.43:0/3835688554 conn(0x561190f7d760 msgr2=0x561191472680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 shutdown_connections 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 1 -- 172.21.15.43:0/3835688554 wait complete. 2024-09-17T05:33:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.338+0000 7f4c8b318000 20 client.9879 trim_cache size 0 max 16384 2024-09-17T05:33:32.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 Processor -- start 2024-09-17T05:33:32.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- start start 2024-09-17T05:33:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x561191479770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x561191475be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56119126b300 con 0x56119136e1c0 2024-09-17T05:33:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56119127d670 con 0x561191357d90 2024-09-17T05:33:32.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56119137ffc0 con 0x561191378b60 2024-09-17T05:33:32.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59596/0 (socket says 172.21.15.43:59596) 2024-09-17T05:33:32.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 -- 172.21.15.43:0/2949166269 learned_addr learned my addr 172.21.15.43:0/2949166269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 -- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 msgr2=0x561191475be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x561191475be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 -- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 msgr2=0x561191479770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x561191479770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 -- 172.21.15.43:0/2949166269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561191345d10 con 0x56119136e1c0 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c7be00640 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4c680f5920 tx=0x7f4c680094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ae8367efb195a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c6cc00640 1 -- 172.21.15.43:0/2949166269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680a58f0 con 0x56119136e1c0 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c6cc00640 1 -- 172.21.15.43:0/2949166269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c680f8db0 con 0x56119136e1c0 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c6cc00640 1 -- 172.21.15.43:0/2949166269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68005620 con 0x56119136e1c0 2024-09-17T05:33:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- 172.21.15.43:0/2949166269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 msgr2=0x5611914756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4c680f5920 tx=0x7f4c680094a0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- 172.21.15.43:0/2949166269 shutdown_connections 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561191378b60 0x561191475be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56119136e1c0 0x5611914756a0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2949166269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x561191479770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- 172.21.15.43:0/2949166269 >> 172.21.15.43:0/2949166269 conn(0x561190f7d760 msgr2=0x561191478c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- 172.21.15.43:0/2949166269 shutdown_connections 2024-09-17T05:33:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.354+0000 7f4c8b318000 1 -- 172.21.15.43:0/2949166269 wait complete. 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 -- start start 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 Processor -- start 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x5611914d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5611912cfa60 con 0x561191378b60 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5611912c31e0 con 0x561191357d90 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5611912c8500 con 0x56119136e1c0 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c8b318000 10 client.0 _ll_register_callbacks cb 0x7f4c680f8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:32.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59610/0 (socket says 172.21.15.43:59610) 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 -- 172.21.15.43:0/2970524773 learned_addr learned my addr 172.21.15.43:0/2970524773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81200640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x5611914d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c7be00640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 -- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 msgr2=0x5611914d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 -- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 msgr2=0x5611914d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x5611914d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 -- 172.21.15.43:0/2970524773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561191276e50 con 0x561191378b60 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c81c00640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4c74007000 tx=0x7f4c74006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8151e03047a0679 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.358+0000 7f4c62000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74018040 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f4c7400db10 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c680f8db0 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=293) ==== 24+0+0 (secure 0 0 0) 0x7f4c7402c050 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 10 client.9885 fetch_fsmap learned FSMap version 293 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=293,monmap=2+}) -- 0x5611912652d0 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c680f8db0 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 293) ==== 3502+0+0 (secure 0 0 0) 0x7f4c7400dd10 con 0x561191378b60 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 10 client.9885 fetch_fsmap finished waiting for FSMap version 293 2024-09-17T05:33:32.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 10 client.9885 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 20 client.9885 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f50b30 0x56119153b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 4 client.9885 mds_command: new command op to 9786 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 --> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5611912ce690 con 0x561190f50b30 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c81200640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f50b30 0x56119153b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 10 client.9885 ms_handle_connect on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c81200640 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f50b30 0x56119153b230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f8549ac5172dac server_cookie=74f005009ff273d6 in_seq=0 out_seq=0 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 1 -- 172.21.15.43:0/2970524773 <== mds.0 v2:172.21.15.43:6833/1507988053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x5611912ce690 con 0x561190f50b30 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 10 client.9885 handle_command_reply: tid=0 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 client.9885 shutdown 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 msgr2=0x5611914efdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4c74007000 tx=0x7f4c74006d50 comp rx=0 tx=0).stop 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 shutdown_connections 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:6833/1507988053,v1:172.21.15.43:6835/1507988053] conn(0x561190f50b30 0x56119153b230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561191378b60 0x5611914efdc0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56119136e1c0 0x5611914d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 --2- 172.21.15.43:0/2970524773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561191357d90 0x5611914d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c62000640 0 client.9885 ms_handle_reset on v2:172.21.15.43:6833/1507988053 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 >> 172.21.15.43:0/2970524773 conn(0x561190f7d760 msgr2=0x561191479600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 shutdown_connections 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 1 -- 172.21.15.43:0/2970524773 wait complete. 2024-09-17T05:33:32.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c8b318000 20 client.9885 trim_cache size 0 max 16384 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "dentry", 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 3233622024, 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1, 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "frag": "*", 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "dname": "subdir", 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_id": "head", 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/subdir" 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:33:32.370 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:33:32.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 msgr2=0x7f4c5c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 0x7f4c5c075740 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4c70005fd0 tx=0x7f4c70005de0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 msgr2=0x7f4c7c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000c10 tx=0x7f4c680a5710 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 shutdown_connections 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4c5c073230 0x7f4c5c075740 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c7c155d70 0x7f4c7c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7c0b3920 0x7f4c7c0cfa70 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 --2- 172.21.15.43:0/1109996436 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4c7c06f110 0x7f4c7c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 >> 172.21.15.43:0/1109996436 conn(0x7f4c7c0b23d0 msgr2=0x7f4c7c09db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 shutdown_connections 2024-09-17T05:33:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.362+0000 7f4c83c00640 1 -- 172.21.15.43:0/1109996436 wait complete. 2024-09-17T05:33:32.386 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-09-17T05:33:32.386 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:32.386 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:32.395 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:32.395 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:32.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:33:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 -- 172.21.15.43:0/2247686441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 msgr2=0x7f9b34004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2247686441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b34004c20 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f9b380016c0 tx=0x7f9b3801c790 comp rx=0 tx=0).stop 2024-09-17T05:33:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b4e800640 1 -- 172.21.15.43:0/2247686441 reap_dead start 2024-09-17T05:33:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 -- 172.21.15.43:0/2247686441 shutdown_connections 2024-09-17T05:33:32.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 -- 172.21.15.43:0/2247686441 >> 172.21.15.43:0/2247686441 conn(0x7f9b500b23d0 msgr2=0x7f9b500d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 -- 172.21.15.43:0/2247686441 shutdown_connections 2024-09-17T05:33:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.646+0000 7f9b54a00640 1 -- 172.21.15.43:0/2247686441 wait complete. 2024-09-17T05:33:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 Processor -- start 2024-09-17T05:33:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 -- start start 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5015df40 0x7f9b50159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b50159c70 0x7f9b501580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50168a70 con 0x7f9b34004810 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501688f0 con 0x7f9b5015df40 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b54a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9b50168bf0 con 0x7f9b50159c70 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59620/0 (socket says 172.21.15.43:59620) 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 -- 172.21.15.43:0/2386139337 learned_addr learned my addr 172.21.15.43:0/2386139337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b47e00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5015df40 0x7f9b50159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4e800640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b50159c70 0x7f9b501580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 -- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b50159c70 msgr2=0x7f9b501580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b50159c70 0x7f9b501580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 -- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5015df40 msgr2=0x7f9b50159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5015df40 0x7f9b50159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.650+0000 7f9b4de00640 1 -- 172.21.15.43:0/2386139337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5015a080 con 0x7f9b34004810 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b4de00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e590 tx=0x7f9b4000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40015070 con 0x7f9b34004810 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b400109a0 con 0x7f9b34004810 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b501f8f20 con 0x7f9b34004810 2024-09-17T05:33:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40010df0 con 0x7f9b34004810 2024-09-17T05:33:32.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501f94e0 con 0x7f9b34004810 2024-09-17T05:33:32.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b500730b0 con 0x7f9b34004810 2024-09-17T05:33:32.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b40018070 con 0x7f9b34004810 2024-09-17T05:33:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 0x7f9b2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.654+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7f9b40095150 con 0x7f9b34004810 2024-09-17T05:33:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.658+0000 7f9b47e00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 0x7f9b2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.658+0000 7f9b47e00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 0x7f9b2c075710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9b500c3a70 tx=0x7f9b380b1040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.662+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9b5015a080 con 0x7f9b34004810 2024-09-17T05:33:32.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9b5015f8d0 con 0x7f9b34004810 2024-09-17T05:33:32.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b46000640 1 -- 172.21.15.43:0/2386139337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v122) ==== 81+0+4012 (secure 0 0 0) 0x7f9b4005ef60 con 0x7f9b34004810 2024-09-17T05:33:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:33:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:32.857 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:32.858 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:32.858 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:listed 66 entries 2024-09-17T05:33:32.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 msgr2=0x7f9b2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 0x7f9b2c075710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9b500c3a70 tx=0x7f9b380b1040 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 msgr2=0x7f9b50154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000e590 tx=0x7f9b4000e990 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 shutdown_connections 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9b2c073200 0x7f9b2c075710 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5015df40 0x7f9b50159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9b50159c70 0x7f9b501580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 --2- 172.21.15.43:0/2386139337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b34004810 0x7f9b50154430 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 >> 172.21.15.43:0/2386139337 conn(0x7f9b500b23d0 msgr2=0x7f9b5015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 shutdown_connections 2024-09-17T05:33:32.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:32.850+0000 7f9b54a00640 1 -- 172.21.15.43:0/2386139337 wait complete. 2024-09-17T05:33:32.875 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:33:32.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:32.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:33:32.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:33:32.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:32.899 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:32.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 -- 172.21.15.43:0/794794420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b2330 msgr2=0x7fa4380b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 --2- 172.21.15.43:0/794794420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b2330 0x7fa4380b2740 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa42401c790 comp rx=0 tx=0).stop 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 -- 172.21.15.43:0/794794420 shutdown_connections 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 --2- 172.21.15.43:0/794794420 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380cd9a0 0x7fa4380cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 --2- 172.21.15.43:0/794794420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b2330 0x7fa4380b2740 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 -- 172.21.15.43:0/794794420 >> 172.21.15.43:0/794794420 conn(0x7fa4380b2e80 msgr2=0x7fa4380b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 -- 172.21.15.43:0/794794420 shutdown_connections 2024-09-17T05:33:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.150+0000 7fa440600640 1 -- 172.21.15.43:0/794794420 wait complete. 2024-09-17T05:33:33.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 Processor -- start 2024-09-17T05:33:33.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- start start 2024-09-17T05:33:33.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380b2330 0x7fa4380b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4380cd9a0 0x7fa4381fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa43816a180 con 0x7fa4380b6b10 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa43816a000 con 0x7fa4380cd9a0 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa43816a300 con 0x7fa4380b2330 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59638/0 (socket says 172.21.15.43:59638) 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 -- 172.21.15.43:0/2418785923 learned_addr learned my addr 172.21.15.43:0/2418785923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43e600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4380cd9a0 0x7fa4381fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43dc00640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380b2330 0x7fa4380b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 -- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380b2330 msgr2=0x7fa4380b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380b2330 0x7fa4380b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 -- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4380cd9a0 msgr2=0x7fa4381fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4380cd9a0 0x7fa4381fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 -- 172.21.15.43:0/2418785923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4380af4c0 con 0x7fa4380b6b10 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa43d200640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa42401a3c0 tx=0x7fa4240a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240bb070 con 0x7fa4380b6b10 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa424005030 con 0x7fa4380b6b10 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b2040 con 0x7fa4380b6b10 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4380af7a0 con 0x7fa4380b6b10 2024-09-17T05:33:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.154+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4380afa80 con 0x7fa4380b6b10 2024-09-17T05:33:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4380b2740 con 0x7fa4380b6b10 2024-09-17T05:33:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4240be070 con 0x7fa4380b6b10 2024-09-17T05:33:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa435600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 0x7fa418075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8547+0+0 (secure 0 0 0) 0x7fa42412e3b0 con 0x7fa4380b6b10 2024-09-17T05:33:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa43dc00640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 0x7fa418075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.158+0000 7fa43dc00640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 0x7fa418075710 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa42c001400 tx=0x7fa42c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.162+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa424016410 con 0x7fa4380b6b10 2024-09-17T05:33:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.370+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa43806dd50 con 0x7fa4380b6b10 2024-09-17T05:33:33.376 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:33.374+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:33.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.374+0000 7fa435600640 1 -- 172.21.15.43:0/2418785923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v294) ==== 161+0+0 (secure 0 0 0) 0x7fa4240f89b0 con 0x7fa4380b6b10 2024-09-17T05:33:33.378 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 msgr2=0x7fa418075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 0x7fa418075710 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa42c001400 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2024-09-17T05:33:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 msgr2=0x7fa43815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa42401a3c0 tx=0x7fa4240a5780 comp rx=0 tx=0).stop 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 shutdown_connections 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa418073200 0x7fa418075710 secure :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa42c001400 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4380cd9a0 0x7fa4381fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4380b6b10 0x7fa43815c3e0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 --2- 172.21.15.43:0/2418785923 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4380b2330 0x7fa4380b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 >> 172.21.15.43:0/2418785923 conn(0x7fa4380b2e80 msgr2=0x7fa4380b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 shutdown_connections 2024-09-17T05:33:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.378+0000 7fa440600640 1 -- 172.21.15.43:0/2418785923 wait complete. 2024-09-17T05:33:33.387 INFO:tasks.ceph.mds.c.smithi043.stderr: -570> 2024-09-17T05:33:27.482+0000 7f6d91000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:33.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:33:33.509 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:33:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.562+0000 7f8b93e00640 1 -- 172.21.15.43:0/2098268468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9415e8b0 msgr2=0x7f8b940b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.562+0000 7f8b93e00640 1 --2- 172.21.15.43:0/2098268468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9415e8b0 0x7f8b940b40a0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009840 tx=0x7f8b7c01c760 comp rx=0 tx=0).stop 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- 172.21.15.43:0/2098268468 shutdown_connections 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 --2- 172.21.15.43:0/2098268468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b9415e8b0 0x7f8b940b40a0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- 172.21.15.43:0/2098268468 >> 172.21.15.43:0/2098268468 conn(0x7f8b94068fe0 msgr2=0x7f8b94069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- 172.21.15.43:0/2098268468 shutdown_connections 2024-09-17T05:33:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- 172.21.15.43:0/2098268468 wait complete. 2024-09-17T05:33:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 Processor -- start 2024-09-17T05:33:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- start start 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b940b45e0 0x7f8b94154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9415e8b0 0x7f8b94161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9416d0d0 con 0x7f8b940cdeb0 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9416cf50 con 0x7f8b940b45e0 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.566+0000 7f8b93e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9416d250 con 0x7f8b9415e8b0 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b92a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b940b45e0 0x7f8b94154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59664/0 (socket says 172.21.15.43:59664) 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 -- 172.21.15.43:0/3179862661 learned_addr learned my addr 172.21.15.43:0/3179862661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b93400640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9415e8b0 0x7f8b94161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 -- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9415e8b0 msgr2=0x7f8b94161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9415e8b0 0x7f8b94161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 -- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b940b45e0 msgr2=0x7f8b94154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b940b45e0 0x7f8b94154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 -- 172.21.15.43:0/3179862661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9415a630 con 0x7f8b940cdeb0 2024-09-17T05:33:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8a000640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009790 tx=0x7f8b7c0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0b2040 con 0x7f8b940cdeb0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b7c003600 con 0x7f8b940cdeb0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9415a910 con 0x7f8b940cdeb0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0044f0 con 0x7f8b940cdeb0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b94156630 con 0x7f8b940cdeb0 2024-09-17T05:33:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9415f250 con 0x7f8b940cdeb0 2024-09-17T05:33:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.570+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b7c0bd060 con 0x7f8b940cdeb0 2024-09-17T05:33:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.574+0000 7f8b8b400640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 0x7f8b70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.574+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f8b7c12e7a0 con 0x7f8b940cdeb0 2024-09-17T05:33:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.574+0000 7f8b92a00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 0x7f8b70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.574+0000 7f8b92a00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 0x7f8b70075710 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8b84007fd0 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.578+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b7c016410 con 0x7f8b940cdeb0 2024-09-17T05:33:33.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:33.818+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8b9415af60 con 0x7f8b940cdeb0 2024-09-17T05:33:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b8b400640 1 -- 172.21.15.43:0/3179862661 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v295) ==== 107+0+0 (secure 0 0 0) 0x7f8b7c0f85c0 con 0x7f8b940cdeb0 2024-09-17T05:33:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 msgr2=0x7f8b70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 0x7f8b70075710 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8b84007fd0 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 msgr2=0x7f8b941616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009790 tx=0x7f8b7c0a5780 comp rx=0 tx=0).stop 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 shutdown_connections 2024-09-17T05:33:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b70073200 0x7f8b70075710 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8b9415e8b0 0x7f8b94161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b940cdeb0 0x7f8b941616c0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 --2- 172.21.15.43:0/3179862661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b940b45e0 0x7f8b94154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 >> 172.21.15.43:0/3179862661 conn(0x7f8b94068fe0 msgr2=0x7f8b9406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 shutdown_connections 2024-09-17T05:33:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.378+0000 7f8b93e00640 1 -- 172.21.15.43:0/3179862661 wait complete. 2024-09-17T05:33:34.405 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 Processor -- start 2024-09-17T05:33:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 -- start start 2024-09-17T05:33:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f01429f9c0 0x55f0142dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f014350010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0141e8a00 con 0x55f0143519c0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f014246f40 con 0x55f01429f9c0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16eca2c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f0142e08c0 con 0x55f0143515b0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50552/0 (socket says 172.21.15.43:50552) 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 -- 172.21.15.43:0/217788191 learned_addr learned my addr 172.21.15.43:0/217788191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16dfe00640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f014350010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e4c00640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f01429f9c0 0x55f0142dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 -- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 msgr2=0x55f014350010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f014350010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 -- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f01429f9c0 msgr2=0x55f0142dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f01429f9c0 0x55f0142dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 -- 172.21.15.43:0/217788191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0141c1c10 con 0x55f0143519c0 2024-09-17T05:33:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16dfe00640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f014350010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16e5600640 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f16e0019b60 tx=0x7f16e0018250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34819541fcd6a43c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16dea00640 1 -- 172.21.15.43:0/217788191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00bb020 con 0x55f0143519c0 2024-09-17T05:33:34.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16dea00640 1 -- 172.21.15.43:0/217788191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16e00b1040 con 0x55f0143519c0 2024-09-17T05:33:34.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.462+0000 7f16dea00640 1 -- 172.21.15.43:0/217788191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00b0070 con 0x55f0143519c0 2024-09-17T05:33:34.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- 172.21.15.43:0/217788191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 msgr2=0x55f014350550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f16e0019b60 tx=0x7f16e0018250 comp rx=0 tx=0).stop 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- 172.21.15.43:0/217788191 shutdown_connections 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f0143519c0 0x55f014350550 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f014350010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- 172.21.15.43:0/217788191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f01429f9c0 0x55f0142dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- 172.21.15.43:0/217788191 >> 172.21.15.43:0/217788191 conn(0x55f014286d80 msgr2=0x55f014370d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- 172.21.15.43:0/217788191 shutdown_connections 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- 172.21.15.43:0/217788191 wait complete. 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 Processor -- start 2024-09-17T05:33:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- start start 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f0142db860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f0143519c0 0x55f01436eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f0142e50c0 con 0x55f01429f9c0 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f0142e4e20 con 0x55f0143519c0 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16eca2c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55f0142e4720 con 0x55f0143515b0 2024-09-17T05:33:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16e4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16e4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50554/0 (socket says 172.21.15.43:50554) 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 learned_addr learned my addr 172.21.15.43:0/876085822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f0142db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.466+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f0143519c0 0x55f01436eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 msgr2=0x55f0142db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55f0143515b0 0x55f0142db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f0143519c0 msgr2=0x55f01436eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f0143519c0 0x55f01436eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f01436d7d0 con 0x55f01429f9c0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f0143519c0 0x55f01436eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f16d8003f00 tx=0x7f16d800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 -- 172.21.15.43:0/876085822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8014020 con 0x55f01429f9c0 2024-09-17T05:33:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 -- 172.21.15.43:0/876085822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16d8001400 con 0x55f01429f9c0 2024-09-17T05:33:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f01436da70 con 0x55f01429f9c0 2024-09-17T05:33:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 -- 172.21.15.43:0/876085822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d800a040 con 0x55f01429f9c0 2024-09-17T05:33:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f01440db90 con 0x55f01429f9c0 2024-09-17T05:33:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 -- 172.21.15.43:0/876085822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f16d8017020 con 0x55f01429f9c0 2024-09-17T05:33:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 0x7f16c40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.470+0000 7f16dd600640 1 -- 172.21.15.43:0/876085822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f16d8093610 con 0x55f01429f9c0 2024-09-17T05:33:34.475 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 0x55f014353f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0142dbda0 con 0x55f01436f090 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 0x7f16c40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 0x55f014353f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 0x7f16c40740f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c650 tx=0x7f16d0004a30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 0x55f014353f00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v123'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.474+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014354440 con 0x55f01436f090 2024-09-17T05:33:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.478+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v123'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.478+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014416bf0 con 0x55f01436f090 2024-09-17T05:33:34.482 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.478+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v123'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.478+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417fa0 con 0x55f01436f090 2024-09-17T05:33:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014418340 con 0x55f01436f090 2024-09-17T05:33:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014416f90 con 0x55f01436f090 2024-09-17T05:33:34.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.482+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417380 con 0x55f01436f090 2024-09-17T05:33:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.486+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v123'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.486+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.486+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v123'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.486+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.491 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v123'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.490+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.494+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.500 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:34.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.498+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.498+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.498+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.498+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417720 con 0x55f01436f090 2024-09-17T05:33:34.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417b20 con 0x55f01436f090 2024-09-17T05:33:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.502+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014417b00 con 0x55f01436f090 2024-09-17T05:33:34.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.506+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v123'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.506+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.506+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v123'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.506+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.512 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:34.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v123'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.510+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.514+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.514+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.514+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v123'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.514+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v123'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.521 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:34.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v123'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.518+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f01436f090 2024-09-17T05:33:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f014418bf0 0x55f01441b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144179a0 con 0x55f014418bf0 2024-09-17T05:33:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f014418bf0 0x55f01441b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.522+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f014418bf0 0x55f01441b010 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.526+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'751 uv730 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.526+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01441b550 con 0x55f014418bf0 2024-09-17T05:33:34.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.526+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'752 uv752 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.526+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441b8f0 con 0x55f014418bf0 2024-09-17T05:33:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.530+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.533 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.530+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441c140 con 0x55f014418bf0 2024-09-17T05:33:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.530+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.530+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441c4e0 con 0x55f014418bf0 2024-09-17T05:33:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441c880 con 0x55f014418bf0 2024-09-17T05:33:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v123'756 uv749 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01441cc20 con 0x55f014418bf0 2024-09-17T05:33:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v123'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.534+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.538+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v123'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.542 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.538+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.542+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.542+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.546+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.546+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.546+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v123'761 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.546+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.550+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v123'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.550+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.554 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.550+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v123'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.550+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014421190 con 0x55f014418bf0 2024-09-17T05:33:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.554+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v123'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.554+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014421190 con 0x55f014418bf0 2024-09-17T05:33:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.554+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.554+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v123'766 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v123'767 uv767 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.562 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:34.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v123'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.558+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441dfd0 con 0x55f014418bf0 2024-09-17T05:33:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441dfd0 con 0x55f014418bf0 2024-09-17T05:33:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441cfc0 con 0x55f014418bf0 2024-09-17T05:33:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f014418bf0 2024-09-17T05:33:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55f01441d4d0 0x55f01441f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.562+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441fe30 con 0x55f01441d4d0 2024-09-17T05:33:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.566+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55f01441d4d0 0x55f01441f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.566+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55f01441d4d0 0x55f01441f8f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.566+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v123'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.566+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f0144202f0 con 0x55f01441d4d0 2024-09-17T05:33:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v123'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01441bd80 con 0x55f01441d4d0 2024-09-17T05:33:34.573 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:34.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v123'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014422100 con 0x55f01441d4d0 2024-09-17T05:33:34.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.570+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144224a0 con 0x55f01441d4d0 2024-09-17T05:33:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144210f0 con 0x55f01441d4d0 2024-09-17T05:33:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144214e0 con 0x55f01436f090 2024-09-17T05:33:34.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'145 uv144 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.574+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f0144214e0 con 0x55f01436f090 2024-09-17T05:33:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.578+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'146 uv146 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.578+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144214e0 con 0x55f01436f090 2024-09-17T05:33:34.582 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.578+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'147 uv147 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.578+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144214e0 con 0x55f01436f090 2024-09-17T05:33:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'148 uv148 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 0x55f014424c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144251a0 con 0x55f014422840 2024-09-17T05:33:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 0x55f014424c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16e5600640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 0x55f014424c60 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v123'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.582+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f0144251a0 con 0x55f014422840 2024-09-17T05:33:34.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v123'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144251a0 con 0x55f014422840 2024-09-17T05:33:34.589 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v123'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144261b0 con 0x55f014422840 2024-09-17T05:33:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.586+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144261b0 con 0x55f014422840 2024-09-17T05:33:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144251a0 con 0x55f014422840 2024-09-17T05:33:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55f0144256b0 0x55f014427ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f0144281d0 con 0x55f0144256b0 2024-09-17T05:33:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55f0144256b0 0x55f014427ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16e4c00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55f0144256b0 0x55f014427ad0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v123'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.590+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014428210 con 0x55f0144256b0 2024-09-17T05:33:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.594+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v123'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.594+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.598 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.594+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v123'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.594+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014429640 con 0x55f0144256b0 2024-09-17T05:33:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014429640 con 0x55f0144256b0 2024-09-17T05:33:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.598+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v123'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v123'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.606 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v123'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.602+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014428630 con 0x55f0144256b0 2024-09-17T05:33:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16e4c00640 1 -- 172.21.15.43:0/876085822 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d8059620 con 0x55f0144256b0 2024-09-17T05:33:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55f014428cb0 0x55f01442b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442b6e0 con 0x55f014428cb0 2024-09-17T05:33:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55f014428cb0 0x55f01442b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.606+0000 7f16dfe00640 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55f014428cb0 0x55f01442b0d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v123'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442bab0 con 0x55f014428cb0 2024-09-17T05:33:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v123'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f014428cb0 2024-09-17T05:33:34.614 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v123'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.610+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f014428cb0 2024-09-17T05:33:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f014428cb0 2024-09-17T05:33:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01436f090 2024-09-17T05:33:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v123'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.614+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01436f090 2024-09-17T05:33:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v123'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01436f090 2024-09-17T05:33:34.620 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v123'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442ceb0 con 0x55f01436f090 2024-09-17T05:33:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.618+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442ceb0 con 0x55f01436f090 2024-09-17T05:33:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01436f090 2024-09-17T05:33:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f01436f090 2024-09-17T05:33:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01441d4d0 2024-09-17T05:33:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v123'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.622+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01441d4d0 2024-09-17T05:33:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v123'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442bea0 con 0x55f01441d4d0 2024-09-17T05:33:34.629 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v123'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.626+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.630+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.630+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.630+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v123'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.630+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v123'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.637 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v123'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c290 con 0x55f01441d4d0 2024-09-17T05:33:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.634+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442f0a0 con 0x55f01441d4d0 2024-09-17T05:33:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.638+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f01441d4d0 2024-09-17T05:33:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.638+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c740 con 0x55f014422840 2024-09-17T05:33:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.642+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v123'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.642+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442c740 con 0x55f014422840 2024-09-17T05:33:34.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.642+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.642+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v123'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.648 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:34.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v123'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.646+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v123'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v123'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.654 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442d5d0 con 0x55f014422840 2024-09-17T05:33:34.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.650+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442d5d0 con 0x55f014422840 2024-09-17T05:33:34.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014422840 2024-09-17T05:33:34.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16e0002050 con 0x55f014422840 2024-09-17T05:33:34.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014428cb0 2024-09-17T05:33:34.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v123'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.654+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014428cb0 2024-09-17T05:33:34.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v123'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.661 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:34.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v123'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.658+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v123'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v123'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.662+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420d30 con 0x55f014428cb0 2024-09-17T05:33:34.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v123'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.669 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:34.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c610 con 0x55f014428cb0 2024-09-17T05:33:34.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c610 con 0x55f014428cb0 2024-09-17T05:33:34.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.666+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014428cb0 2024-09-17T05:33:34.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v123'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014428cb0 2024-09-17T05:33:34.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v123'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420c90 con 0x55f014428cb0 2024-09-17T05:33:34.675 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:34.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v123'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.670+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420c90 con 0x55f014428cb0 2024-09-17T05:33:34.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.674+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.674+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420c90 con 0x55f014428cb0 2024-09-17T05:33:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.674+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.674+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f014420c90 con 0x55f014428cb0 2024-09-17T05:33:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.674+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f01442c5c0 con 0x55f014428cb0 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16dfe00640 1 -- 172.21.15.43:0/876085822 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f16d0008ac0 con 0x55f014428cb0 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f014418bf0 msgr2=0x55f01441b010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55f014418bf0 0x55f01441b010 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55f0144256b0 msgr2=0x55f014427ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55f0144256b0 0x55f014427ad0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55f014428cb0 msgr2=0x55f01442b0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55f014428cb0 0x55f01442b0d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55f01441d4d0 msgr2=0x55f01441f8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55f01441d4d0 0x55f01441f8f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 msgr2=0x55f014353f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16e5600640 1 -- 172.21.15.43:0/876085822 reap_dead start 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 0x55f014353f00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 msgr2=0x55f014424c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 0x55f014424c60 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 msgr2=0x7f16c40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 0x7f16c40740f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c650 tx=0x7f16d0004a30 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 msgr2=0x55f01434e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f16d8003f00 tx=0x7f16d800b5c0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 shutdown_connections 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55f014422840 0x55f014424c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16c4071be0 0x7f16c40740f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55f01436f090 0x55f014353f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 --2- 172.21.15.43:0/876085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f01429f9c0 0x55f01434e540 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 >> 172.21.15.43:0/876085822 conn(0x55f014286d80 msgr2=0x55f0142dc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 shutdown_connections 2024-09-17T05:33:34.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.678+0000 7f16eca2c080 1 -- 172.21.15.43:0/876085822 wait complete. 2024-09-17T05:33:34.686 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 Processor -- start 2024-09-17T05:33:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 -- start start 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befada890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befadadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559befa88f40 con 0x7f6fbc003100 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559befa95d00 con 0x559befb22090 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559befa2aa00 con 0x559befb929e0 2024-09-17T05:33:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50582/0 (socket says 172.21.15.43:50582) 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 -- 172.21.15.43:0/363919148 learned_addr learned my addr 172.21.15.43:0/363919148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7000640 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befada890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd8400640 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befadadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 -- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 msgr2=0x559befadadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befadadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 -- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 msgr2=0x559befada890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befada890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 -- 172.21.15.43:0/363919148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559befb25080 con 0x7f6fbc003100 2024-09-17T05:33:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b780 tx=0x7f6fc800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa71432a5a45f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd5c00640 1 -- 172.21.15.43:0/363919148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc801a070 con 0x7f6fbc003100 2024-09-17T05:33:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd5c00640 1 -- 172.21.15.43:0/363919148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc800ae30 con 0x7f6fbc003100 2024-09-17T05:33:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fd5c00640 1 -- 172.21.15.43:0/363919148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8008680 con 0x7f6fbc003100 2024-09-17T05:33:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 -- 172.21.15.43:0/363919148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 msgr2=0x559befb21b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.786+0000 7f6fde845080 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b780 tx=0x7f6fc800bc40 comp rx=0 tx=0).stop 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- 172.21.15.43:0/363919148 shutdown_connections 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befada890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befadadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- 172.21.15.43:0/363919148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befb21b50 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- 172.21.15.43:0/363919148 >> 172.21.15.43:0/363919148 conn(0x559befac8d80 msgr2=0x559befb0b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- 172.21.15.43:0/363919148 shutdown_connections 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- 172.21.15.43:0/363919148 wait complete. 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 Processor -- start 2024-09-17T05:33:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- start start 2024-09-17T05:33:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befc3a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befc4c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befc3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559befbb10f0 con 0x7f6fbc003100 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559befbb1390 con 0x559befb22090 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fde845080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559befc4b850 con 0x559befb929e0 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befc3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befc3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50590/0 (socket says 172.21.15.43:50590) 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 -- 172.21.15.43:0/1119759737 learned_addr learned my addr 172.21.15.43:0/1119759737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd7000640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befc4c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befc3a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 -- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 msgr2=0x559befc4c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559befb929e0 0x559befc4c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 -- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 msgr2=0x559befc3a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559befb22090 0x559befc3a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 -- 172.21.15.43:0/1119759737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559befc4ba80 con 0x7f6fbc003100 2024-09-17T05:33:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fd8400640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befc3ada0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005870 tx=0x7f6fd0005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.790+0000 7f6fcfe00640 1 -- 172.21.15.43:0/1119759737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd001b070 con 0x7f6fbc003100 2024-09-17T05:33:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fcfe00640 1 -- 172.21.15.43:0/1119759737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd0009e10 con 0x7f6fbc003100 2024-09-17T05:33:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fcfe00640 1 -- 172.21.15.43:0/1119759737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00077d0 con 0x7f6fbc003100 2024-09-17T05:33:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559befc4bd20 con 0x7f6fbc003100 2024-09-17T05:33:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559befc4c260 con 0x7f6fbc003100 2024-09-17T05:33:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fcfe00640 1 -- 172.21.15.43:0/1119759737 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6fd001e070 con 0x7f6fbc003100 2024-09-17T05:33:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fcfe00640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fb8071be0 0x7f6fb80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fcfe00640 1 -- 172.21.15.43:0/1119759737 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f6fd0093560 con 0x7f6fbc003100 2024-09-17T05:33:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fb8071be0 0x7f6fb80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fde845080 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559befadada0 0x559befc37e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.794+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x559befc38370 con 0x559befadada0 2024-09-17T05:33:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fd7000640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559befadada0 0x559befc37e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fd7a00640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fb8071be0 0x7f6fb80740f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b720 tx=0x7f6fc8003030 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fd7000640 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559befadada0 0x559befc37e30 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fd7000640 1 -- 172.21.15.43:0/1119759737 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v123'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6fc40b3040 con 0x559befadada0 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559befadada0 msgr2=0x559befc37e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559befadada0 0x559befc37e30 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fb8071be0 msgr2=0x7f6fb80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6fb8071be0 0x7f6fb80740f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b720 tx=0x7f6fc8003030 comp rx=0 tx=0).stop 2024-09-17T05:33:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 msgr2=0x559befc3ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 --2- 172.21.15.43:0/1119759737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc003100 0x559befc3ada0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005870 tx=0x7f6fd0005960 comp rx=0 tx=0).stop 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fd8400640 1 -- 172.21.15.43:0/1119759737 reap_dead start 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 shutdown_connections 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 >> 172.21.15.43:0/1119759737 conn(0x559befac8d80 msgr2=0x559befb0b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 shutdown_connections 2024-09-17T05:33:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.798+0000 7f6fde845080 1 -- 172.21.15.43:0/1119759737 wait complete. 2024-09-17T05:33:34.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- 172.21.15.43:0/533714967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac06fc90 msgr2=0x7f14ac0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 --2- 172.21.15.43:0/533714967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac06fc90 0x7f14ac0700a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0d0e90 tx=0x7f149401d200 comp rx=0 tx=0).stop 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- 172.21.15.43:0/533714967 shutdown_connections 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 --2- 172.21.15.43:0/533714967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac06fc90 0x7f14ac0700a0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 --2- 172.21.15.43:0/533714967 >> v2:172.21.15.62:3300/0 conn(0x7f14ac0730b0 0x7f14ac0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 --2- 172.21.15.43:0/533714967 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 0x7f14ac0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- 172.21.15.43:0/533714967 >> 172.21.15.43:0/533714967 conn(0x7f14ac0b23d0 msgr2=0x7f14ac0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:34.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- 172.21.15.43:0/533714967 shutdown_connections 2024-09-17T05:33:34.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- 172.21.15.43:0/533714967 wait complete. 2024-09-17T05:33:34.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 Processor -- start 2024-09-17T05:33:34.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.966+0000 7f14b1c00640 1 -- start start 2024-09-17T05:33:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ac06fc90 0x7f14ac1f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 0x7f14ac0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0cea10 con 0x7f14ac0730b0 2024-09-17T05:33:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac0ce890 con 0x7f14ac06fc90 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac0ceb90 con 0x7f14ac15fe10 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50608/0 (socket says 172.21.15.43:50608) 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 -- 172.21.15.43:0/2290752156 learned_addr learned my addr 172.21.15.43:0/2290752156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14aaa00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ac06fc90 0x7f14ac1f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14ab400640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 0x7f14ac0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 -- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 msgr2=0x7f14ac0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 0x7f14ac0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 -- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ac06fc90 msgr2=0x7f14ac1f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ac06fc90 0x7f14ac1f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 -- 172.21.15.43:0/2290752156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac162a80 con 0x7f14ac0730b0 2024-09-17T05:33:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a3e00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f14940b0040 tx=0x7f1494005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940bc020 con 0x7f14ac0730b0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1494004130 con 0x7f14ac0730b0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac162d60 con 0x7f14ac0730b0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940b2040 con 0x7f14ac0730b0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac163320 con 0x7f14ac0730b0 2024-09-17T05:33:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.970+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac0734c0 con 0x7f14ac0730b0 2024-09-17T05:33:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.974+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1494003600 con 0x7f14ac0730b0 2024-09-17T05:33:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.974+0000 7f14a2a00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 0x7f1488075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.974+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f149412f060 con 0x7f14ac0730b0 2024-09-17T05:33:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.974+0000 7f14aaa00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 0x7f1488075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.974+0000 7f14aaa00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 0x7f1488075710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f149c009b40 tx=0x7f149c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:34.978+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14940c6060 con 0x7f14ac0730b0 2024-09-17T05:33:35.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.190+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f14ac1f6b80 con 0x7f14ac0730b0 2024-09-17T05:33:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.550+0000 7f14a2a00640 1 -- 172.21.15.43:0/2290752156 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v296) ==== 162+0+0 (secure 0 0 0) 0x7f14940164b0 con 0x7f14ac0730b0 2024-09-17T05:33:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 msgr2=0x7f1488075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 0x7f1488075710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f149c009b40 tx=0x7f149c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 msgr2=0x7f14ac099280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f14940b0040 tx=0x7f1494005760 comp rx=0 tx=0).stop 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 shutdown_connections 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1488073200 0x7f1488075710 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14ac15fe10 0x7f14ac0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14ac0730b0 0x7f14ac099280 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 --2- 172.21.15.43:0/2290752156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ac06fc90 0x7f14ac1f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 >> 172.21.15.43:0/2290752156 conn(0x7f14ac0b23d0 msgr2=0x7f14ac15b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 shutdown_connections 2024-09-17T05:33:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:35.554+0000 7f14b1c00640 1 -- 172.21.15.43:0/2290752156 wait complete. 2024-09-17T05:33:35.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 -- 172.21.15.43:0/2171955884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 msgr2=0x7fb80c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 --2- 172.21.15.43:0/2171955884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0ad0b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0af0a0 tx=0x7fb80401cac0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 -- 172.21.15.43:0/2171955884 shutdown_connections 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 --2- 172.21.15.43:0/2171955884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0ad0b0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 --2- 172.21.15.43:0/2171955884 >> v2:172.21.15.62:3300/0 conn(0x7fb80c0b3af0 0x7fb80c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 -- 172.21.15.43:0/2171955884 >> 172.21.15.43:0/2171955884 conn(0x7fb80c0b23d0 msgr2=0x7fb80c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 -- 172.21.15.43:0/2171955884 shutdown_connections 2024-09-17T05:33:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.142+0000 7fb812400640 1 -- 172.21.15.43:0/2171955884 wait complete. 2024-09-17T05:33:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 Processor -- start 2024-09-17T05:33:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- start start 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb80c155d70 0x7fb80c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 0x7fb80c1f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c168b90 con 0x7fb80c06f110 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c168a10 con 0x7fb80c1f7fc0 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb80c168d10 con 0x7fb80c155d70 2024-09-17T05:33:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50624/0 (socket says 172.21.15.43:50624) 2024-09-17T05:33:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 -- 172.21.15.43:0/406171592 learned_addr learned my addr 172.21.15.43:0/406171592 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80be00640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 0x7fb80c1f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb803e00640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb80c155d70 0x7fb80c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 -- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb80c155d70 msgr2=0x7fb80c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb80c155d70 0x7fb80c0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 -- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 msgr2=0x7fb80c1f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 0x7fb80c1f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 -- 172.21.15.43:0/406171592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c0cd860 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80be00640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 0x7fb80c1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb80b400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00c430 tx=0x7fb7fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc011790 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7fc017070 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc011cc0 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c0c6b80 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c0cd060 con 0x7fb80c06f110 2024-09-17T05:33:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.146+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c0ad0b0 con 0x7fb80c06f110 2024-09-17T05:33:36.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.150+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb7fc01c460 con 0x7fb80c06f110 2024-09-17T05:33:36.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.150+0000 7fb803400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 0x7fb7e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.150+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7fb7fc0948c0 con 0x7fb80c06f110 2024-09-17T05:33:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.150+0000 7fb803e00640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 0x7fb7e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.150+0000 7fb803e00640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 0x7fb7e8075710 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0cfc10 tx=0x7fb7f40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.154+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb7fc021020 con 0x7fb80c06f110 2024-09-17T05:33:36.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.366+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb80c15cab0 con 0x7fb80c06f110 2024-09-17T05:33:36.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.366+0000 7fb803400640 1 -- 172.21.15.43:0/406171592 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) ==== 95+0+5218 (secure 0 0 0) 0x7fb7fc05ede0 con 0x7fb80c06f110 2024-09-17T05:33:36.371 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:36.371 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":297,"btime":"2024-09-17T05:33:35:556812+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":297,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:35.556794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9909},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9909":{"gid":9909,"name":"c","rank":0,"incarnation":297,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/2696141215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2696141215},{"type":"v1","addr":"172.21.15.43:6835","nonce":2696141215}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:36.371 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 297 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 msgr2=0x7fb7e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 0x7fb7e8075710 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0cfc10 tx=0x7fb7f40091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 msgr2=0x7fb80c0cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00c430 tx=0x7fb7fc00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 shutdown_connections 2024-09-17T05:33:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb7e8073200 0x7fb7e8075710 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb80c1f7fc0 0x7fb80c1f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb80c155d70 0x7fb80c0cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 --2- 172.21.15.43:0/406171592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c06f110 0x7fb80c0cc260 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 >> 172.21.15.43:0/406171592 conn(0x7fb80c0b23d0 msgr2=0x7fb80c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 shutdown_connections 2024-09-17T05:33:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:36.370+0000 7fb812400640 1 -- 172.21.15.43:0/406171592 wait complete. 2024-09-17T05:33:36.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 297, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:35.556794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9909}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9909': {'gid': 9909, 'name': 'c', 'rank': 0, 'incarnation': 297, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/2696141215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 2696141215}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2696141215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:33:36.394 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9909:up:replay 2024-09-17T05:33:36.394 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:33:39.390 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:39.386+0000 7f6c5c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:41.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 -- 172.21.15.43:0/7201668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c406fc90 msgr2=0x7f56c40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 --2- 172.21.15.43:0/7201668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c406fc90 0x7f56c40700a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f56c40d0e90 tx=0x7f56ac01d200 comp rx=0 tx=0).stop 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 -- 172.21.15.43:0/7201668 shutdown_connections 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 --2- 172.21.15.43:0/7201668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c406fc90 0x7f56c40700a0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 --2- 172.21.15.43:0/7201668 >> v2:172.21.15.62:3300/0 conn(0x7f56c40730b0 0x7f56c40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 --2- 172.21.15.43:0/7201668 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c415fe10 0x7f56c40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 -- 172.21.15.43:0/7201668 >> 172.21.15.43:0/7201668 conn(0x7f56c40b23d0 msgr2=0x7f56c40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 -- 172.21.15.43:0/7201668 shutdown_connections 2024-09-17T05:33:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.966+0000 7f56cbc00640 1 -- 172.21.15.43:0/7201668 wait complete. 2024-09-17T05:33:41.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 Processor -- start 2024-09-17T05:33:41.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- start start 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 0x7f56c40c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c40730b0 0x7f56c409d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40cea10 con 0x7f56c415fe10 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ce890 con 0x7f56c40730b0 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f56c40ceb90 con 0x7f56c406fc90 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50656/0 (socket says 172.21.15.43:50656) 2024-09-17T05:33:41.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 -- 172.21.15.43:0/1818086440 learned_addr learned my addr 172.21.15.43:0/1818086440 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56bbe00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c40730b0 0x7f56c409d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9200640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 0x7f56c40c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 -- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 msgr2=0x7f56c40c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 0x7f56c40c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 -- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c40730b0 msgr2=0x7f56c409d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c40730b0 0x7f56c409d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 -- 172.21.15.43:0/1818086440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c40c8370 con 0x7f56c415fe10 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9200640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 0x7f56c40c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c9c00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e990 tx=0x7f56bc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004d30 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc00caa0 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c40c8650 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc005020 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.970+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c40c8c10 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.974+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c40734c0 con 0x7f56c415fe10 2024-09-17T05:33:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.974+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56bc0051c0 con 0x7f56c415fe10 2024-09-17T05:33:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.974+0000 7f56c1600640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 0x7f56a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.974+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f56bc095570 con 0x7f56c415fe10 2024-09-17T05:33:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.974+0000 7f56c9200640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 0x7f56a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.978+0000 7f56c9200640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 0x7f56a0075710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005fd0 tx=0x7f56b400a540 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:41.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:41.978+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f56bc016030 con 0x7f56c415fe10 2024-09-17T05:33:42.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c409e420 con 0x7f56c415fe10 2024-09-17T05:33:42.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56c1600640 1 -- 172.21.15.43:0/1818086440 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 300 v300) ==== 95+0+5225 (secure 0 0 0) 0x7f56bc05f440 con 0x7f56c415fe10 2024-09-17T05:33:42.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:42.193 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":300,"btime":"2024-09-17T05:33:38:576115+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":300,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:38.576108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9909},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9909":{"gid":9909,"name":"c","rank":0,"incarnation":297,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2696141215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2696141215},{"type":"v1","addr":"172.21.15.43:6835","nonce":2696141215}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9909,"qdb_cluster":[9909]},"id":6}]} 2024-09-17T05:33:42.193 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 300 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 msgr2=0x7f56a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 0x7f56a0075710 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005fd0 tx=0x7f56b400a540 comp rx=0 tx=0).stop 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 msgr2=0x7f56c40b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e990 tx=0x7f56bc00ee50 comp rx=0 tx=0).stop 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 shutdown_connections 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f56a0073200 0x7f56a0075710 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56c40730b0 0x7f56c409d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56c415fe10 0x7f56c40b0e70 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 --2- 172.21.15.43:0/1818086440 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f56c406fc90 0x7f56c40c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 >> 172.21.15.43:0/1818086440 conn(0x7f56c40b23d0 msgr2=0x7f56c40b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 shutdown_connections 2024-09-17T05:33:42.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.190+0000 7f56cbc00640 1 -- 172.21.15.43:0/1818086440 wait complete. 2024-09-17T05:33:42.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 300, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:38.576108+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9909}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9909': {'gid': 9909, 'name': 'c', 'rank': 0, 'incarnation': 297, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/2696141215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 2696141215}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2696141215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9909, 'qdb_cluster': [9909]} 2024-09-17T05:33:42.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:33:42.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 -- 172.21.15.43:0/1591586411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 msgr2=0x7f82c8074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1591586411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c8074070 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009bc0 tx=0x7f82c001cb70 comp rx=0 tx=0).stop 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 -- 172.21.15.43:0/1591586411 shutdown_connections 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1591586411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c8074070 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1591586411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c806d570 0x7f82c80713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 -- 172.21.15.43:0/1591586411 >> 172.21.15.43:0/1591586411 conn(0x7f82c80d1100 msgr2=0x7f82c80d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 -- 172.21.15.43:0/1591586411 shutdown_connections 2024-09-17T05:33:42.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.374+0000 7f82c7e00640 1 -- 172.21.15.43:0/1591586411 wait complete. 2024-09-17T05:33:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 Processor -- start 2024-09-17T05:33:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- start start 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c80cd960 0x7f82c8166490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c81669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8169c00 con 0x7f82c80b1df0 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8169a80 con 0x7f82c80d0b30 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f82c8169d80 con 0x7f82c80cd960 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50664/0 (socket says 172.21.15.43:50664) 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 -- 172.21.15.43:0/1279157920 learned_addr learned my addr 172.21.15.43:0/1279157920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c6a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c80cd960 0x7f82c8166490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82bfe00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c81669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 -- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c80cd960 msgr2=0x7f82c8166490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c80cd960 0x7f82c8166490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 -- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 msgr2=0x7f82c81669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c81669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 -- 172.21.15.43:0/1279157920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c80b2740 con 0x7f82c80b1df0 2024-09-17T05:33:42.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7400640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f82c00b0040 tx=0x7f82c0005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82bf400640 1 -- 172.21.15.43:0/1279157920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00b2040 con 0x7f82c80b1df0 2024-09-17T05:33:42.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82bf400640 1 -- 172.21.15.43:0/1279157920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82c00041f0 con 0x7f82c80b1df0 2024-09-17T05:33:42.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81ffad0 con 0x7f82c80b1df0 2024-09-17T05:33:42.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82bf400640 1 -- 172.21.15.43:0/1279157920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004620 con 0x7f82c80b1df0 2024-09-17T05:33:42.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.378+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c8200070 con 0x7f82c80b1df0 2024-09-17T05:33:42.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82bf400640 1 -- 172.21.15.43:0/1279157920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82c00bd030 con 0x7f82c80b1df0 2024-09-17T05:33:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82bf400640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 0x7f82a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82bf400640 1 -- 172.21.15.43:0/1279157920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f82c012ef20 con 0x7f82c80b1df0 2024-09-17T05:33:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82c6a00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 0x7f82a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82d5617000 1 Processor -- start 2024-09-17T05:33:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.382+0000 7f82d5617000 1 -- start start 2024-09-17T05:33:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c6a00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 0x7f82a4075710 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82b8007fd0 tx=0x7f82b8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6e8950 0x563a2c839be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a2c70e410 con 0x563a2c6d1ca0 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a2c64bb40 con 0x563a2c6e8950 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563a2c63af70 con 0x563a2c2f8460 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50668/0 (socket says 172.21.15.43:50668) 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 -- 172.21.15.43:0/4003430059 learned_addr learned my addr 172.21.15.43:0/4003430059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c6a00640 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82bfe00640 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6e8950 0x563a2c839be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 -- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 msgr2=0x563a2c8396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 -- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6e8950 msgr2=0x563a2c839be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6e8950 0x563a2c839be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 -- 172.21.15.43:0/4003430059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2c6025b0 con 0x563a2c6d1ca0 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82c7400640 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f82c00f60f0 tx=0x7f82c00f4e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996b23232bf5bbad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82b7400640 1 -- 172.21.15.43:0/4003430059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5b30 con 0x563a2c6d1ca0 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82b7400640 1 -- 172.21.15.43:0/4003430059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82c00f8990 con 0x563a2c6d1ca0 2024-09-17T05:33:42.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82b7400640 1 -- 172.21.15.43:0/4003430059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00f8e80 con 0x563a2c6d1ca0 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- 172.21.15.43:0/4003430059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 msgr2=0x563a2c83a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f82c00f60f0 tx=0x7f82c00f4e50 comp rx=0 tx=0).stop 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- 172.21.15.43:0/4003430059 shutdown_connections 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6e8950 0x563a2c839be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6d1ca0 0x563a2c83a120 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 --2- 172.21.15.43:0/4003430059 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- 172.21.15.43:0/4003430059 >> 172.21.15.43:0/4003430059 conn(0x563a2c7bed90 msgr2=0x563a2c7ee690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- 172.21.15.43:0/4003430059 shutdown_connections 2024-09-17T05:33:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.386+0000 7f82d5617000 1 -- 172.21.15.43:0/4003430059 wait complete. 2024-09-17T05:33:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- start start 2024-09-17T05:33:42.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 Processor -- start 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 0x563a2c840920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c6d1ca0 0x563a2c840e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a2c5d55f0 con 0x563a2c6e8950 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a2c63ed80 con 0x563a2c2f8460 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563a2c5e1520 con 0x563a2c6d1ca0 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 10 client.0 _ll_register_callbacks cb 0x563a2c5e5830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50680/0 (socket says 172.21.15.43:50680) 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 -- 172.21.15.43:0/2897298870 learned_addr learned my addr 172.21.15.43:0/2897298870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82bfe00640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c6d1ca0 0x563a2c840e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c6a00640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 0x563a2c840920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 -- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c6d1ca0 msgr2=0x563a2c840e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c6d1ca0 0x563a2c840e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 -- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 msgr2=0x563a2c840920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 0x563a2c840920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 -- 172.21.15.43:0/2897298870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2c5d9190 con 0x563a2c6e8950 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c6a00640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 0x563a2c840920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82c7400640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009bc0 tx=0x7f82c0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0245361c12ea652 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82abe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00fd8b0 con 0x563a2c6e8950 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a2c5d3110 con 0x563a2c6e8950 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f82c01022d0 con 0x563a2c6e8950 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=300) ==== 24+0+0 (secure 0 0 0) 0x7f82c0102760 con 0x563a2c6e8950 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 10 client.9966 fetch_fsmap learned FSMap version 300 2024-09-17T05:33:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=300,monmap=2+}) -- 0x563a2c6bff10 con 0x563a2c6e8950 2024-09-17T05:33:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.390+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563a2c5d3110 con 0x563a2c6e8950 2024-09-17T05:33:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 300) ==== 3502+0+0 (secure 0 0 0) 0x7f82c00b5070 con 0x563a2c6e8950 2024-09-17T05:33:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 10 client.9966 fetch_fsmap finished waiting for FSMap version 300 2024-09-17T05:33:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 10 client.9966 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 20 client.9966 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2dd7d0 0x563a2c84ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 4 client.9966 mds_command: new command op to 9909 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 --> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a2c64da90 con 0x563a2c2dd7d0 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82c6a00640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2dd7d0 0x563a2c84ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82abe00640 10 client.9966 ms_handle_connect on v2:172.21.15.43:6833/2696141215 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82c6a00640 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2dd7d0 0x563a2c84ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72b12a55d0191f3 server_cookie=37af2a6cbb58c6ed in_seq=0 out_seq=0 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82abe00640 1 -- 172.21.15.43:0/2897298870 <== mds.0 v2:172.21.15.43:6833/2696141215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a2c64da90 con 0x563a2c2dd7d0 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82abe00640 10 client.9966 handle_command_reply: tid=0 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 client.9966 shutdown 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 msgr2=0x563a2c8413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009bc0 tx=0x7f82c0009280 comp rx=0 tx=0).stop 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 shutdown_connections 2024-09-17T05:33:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2dd7d0 0x563a2c84ad00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8413a0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c6d1ca0 0x563a2c840e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 --2- 172.21.15.43:0/2897298870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c2f8460 0x563a2c840920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82abe00640 0 client.9966 ms_handle_reset on v2:172.21.15.43:6833/2696141215 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 >> 172.21.15.43:0/2897298870 conn(0x563a2c7bed90 msgr2=0x563a2c7f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 shutdown_connections 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 1 -- 172.21.15.43:0/2897298870 wait complete. 2024-09-17T05:33:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.394+0000 7f82d5617000 20 client.9966 trim_cache size 0 max 16384 2024-09-17T05:33:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 Processor -- start 2024-09-17T05:33:42.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 -- start start 2024-09-17T05:33:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82c820aaf0 con 0x563a2c6e8950 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8209af0 con 0x563a2c6d1ca0 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f82c8208e50 con 0x563a2c2f8460 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50682/0 (socket says 172.21.15.43:50682) 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82c7400640 1 -- 172.21.15.43:0/3399855693 learned_addr learned my addr 172.21.15.43:0/3399855693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82bfe00640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.414+0000 7f82c6a00640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 -- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 msgr2=0x563a2c8ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 -- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 msgr2=0x563a2c8abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 -- 172.21.15.43:0/3399855693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2c696db0 con 0x563a2c6e8950 2024-09-17T05:33:42.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c6a00640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:42.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82c7400640 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f82c001d050 tx=0x7f82c00f8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f163f6266ddea45f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82b6a00640 1 -- 172.21.15.43:0/3399855693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00f8990 con 0x563a2c6e8950 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82b6a00640 1 -- 172.21.15.43:0/3399855693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82c00fd8b0 con 0x563a2c6e8950 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82b6a00640 1 -- 172.21.15.43:0/3399855693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c011e1d0 con 0x563a2c6e8950 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- 172.21.15.43:0/3399855693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 msgr2=0x563a2c8ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f82c001d050 tx=0x7f82c00f8ed0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- 172.21.15.43:0/3399855693 shutdown_connections 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8ac0b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 --2- 172.21.15.43:0/3399855693 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- 172.21.15.43:0/3399855693 >> 172.21.15.43:0/3399855693 conn(0x563a2c7bed90 msgr2=0x563a2c7edcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- 172.21.15.43:0/3399855693 shutdown_connections 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- 172.21.15.43:0/3399855693 wait complete. 2024-09-17T05:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 -- start start 2024-09-17T05:33:42.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.418+0000 7f82d5617000 1 Processor -- start 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a2c5e1520 con 0x563a2c6e8950 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c82094a0 con 0x563a2c6d1ca0 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x563a2c5d55f0 con 0x563a2c2f8460 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 10 client.0 _ll_register_callbacks cb 0x7f82c00f4d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50690/0 (socket says 172.21.15.43:50690) 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 -- 172.21.15.43:0/126845446 learned_addr learned my addr 172.21.15.43:0/126845446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82bfe00640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c6a00640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 -- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 msgr2=0x563a2c8af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 -- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 msgr2=0x563a2c8af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 -- 172.21.15.43:0/126845446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2c601ab0 con 0x563a2c6e8950 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c7400640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f82c00f8f40 tx=0x7f82c00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9c957d03c439ec server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c011e3e0 con 0x563a2c6e8950 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f82c0100ba0 con 0x563a2c6e8950 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c00fd8b0 con 0x563a2c6e8950 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=300) ==== 24+0+0 (secure 0 0 0) 0x7f82c0107050 con 0x563a2c6e8950 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 10 client.9972 fetch_fsmap learned FSMap version 300 2024-09-17T05:33:42.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=300,monmap=2+}) -- 0x563a2c5e0070 con 0x563a2c6e8950 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82c00fd8b0 con 0x563a2c6e8950 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 300) ==== 3502+0+0 (secure 0 0 0) 0x7f82c00a5ce0 con 0x563a2c6e8950 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 10 client.9972 fetch_fsmap finished waiting for FSMap version 300 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 10 client.9972 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 20 client.9972 populate_metadata read hostname 'smithi043' 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2cb830 0x563a2c86a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 4 client.9972 mds_command: new command op to 9909 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 --> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] -- command(tid 0: {"prefix": "status"}) -- 0x563a2c649540 con 0x563a2c2cb830 2024-09-17T05:33:42.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c6a00640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2cb830 0x563a2c86a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82c6a00640 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2cb830 0x563a2c86a6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1028936c6f2b7517 server_cookie=b300dd44af7cc81a in_seq=0 out_seq=0 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 10 client.9972 ms_handle_connect on v2:172.21.15.43:6833/2696141215 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 1 -- 172.21.15.43:0/126845446 <== mds.0 v2:172.21.15.43:6833/2696141215 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x563a2c649540 con 0x563a2c2cb830 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 10 client.9972 handle_command_reply: tid=0 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 client.9972 shutdown 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 msgr2=0x563a2c8afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f82c00f8f40 tx=0x7f82c00098a0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 shutdown_connections 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:6833/2696141215,v1:172.21.15.43:6835/2696141215] conn(0x563a2c2cb830 0x563a2c86a6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a2c6d1ca0 0x563a2c8af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a2c6e8950 0x563a2c8afab0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 --2- 172.21.15.43:0/126845446 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563a2c2f8460 0x563a2c8af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82abe00640 0 client.9972 ms_handle_reset on v2:172.21.15.43:6833/2696141215 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 >> 172.21.15.43:0/126845446 conn(0x563a2c7bed90 msgr2=0x563a2c84a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.422+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 shutdown_connections 2024-09-17T05:33:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82d5617000 1 -- 172.21.15.43:0/126845446 wait complete. 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82d5617000 20 client.9972 trim_cache size 0 max 16384 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9909, 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8644657560000004, 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 300, 2024-09-17T05:33:42.431 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 123, 2024-09-17T05:33:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 123, 2024-09-17T05:33:42.432 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 8.9148114829999994 2024-09-17T05:33:42.432 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:33:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 msgr2=0x7f82a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 0x7f82a4075710 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f82b8007fd0 tx=0x7f82b8009040 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 msgr2=0x7f82c80b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f82c00b0040 tx=0x7f82c0005a60 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 shutdown_connections 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f82a4073200 0x7f82a4075710 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80d0b30 0x7f82c81669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c80b1df0 0x7f82c80b2200 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 --2- 172.21.15.43:0/1279157920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f82c80cd960 0x7f82c8166490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 >> 172.21.15.43:0/1279157920 conn(0x7f82c80d1100 msgr2=0x7f82c8143120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 shutdown_connections 2024-09-17T05:33:42.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.426+0000 7f82c7e00640 1 -- 172.21.15.43:0/1279157920 wait complete. 2024-09-17T05:33:42.447 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:33:42.448 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-09-17T05:33:42.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:33:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 -- 172.21.15.43:0/49790172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0240 msgr2=0x7fad340cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 --2- 172.21.15.43:0/49790172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0240 0x7fad340cd040 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fad20009840 tx=0x7fad2001c820 comp rx=0 tx=0).stop 2024-09-17T05:33:42.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 -- 172.21.15.43:0/49790172 shutdown_connections 2024-09-17T05:33:42.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 --2- 172.21.15.43:0/49790172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0240 0x7fad340cd040 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 -- 172.21.15.43:0/49790172 >> 172.21.15.43:0/49790172 conn(0x7fad340cee60 msgr2=0x7fad340cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.602+0000 7fad3c600640 1 -- 172.21.15.43:0/49790172 shutdown_connections 2024-09-17T05:33:42.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 -- 172.21.15.43:0/49790172 wait complete. 2024-09-17T05:33:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 Processor -- start 2024-09-17T05:33:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 -- start start 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3409ba60 0x7fad3414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 0x7fad34154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad3416aae0 con 0x7fad340d0650 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad3416a960 con 0x7fad3409ba60 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3c600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fad3416ac60 con 0x7fad340d0240 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50702/0 (socket says 172.21.15.43:50702) 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.606+0000 7fad3a600640 1 -- 172.21.15.43:0/430786947 learned_addr learned my addr 172.21.15.43:0/430786947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad39c00640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3409ba60 0x7fad3414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad39200640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 0x7fad34154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 -- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 msgr2=0x7fad34154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 0x7fad34154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 -- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3409ba60 msgr2=0x7fad3414e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3409ba60 0x7fad3414e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 -- 172.21.15.43:0/430786947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad34154f70 con 0x7fad340d0650 2024-09-17T05:33:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad39200640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 0x7fad34154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:42.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3a600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c700 tx=0x7fad2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c017070 con 0x7fad340d0650 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad2c00ce10 con 0x7fad340d0650 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad34151df0 con 0x7fad340d0650 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0052d0 con 0x7fad340d0650 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad341fbe70 con 0x7fad340d0650 2024-09-17T05:33:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad340b3130 con 0x7fad340d0650 2024-09-17T05:33:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.610+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad2c01a070 con 0x7fad340d0650 2024-09-17T05:33:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.614+0000 7fad31600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 0x7fad14075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.614+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7fad2c0943b0 con 0x7fad340d0650 2024-09-17T05:33:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.614+0000 7fad39c00640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 0x7fad14075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.614+0000 7fad39c00640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 0x7fad14075740 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b810 tx=0x7fad28008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.618+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad2c05e840 con 0x7fad340d0650 2024-09-17T05:33:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.826+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fad341590f0 con 0x7fad340d0650 2024-09-17T05:33:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.826+0000 7fad31600640 1 -- 172.21.15.43:0/430786947 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v300) ==== 53+0+73 (secure 0 0 0) 0x7fad2c05e140 con 0x7fad340d0650 2024-09-17T05:33:42.831 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:33:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 msgr2=0x7fad14075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 0x7fad14075740 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b810 tx=0x7fad28008040 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 msgr2=0x7fad3406df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c700 tx=0x7fad2c00cbc0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 shutdown_connections 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad14073230 0x7fad14075740 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad340d0650 0x7fad3406df70 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad340d0240 0x7fad34154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 --2- 172.21.15.43:0/430786947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad3409ba60 0x7fad3414e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 >> 172.21.15.43:0/430786947 conn(0x7fad340cee60 msgr2=0x7fad340cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 shutdown_connections 2024-09-17T05:33:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:42.830+0000 7fad3c600640 1 -- 172.21.15.43:0/430786947 wait complete. 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:42.853 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:33:42.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:33:42.854 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:33:42.854 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:42.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:42.854 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:33:42.864 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41821sec preferred_lft 41821sec 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:33:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:33:42.866 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:33:42.867 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:33:42.867 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:33:42.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:42.867 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:33:42.929 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:33:42.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:42.931 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:33:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:33:42.989 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:33:42.990 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:33:42.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:33:42.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:33:42.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:33:42.990 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:33:43.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:33:43.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:43.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:43.130 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:33:43.138 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:33:43.138 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:33:43.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:43.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:43.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:33:43.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:43.203 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:43.255 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:33:43.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:33:43.256 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:43.310 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:33:43.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:43.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:43.415 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:43.415 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:43.415 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:43.443 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:33:43.443 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:33:43.444 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:33:43.447 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:33:43.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:43.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:43.466 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:33:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:33:43.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:43.565 INFO:teuthology.orchestra.run.smithi043.stdout:client9984 2024-09-17T05:33:43.565 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:43.565 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:43.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:43.615 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:33:43.662 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:33:43.669 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:43.666+0000 7f6c57000640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:33:43.669 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:33:43.670 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:33:43.670 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:43.670 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:33:43.671 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-09-17T05:33:43.671 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-09-17T05:33:43.671 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:43.671 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:43.725 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:43.725 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:43.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:33:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 -- 172.21.15.43:0/371686502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 msgr2=0x7f67fc0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 --2- 172.21.15.43:0/371686502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0b33e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009840 tx=0x7f67e401c760 comp rx=0 tx=0).stop 2024-09-17T05:33:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 -- 172.21.15.43:0/371686502 shutdown_connections 2024-09-17T05:33:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 --2- 172.21.15.43:0/371686502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0b33e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 --2- 172.21.15.43:0/371686502 >> v2:172.21.15.62:3300/0 conn(0x7f67fc155d70 0x7f67fc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 -- 172.21.15.43:0/371686502 >> 172.21.15.43:0/371686502 conn(0x7f67fc0b23d0 msgr2=0x7f67fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 -- 172.21.15.43:0/371686502 shutdown_connections 2024-09-17T05:33:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.974+0000 7f6802400640 1 -- 172.21.15.43:0/371686502 wait complete. 2024-09-17T05:33:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 Processor -- start 2024-09-17T05:33:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 -- start start 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67fc0b3920 0x7f67fc0d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc168a60 con 0x7f67fc155d70 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1688e0 con 0x7f67fc0b3920 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f6802400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f67fc168be0 con 0x7f67fc06f110 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56226/0 (socket says 172.21.15.43:56226) 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67fc0b3920 0x7f67fc0d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 -- 172.21.15.43:0/3265850949 learned_addr learned my addr 172.21.15.43:0/3265850949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:43.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fb400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 -- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 msgr2=0x7f67fc0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 -- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67fc0b3920 msgr2=0x7f67fc0d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67fc0b3920 0x7f67fc0d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 -- 172.21.15.43:0/3265850949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67fc0cdb90 con 0x7f67fc155d70 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.978+0000 7f67fbe00640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c730 tx=0x7f67f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4003b80 con 0x7f67fc155d70 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67f4003d40 con 0x7f67fc155d70 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f40052d0 con 0x7f67fc155d70 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67fc1f74f0 con 0x7f67fc155d70 2024-09-17T05:33:43.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67fc0cd390 con 0x7f67fc155d70 2024-09-17T05:33:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67fc0b33e0 con 0x7f67fc155d70 2024-09-17T05:33:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67f4005470 con 0x7f67fc155d70 2024-09-17T05:33:43.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f67f4094a30 con 0x7f67fc155d70 2024-09-17T05:33:43.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.982+0000 7f67fb400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.986+0000 7f67fb400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 0x7f67d8075740 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0096c0 tx=0x7f67ec009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:43.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:43.986+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f67fc0b3d30 con 0x7f67fc155d70 2024-09-17T05:33:44.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.174+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67fc06e070 con 0x7f67fc155d70 2024-09-17T05:33:44.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f67f3400640 1 -- 172.21.15.43:0/3265850949 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v123) ==== 81+0+4134 (secure 0 0 0) 0x7f67f405e900 con 0x7f67fc155d70 2024-09-17T05:33:44.180 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1507988053 2024-09-18T05:33:33.376586+0000 2024-09-17T05:33:44.180 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:33:44.180 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:33:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:33:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:33:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:44.182 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:44.182 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:44.182 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:44.182 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:44.182 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:44.183 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:44.183 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:33:44.183 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:44.184 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:33:44.184 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:44.184 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:44.184 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:33:44.185 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:44.185 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:44.185 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:44.185 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:33:44.185 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:33:44.187 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:44.188 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:33:44.188 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:33:44.188 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:44.189 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:33:44.189 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:33:44.189 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:44.189 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1507988053 2024-09-18T05:33:33.376586+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:33:44.190 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:33:44.191 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:33:44.192 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:33:44.192 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:44.192 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:33:44.192 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:listed 68 entries 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 msgr2=0x7f67d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 0x7f67d8075740 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0096c0 tx=0x7f67ec009260 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 msgr2=0x7f67fc0c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c730 tx=0x7f67f400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 shutdown_connections 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f67d8073230 0x7f67d8075740 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67fc0b3920 0x7f67fc0d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67fc155d70 0x7f67fc0c5150 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 --2- 172.21.15.43:0/3265850949 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f67fc06f110 0x7f67fc0cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 >> 172.21.15.43:0/3265850949 conn(0x7f67fc0b23d0 msgr2=0x7f67fc09d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 shutdown_connections 2024-09-17T05:33:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.178+0000 7f6802400640 1 -- 172.21.15.43:0/3265850949 wait complete. 2024-09-17T05:33:44.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:33:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:44.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:33:44.226 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:33:44.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:44.226 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:44.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 -- 172.21.15.43:0/1335317178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7806fc90 msgr2=0x7f6c780700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 --2- 172.21.15.43:0/1335317178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7806fc90 0x7f6c780700a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f6c780ccb60 tx=0x7f6c6401d200 comp rx=0 tx=0).stop 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 -- 172.21.15.43:0/1335317178 shutdown_connections 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 --2- 172.21.15.43:0/1335317178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7806fc90 0x7f6c780700a0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 --2- 172.21.15.43:0/1335317178 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c780730b0 0x7f6c780734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 --2- 172.21.15.43:0/1335317178 >> v2:172.21.15.62:3300/0 conn(0x7f6c7815fe10 0x7f6c780b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 -- 172.21.15.43:0/1335317178 >> 172.21.15.43:0/1335317178 conn(0x7f6c780b23d0 msgr2=0x7f6c780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 -- 172.21.15.43:0/1335317178 shutdown_connections 2024-09-17T05:33:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.478+0000 7f6c80000640 1 -- 172.21.15.43:0/1335317178 wait complete. 2024-09-17T05:33:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 Processor -- start 2024-09-17T05:33:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 -- start start 2024-09-17T05:33:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c7806fc90 0x7f6c78155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 0x7f6c781f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c780cea60 con 0x7f6c7815fe10 2024-09-17T05:33:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c780ce8e0 con 0x7f6c780730b0 2024-09-17T05:33:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c80000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f6c780cebe0 con 0x7f6c7806fc90 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56248/0 (socket says 172.21.15.43:56248) 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 -- 172.21.15.43:0/1708259681 learned_addr learned my addr 172.21.15.43:0/1708259681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7d600640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c7806fc90 0x7f6c78155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7cc00640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 0x7f6c781f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 -- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c7806fc90 msgr2=0x7f6c78155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c7806fc90 0x7f6c78155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 -- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 msgr2=0x7f6c781f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 0x7f6c781f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 -- 172.21.15.43:0/1708259681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c781f8ef0 con 0x7f6c7815fe10 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7cc00640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 0x7f6c781f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c7e000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e990 tx=0x7f6c7000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004d30 con 0x7f6c7815fe10 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c7000caa0 con 0x7f6c7815fe10 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.482+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x7f6c7815fe10 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c78074bb0 con 0x7f6c7815fe10 2024-09-17T05:33:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781fd3f0 con 0x7f6c7815fe10 2024-09-17T05:33:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c780734c0 con 0x7f6c7815fe10 2024-09-17T05:33:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c700051c0 con 0x7f6c7815fe10 2024-09-17T05:33:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c75600640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 0x7f6c58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8633+0+0 (secure 0 0 0) 0x7f6c70095510 con 0x7f6c7815fe10 2024-09-17T05:33:44.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.486+0000 7f6c7d600640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 0x7f6c58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.490+0000 7f6c7d600640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 0x7f6c58075710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00bb50 tx=0x7f6c6c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:44.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.490+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6c70016030 con 0x7f6c7815fe10 2024-09-17T05:33:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.702+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c780aed00 con 0x7f6c7815fe10 2024-09-17T05:33:44.705 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:44.702+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c75600640 1 -- 172.21.15.43:0/1708259681 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v301) ==== 161+0+0 (secure 0 0 0) 0x7f6c7005f350 con 0x7f6c7815fe10 2024-09-17T05:33:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:44.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 msgr2=0x7f6c58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 0x7f6c58075710 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00bb50 tx=0x7f6c6c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 msgr2=0x7f6c780996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000e990 tx=0x7f6c7000ee50 comp rx=0 tx=0).stop 2024-09-17T05:33:44.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 shutdown_connections 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6c58073200 0x7f6c58075710 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c780730b0 0x7f6c781f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c7815fe10 0x7f6c780996b0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 --2- 172.21.15.43:0/1708259681 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f6c7806fc90 0x7f6c78155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.706+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 >> 172.21.15.43:0/1708259681 conn(0x7f6c780b23d0 msgr2=0x7f6c78159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.710+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 shutdown_connections 2024-09-17T05:33:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.710+0000 7f6c80000640 1 -- 172.21.15.43:0/1708259681 wait complete. 2024-09-17T05:33:44.716 INFO:tasks.ceph.mds.c.smithi043.stderr: -692> 2024-09-17T05:33:39.386+0000 7f6c5c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:44.717 INFO:tasks.ceph.mds.c.smithi043.stderr: -128> 2024-09-17T05:33:43.666+0000 7f6c57000640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:33:44.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:33:44.838 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:33:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- 172.21.15.43:0/639704083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0650 msgr2=0x7f433c0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 --2- 172.21.15.43:0/639704083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0650 0x7f433c0cd750 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f4334009c80 tx=0x7f433401cda0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- 172.21.15.43:0/639704083 shutdown_connections 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 --2- 172.21.15.43:0/639704083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0650 0x7f433c0cd750 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 --2- 172.21.15.43:0/639704083 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c09ba60 0x7f433c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- 172.21.15.43:0/639704083 >> 172.21.15.43:0/639704083 conn(0x7f433c0cee60 msgr2=0x7f433c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- 172.21.15.43:0/639704083 shutdown_connections 2024-09-17T05:33:44.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- 172.21.15.43:0/639704083 wait complete. 2024-09-17T05:33:44.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 Processor -- start 2024-09-17T05:33:44.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.894+0000 7f4341e00640 1 -- start start 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 0x7f433c14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 0x7f433c0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c0d0650 0x7f433c0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0b2b80 con 0x7f433c0d0240 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f433c0b2a00 con 0x7f433c09ba60 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f433c169e30 con 0x7f433c0d0650 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 0x7f433c0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f433aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 0x7f433c14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f433aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 0x7f433c14e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:33950/0 (socket says 172.21.15.43:33950) 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f433aa00640 1 -- 172.21.15.43:0/2437868359 learned_addr learned my addr 172.21.15.43:0/2437868359 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f433b400640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c0d0650 0x7f433c0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 -- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c0d0650 msgr2=0x7f433c0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c0d0650 0x7f433c0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 -- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 msgr2=0x7f433c14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 0x7f433c14e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:44.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 -- 172.21.15.43:0/2437868359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c0b0020 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4332000640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 0x7f433c0aef70 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f432400c3c0 tx=0x7f432400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324017070 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43240053c0 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324003860 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c14e990 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c0b05a0 con 0x7f433c0d0240 2024-09-17T05:33:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.898+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c0b3130 con 0x7f433c0d0240 2024-09-17T05:33:44.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.902+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f432401a070 con 0x7f433c0d0240 2024-09-17T05:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.902+0000 7f4333e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 0x7f4318075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.902+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8719+0+0 (secure 0 0 0) 0x7f43240941b0 con 0x7f433c0d0240 2024-09-17T05:33:44.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.902+0000 7f433aa00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 0x7f4318075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.902+0000 7f433aa00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 0x7f4318075710 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f432c005fd0 tx=0x7f432c00b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:44.906+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f432405dfb0 con 0x7f433c0d0240 2024-09-17T05:33:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.142+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f433c15d360 con 0x7f433c0d0240 2024-09-17T05:33:45.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.706+0000 7f4333e00640 1 -- 172.21.15.43:0/2437868359 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v302) ==== 107+0+0 (secure 0 0 0) 0x7f43240629e0 con 0x7f433c0d0240 2024-09-17T05:33:45.710 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:33:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 msgr2=0x7f4318075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 0x7f4318075710 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f432c005fd0 tx=0x7f432c00b390 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 msgr2=0x7f433c0aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 0x7f433c0aef70 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f432400c3c0 tx=0x7f432400c880 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 shutdown_connections 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4318073200 0x7f4318075710 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f433c0d0650 0x7f433c0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f433c0d0240 0x7f433c0aef70 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 --2- 172.21.15.43:0/2437868359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c09ba60 0x7f433c14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 >> 172.21.15.43:0/2437868359 conn(0x7f433c0cee60 msgr2=0x7f433c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 shutdown_connections 2024-09-17T05:33:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.710+0000 7f4341e00640 1 -- 172.21.15.43:0/2437868359 wait complete. 2024-09-17T05:33:45.735 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:45.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 Processor -- start 2024-09-17T05:33:45.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 -- start start 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634569659c0 0x563456a3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x563456a36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5634568aea00 con 0x5634569659c0 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56345690cf40 con 0x563456a370e0 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.790+0000 7fe5373ff080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5634569a6f00 con 0x563456a175b0 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:33966/0 (socket says 172.21.15.43:33966) 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 -- 172.21.15.43:0/3401031174 learned_addr learned my addr 172.21.15.43:0/3401031174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe52fc00640 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x563456a36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe530600640 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634569659c0 0x563456a3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 -- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 msgr2=0x563456a36ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x563456a36ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 -- 172.21.15.43:0/3401031174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634569659c0 msgr2=0x563456a3c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634569659c0 0x563456a3c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 -- 172.21.15.43:0/3401031174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563456887c10 con 0x563456a370e0 2024-09-17T05:33:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe531000640 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b790 tx=0x7fe52800bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86667195c5beab1e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe52e800640 1 -- 172.21.15.43:0/3401031174 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800db30 con 0x563456a370e0 2024-09-17T05:33:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe52e800640 1 -- 172.21.15.43:0/3401031174 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe52800ddc0 con 0x563456a370e0 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe52e800640 1 -- 172.21.15.43:0/3401031174 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800f690 con 0x563456a370e0 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 msgr2=0x563456961440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b790 tx=0x7fe52800bc50 comp rx=0 tx=0).stop 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- 172.21.15.43:0/3401031174 shutdown_connections 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563456a370e0 0x563456961440 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x563456a36ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 --2- 172.21.15.43:0/3401031174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634569659c0 0x563456a3c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- 172.21.15.43:0/3401031174 >> 172.21.15.43:0/3401031174 conn(0x56345694cd80 msgr2=0x5634569a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- 172.21.15.43:0/3401031174 shutdown_connections 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- 172.21.15.43:0/3401031174 wait complete. 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 Processor -- start 2024-09-17T05:33:45.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.794+0000 7fe5373ff080 1 -- start start 2024-09-17T05:33:45.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634569659c0 0x563456a33540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x5634569a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5634569ab3d0 con 0x563456a370e0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634569a1cb0 con 0x5634569659c0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5634569aae90 con 0x563456a175b0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56316/0 (socket says 172.21.15.43:56316) 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 learned_addr learned my addr 172.21.15.43:0/2115316220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634569659c0 0x563456a33540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x5634569a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 msgr2=0x5634569a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x563456a175b0 0x5634569a9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634569659c0 msgr2=0x563456a33540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634569659c0 0x563456a33540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563456ad0500 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe531000640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe52800dbd0 tx=0x7fe52800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe52d400640 1 -- 172.21.15.43:0/2115316220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280107d0 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563456ad07a0 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe52d400640 1 -- 172.21.15.43:0/2115316220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5280109b0 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe52d400640 1 -- 172.21.15.43:0/2115316220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528010da0 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.798+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563456ad0ce0 con 0x563456a370e0 2024-09-17T05:33:45.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe52d400640 1 -- 172.21.15.43:0/2115316220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe52802b430 con 0x563456a370e0 2024-09-17T05:33:45.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe52d400640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 0x7fe5100740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe52d400640 1 -- 172.21.15.43:0/2115316220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8719+0+0 (secure 0 0 0) 0x7fe528095ad0 con 0x563456a370e0 2024-09-17T05:33:45.805 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 0x7fe5100740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563456ad1420 0x563456ad1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ad1d70 con 0x563456ad1420 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563456ad1420 0x563456ad1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 0x7fe5100740f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ab000 tx=0x7fe520002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.802+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563456ad1420 0x563456ad1830 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.806+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v124'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.806+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ad2140 con 0x563456ad1420 2024-09-17T05:33:45.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.806+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v124'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.806+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ad24e0 con 0x563456ad1420 2024-09-17T05:33:45.812 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.810+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v124'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.810+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adda70 con 0x563456ad1420 2024-09-17T05:33:45.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.810+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.810+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adde10 con 0x563456ad1420 2024-09-17T05:33:45.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adca60 con 0x563456ad1420 2024-09-17T05:33:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adce50 con 0x563456ad1420 2024-09-17T05:33:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v124'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.814+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456add240 con 0x563456ad1420 2024-09-17T05:33:45.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.818+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v124'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.818+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add240 con 0x563456ad1420 2024-09-17T05:33:45.822 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.818+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v124'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.818+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.822+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.826+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.826+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.826+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.826+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.831 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:45.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.826+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.830+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.830+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.830+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.830+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.830+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456add1f0 con 0x563456ad1420 2024-09-17T05:33:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v124'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.834+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.838+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v124'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.838+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.842 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:45.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.838+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v124'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.838+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adf1c0 con 0x563456ad1420 2024-09-17T05:33:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adf1c0 con 0x563456ad1420 2024-09-17T05:33:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.842+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v124'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v124'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.851 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v124'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.846+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adf1c0 con 0x563456ad1420 2024-09-17T05:33:45.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.850+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.850+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456adf1c0 con 0x563456ad1420 2024-09-17T05:33:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.850+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.850+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ade1b0 con 0x563456ad1420 2024-09-17T05:33:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563456ade6c0 0x563456ae0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae1020 con 0x563456ade6c0 2024-09-17T05:33:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563456ade6c0 0x563456ae0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563456ade6c0 0x563456ae0ae0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'776 uv755 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.854+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae13c0 con 0x563456ade6c0 2024-09-17T05:33:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.858+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'777 uv777 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.858+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae1760 con 0x563456ade6c0 2024-09-17T05:33:45.861 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.858+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.858+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae1fb0 con 0x563456ade6c0 2024-09-17T05:33:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.862+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.862+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2350 con 0x563456ade6c0 2024-09-17T05:33:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.862+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.862+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae26f0 con 0x563456ade6c0 2024-09-17T05:33:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v124'781 uv774 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae2a90 con 0x563456ade6c0 2024-09-17T05:33:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v124'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v124'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.871 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.866+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.870+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.870+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.870+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.870+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.874+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v124'786 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.874+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.874+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v124'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.874+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.878+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v124'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.880 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:45.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.878+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae7000 con 0x563456ade6c0 2024-09-17T05:33:45.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.878+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v124'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.878+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae7000 con 0x563456ade6c0 2024-09-17T05:33:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.882+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.882+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.882+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v124'791 uv771 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.882+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v124'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v124'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.889 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae3e40 con 0x563456ade6c0 2024-09-17T05:33:45.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.886+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae3e40 con 0x563456ade6c0 2024-09-17T05:33:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae2e30 con 0x563456ade6c0 2024-09-17T05:33:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456ade6c0 2024-09-17T05:33:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563456ae34b0 0x563456ae58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae5ee0 con 0x563456ae34b0 2024-09-17T05:33:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.890+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563456ae34b0 0x563456ae58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.894+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563456ae34b0 0x563456ae58d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.894+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v124'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.894+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae5fa0 con 0x563456ae34b0 2024-09-17T05:33:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.894+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v124'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.894+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ae34b0 2024-09-17T05:33:45.899 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v124'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae73d0 con 0x563456ae34b0 2024-09-17T05:33:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae73d0 con 0x563456ae34b0 2024-09-17T05:33:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.898+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ae34b0 2024-09-17T05:33:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.902+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.902+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ad1420 2024-09-17T05:33:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.902+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'150 uv149 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.902+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ad1420 2024-09-17T05:33:45.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.906+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'151 uv151 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.906+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ad1420 2024-09-17T05:33:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.906+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.906+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae63c0 con 0x563456ad1420 2024-09-17T05:33:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563456ae6a40 0x563456ae8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae9430 con 0x563456ae6a40 2024-09-17T05:33:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe531000640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563456ae6a40 0x563456ae8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe531000640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563456ae6a40 0x563456ae8e60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v124'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.910+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456ae98f0 con 0x563456ae6a40 2024-09-17T05:33:45.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v124'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456ae9dd0 con 0x563456ae6a40 2024-09-17T05:33:45.917 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:45.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v124'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aeb680 con 0x563456ae6a40 2024-09-17T05:33:45.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.914+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aeba20 con 0x563456ae6a40 2024-09-17T05:33:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aea670 con 0x563456ae6a40 2024-09-17T05:33:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563456aeaf70 0x563456aeddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee310 con 0x563456aeaf70 2024-09-17T05:33:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563456aeaf70 0x563456aeddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.918+0000 7fe530600640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563456aeaf70 0x563456aeddd0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v124'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456aee310 con 0x563456aeaf70 2024-09-17T05:33:45.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v124'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee390 con 0x563456aeaf70 2024-09-17T05:33:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v124'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.922+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aef790 con 0x563456aeaf70 2024-09-17T05:33:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aef790 con 0x563456aeaf70 2024-09-17T05:33:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.926+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v124'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v124'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.935 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:45.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v124'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.930+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456aee780 con 0x563456aeaf70 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe530600640 1 -- 172.21.15.43:0/2115316220 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe5200acc60 con 0x563456aeaf70 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563456aeec90 0x563456af10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af16d0 con 0x563456aeec90 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563456aeec90 0x563456af10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.934+0000 7fe52fc00640 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563456aeec90 0x563456af10b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v124'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af16d0 con 0x563456aeec90 2024-09-17T05:33:45.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v124'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456aeec90 2024-09-17T05:33:45.943 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v124'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.938+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456aeec90 2024-09-17T05:33:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456aeec90 2024-09-17T05:33:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ad1420 2024-09-17T05:33:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v124'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.942+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ad1420 2024-09-17T05:33:45.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.946+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v124'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.946+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ad1420 2024-09-17T05:33:45.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.946+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v124'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.950 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:45.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.946+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af2b30 con 0x563456ad1420 2024-09-17T05:33:45.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af2b30 con 0x563456ad1420 2024-09-17T05:33:45.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ad1420 2024-09-17T05:33:45.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ad1420 2024-09-17T05:33:45.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.950+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.954+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v124'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.954+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.954+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v124'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.954+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.954+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v124'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.961 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.958+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.958+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.958+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.958+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.958+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v124'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v124'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v124'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.967 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.962+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1b20 con 0x563456ae34b0 2024-09-17T05:33:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.966+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.966+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af4930 con 0x563456ae34b0 2024-09-17T05:33:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.966+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456ae34b0 2024-09-17T05:33:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.966+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v124'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.974 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v124'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.970+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.974+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.974+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.974+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v124'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.974+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v124'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v124'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.981 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:45.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af2fe0 con 0x563456ae6a40 2024-09-17T05:33:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.978+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af2fe0 con 0x563456ae6a40 2024-09-17T05:33:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456ae6a40 2024-09-17T05:33:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe528002030 con 0x563456ae6a40 2024-09-17T05:33:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v124'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.982+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v124'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v124'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.990 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.986+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.990+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.990+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.990+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v124'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.990+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v124'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.996 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v124'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.994+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:45.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v124'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v124'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:45.998+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v124'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.004 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.002+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af1fd0 con 0x563456aeec90 2024-09-17T05:33:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x563456af2390 con 0x563456aeec90 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe52fc00640 1 -- 172.21.15.43:0/2115316220 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe51c00d040 con 0x563456aeec90 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563456ade6c0 msgr2=0x563456ae0ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x563456ade6c0 0x563456ae0ae0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563456aeaf70 msgr2=0x563456aeddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x563456aeaf70 0x563456aeddd0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563456aeec90 msgr2=0x563456af10b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x563456aeec90 0x563456af10b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563456ae34b0 msgr2=0x563456ae58d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x563456ae34b0 0x563456ae58d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563456ad1420 msgr2=0x563456ad1830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x563456ad1420 0x563456ad1830 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563456ae6a40 msgr2=0x563456ae8e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x563456ae6a40 0x563456ae8e60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 reap_dead start 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe531000640 1 -- 172.21.15.43:0/2115316220 reap_dead start 2024-09-17T05:33:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 msgr2=0x7fe5100740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 0x7fe5100740f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ab000 tx=0x7fe520002040 comp rx=0 tx=0).stop 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 msgr2=0x563456961a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe52800dbd0 tx=0x7fe52800dc00 comp rx=0 tx=0).stop 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 shutdown_connections 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe510071be0 0x7fe5100740f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 --2- 172.21.15.43:0/2115316220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563456a370e0 0x563456961a00 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 >> 172.21.15.43:0/2115316220 conn(0x56345694cd80 msgr2=0x5634569a2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 shutdown_connections 2024-09-17T05:33:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.006+0000 7fe5373ff080 1 -- 172.21.15.43:0/2115316220 wait complete. 2024-09-17T05:33:46.015 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:46.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.070+0000 7fc7da090080 1 Processor -- start 2024-09-17T05:33:46.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.070+0000 7fc7da090080 1 -- start start 2024-09-17T05:33:46.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e5038280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e501c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1e4f10f40 con 0x55e1e4f67720 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e1e4f1dd00 con 0x7fc7b8002350 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e1e4eb2a00 con 0x55e1e50378c0 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e5038280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e501c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d2800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:52690/0 (socket says 172.21.15.43:52690) 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d2800640 1 -- 172.21.15.43:0/3241956948 learned_addr learned my addr 172.21.15.43:0/3241956948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 -- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 msgr2=0x55e1e501c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 -- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 msgr2=0x55e1e5038280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e5038280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 -- 172.21.15.43:0/3241956948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e4fa8290 con 0x55e1e4f67720 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d2800640 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e501c9f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800af50 tx=0x7fc7cc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396d866b25729bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d1400640 1 -- 172.21.15.43:0/3241956948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc01d420 con 0x55e1e4f67720 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d1400640 1 -- 172.21.15.43:0/3241956948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7cc01d5c0 con 0x55e1e4f67720 2024-09-17T05:33:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7d1400640 1 -- 172.21.15.43:0/3241956948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc01da10 con 0x55e1e4f67720 2024-09-17T05:33:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- 172.21.15.43:0/3241956948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 msgr2=0x55e1e501c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e501c9f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800af50 tx=0x7fc7cc00b7b0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- 172.21.15.43:0/3241956948 shutdown_connections 2024-09-17T05:33:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e501c9f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e501c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 --2- 172.21.15.43:0/3241956948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e5038280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- 172.21.15.43:0/3241956948 >> 172.21.15.43:0/3241956948 conn(0x55e1e4f50d80 msgr2=0x55e1e4fad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- 172.21.15.43:0/3241956948 shutdown_connections 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.074+0000 7fc7da090080 1 -- 172.21.15.43:0/3241956948 wait complete. 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 Processor -- start 2024-09-17T05:33:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- start start 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e4faa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e50d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e50d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e1e50387c0 con 0x55e1e4f67720 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e1e4f86ba0 con 0x7fc7b8002350 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e1e4faad40 con 0x55e1e50378c0 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e4faa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e4faa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56336/0 (socket says 172.21.15.43:56336) 2024-09-17T05:33:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 -- 172.21.15.43:0/1292657802 learned_addr learned my addr 172.21.15.43:0/1292657802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d2800640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e50d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e50d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 -- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 msgr2=0x55e1e50d34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e1e50378c0 0x55e1e50d34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 -- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 msgr2=0x55e1e50d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8002350 0x55e1e50d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 -- 172.21.15.43:0/1292657802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e4f86dd0 con 0x55e1e4f67720 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7d3200640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e4faa800 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4096a30 tx=0x7fc7c40ab4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7cbe00640 1 -- 172.21.15.43:0/1292657802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40bc070 con 0x55e1e4f67720 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7cbe00640 1 -- 172.21.15.43:0/1292657802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7c40abb20 con 0x55e1e4f67720 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1e4f87070 con 0x55e1e4f67720 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7cbe00640 1 -- 172.21.15.43:0/1292657802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40b0040 con 0x55e1e4f67720 2024-09-17T05:33:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.078+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e4f875b0 con 0x55e1e4f67720 2024-09-17T05:33:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7cbe00640 1 -- 172.21.15.43:0/1292657802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7c40aa3d0 con 0x55e1e4f67720 2024-09-17T05:33:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7cbe00640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7b4071be0 0x7fc7b40740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7cbe00640 1 -- 172.21.15.43:0/1292657802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8719+0+0 (secure 0 0 0) 0x7fc7c412dfc0 con 0x55e1e4f67720 2024-09-17T05:33:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7d2800640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7b4071be0 0x7fc7b40740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7da090080 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e1e4f87cf0 0x55e1e50cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55e1e50d0b90 con 0x55e1e4f87cf0 2024-09-17T05:33:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7d2800640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7b4071be0 0x7fc7b40740f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0001870 tx=0x7fc7c0003040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.082+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e1e4f87cf0 0x55e1e50cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7d3c00640 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e1e4f87cf0 0x55e1e50cfd30 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7d3c00640 1 -- 172.21.15.43:0/1292657802 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v124'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc7cc008030 con 0x55e1e4f87cf0 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e1e4f87cf0 msgr2=0x55e1e50cfd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e1e4f87cf0 0x55e1e50cfd30 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7b4071be0 msgr2=0x7fc7b40740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc7b4071be0 0x7fc7b40740f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0001870 tx=0x7fc7c0003040 comp rx=0 tx=0).stop 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 msgr2=0x55e1e4faa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 --2- 172.21.15.43:0/1292657802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e1e4f67720 0x55e1e4faa800 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4096a30 tx=0x7fc7c40ab4b0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7d3c00640 1 -- 172.21.15.43:0/1292657802 reap_dead start 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 shutdown_connections 2024-09-17T05:33:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 >> 172.21.15.43:0/1292657802 conn(0x55e1e4f50d80 msgr2=0x55e1e5039250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 shutdown_connections 2024-09-17T05:33:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.086+0000 7fc7da090080 1 -- 172.21.15.43:0/1292657802 wait complete. 2024-09-17T05:33:46.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- 172.21.15.43:0/186306130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92540b3af0 msgr2=0x7f92540ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 --2- 172.21.15.43:0/186306130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92540b3af0 0x7f92540ad0b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f924c009c80 tx=0x7f924c01cce0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- 172.21.15.43:0/186306130 shutdown_connections 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 --2- 172.21.15.43:0/186306130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92540b3af0 0x7f92540ad0b0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 --2- 172.21.15.43:0/186306130 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9254155d70 0x7f925415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- 172.21.15.43:0/186306130 >> 172.21.15.43:0/186306130 conn(0x7f92540b23d0 msgr2=0x7f92540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- 172.21.15.43:0/186306130 shutdown_connections 2024-09-17T05:33:46.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- 172.21.15.43:0/186306130 wait complete. 2024-09-17T05:33:46.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 Processor -- start 2024-09-17T05:33:46.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.254+0000 7f925be00640 1 -- start start 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406f110 0x7f925415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 0x7f925409cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9254168c80 con 0x7f925415dbd0 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9254168b00 con 0x7f9254155d70 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9254168e00 con 0x7f925406f110 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56358/0 (socket says 172.21.15.43:56358) 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 -- 172.21.15.43:0/3500300486 learned_addr learned my addr 172.21.15.43:0/3500300486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259400640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406f110 0x7f925415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9258a00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 0x7f925409cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 -- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406f110 msgr2=0x7f925415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406f110 0x7f925415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 -- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 msgr2=0x7f925409cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 0x7f925409cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 -- 172.21.15.43:0/3500300486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92541f9ad0 con 0x7f925415dbd0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9258a00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 0x7f925409cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9259e00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f924c01b390 tx=0x7f924c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0bb070 con 0x7f925415dbd0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f924c004b40 con 0x7f925415dbd0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0b2040 con 0x7f925415dbd0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92541fcf00 con 0x7f925415dbd0 2024-09-17T05:33:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925409dd50 con 0x7f925415dbd0 2024-09-17T05:33:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.258+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92540ad0b0 con 0x7f925415dbd0 2024-09-17T05:33:46.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.262+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f924c004000 con 0x7f925415dbd0 2024-09-17T05:33:46.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.262+0000 7f9251600640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 0x7f9234075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:46.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.262+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8719+0+0 (secure 0 0 0) 0x7f924c12ed30 con 0x7f925415dbd0 2024-09-17T05:33:46.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.262+0000 7f9259400640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 0x7f9234075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:46.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.262+0000 7f9259400640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 0x7f9234075740 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f92540715c0 tx=0x7f92480091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.266+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f924c016990 con 0x7f925415dbd0 2024-09-17T05:33:46.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:46.478+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f92540ca540 con 0x7f925415dbd0 2024-09-17T05:33:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.070+0000 7f9251600640 1 -- 172.21.15.43:0/3500300486 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v303) ==== 162+0+0 (secure 0 0 0) 0x7f924c0f90e0 con 0x7f925415dbd0 2024-09-17T05:33:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 msgr2=0x7f9234075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 0x7f9234075740 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f92540715c0 tx=0x7f92480091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 msgr2=0x7f9254164770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f924c01b390 tx=0x7f924c005b00 comp rx=0 tx=0).stop 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 shutdown_connections 2024-09-17T05:33:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9234073230 0x7f9234075740 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9254155d70 0x7f925409cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f925415dbd0 0x7f9254164770 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 --2- 172.21.15.43:0/3500300486 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f925406f110 0x7f925415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 >> 172.21.15.43:0/3500300486 conn(0x7f92540b23d0 msgr2=0x7f92540d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 shutdown_connections 2024-09-17T05:33:47.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.074+0000 7f925be00640 1 -- 172.21.15.43:0/3500300486 wait complete. 2024-09-17T05:33:47.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- 172.21.15.43:0/2747152396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 msgr2=0x7f16b40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 --2- 172.21.15.43:0/2747152396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b40ad0b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f16b4072b60 tx=0x7f16ac01ccc0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- 172.21.15.43:0/2747152396 shutdown_connections 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 --2- 172.21.15.43:0/2747152396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b40ad0b0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 --2- 172.21.15.43:0/2747152396 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 0x7f16b40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 --2- 172.21.15.43:0/2747152396 >> v2:172.21.15.62:3300/0 conn(0x7f16b4155d70 0x7f16b415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- 172.21.15.43:0/2747152396 >> 172.21.15.43:0/2747152396 conn(0x7f16b40b23d0 msgr2=0x7f16b40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- 172.21.15.43:0/2747152396 shutdown_connections 2024-09-17T05:33:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- 172.21.15.43:0/2747152396 wait complete. 2024-09-17T05:33:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 Processor -- start 2024-09-17T05:33:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.266+0000 7f16bb800640 1 -- start start 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 0x7f16b4154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b4155d70 0x7f16b4165b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16b415f8d0 con 0x7f16b40b3920 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16b415f750 con 0x7f16b4155d70 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f16b415fa50 con 0x7f16b406f110 2024-09-17T05:33:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56368/0 (socket says 172.21.15.43:56368) 2024-09-17T05:33:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 -- 172.21.15.43:0/1506689652 learned_addr learned my addr 172.21.15.43:0/1506689652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16b8e00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 0x7f16b4154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16b9800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b4155d70 0x7f16b4165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 -- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 msgr2=0x7f16b4154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 0x7f16b4154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 -- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b4155d70 msgr2=0x7f16b4165b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b4155d70 0x7f16b4165b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 -- 172.21.15.43:0/1506689652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b4074500 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16abe00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d6d0 tx=0x7f169c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c017070 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f169c009b10 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b40747e0 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c010460 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b40ca980 con 0x7f16b40b3920 2024-09-17T05:33:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.270+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b4074270 con 0x7f16b40b3920 2024-09-17T05:33:47.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.274+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f169c01a070 con 0x7f16b40b3920 2024-09-17T05:33:47.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.274+0000 7f16b1600640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 0x7f1690075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.274+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8719+0+0 (secure 0 0 0) 0x7f169c094f90 con 0x7f16b40b3920 2024-09-17T05:33:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.274+0000 7f16b8e00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 0x7f1690075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.274+0000 7f16b8e00640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 0x7f1690075740 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005fd0 tx=0x7f16a400b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.278+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f169c05ec70 con 0x7f16b40b3920 2024-09-17T05:33:47.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.490+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16b415ff00 con 0x7f16b40b3920 2024-09-17T05:33:47.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.490+0000 7f16b1600640 1 -- 172.21.15.43:0/1506689652 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+5221 (secure 0 0 0) 0x7f169c0636a0 con 0x7f16b40b3920 2024-09-17T05:33:47.494 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:47.494 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":304,"btime":"2024-09-17T05:33:47:078677+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:47.078655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10002":{"gid":10002,"name":"c","rank":0,"incarnation":304,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/3094176220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3094176220},{"type":"v1","addr":"172.21.15.43:6835","nonce":3094176220}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:47.494 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 304 2024-09-17T05:33:47.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 msgr2=0x7f1690075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 0x7f1690075740 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005fd0 tx=0x7f16a400b390 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 msgr2=0x7f16b41655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f169c00d6d0 tx=0x7f169c00db90 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 shutdown_connections 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1690073230 0x7f1690075740 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b4155d70 0x7f16b4165b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16b40b3920 0x7f16b41655f0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 --2- 172.21.15.43:0/1506689652 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16b406f110 0x7f16b4154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 >> 172.21.15.43:0/1506689652 conn(0x7f16b40b23d0 msgr2=0x7f16b40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 shutdown_connections 2024-09-17T05:33:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:47.494+0000 7f16bb800640 1 -- 172.21.15.43:0/1506689652 wait complete. 2024-09-17T05:33:47.517 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:33:50.111 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:50.106+0000 7fc8c4800640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:33:50.111 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:50.106+0000 7fc8c4800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-17T05:33:50.112 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:50.110+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:50.112 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:33:50.110+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:33:50.719 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:50.718+0000 7fc8c9800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:52.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:52.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:52.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:52.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 -- 172.21.15.43:0/2123465712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f406fc90 msgr2=0x7f68f40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 --2- 172.21.15.43:0/2123465712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f406fc90 0x7f68f40700a0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009840 tx=0x7f68e001c700 comp rx=0 tx=0).stop 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 -- 172.21.15.43:0/2123465712 shutdown_connections 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 --2- 172.21.15.43:0/2123465712 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68f40730b0 0x7f68f40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 --2- 172.21.15.43:0/2123465712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f406fc90 0x7f68f40700a0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 -- 172.21.15.43:0/2123465712 >> 172.21.15.43:0/2123465712 conn(0x7f68f40b23d0 msgr2=0x7f68f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 -- 172.21.15.43:0/2123465712 shutdown_connections 2024-09-17T05:33:52.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.690+0000 7f68f9a00640 1 -- 172.21.15.43:0/2123465712 wait complete. 2024-09-17T05:33:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 Processor -- start 2024-09-17T05:33:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- start start 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68f406fc90 0x7f68f4156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 0x7f68f41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68f416a010 con 0x7f68f415fe10 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4169e90 con 0x7f68f40730b0 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f68f416a190 con 0x7f68f406fc90 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56390/0 (socket says 172.21.15.43:56390) 2024-09-17T05:33:52.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 -- 172.21.15.43:0/4240539304 learned_addr learned my addr 172.21.15.43:0/4240539304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f2000640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 0x7f68f41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 -- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68f406fc90 msgr2=0x7f68f4156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68f406fc90 0x7f68f4156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 -- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 msgr2=0x7f68f41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 0x7f68f41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 -- 172.21.15.43:0/4240539304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f4154f90 con 0x7f68f415fe10 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f2000640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 0x7f68f41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f3400640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fd0 tx=0x7f68ec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00ce20 con 0x7f68f415fe10 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68ec004b30 con 0x7f68f415fe10 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f4154ae0 con 0x7f68f415fe10 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec005020 con 0x7f68f415fe10 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.694+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f40aedb0 con 0x7f68f415fe10 2024-09-17T05:33:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f4155230 con 0x7f68f415fe10 2024-09-17T05:33:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68ec0051c0 con 0x7f68f415fe10 2024-09-17T05:33:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68eaa00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 0x7f68d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f68ec095ab0 con 0x7f68f415fe10 2024-09-17T05:33:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68f2a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 0x7f68d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.698+0000 7f68f2a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 0x7f68d4075710 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f68e40046c0 tx=0x7f68e40012c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:52.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.702+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f68ec05fe90 con 0x7f68f415fe10 2024-09-17T05:33:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.918+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68f4154dc0 con 0x7f68f415fe10 2024-09-17T05:33:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.918+0000 7f68eaa00640 1 -- 172.21.15.43:0/4240539304 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 308 v308) ==== 95+0+4383 (secure 0 0 0) 0x7f68ec05f790 con 0x7f68f415fe10 2024-09-17T05:33:52.923 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:52.923 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":308,"btime":"2024-09-17T05:33:50:112398+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":308,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:50.112393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 308 2024-09-17T05:33:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 msgr2=0x7f68d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 0x7f68d4075710 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f68e40046c0 tx=0x7f68e40012c0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 msgr2=0x7f68f4153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009fd0 tx=0x7f68ec00ef10 comp rx=0 tx=0).stop 2024-09-17T05:33:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 shutdown_connections 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f68d4073200 0x7f68d4075710 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f40730b0 0x7f68f41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68f415fe10 0x7f68f4153900 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 --2- 172.21.15.43:0/4240539304 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68f406fc90 0x7f68f4156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 >> 172.21.15.43:0/4240539304 conn(0x7f68f40b23d0 msgr2=0x7f68f40ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 shutdown_connections 2024-09-17T05:33:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:52.922+0000 7f68f9a00640 1 -- 172.21.15.43:0/4240539304 wait complete. 2024-09-17T05:33:52.945 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:33:52.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 -- 172.21.15.43:0/3573867018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61006fc90 msgr2=0x7fa6100700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 --2- 172.21.15.43:0/3573867018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61006fc90 0x7fa6100700a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 -- 172.21.15.43:0/3573867018 shutdown_connections 2024-09-17T05:33:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 --2- 172.21.15.43:0/3573867018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa6100734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 --2- 172.21.15.43:0/3573867018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61006fc90 0x7fa6100700a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 -- 172.21.15.43:0/3573867018 >> 172.21.15.43:0/3573867018 conn(0x7fa6100b23d0 msgr2=0x7fa6100d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 -- 172.21.15.43:0/3573867018 shutdown_connections 2024-09-17T05:33:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.102+0000 7fa616e00640 1 -- 172.21.15.43:0/3573867018 wait complete. 2024-09-17T05:33:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 Processor -- start 2024-09-17T05:33:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 -- start start 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa61015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61015fe10 0x7fa61015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa61016a060 con 0x7fa61006fc90 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa610169ee0 con 0x7fa61015fe10 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa616e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa61016a1e0 con 0x7fa6100730b0 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56394/0 (socket says 172.21.15.43:56394) 2024-09-17T05:33:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 -- 172.21.15.43:0/2805545812 learned_addr learned my addr 172.21.15.43:0/2805545812 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa614e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61015fe10 0x7fa61015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa607e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa61015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 -- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 msgr2=0x7fa61015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa61015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 -- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61015fe10 msgr2=0x7fa61015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61015fe10 0x7fa61015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 -- 172.21.15.43:0/2805545812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa610157f20 con 0x7fa61006fc90 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa607e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa61015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.106+0000 7fa60fe00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c400 tx=0x7fa60000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600017070 con 0x7fa61006fc90 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa600003600 con 0x7fa61006fc90 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa610158200 con 0x7fa61006fc90 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600003a50 con 0x7fa61006fc90 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa610154860 con 0x7fa61006fc90 2024-09-17T05:33:53.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6100700a0 con 0x7fa61006fc90 2024-09-17T05:33:53.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa60001a070 con 0x7fa61006fc90 2024-09-17T05:33:53.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 0x7fa5ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.110+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7fa600093ff0 con 0x7fa61006fc90 2024-09-17T05:33:53.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.114+0000 7fa607e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 0x7fa5ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.114+0000 7fa607e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 0x7fa5ec075740 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80af000 tx=0x7fa5f801b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.114+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa610073a00 con 0x7fa61006fc90 2024-09-17T05:33:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa610074e90 con 0x7fa61006fc90 2024-09-17T05:33:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa60cc00640 1 -- 172.21.15.43:0/2805545812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 308 v308) ==== 95+0+4383 (secure 0 0 0) 0x7fa60005de30 con 0x7fa61006fc90 2024-09-17T05:33:53.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:53.329 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":308,"btime":"2024-09-17T05:33:50:112398+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":308,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:50.112393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 308 2024-09-17T05:33:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 msgr2=0x7fa5ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 0x7fa5ec075740 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80af000 tx=0x7fa5f801b000 comp rx=0 tx=0).stop 2024-09-17T05:33:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 msgr2=0x7fa610099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c400 tx=0x7fa60000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 shutdown_connections 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa5ec073230 0x7fa5ec075740 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa61015fe10 0x7fa61015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa6100730b0 0x7fa61015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 --2- 172.21.15.43:0/2805545812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61006fc90 0x7fa610099b20 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 >> 172.21.15.43:0/2805545812 conn(0x7fa6100b23d0 msgr2=0x7fa6100702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 shutdown_connections 2024-09-17T05:33:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.326+0000 7fa616e00640 1 -- 172.21.15.43:0/2805545812 wait complete. 2024-09-17T05:33:53.352 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:33:53.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 -- 172.21.15.43:0/3978975278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cf220 msgr2=0x7fd6b80cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 --2- 172.21.15.43:0/3978975278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cf220 0x7fd6b80cf630 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fd6b809f3a0 tx=0x7fd6b001cd50 comp rx=0 tx=0).stop 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 -- 172.21.15.43:0/3978975278 shutdown_connections 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 --2- 172.21.15.43:0/3978975278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cf220 0x7fd6b80cf630 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 --2- 172.21.15.43:0/3978975278 >> v2:172.21.15.62:3300/0 conn(0x7fd6b80cc1f0 0x7fd6b80cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 -- 172.21.15.43:0/3978975278 >> 172.21.15.43:0/3978975278 conn(0x7fd6b80b2de0 msgr2=0x7fd6b80b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 -- 172.21.15.43:0/3978975278 shutdown_connections 2024-09-17T05:33:53.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.506+0000 7fd6bee00640 1 -- 172.21.15.43:0/3978975278 wait complete. 2024-09-17T05:33:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 Processor -- start 2024-09-17T05:33:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 -- start start 2024-09-17T05:33:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cf220 0x7fd6b8151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6b80d0830 0x7fd6b81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80cff80 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80b23c0 con 0x7fd6b80cf220 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80d0100 con 0x7fd6b80d0830 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56420/0 (socket says 172.21.15.43:56420) 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bda00640 1 -- 172.21.15.43:0/4116383882 learned_addr learned my addr 172.21.15.43:0/4116383882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6bd000640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cf220 0x7fd6b8151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.510+0000 7fd6be400640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6b80d0830 0x7fd6b81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 -- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6b80d0830 msgr2=0x7fd6b81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6b80d0830 0x7fd6b81567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 -- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cf220 msgr2=0x7fd6b8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cf220 0x7fd6b8151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 -- 172.21.15.43:0/4116383882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b81581c0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bda00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c3f0 tx=0x7fd6ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac0052d0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ac007020 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac007470 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b81584a0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b8153ac0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b80cc600 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.514+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6ac0067e0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.518+0000 7fd6b5600640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 0x7fd698075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.518+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7fd6ac094350 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.518+0000 7fd6bd000640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 0x7fd698075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.518+0000 7fd6bd000640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 0x7fd698075710 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd6b81f67c0 tx=0x7fd6a400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.522+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6b80c6770 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b8071de0 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6b5600640 1 -- 172.21.15.43:0/4116383882 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 308 v308) ==== 95+0+4383 (secure 0 0 0) 0x7fd6ac001090 con 0x7fd6b80cc1f0 2024-09-17T05:33:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:53.733 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":308,"btime":"2024-09-17T05:33:50:112398+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":308,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:50.112393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:53.733 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 308 2024-09-17T05:33:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 msgr2=0x7fd698075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 0x7fd698075710 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd6b81f67c0 tx=0x7fd6a400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 msgr2=0x7fd6b8155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c3f0 tx=0x7fd6ac00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 shutdown_connections 2024-09-17T05:33:53.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd698073200 0x7fd698075710 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cf220 0x7fd6b8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd6b80d0830 0x7fd6b81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 --2- 172.21.15.43:0/4116383882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b80cc1f0 0x7fd6b8155e00 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 >> 172.21.15.43:0/4116383882 conn(0x7fd6b80b2de0 msgr2=0x7fd6b80b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 shutdown_connections 2024-09-17T05:33:53.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:53.730+0000 7fd6bee00640 1 -- 172.21.15.43:0/4116383882 wait complete. 2024-09-17T05:33:53.756 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-09-17T05:33:53.756 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-09-17T05:33:53.756 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:33:53.756 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:53.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:33:53.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:33:53.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:53.766 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:53.815 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:33:53.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:33:53.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 -- 172.21.15.43:0/2621203302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156006ce60 msgr2=0x7f156006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 --2- 172.21.15.43:0/2621203302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156006ce60 0x7f156006d270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f154c009840 tx=0x7f154c01c790 comp rx=0 tx=0).stop 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 -- 172.21.15.43:0/2621203302 shutdown_connections 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 --2- 172.21.15.43:0/2621203302 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f156006ce60 0x7f156006d270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 --2- 172.21.15.43:0/2621203302 >> v2:172.21.15.62:3300/0 conn(0x7f1560070620 0x7f15600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 -- 172.21.15.43:0/2621203302 >> 172.21.15.43:0/2621203302 conn(0x7f15600b23d0 msgr2=0x7f15600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 -- 172.21.15.43:0/2621203302 shutdown_connections 2024-09-17T05:33:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.014+0000 7f1567800640 1 -- 172.21.15.43:0/2621203302 wait complete. 2024-09-17T05:33:54.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 Processor -- start 2024-09-17T05:33:54.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 -- start start 2024-09-17T05:33:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1560070620 0x7f15601597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600cb860 0x7f15600c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15600ce5b0 con 0x7f156006ce60 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15600ce430 con 0x7f15600cb860 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1567800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f15600ce730 con 0x7f1560070620 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56070/0 (socket says 172.21.15.43:56070) 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 -- 172.21.15.43:0/2078593397 learned_addr learned my addr 172.21.15.43:0/2078593397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f155fe00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1560070620 0x7f15601597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1565800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600cb860 0x7f15600c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 -- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1560070620 msgr2=0x7f15601597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1560070620 0x7f15601597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 -- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600cb860 msgr2=0x7f15600c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600cb860 0x7f15600c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 -- 172.21.15.43:0/2078593397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600c5120 con 0x7f156006ce60 2024-09-17T05:33:54.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.018+0000 7f1564e00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f155400e9a0 tx=0x7f155400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400cd80 con 0x7f156006ce60 2024-09-17T05:33:54.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15600c5400 con 0x7f156006ce60 2024-09-17T05:33:54.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1554004020 con 0x7f156006ce60 2024-09-17T05:33:54.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554004450 con 0x7f156006ce60 2024-09-17T05:33:54.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15600c59c0 con 0x7f156006ce60 2024-09-17T05:33:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15600cecc0 con 0x7f156006ce60 2024-09-17T05:33:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1554003600 con 0x7f156006ce60 2024-09-17T05:33:54.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 0x7f1540075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f15540958d0 con 0x7f156006ce60 2024-09-17T05:33:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.022+0000 7f155fe00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 0x7f1540075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.026+0000 7f155fe00640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 0x7f1540075710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f154c0af000 tx=0x7f154c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.030+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f155405fc30 con 0x7f156006ce60 2024-09-17T05:33:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:54.238+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1560159cf0 con 0x7f156006ce60 2024-09-17T05:33:54.865 INFO:tasks.ceph.mds.c.smithi043.stderr: -54> 2024-09-17T05:33:50.106+0000 7fc8c4800640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:33:54.865 INFO:tasks.ceph.mds.c.smithi043.stderr: -51> 2024-09-17T05:33:50.106+0000 7fc8c4800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-17T05:33:54.865 INFO:tasks.ceph.mds.c.smithi043.stderr: -37> 2024-09-17T05:33:50.718+0000 7fc8c9800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:33:54.989 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:33:55.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f155cc00640 1 -- 172.21.15.43:0/2078593397 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v309) ==== 161+0+0 (secure 0 0 0) 0x7f155405f530 con 0x7f156006ce60 2024-09-17T05:33:55.124 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 msgr2=0x7f1540075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 0x7f1540075710 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f154c0af000 tx=0x7f154c0b2040 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 msgr2=0x7f1560160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f155400e9a0 tx=0x7f155400ee60 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 shutdown_connections 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1540073200 0x7f1540075710 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15600cb860 0x7f15600c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1560070620 0x7f15601597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 --2- 172.21.15.43:0/2078593397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f156006ce60 0x7f1560160df0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 >> 172.21.15.43:0/2078593397 conn(0x7f15600b23d0 msgr2=0x7f15600b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 shutdown_connections 2024-09-17T05:33:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.122+0000 7f1567800640 1 -- 172.21.15.43:0/2078593397 wait complete. 2024-09-17T05:33:55.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:33:55.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- 172.21.15.43:0/268503067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b3af0 msgr2=0x7f520c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 --2- 172.21.15.43:0/268503067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b3af0 0x7f520c0ad0b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f520c072b60 tx=0x7f520401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- 172.21.15.43:0/268503067 shutdown_connections 2024-09-17T05:33:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 --2- 172.21.15.43:0/268503067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b3af0 0x7f520c0ad0b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 --2- 172.21.15.43:0/268503067 >> v2:172.21.15.62:3300/0 conn(0x7f520c155d70 0x7f520c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- 172.21.15.43:0/268503067 >> 172.21.15.43:0/268503067 conn(0x7f520c0b23d0 msgr2=0x7f520c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- 172.21.15.43:0/268503067 shutdown_connections 2024-09-17T05:33:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- 172.21.15.43:0/268503067 wait complete. 2024-09-17T05:33:55.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 Processor -- start 2024-09-17T05:33:55.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.306+0000 7f5212600640 1 -- start start 2024-09-17T05:33:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520c06f110 0x7f520c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f520c09c0e0 0x7f520c09c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f520c168c30 con 0x7f520c155d70 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f520c168ab0 con 0x7f520c06f110 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f520c168db0 con 0x7f520c09c0e0 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56096/0 (socket says 172.21.15.43:56096) 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 -- 172.21.15.43:0/2245583194 learned_addr learned my addr 172.21.15.43:0/2245583194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f520b400640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520c06f110 0x7f520c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f520be00640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f520c09c0e0 0x7f520c09c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 -- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f520c09c0e0 msgr2=0x7f520c09c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f520c09c0e0 0x7f520c09c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 -- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520c06f110 msgr2=0x7f520c15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520c06f110 0x7f520c15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 -- 172.21.15.43:0/2245583194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c09ca30 con 0x7f520c155d70 2024-09-17T05:33:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5202a00640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c3f0 tx=0x7f51f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017070 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51f40053c0 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4003860 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c0c9030 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c0c95f0 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.310+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c0c8da0 con 0x7f520c155d70 2024-09-17T05:33:55.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.314+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f51f401a070 con 0x7f520c155d70 2024-09-17T05:33:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.314+0000 7f5203e00640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 0x7f51e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.314+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f51f4094100 con 0x7f520c155d70 2024-09-17T05:33:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.314+0000 7f520b400640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 0x7f51e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.314+0000 7f520b400640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 0x7f51e8075740 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f520c0715c0 tx=0x7f51fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.318+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f520c09ca30 con 0x7f520c155d70 2024-09-17T05:33:55.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:55.554+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f520c0ad0b0 con 0x7f520c155d70 2024-09-17T05:33:56.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.126+0000 7f5203e00640 1 -- 172.21.15.43:0/2245583194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v310) ==== 101+0+0 (secure 0 0 0) 0x7f51f405de90 con 0x7f520c155d70 2024-09-17T05:33:56.130 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:33:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 msgr2=0x7f51e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 0x7f51e8075740 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f520c0715c0 tx=0x7f51fc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 msgr2=0x7f520c073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c3f0 tx=0x7f51f400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 shutdown_connections 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f51e8073230 0x7f51e8075740 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f520c09c0e0 0x7f520c09c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c155d70 0x7f520c073dc0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 --2- 172.21.15.43:0/2245583194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520c06f110 0x7f520c15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 >> 172.21.15.43:0/2245583194 conn(0x7f520c0b23d0 msgr2=0x7f520c15de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 shutdown_connections 2024-09-17T05:33:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.130+0000 7f5212600640 1 -- 172.21.15.43:0/2245583194 wait complete. 2024-09-17T05:33:56.156 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:33:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 Processor -- start 2024-09-17T05:33:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 -- start start 2024-09-17T05:33:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 0x55d86b30a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b2ed010 0x55d86b2ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d86b1e1f40 con 0x55d86b2ecbc0 2024-09-17T05:33:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d86b1eed00 con 0x55d86b234250 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9848049080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d86b183a00 con 0x55d86b2ed010 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56104/0 (socket says 172.21.15.43:56104) 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 -- 172.21.15.43:0/447361675 learned_addr learned my addr 172.21.15.43:0/447361675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840400640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 0x55d86b30a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f983fa00640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b2ed010 0x55d86b2ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 -- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b2ed010 msgr2=0x55d86b2ed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b2ed010 0x55d86b2ed4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 -- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 msgr2=0x55d86b30a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 0x55d86b30a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 -- 172.21.15.43:0/447361675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86b30e450 con 0x55d86b2ecbc0 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840400640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 0x55d86b30a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.214+0000 7f9840e00640 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f983801b8f0 tx=0x7f98380181b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8888445d364679 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f983e600640 1 -- 172.21.15.43:0/447361675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380af030 con 0x55d86b2ecbc0 2024-09-17T05:33:56.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f983e600640 1 -- 172.21.15.43:0/447361675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838004110 con 0x55d86b2ecbc0 2024-09-17T05:33:56.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f983e600640 1 -- 172.21.15.43:0/447361675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380aa980 con 0x55d86b2ecbc0 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- 172.21.15.43:0/447361675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 msgr2=0x55d86b2ed9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f983801b8f0 tx=0x7f98380181b0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- 172.21.15.43:0/447361675 shutdown_connections 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ecbc0 0x55d86b2ed9f0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b2ed010 0x55d86b2ed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- 172.21.15.43:0/447361675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b234250 0x55d86b30a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- 172.21.15.43:0/447361675 >> 172.21.15.43:0/447361675 conn(0x55d86b221d80 msgr2=0x55d86b237b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- 172.21.15.43:0/447361675 shutdown_connections 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- 172.21.15.43:0/447361675 wait complete. 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 Processor -- start 2024-09-17T05:33:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- start start 2024-09-17T05:33:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b234250 0x55d86b3a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b2ed010 0x55d86b27dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d86b30cb50 con 0x55d86b2ee8b0 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d86b30df50 con 0x55d86b2ed010 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.218+0000 7f9848049080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d86b2ea360 con 0x55d86b234250 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b234250 0x55d86b3a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b234250 0x55d86b3a8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48324/0 (socket says 172.21.15.43:48324) 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56120/0 (socket says 172.21.15.43:56120) 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 learned_addr learned my addr 172.21.15.43:0/567850944 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b2ed010 0x55d86b27dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b234250 msgr2=0x55d86b3a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d86b234250 0x55d86b3a8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b2ed010 msgr2=0x55d86b27dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d86b2ed010 0x55d86b27dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86b2ea600 con 0x55d86b2ee8b0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9840e00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9838019040 tx=0x7f98380aa980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f983d200640 1 -- 172.21.15.43:0/567850944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380c6070 con 0x55d86b2ee8b0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f983d200640 1 -- 172.21.15.43:0/567850944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98380aad20 con 0x55d86b2ee8b0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f983d200640 1 -- 172.21.15.43:0/567850944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98380c16b0 con 0x55d86b2ee8b0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d86b2ea8a0 con 0x55d86b2ee8b0 2024-09-17T05:33:56.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86b2eade0 con 0x55d86b2ee8b0 2024-09-17T05:33:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.222+0000 7f983d200640 1 -- 172.21.15.43:0/567850944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f98380be070 con 0x55d86b2ee8b0 2024-09-17T05:33:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f983d200640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 0x7f98200740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f983d200640 1 -- 172.21.15.43:0/567850944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f983812eff0 con 0x55d86b2ee8b0 2024-09-17T05:33:56.228 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:33:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d86b235f20 0x55d86b236330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 0x7f98200740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b1230 con 0x55d86b235f20 2024-09-17T05:33:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d86b235f20 0x55d86b236330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 0x7f98200740f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f98300105a0 tx=0x7f9830003940 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.226+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d86b235f20 0x55d86b236330 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.230+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v125'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.230+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b15d0 con 0x55d86b235f20 2024-09-17T05:33:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.230+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v125'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.230+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b1970 con 0x55d86b235f20 2024-09-17T05:33:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v125'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.235 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:33:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b2d20 con 0x55d86b235f20 2024-09-17T05:33:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b30c0 con 0x55d86b235f20 2024-09-17T05:33:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.234+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b1d10 con 0x55d86b235f20 2024-09-17T05:33:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.238+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.238+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b2100 con 0x55d86b235f20 2024-09-17T05:33:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.238+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v125'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.238+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b24f0 con 0x55d86b235f20 2024-09-17T05:33:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v125'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24f0 con 0x55d86b235f20 2024-09-17T05:33:56.244 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:33:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v125'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.242+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.246+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.250+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.250+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.254 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:33:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.250+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.250+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.254+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.258+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.258+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.258+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v125'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.258+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v125'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.265 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:33:56.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v125'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.262+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v125'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.266+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.270+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v125'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.270+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.274 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:33:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.270+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v125'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.270+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b235f20 2024-09-17T05:33:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d86b3b2860 0x55d86b3b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b24a0 con 0x55d86b3b2860 2024-09-17T05:33:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.274+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d86b3b2860 0x55d86b3b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.278+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d86b3b2860 0x55d86b3b5980 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.278+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'798 uv780 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.278+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b5ec0 con 0x55d86b3b2860 2024-09-17T05:33:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.278+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'799 uv799 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.278+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b6260 con 0x55d86b3b2860 2024-09-17T05:33:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.282+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.285 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:33:56.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.282+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b6ab0 con 0x55d86b3b2860 2024-09-17T05:33:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.282+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.282+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b6e50 con 0x55d86b3b2860 2024-09-17T05:33:56.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b71f0 con 0x55d86b3b2860 2024-09-17T05:33:56.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v125'803 uv785 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b75e0 con 0x55d86b3b2860 2024-09-17T05:33:56.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v125'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.286+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b79d0 con 0x55d86b3b2860 2024-09-17T05:33:56.292 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:33:56.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v125'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.290+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.294+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v125'808 uv790 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.294+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.294+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v125'809 uv809 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.294+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v125'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.300 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:33:56.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v125'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.298+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.302+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v125'813 uv796 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.302+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.302+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v125'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.302+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.302+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v125'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.307 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:33:56.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.306+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.306+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.306+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.306+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.306+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3b66e0 con 0x55d86b3b2860 2024-09-17T05:33:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3b2860 2024-09-17T05:33:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d86b3b8340 0x55d86b3ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3baca0 con 0x55d86b3b8340 2024-09-17T05:33:56.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d86b3b8340 0x55d86b3ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d86b3b8340 0x55d86b3ba760 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v125'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.310+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3baca0 con 0x55d86b3b8340 2024-09-17T05:33:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v125'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b3b8340 2024-09-17T05:33:56.317 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:33:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v125'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bc0d0 con 0x55d86b3b8340 2024-09-17T05:33:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.314+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bc0d0 con 0x55d86b3b8340 2024-09-17T05:33:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b3b8340 2024-09-17T05:33:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b3b8340 2024-09-17T05:33:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b235f20 2024-09-17T05:33:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'154 uv153 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.318+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b235f20 2024-09-17T05:33:56.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.322+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.322+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b235f20 2024-09-17T05:33:56.326 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:33:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.322+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.322+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bb0c0 con 0x55d86b235f20 2024-09-17T05:33:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'157 uv157 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f982c0106a0 con 0x55d86b235f20 2024-09-17T05:33:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 0x55d86b3bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3be170 con 0x55d86b3bb740 2024-09-17T05:33:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9840e00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 0x55d86b3bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9840e00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 0x55d86b3bdbe0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v125'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.326+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3be660 con 0x55d86b3bb740 2024-09-17T05:33:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v125'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3beaf0 con 0x55d86b3bb740 2024-09-17T05:33:56.333 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:33:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v125'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c03a0 con 0x55d86b3bb740 2024-09-17T05:33:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.330+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c0740 con 0x55d86b3bb740 2024-09-17T05:33:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bf390 con 0x55d86b3bb740 2024-09-17T05:33:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d86b3bfcb0 0x55d86b3c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3030 con 0x55d86b3bfcb0 2024-09-17T05:33:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d86b3bfcb0 0x55d86b3c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.334+0000 7f9840400640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d86b3bfcb0 0x55d86b3c2af0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v125'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c3030 con 0x55d86b3bfcb0 2024-09-17T05:33:56.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v125'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3030 con 0x55d86b3bfcb0 2024-09-17T05:33:56.342 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:33:56.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v125'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.338+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c4430 con 0x55d86b3bfcb0 2024-09-17T05:33:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c4430 con 0x55d86b3bfcb0 2024-09-17T05:33:56.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.342+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v125'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v125'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.350 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:33:56.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v125'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.346+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c3420 con 0x55d86b3bfcb0 2024-09-17T05:33:56.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f9840400640 1 -- 172.21.15.43:0/567850944 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983001b040 con 0x55d86b3bfcb0 2024-09-17T05:33:56.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d86b3c3930 0x55d86b3c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c6450 con 0x55d86b3c3930 2024-09-17T05:33:56.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d86b3c3930 0x55d86b3c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.350+0000 7f983fa00640 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d86b3c3930 0x55d86b3c5d50 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v125'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c6450 con 0x55d86b3c3930 2024-09-17T05:33:56.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v125'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3c3930 2024-09-17T05:33:56.358 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:33:56.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v125'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.354+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3c3930 2024-09-17T05:33:56.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3c3930 2024-09-17T05:33:56.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b235f20 2024-09-17T05:33:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v125'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.358+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b235f20 2024-09-17T05:33:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v125'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b235f20 2024-09-17T05:33:56.365 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:33:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v125'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c78b0 con 0x55d86b235f20 2024-09-17T05:33:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.362+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c78b0 con 0x55d86b235f20 2024-09-17T05:33:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b235f20 2024-09-17T05:33:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b235f20 2024-09-17T05:33:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3b8340 2024-09-17T05:33:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v125'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.366+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3b8340 2024-09-17T05:33:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.370+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v125'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.370+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c68a0 con 0x55d86b3b8340 2024-09-17T05:33:56.373 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:33:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.370+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v125'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.370+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v125'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.374+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.378+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v125'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.378+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.382 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:33:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.378+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v125'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.378+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3bef80 con 0x55d86b3b8340 2024-09-17T05:33:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c96f0 con 0x55d86b3b8340 2024-09-17T05:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3b8340 2024-09-17T05:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c6d50 con 0x55d86b3bb740 2024-09-17T05:33:56.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v125'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.382+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c7140 con 0x55d86b3bb740 2024-09-17T05:33:56.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.386+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.386+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c7140 con 0x55d86b3bb740 2024-09-17T05:33:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:33:56.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.386+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v125'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.386+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v125'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v125'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.390+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.397 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:33:56.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.394+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v125'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.394+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.394+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.394+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.398+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.398+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3bb740 2024-09-17T05:33:56.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.398+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9838002050 con 0x55d86b3bb740 2024-09-17T05:33:56.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.398+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.398+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v125'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.402+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.402+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v125'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.402+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.406 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:33:56.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.402+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v125'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.402+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v125'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.406+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v125'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.413 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:33:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v125'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.410+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v125'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v125'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.414+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v125'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.420 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:33:56.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c8100 con 0x55d86b3c3930 2024-09-17T05:33:56.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c8100 con 0x55d86b3c3930 2024-09-17T05:33:56.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.418+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c70f0 con 0x55d86b3c3930 2024-09-17T05:33:56.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55d86b3c75a0 con 0x55d86b3c3930 2024-09-17T05:33:56.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f983fa00640 1 -- 172.21.15.43:0/567850944 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f982c027040 con 0x55d86b3c3930 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d86b3b2860 msgr2=0x55d86b3b5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55d86b3b2860 0x55d86b3b5980 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d86b3bfcb0 msgr2=0x55d86b3c2af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55d86b3bfcb0 0x55d86b3c2af0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d86b3c3930 msgr2=0x55d86b3c5d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55d86b3c3930 0x55d86b3c5d50 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d86b3b8340 msgr2=0x55d86b3ba760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55d86b3b8340 0x55d86b3ba760 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 reap_dead start 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d86b235f20 msgr2=0x55d86b236330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55d86b235f20 0x55d86b236330 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9840e00640 1 -- 172.21.15.43:0/567850944 reap_dead start 2024-09-17T05:33:56.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 msgr2=0x55d86b3bdbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 0x55d86b3bdbe0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 msgr2=0x7f98200740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 0x7f98200740f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f98300105a0 tx=0x7f9830003940 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 msgr2=0x55d86b2eecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9838019040 tx=0x7f98380aa980 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 shutdown_connections 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d86b3bb740 0x55d86b3bdbe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9820071be0 0x7f98200740f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 --2- 172.21.15.43:0/567850944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d86b2ee8b0 0x55d86b2eecc0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 >> 172.21.15.43:0/567850944 conn(0x55d86b221d80 msgr2=0x55d86b237b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 shutdown_connections 2024-09-17T05:33:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.422+0000 7f9848049080 1 -- 172.21.15.43:0/567850944 wait complete. 2024-09-17T05:33:56.432 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:33:56.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.486+0000 7fcae804b080 1 Processor -- start 2024-09-17T05:33:56.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.486+0000 7fcae804b080 1 -- start start 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a719b5cb0 0x559a719ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a719cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a71a3d8e0 con 0x559a71986d70 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a718d4a00 con 0x559a71a3ce40 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559a71a3db30 con 0x559a719b5cb0 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56140/0 (socket says 172.21.15.43:56140) 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 -- 172.21.15.43:0/41080854 learned_addr learned my addr 172.21.15.43:0/41080854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:56.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae1000640 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a719cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcadfc00640 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a719b5cb0 0x559a719ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 -- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a719b5cb0 msgr2=0x559a719ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a719b5cb0 0x559a719ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 -- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 msgr2=0x559a719cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a719cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 -- 172.21.15.43:0/41080854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a718d3a20 con 0x559a71986d70 2024-09-17T05:33:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae0600640 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c670 tx=0x7fcad000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c14b41d5628e023 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcade800640 1 -- 172.21.15.43:0/41080854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000add0 con 0x559a71986d70 2024-09-17T05:33:56.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcade800640 1 -- 172.21.15.43:0/41080854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcad0008450 con 0x559a71986d70 2024-09-17T05:33:56.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcade800640 1 -- 172.21.15.43:0/41080854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00088c0 con 0x559a71986d70 2024-09-17T05:33:56.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- 172.21.15.43:0/41080854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 msgr2=0x559a719c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c670 tx=0x7fcad000cb30 comp rx=0 tx=0).stop 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- 172.21.15.43:0/41080854 shutdown_connections 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a719cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a719b5cb0 0x559a719ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 --2- 172.21.15.43:0/41080854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a71986d70 0x559a719c9b20 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- 172.21.15.43:0/41080854 >> 172.21.15.43:0/41080854 conn(0x559a71972d80 msgr2=0x559a71984890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- 172.21.15.43:0/41080854 shutdown_connections 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.490+0000 7fcae804b080 1 -- 172.21.15.43:0/41080854 wait complete. 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 Processor -- start 2024-09-17T05:33:56.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- start start 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a71986d70 0x559a719cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a719b5cb0 0x559a71af6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a71af37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a71974ec0 con 0x559a719b5cb0 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a71a58af0 con 0x559a71a3ce40 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559a71a58d90 con 0x559a71986d70 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a719b5cb0 0x559a71af6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a719b5cb0 0x559a71af6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56146/0 (socket says 172.21.15.43:56146) 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadfc00640 1 -- 172.21.15.43:0/3075790321 learned_addr learned my addr 172.21.15.43:0/3075790321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a71986d70 0x559a719cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae1000640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a71af37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 -- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 msgr2=0x559a71af37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a71a3ce40 0x559a71af37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 -- 172.21.15.43:0/3075790321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a719b5cb0 msgr2=0x559a71af6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a719b5cb0 0x559a71af6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 -- 172.21.15.43:0/3075790321 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a718adc10 con 0x559a71986d70 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae0600640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a71986d70 0x559a719cefc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcad0014000 tx=0x7fcad0016ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadd400640 1 -- 172.21.15.43:0/3075790321 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad001a070 con 0x559a71986d70 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadd400640 1 -- 172.21.15.43:0/3075790321 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcad0020640 con 0x559a71986d70 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcadd400640 1 -- 172.21.15.43:0/3075790321 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0020a90 con 0x559a71986d70 2024-09-17T05:33:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a719ca590 con 0x559a71986d70 2024-09-17T05:33:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.494+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x559a719caad0 con 0x559a71986d70 2024-09-17T05:33:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcadd400640 1 -- 172.21.15.43:0/3075790321 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcad0008450 con 0x559a71986d70 2024-09-17T05:33:56.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcadd400640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcac0071be0 0x7fcac00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcadd400640 1 -- 172.21.15.43:0/3075790321 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7fcad0096090 con 0x559a71986d70 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcadfc00640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcac0071be0 0x7fcac00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcae804b080 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559a719cb4b0 0x559a719cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x559a719ad220 con 0x559a719cb4b0 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcae1000640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559a719cb4b0 0x559a719cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcadfc00640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcac0071be0 0x7fcac00740f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcacc000c10 tx=0x7fcacc0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.498+0000 7fcae1000640 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559a719cb4b0 0x559a719cb8c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae1000640 1 -- 172.21.15.43:0/3075790321 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v125'158 uv158 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fcad800b040 con 0x559a719cb4b0 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559a719cb4b0 msgr2=0x559a719cb8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559a719cb4b0 0x559a719cb8c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcac0071be0 msgr2=0x7fcac00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcac0071be0 0x7fcac00740f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcacc000c10 tx=0x7fcacc0b2040 comp rx=0 tx=0).stop 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a71986d70 msgr2=0x559a719cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 --2- 172.21.15.43:0/3075790321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a71986d70 0x559a719cefc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcad0014000 tx=0x7fcad0016ae0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae1000640 1 -- 172.21.15.43:0/3075790321 reap_dead start 2024-09-17T05:33:56.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 shutdown_connections 2024-09-17T05:33:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 >> 172.21.15.43:0/3075790321 conn(0x559a71972d80 msgr2=0x559a71984a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 shutdown_connections 2024-09-17T05:33:56.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.502+0000 7fcae804b080 1 -- 172.21.15.43:0/3075790321 wait complete. 2024-09-17T05:33:56.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:33:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 -- 172.21.15.43:0/1781532060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2614155d70 msgr2=0x7f261415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 --2- 172.21.15.43:0/1781532060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2614155d70 0x7f261415c7e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260401c820 comp rx=0 tx=0).stop 2024-09-17T05:33:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 -- 172.21.15.43:0/1781532060 shutdown_connections 2024-09-17T05:33:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 --2- 172.21.15.43:0/1781532060 >> v2:172.21.15.62:3300/0 conn(0x7f26140b3af0 0x7f26140ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 --2- 172.21.15.43:0/1781532060 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2614155d70 0x7f261415c7e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 -- 172.21.15.43:0/1781532060 >> 172.21.15.43:0/1781532060 conn(0x7f26140b23d0 msgr2=0x7f26140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 -- 172.21.15.43:0/1781532060 shutdown_connections 2024-09-17T05:33:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.666+0000 7f261b800640 1 -- 172.21.15.43:0/1781532060 wait complete. 2024-09-17T05:33:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 Processor -- start 2024-09-17T05:33:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 -- start start 2024-09-17T05:33:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2614155d70 0x7f2614155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 0x7f26140c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2614168c80 con 0x7f261406f110 2024-09-17T05:33:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2614168b00 con 0x7f2614155d70 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f261b800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2614168e00 con 0x7f26140c9c50 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56156/0 (socket says 172.21.15.43:56156) 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 -- 172.21.15.43:0/1328447907 learned_addr learned my addr 172.21.15.43:0/1328447907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f260be00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2614155d70 0x7f2614155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2619800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 0x7f26140c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 -- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 msgr2=0x7f26140c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 0x7f26140c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 -- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2614155d70 msgr2=0x7f2614155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2614155d70 0x7f2614155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2618e00640 1 -- 172.21.15.43:0/1328447907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141f75f0 con 0x7f261406f110 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.670+0000 7f2619800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 0x7f26140c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2618e00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f2604009850 tx=0x7f26040a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040b2040 con 0x7f261406f110 2024-09-17T05:33:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2604004340 con 0x7f261406f110 2024-09-17T05:33:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26140ca1f0 con 0x7f261406f110 2024-09-17T05:33:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040044f0 con 0x7f261406f110 2024-09-17T05:33:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26140c40c0 con 0x7f261406f110 2024-09-17T05:33:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26140ad0b0 con 0x7f261406f110 2024-09-17T05:33:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2604005290 con 0x7f261406f110 2024-09-17T05:33:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.674+0000 7f2611600640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 0x7f25f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.678+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f260412e8f0 con 0x7f261406f110 2024-09-17T05:33:56.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.678+0000 7f260be00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 0x7f25f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.678+0000 7f260be00640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 0x7f25f0075710 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f26141f4220 tx=0x7f25fc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.682+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26141f75f0 con 0x7f261406f110 2024-09-17T05:33:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:56.890+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26140af9a0 con 0x7f261406f110 2024-09-17T05:33:57.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f2611600640 1 -- 172.21.15.43:0/1328447907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v311) ==== 162+0+0 (secure 0 0 0) 0x7f26040164d0 con 0x7f261406f110 2024-09-17T05:33:57.132 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 msgr2=0x7f25f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 0x7f25f0075710 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f26141f4220 tx=0x7f25fc009040 comp rx=0 tx=0).stop 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 msgr2=0x7f26141f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f2604009850 tx=0x7f26040a57f0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 shutdown_connections 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f25f0073200 0x7f25f0075710 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2614155d70 0x7f2614155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26140c9c50 0x7f26140c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 --2- 172.21.15.43:0/1328447907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261406f110 0x7f26141f70b0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 >> 172.21.15.43:0/1328447907 conn(0x7f26140b23d0 msgr2=0x7f261415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 shutdown_connections 2024-09-17T05:33:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.130+0000 7f261b800640 1 -- 172.21.15.43:0/1328447907 wait complete. 2024-09-17T05:33:57.143 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:33:57.142+0000 7f4096e00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-17T05:33:57.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:57.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- 172.21.15.43:0/706491495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 msgr2=0x7f52dc0b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 --2- 172.21.15.43:0/706491495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc0b4b80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009840 tx=0x7f52cc01c700 comp rx=0 tx=0).stop 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- 172.21.15.43:0/706491495 shutdown_connections 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 --2- 172.21.15.43:0/706491495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc06fc90 0x7f52dc0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 --2- 172.21.15.43:0/706491495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc0b4b80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- 172.21.15.43:0/706491495 >> 172.21.15.43:0/706491495 conn(0x7f52dc0b23d0 msgr2=0x7f52dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- 172.21.15.43:0/706491495 shutdown_connections 2024-09-17T05:33:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- 172.21.15.43:0/706491495 wait complete. 2024-09-17T05:33:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 Processor -- start 2024-09-17T05:33:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.734+0000 7f52e1000640 1 -- start start 2024-09-17T05:33:57.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc0730b0 0x7f52dc157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc169fa0 con 0x7f52dc06fc90 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc169e20 con 0x7f52dc15fe10 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc16a120 con 0x7f52dc0730b0 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56158/0 (socket says 172.21.15.43:56158) 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 -- 172.21.15.43:0/399225274 learned_addr learned my addr 172.21.15.43:0/399225274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d3e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc0730b0 0x7f52dc157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52da800640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 -- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc0730b0 msgr2=0x7f52dc157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc0730b0 0x7f52dc157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 -- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 msgr2=0x7f52dc1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 -- 172.21.15.43:0/399225274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc153910 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d9e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f52cc01adb0 tx=0x7f52cc0a66c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0042c0 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52cc0bd070 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc153bf0 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc003860 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1562d0 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.738+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0700a0 con 0x7f52dc06fc90 2024-09-17T05:33:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.742+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f52cc003a00 con 0x7f52dc06fc90 2024-09-17T05:33:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.742+0000 7f52d2000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 0x7f52b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.742+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f52cc12fe90 con 0x7f52dc06fc90 2024-09-17T05:33:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.742+0000 7f52d3e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 0x7f52b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.742+0000 7f52d3e00640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 0x7f52b8075740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f52dc073bc0 tx=0x7f52c4009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.746+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52cc016410 con 0x7f52dc06fc90 2024-09-17T05:33:57.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52dc075130 con 0x7f52dc06fc90 2024-09-17T05:33:57.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52d2000640 1 -- 172.21.15.43:0/399225274 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) ==== 95+0+5221 (secure 0 0 0) 0x7f52cc0fa280 con 0x7f52dc06fc90 2024-09-17T05:33:57.957 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:33:57.957 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":312,"btime":"2024-09-17T05:33:57:135564+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":312,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:33:57.135547+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10065":{"gid":10065,"name":"c","rank":0,"incarnation":312,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/4105315924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4105315924},{"type":"v1","addr":"172.21.15.43:6835","nonce":4105315924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:33:57.957 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 312 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 msgr2=0x7f52b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 0x7f52b8075740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f52dc073bc0 tx=0x7f52c4009260 comp rx=0 tx=0).stop 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 msgr2=0x7f52dc152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f52cc01adb0 tx=0x7f52cc0a66c0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 shutdown_connections 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52b8073230 0x7f52b8075740 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52dc15fe10 0x7f52dc1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52dc0730b0 0x7f52dc157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 --2- 172.21.15.43:0/399225274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc06fc90 0x7f52dc152ed0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 >> 172.21.15.43:0/399225274 conn(0x7f52dc0b23d0 msgr2=0x7f52dc0addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 shutdown_connections 2024-09-17T05:33:57.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:33:57.954+0000 7f52e1000640 1 -- 172.21.15.43:0/399225274 wait complete. 2024-09-17T05:33:57.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 312, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:33:57.135547+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10065': {'gid': 10065, 'name': 'c', 'rank': 0, 'incarnation': 312, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/4105315924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 4105315924}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 4105315924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:33:57.980 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10065:up:replay 2024-09-17T05:33:57.980 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:34:03.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 -- 172.21.15.43:0/2600262859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 msgr2=0x7f40980700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 --2- 172.21.15.43:0/2600262859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f40980700a0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f408401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 -- 172.21.15.43:0/2600262859 shutdown_connections 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 --2- 172.21.15.43:0/2600262859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f40980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 --2- 172.21.15.43:0/2600262859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f40980700a0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 -- 172.21.15.43:0/2600262859 >> 172.21.15.43:0/2600262859 conn(0x7f40980b23d0 msgr2=0x7f40980d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 -- 172.21.15.43:0/2600262859 shutdown_connections 2024-09-17T05:34:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.554+0000 7f409f800640 1 -- 172.21.15.43:0/2600262859 wait complete. 2024-09-17T05:34:03.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 Processor -- start 2024-09-17T05:34:03.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- start start 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f4098157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409815fe10 0x7f4098159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f409816a060 con 0x7f409806fc90 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4098169ee0 con 0x7f409815fe10 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f409816a1e0 con 0x7f40980730b0 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56180/0 (socket says 172.21.15.43:56180) 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 -- 172.21.15.43:0/4014429535 learned_addr learned my addr 172.21.15.43:0/4014429535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f4097e00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f4098157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409d800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409815fe10 0x7f4098159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 -- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 msgr2=0x7f4098157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f4098157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 -- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409815fe10 msgr2=0x7f4098159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409815fe10 0x7f4098159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 -- 172.21.15.43:0/4014429535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4098154a10 con 0x7f409806fc90 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f4097e00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f4098157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409ce00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c3f0 tx=0x7f408c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c017070 con 0x7f409806fc90 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f408c0053c0 con 0x7f409806fc90 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c003860 con 0x7f409806fc90 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409815afc0 con 0x7f409806fc90 2024-09-17T05:34:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.558+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4098156a60 con 0x7f409806fc90 2024-09-17T05:34:03.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.562+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40981f95c0 con 0x7f409806fc90 2024-09-17T05:34:03.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.562+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f408c01a070 con 0x7f409806fc90 2024-09-17T05:34:03.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.562+0000 7f4094c00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 0x7f4078075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.562+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f408c094250 con 0x7f409806fc90 2024-09-17T05:34:03.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.562+0000 7f4097e00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 0x7f4078075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.566+0000 7f4097e00640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 0x7f4078075710 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f40840af000 tx=0x7f408401b000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.566+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4098073a00 con 0x7f409806fc90 2024-09-17T05:34:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40980700a0 con 0x7f409806fc90 2024-09-17T05:34:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f4094c00640 1 -- 172.21.15.43:0/4014429535 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 315 v315) ==== 95+0+5230 (secure 0 0 0) 0x7f408c05e630 con 0x7f409806fc90 2024-09-17T05:34:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:03.781 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":315,"btime":"2024-09-17T05:34:00:155096+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":315,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:00.155085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10065":{"gid":10065,"name":"c","rank":0,"incarnation":312,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/4105315924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4105315924},{"type":"v1","addr":"172.21.15.43:6835","nonce":4105315924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10065,"qdb_cluster":[10065]},"id":6}]} 2024-09-17T05:34:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 315 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 msgr2=0x7f4078075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 0x7f4078075710 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f40840af000 tx=0x7f408401b000 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 msgr2=0x7f4098156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c3f0 tx=0x7f408c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 shutdown_connections 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4078073200 0x7f4078075710 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f409815fe10 0x7f4098159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f40980730b0 0x7f4098157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 --2- 172.21.15.43:0/4014429535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f409806fc90 0x7f4098156f30 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 >> 172.21.15.43:0/4014429535 conn(0x7f40980b23d0 msgr2=0x7f4098075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 shutdown_connections 2024-09-17T05:34:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.778+0000 7f409f800640 1 -- 172.21.15.43:0/4014429535 wait complete. 2024-09-17T05:34:03.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 315, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:29:42.776833+0000', 'modified': '2024-09-17T05:34:00.155085+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10065': {'gid': 10065, 'name': 'c', 'rank': 0, 'incarnation': 312, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/4105315924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 4105315924}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 4105315924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10065, 'qdb_cluster': [10065]} 2024-09-17T05:34:03.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:34:03.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:34:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 -- 172.21.15.43:0/3381959890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0240 msgr2=0x7f04300cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 --2- 172.21.15.43:0/3381959890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0240 0x7f04300cd040 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f0418009900 tx=0x7f041801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 -- 172.21.15.43:0/3381959890 shutdown_connections 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 --2- 172.21.15.43:0/3381959890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0240 0x7f04300cd040 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 --2- 172.21.15.43:0/3381959890 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 0x7f04300b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.958+0000 7f0437000640 1 -- 172.21.15.43:0/3381959890 >> 172.21.15.43:0/3381959890 conn(0x7f04300cee60 msgr2=0x7f04300cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- 172.21.15.43:0/3381959890 shutdown_connections 2024-09-17T05:34:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- 172.21.15.43:0/3381959890 wait complete. 2024-09-17T05:34:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 Processor -- start 2024-09-17T05:34:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- start start 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 0x7f043006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04300d0240 0x7f043015fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04300b2b00 con 0x7f04300d0650 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04300b2980 con 0x7f04300d0240 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0437000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0430169e30 con 0x7f043009ba60 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.962+0000 7f0435000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49026/0 (socket says 172.21.15.43:49026) 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 -- 172.21.15.43:0/524939452 learned_addr learned my addr 172.21.15.43:0/524939452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0427e00640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04300d0240 0x7f043015fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f042fe00640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 0x7f043006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 -- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 msgr2=0x7f043006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 0x7f043006df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 -- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04300d0240 msgr2=0x7f043015fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04300d0240 0x7f043015fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 -- 172.21.15.43:0/524939452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04301678e0 con 0x7f04300d0650 2024-09-17T05:34:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0435000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f042800c730 tx=0x7f042800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f042cc00640 1 -- 172.21.15.43:0/524939452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428003b80 con 0x7f04300d0650 2024-09-17T05:34:03.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f042cc00640 1 -- 172.21.15.43:0/524939452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0428003d20 con 0x7f04300d0650 2024-09-17T05:34:03.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0430167320 con 0x7f04300d0650 2024-09-17T05:34:03.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f042cc00640 1 -- 172.21.15.43:0/524939452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428005330 con 0x7f04300d0650 2024-09-17T05:34:03.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.966+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04300c7a10 con 0x7f04300d0650 2024-09-17T05:34:03.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042cc00640 1 -- 172.21.15.43:0/524939452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04280054d0 con 0x7f04300d0650 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042cc00640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 0x7f040c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042cc00640 1 -- 172.21.15.43:0/524939452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f0428094a90 con 0x7f04300d0650 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 Processor -- start 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 -- start start 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042fe00640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 0x7f040c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042fe00640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 0x7f040c075710 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b40 tx=0x7f04200091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd1b1460 0x5564bd6f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 0x5564bd6f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564bd5c7410 con 0x5564bd5a1950 2024-09-17T05:34:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5564bd504b40 con 0x5564bd1b1460 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5564bd4f3f70 con 0x5564bd58aca0 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0427e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0427e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49030/0 (socket says 172.21.15.43:49030) 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0427e00640 1 -- 172.21.15.43:0/2053788526 learned_addr learned my addr 172.21.15.43:0/2053788526 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0435000640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 0x5564bd6f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f042fe00640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd1b1460 0x5564bd6f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0435000640 1 -- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd1b1460 msgr2=0x5564bd6f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0435000640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd1b1460 0x5564bd6f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0435000640 1 -- 172.21.15.43:0/2053788526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 msgr2=0x5564bd6f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.970+0000 7f0435000640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0435000640 1 -- 172.21.15.43:0/2053788526 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564bd4bb5b0 con 0x5564bd58aca0 2024-09-17T05:34:03.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0427e00640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:03.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0435000640 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 0x5564bd6f3120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04280922f0 tx=0x7f0428001260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c6d1ea16e755a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0424c00640 1 -- 172.21.15.43:0/2053788526 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042805e8a0 con 0x5564bd58aca0 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0424c00640 1 -- 172.21.15.43:0/2053788526 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0428063250 con 0x5564bd58aca0 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f0424c00640 1 -- 172.21.15.43:0/2053788526 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042805dd40 con 0x5564bd58aca0 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 msgr2=0x5564bd6f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 0x5564bd6f3120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04280922f0 tx=0x7f0428001260 comp rx=0 tx=0).stop 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- 172.21.15.43:0/2053788526 shutdown_connections 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd58aca0 0x5564bd6f3120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd6f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 --2- 172.21.15.43:0/2053788526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd1b1460 0x5564bd6f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- 172.21.15.43:0/2053788526 >> 172.21.15.43:0/2053788526 conn(0x5564bd677d90 msgr2=0x5564bd6a7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- 172.21.15.43:0/2053788526 shutdown_connections 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- 172.21.15.43:0/2053788526 wait complete. 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 -- start start 2024-09-17T05:34:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.974+0000 7f043e56c000 1 Processor -- start 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd6f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd5a1950 0x5564bd6fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564bd48e5f0 con 0x5564bd1b1460 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5564bd4f7d80 con 0x5564bd58aca0 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5564bd49a520 con 0x5564bd5a1950 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 10 client.0 _ll_register_callbacks cb 0x5564bd49e830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49036/0 (socket says 172.21.15.43:49036) 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 -- 172.21.15.43:0/503203787 learned_addr learned my addr 172.21.15.43:0/503203787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f0435000640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd5a1950 0x5564bd6fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f0427e00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd6f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 -- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd5a1950 msgr2=0x5564bd6fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd5a1950 0x5564bd6fa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 -- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 msgr2=0x5564bd6f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd6f9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 -- 172.21.15.43:0/503203787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564bd492190 con 0x5564bd1b1460 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f042fe00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0420004600 tx=0x7f0420004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ba36c98fdfad02 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042001fa40 con 0x5564bd1b1460 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f042001fbe0 con 0x5564bd1b1460 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564bd48c110 con 0x5564bd1b1460 2024-09-17T05:34:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=315) ==== 24+0+0 (secure 0 0 0) 0x7f042001fe20 con 0x5564bd1b1460 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 10 client.10116 fetch_fsmap learned FSMap version 315 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=315,monmap=2+}) -- 0x5564bd578f10 con 0x5564bd1b1460 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5564bd48c110 con 0x5564bd1b1460 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 315) ==== 3502+0+0 (secure 0 0 0) 0x7f0420026070 con 0x5564bd1b1460 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 10 client.10116 fetch_fsmap finished waiting for FSMap version 315 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 10 client.10116 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 20 client.10116 populate_metadata read hostname 'smithi043' 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd1967d0 0x5564bd703d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 4 client.10116 mds_command: new command op to 10065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 --> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564bd506a90 con 0x5564bd1967d0 2024-09-17T05:34:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f0427e00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd1967d0 0x5564bd703d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f0427e00640 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd1967d0 0x5564bd703d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffd8d279e86b871 server_cookie=70ec3299822ef63 in_seq=0 out_seq=0 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 10 client.10116 ms_handle_connect on v2:172.21.15.43:6833/4105315924 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 1 -- 172.21.15.43:0/503203787 <== mds.0 v2:172.21.15.43:6833/4105315924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564bd506a90 con 0x5564bd1967d0 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.978+0000 7f041d600640 10 client.10116 handle_command_reply: tid=0 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 client.10116 shutdown 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 msgr2=0x5564bd6f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f0420004600 tx=0x7f0420004740 comp rx=0 tx=0).stop 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 shutdown_connections 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd1967d0 0x5564bd703d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd6f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd5a1950 0x5564bd6fa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f041d600640 0 client.10116 ms_handle_reset on v2:172.21.15.43:6833/4105315924 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 --2- 172.21.15.43:0/503203787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd1b1460 0x5564bd6f9920 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 >> 172.21.15.43:0/503203787 conn(0x5564bd677d90 msgr2=0x5564bd6ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 shutdown_connections 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 1 -- 172.21.15.43:0/503203787 wait complete. 2024-09-17T05:34:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.982+0000 7f043e56c000 20 client.10116 trim_cache size 0 max 16384 2024-09-17T05:34:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.998+0000 7f043e56c000 1 Processor -- start 2024-09-17T05:34:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:03.998+0000 7f043e56c000 1 -- start start 2024-09-17T05:34:04.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd764620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd5a1950 0x5564bd7650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0430207160 con 0x5564bd58aca0 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0430206140 con 0x5564bd5a1950 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f04302054d0 con 0x5564bd1b1460 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49038/0 (socket says 172.21.15.43:49038) 2024-09-17T05:34:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 -- 172.21.15.43:0/2206370588 learned_addr learned my addr 172.21.15.43:0/2206370588 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0435000640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd5a1950 0x5564bd7650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f042fe00640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd764620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 -- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 msgr2=0x5564bd764620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd764620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 -- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd5a1950 msgr2=0x5564bd7650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd5a1950 0x5564bd7650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 -- 172.21.15.43:0/2206370588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564bd54fdb0 con 0x5564bd58aca0 2024-09-17T05:34:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f042fe00640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd764620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f0427e00640 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f041801c710 tx=0x7f04180a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485734e18945fe7c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f041fe00640 1 -- 172.21.15.43:0/2206370588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418004100 con 0x5564bd58aca0 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f041fe00640 1 -- 172.21.15.43:0/2206370588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04180042a0 con 0x5564bd58aca0 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f041fe00640 1 -- 172.21.15.43:0/2206370588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04180a6940 con 0x5564bd58aca0 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- 172.21.15.43:0/2206370588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 msgr2=0x5564bd764b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f041801c710 tx=0x7f04180a8040 comp rx=0 tx=0).stop 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- 172.21.15.43:0/2206370588 shutdown_connections 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd5a1950 0x5564bd7650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd58aca0 0x5564bd764b60 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 --2- 172.21.15.43:0/2206370588 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd764620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- 172.21.15.43:0/2206370588 >> 172.21.15.43:0/2206370588 conn(0x5564bd677d90 msgr2=0x5564bd6f0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- 172.21.15.43:0/2206370588 shutdown_connections 2024-09-17T05:34:04.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.002+0000 7f043e56c000 1 -- 172.21.15.43:0/2206370588 wait complete. 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- start start 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 Processor -- start 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd724220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd724760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0430205b20 con 0x5564bd5a1950 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5564bd4bfee0 con 0x5564bd58aca0 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5564bd4f7090 con 0x5564bd1b1460 2024-09-17T05:34:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 10 client.0 _ll_register_callbacks cb 0x7f0420011c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49046/0 (socket says 172.21.15.43:49046) 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 -- 172.21.15.43:0/4047394303 learned_addr learned my addr 172.21.15.43:0/4047394303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f042fe00640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd724220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0427e00640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd724760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 -- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 msgr2=0x5564bd724220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd724220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 -- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 msgr2=0x5564bd724760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd724760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 -- 172.21.15.43:0/4047394303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564bd4baab0 con 0x5564bd5a1950 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f0435000640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f042805e830 tx=0x7f042805dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3710f1959a90cd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:34:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042805aef0 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04180042a0 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f042805fb00 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=315) ==== 24+0+0 (secure 0 0 0) 0x7f042805de20 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 10 client.10122 fetch_fsmap learned FSMap version 315 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=315,monmap=2+}) -- 0x5564bd499070 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04180042a0 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 315) ==== 3502+0+0 (secure 0 0 0) 0x7f042805a1f0 con 0x5564bd5a1950 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 10 client.10122 fetch_fsmap finished waiting for FSMap version 315 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 10 client.10122 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 20 client.10122 populate_metadata read hostname 'smithi043' 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd184830 0x5564bd7251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 4 client.10122 mds_command: new command op to 10065 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:34:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.006+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 --> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] -- command(tid 0: {"prefix": "status"}) -- 0x5564bd502540 con 0x5564bd184830 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f042fe00640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd184830 0x5564bd7251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f042fe00640 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd184830 0x5564bd7251e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760e20865abbfbb8 server_cookie=2efed99aff05eb4f in_seq=0 out_seq=0 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f041d600640 10 client.10122 ms_handle_connect on v2:172.21.15.43:6833/4105315924 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f041d600640 1 -- 172.21.15.43:0/4047394303 <== mds.0 v2:172.21.15.43:6833/4105315924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5564bd502540 con 0x5564bd184830 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f041d600640 10 client.10122 handle_command_reply: tid=0 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 client.10122 shutdown 2024-09-17T05:34:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 msgr2=0x5564bd724ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f042805e830 tx=0x7f042805dd30 comp rx=0 tx=0).stop 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 shutdown_connections 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:6833/4105315924,v1:172.21.15.43:6835/4105315924] conn(0x5564bd184830 0x5564bd7251e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564bd58aca0 0x5564bd724760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564bd5a1950 0x5564bd724ca0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 --2- 172.21.15.43:0/4047394303 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5564bd1b1460 0x5564bd724220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f041d600640 0 client.10122 ms_handle_reset on v2:172.21.15.43:6833/4105315924 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 >> 172.21.15.43:0/4047394303 conn(0x5564bd677d90 msgr2=0x5564bd6f0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 shutdown_connections 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 1 -- 172.21.15.43:0/4047394303 wait complete. 2024-09-17T05:34:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f043e56c000 20 client.10122 trim_cache size 0 max 16384 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10065, 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8749999260000001, 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 315, 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 125, 2024-09-17T05:34:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 125, 2024-09-17T05:34:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 9.0196765370000005 2024-09-17T05:34:04.018 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:34:04.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 msgr2=0x7f040c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 0x7f040c075710 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0420009b40 tx=0x7f04200091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 msgr2=0x7f04300c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f042800c730 tx=0x7f042800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 shutdown_connections 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f040c073200 0x7f040c075710 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04300d0240 0x7f043015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04300d0650 0x7f04300c8210 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 --2- 172.21.15.43:0/524939452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f043009ba60 0x7f043006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 >> 172.21.15.43:0/524939452 conn(0x7f04300cee60 msgr2=0x7f04300726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 shutdown_connections 2024-09-17T05:34:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.010+0000 7f0437000640 1 -- 172.21.15.43:0/524939452 wait complete. 2024-09-17T05:34:04.034 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:34:04.034 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-09-17T05:34:04.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- 172.21.15.43:0/3531477823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b45a0 msgr2=0x7ff45c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 --2- 172.21.15.43:0/3531477823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0b6130 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7ff45c073e60 tx=0x7ff44c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- 172.21.15.43:0/3531477823 shutdown_connections 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 --2- 172.21.15.43:0/3531477823 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 0x7ff45c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 --2- 172.21.15.43:0/3531477823 >> v2:172.21.15.62:3300/0 conn(0x7ff45c0b6670 0x7ff45c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 --2- 172.21.15.43:0/3531477823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0b6130 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- 172.21.15.43:0/3531477823 >> 172.21.15.43:0/3531477823 conn(0x7ff45c0b2e80 msgr2=0x7ff45c0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- 172.21.15.43:0/3531477823 shutdown_connections 2024-09-17T05:34:04.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- 172.21.15.43:0/3531477823 wait complete. 2024-09-17T05:34:04.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 Processor -- start 2024-09-17T05:34:04.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.194+0000 7ff461c00640 1 -- start start 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 0x7ff45c0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0ccfa0 con 0x7ff45c0b6670 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c06dbb0 con 0x7ff45c0b45a0 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c0cd120 con 0x7ff45c15fa70 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49054/0 (socket says 172.21.15.43:49054) 2024-09-17T05:34:04.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 -- 172.21.15.43:0/2159753425 learned_addr learned my addr 172.21.15.43:0/2159753425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff45b400640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 0x7ff45c0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff45aa00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 -- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 msgr2=0x7ff45c0b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 0x7ff45c0b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 -- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c0b45a0 msgr2=0x7ff45c0c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 -- 172.21.15.43:0/2159753425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1fc290 con 0x7ff45c0b6670 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff452000640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ea20 tx=0x7ff44400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44400cd80 con 0x7ff45c0b6670 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff444004020 con 0x7ff45c0b6670 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c1fc570 con 0x7ff45c0b6670 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004470 con 0x7ff45c0b6670 2024-09-17T05:34:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1fcb30 con 0x7ff45c0b6670 2024-09-17T05:34:04.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.198+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c15f530 con 0x7ff45c0b6670 2024-09-17T05:34:04.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.202+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff444003600 con 0x7ff45c0b6670 2024-09-17T05:34:04.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.202+0000 7ff453400640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 0x7ff438075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.202+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7ff4440956d0 con 0x7ff45c0b6670 2024-09-17T05:34:04.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.202+0000 7ff45aa00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 0x7ff438075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.202+0000 7ff45aa00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 0x7ff438075710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0af000 tx=0x7ff44c0a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.206+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff44405f3d0 con 0x7ff45c0b6670 2024-09-17T05:34:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff45c0b7120 con 0x7ff45c0b6670 2024-09-17T05:34:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff453400640 1 -- 172.21.15.43:0/2159753425 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v315) ==== 53+0+73 (secure 0 0 0) 0x7ff444063e00 con 0x7ff45c0b6670 2024-09-17T05:34:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 msgr2=0x7ff438075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 0x7ff438075710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0af000 tx=0x7ff44c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 msgr2=0x7ff45c148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ea20 tx=0x7ff44400eee0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 shutdown_connections 2024-09-17T05:34:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff438073200 0x7ff438075710 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff45c15fa70 0x7ff45c0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff45c0b6670 0x7ff45c148530 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 --2- 172.21.15.43:0/2159753425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c0b45a0 0x7ff45c0c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 >> 172.21.15.43:0/2159753425 conn(0x7ff45c0b2e80 msgr2=0x7ff45c15b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 shutdown_connections 2024-09-17T05:34:04.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04.418+0000 7ff461c00640 1 -- 172.21.15.43:0/2159753425 wait complete. 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:34:04.444 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:04.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.445 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:34:04.454 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41799sec preferred_lft 41799sec 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:34:04.455 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.456 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:04.457 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:34:04.457 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:34:04.457 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:04.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.458 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:34:04.517 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:34:04.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.519 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:34:04.572 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:34:04.572 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:34:04.573 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:34:04.573 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:34:04.573 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:34:04.573 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:34:04.573 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:34:04.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:34:04.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:04.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.711 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:34:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:34:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:34:04.721 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:04.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:34:04.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:04.783 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:04.831 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:04.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:04.832 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:04.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:34:04.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:04.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:04.983 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:04.984 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:04.984 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:34:05.002 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:05.013 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:05.013 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:05.013 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:05.014 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:34:05.014 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:05.014 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:34:05.014 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:34:05.014 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:34:05.017 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:34:05.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:05.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:05.037 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:34:05.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:34:05.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:05.132 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:34:05.130+0000 7f409be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:05.136 INFO:teuthology.orchestra.run.smithi043.stdout:client9520 2024-09-17T05:34:05.136 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:05.136 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:05.145 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-17T05:34:05.189 INFO:teuthology.orchestra.run:waiting for 20 2024-09-17T05:34:05.197 INFO:teuthology.orchestra.run.smithi043.stdout:.: 2024-09-17T05:34:05.197 INFO:teuthology.orchestra.run.smithi043.stdout:subdir 2024-09-17T05:34:05.197 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:05.197 INFO:teuthology.orchestra.run.smithi043.stdout:./subdir: 2024-09-17T05:34:05.197 INFO:teuthology.orchestra.run.smithi043.stdout:sixmegs 2024-09-17T05:34:05.198 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-09-17T05:34:05.198 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-09-17T05:34:05.198 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:05.198 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.251 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:05.251 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- 172.21.15.43:0/846391532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 msgr2=0x7f7acc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 --2- 172.21.15.43:0/846391532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7acc00b9d0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009f10 tx=0x7f7adc01d5f0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7ae9600640 1 -- 172.21.15.43:0/846391532 reap_dead start 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- 172.21.15.43:0/846391532 shutdown_connections 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- 172.21.15.43:0/846391532 >> 172.21.15.43:0/846391532 conn(0x7f7ae40b23d0 msgr2=0x7f7ae40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- 172.21.15.43:0/846391532 shutdown_connections 2024-09-17T05:34:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- 172.21.15.43:0/846391532 wait complete. 2024-09-17T05:34:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 Processor -- start 2024-09-17T05:34:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.506+0000 7f7aeb600640 1 -- start start 2024-09-17T05:34:05.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 0x7f7ae4074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 0x7f7ae4074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae40ad0b0 con 0x7f7acc005160 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae40b3d00 con 0x7f7ae40c9cc0 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae40ad230 con 0x7f7ae40ca0d0 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49068/0 (socket says 172.21.15.43:49068) 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 -- 172.21.15.43:0/231915871 learned_addr learned my addr 172.21.15.43:0/231915871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae9600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 0x7f7ae4074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae3e00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 0x7f7ae4074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 -- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 msgr2=0x7f7ae4074a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 0x7f7ae4074a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 -- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 msgr2=0x7f7ae4074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 0x7f7ae4074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 -- 172.21.15.43:0/231915871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae41580d0 con 0x7f7acc005160 2024-09-17T05:34:05.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae9600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 0x7f7ae4074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:05.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae3e00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 0x7f7ae4074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae8c00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8011ab0 tx=0x7f7ad8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad801e070 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ad800fc70 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8019410 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae409dac0 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae4157150 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.510+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae406f810 con 0x7f7acc005160 2024-09-17T05:34:05.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.514+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7ad8021070 con 0x7f7acc005160 2024-09-17T05:34:05.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.514+0000 7f7ae0c00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 0x7f7ac4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.514+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f7ad80956f0 con 0x7f7acc005160 2024-09-17T05:34:05.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.514+0000 7f7ae3e00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 0x7f7ac4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.514+0000 7f7ae3e00640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 0x7f7ac4075710 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000fc0 tx=0x7f7ad0001e60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.518+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7ad805f450 con 0x7f7acc005160 2024-09-17T05:34:05.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.702+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7ae4157390 con 0x7f7acc005160 2024-09-17T05:34:05.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.702+0000 7f7ae0c00640 1 -- 172.21.15.43:0/231915871 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v125) ==== 81+0+4378 (secure 0 0 0) 0x7f7ad8063e80 con 0x7f7acc005160 2024-09-17T05:34:05.706 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2696141215 2024-09-18T05:33:44.705848+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2696141215 2024-09-18T05:33:44.705848+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1507988053 2024-09-18T05:33:33.376586+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3762350799 2024-09-18T05:33:16.473578+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3094176220 2024-09-18T05:33:50.112138+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/647815137 2024-09-18T05:32:57.599675+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:34:05.707 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3644279875 2024-09-18T05:30:58.496775+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2775198885 2024-09-18T05:30:52.367787+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/279176341 2024-09-18T05:31:05.541294+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/273818866 2024-09-18T05:30:08.113534+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:34:05.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3640890143 2024-09-18T05:30:28.229649+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1587862715 2024-09-18T05:31:49.459141+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1981693955 2024-09-18T05:30:13.152555+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3277842592 2024-09-18T05:30:41.396242+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1578591246 2024-09-18T05:31:27.260942+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/352682704 2024-09-18T05:30:54.467138+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:34:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2023604015 2024-09-18T05:31:41.375774+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/274766899 2024-09-18T05:29:52.620716+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1590794613 2024-09-18T05:30:24.225848+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1077468406 2024-09-18T05:31:34.313578+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3611976391 2024-09-18T05:30:20.206882+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1507988053 2024-09-18T05:33:33.376586+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2584621842 2024-09-18T05:31:45.414409+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3742224456 2024-09-18T05:32:06.064785+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2992252190 2024-09-18T05:31:12.052972+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/1306620592 2024-09-18T05:32:10.088635+0000 2024-09-17T05:34:05.710 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/324676924 2024-09-18T05:32:14.126639+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3094176220 2024-09-18T05:33:50.112138+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2196052200 2024-09-18T05:32:03.861884+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2994345600 2024-09-18T05:31:25.236711+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/2407777034 2024-09-18T05:32:18.163569+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3572479007 2024-09-18T05:32:22.223422+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/1404316783 2024-09-18T05:32:30.299996+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3432953722 2024-09-18T05:32:26.259793+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/1653179583 2024-09-18T05:32:53.543785+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/351071204 2024-09-18T05:33:11.430207+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6834/2354591248 2024-09-18T05:32:43.445225+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:34:05.711 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/3812598362 2024-09-18T05:32:45.466679+0000 2024-09-17T05:34:05.712 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.62:6835/3412634924 2024-09-18T05:32:49.498519+0000 2024-09-17T05:34:05.712 INFO:teuthology.orchestra.run.smithi043.stderr:listed 72 entries 2024-09-17T05:34:05.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 msgr2=0x7f7ac4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 0x7f7ac4075710 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000fc0 tx=0x7f7ad0001e60 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 msgr2=0x7f7ae4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8011ab0 tx=0x7f7ad8011f70 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 shutdown_connections 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7ac4073200 0x7f7ac4075710 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ae40c9cc0 0x7f7ae4074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ae40ca0d0 0x7f7ae4074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 --2- 172.21.15.43:0/231915871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7acc005160 0x7f7ae4156300 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 >> 172.21.15.43:0/231915871 conn(0x7f7ae40b23d0 msgr2=0x7f7ae40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 shutdown_connections 2024-09-17T05:34:05.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:05.706+0000 7f7aeb600640 1 -- 172.21.15.43:0/231915871 wait complete. 2024-09-17T05:34:05.729 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:34:05.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:05.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:34:05.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:34:05.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:05.753 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:34:05.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:34:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- 172.21.15.43:0/1857926109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c340051b0 msgr2=0x7f0c3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1857926109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c340051b0 0x7f0c3400b9d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c610 tx=0x7f0c4801d2a0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c4f400640 1 -- 172.21.15.43:0/1857926109 reap_dead start 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- 172.21.15.43:0/1857926109 shutdown_connections 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- 172.21.15.43:0/1857926109 >> 172.21.15.43:0/1857926109 conn(0x7f0c500b23d0 msgr2=0x7f0c500d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- 172.21.15.43:0/1857926109 shutdown_connections 2024-09-17T05:34:06.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- 172.21.15.43:0/1857926109 wait complete. 2024-09-17T05:34:06.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 Processor -- start 2024-09-17T05:34:06.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.010+0000 7f0c55c00640 1 -- start start 2024-09-17T05:34:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 0x7f0c5009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c50074500 0x7f0c5015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50167540 con 0x7f0c500ca8e0 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501673c0 con 0x7f0c340051b0 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0c501676c0 con 0x7f0c50074500 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49074/0 (socket says 172.21.15.43:49074) 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 -- 172.21.15.43:0/1967679502 learned_addr learned my addr 172.21.15.43:0/1967679502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c47e00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c50074500 0x7f0c5015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4ea00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 0x7f0c5009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 -- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c50074500 msgr2=0x7f0c5015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c50074500 0x7f0c5015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 -- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 msgr2=0x7f0c5009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 0x7f0c5009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 -- 172.21.15.43:0/1967679502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c50074d00 con 0x7f0c500ca8e0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4ea00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 0x7f0c5009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c4f400640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f0c48098d60 tx=0x7f0c48006600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48004e60 con 0x7f0c500ca8e0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c48001880 con 0x7f0c500ca8e0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48002060 con 0x7f0c500ca8e0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c501fbc30 con 0x7f0c500ca8e0 2024-09-17T05:34:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501fc1f0 con 0x7f0c500ca8e0 2024-09-17T05:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.014+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c501fba50 con 0x7f0c500ca8e0 2024-09-17T05:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.018+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c48003980 con 0x7f0c500ca8e0 2024-09-17T05:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.018+0000 7f0c46a00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 0x7f0c2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.018+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8805+0+0 (secure 0 0 0) 0x7f0c4812f590 con 0x7f0c500ca8e0 2024-09-17T05:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.018+0000 7f0c4ea00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 0x7f0c2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.018+0000 7f0c4ea00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 0x7f0c2c075710 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000c10 tx=0x7f0c40006bf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.022+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c501fba50 con 0x7f0c500ca8e0 2024-09-17T05:34:06.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.234+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c500d1150 con 0x7f0c500ca8e0 2024-09-17T05:34:06.236 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:06.234+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.234+0000 7f0c46a00640 1 -- 172.21.15.43:0/1967679502 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v316) ==== 161+0+0 (secure 0 0 0) 0x7f0c500d1150 con 0x7f0c500ca8e0 2024-09-17T05:34:06.239 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:34:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 msgr2=0x7f0c2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 0x7f0c2c075710 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000c10 tx=0x7f0c40006bf0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 msgr2=0x7f0c5015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f0c48098d60 tx=0x7f0c48006600 comp rx=0 tx=0).stop 2024-09-17T05:34:06.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 shutdown_connections 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c2c073200 0x7f0c2c075710 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c500ca8e0 0x7f0c5015ac50 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c50074500 0x7f0c5015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 --2- 172.21.15.43:0/1967679502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c340051b0 0x7f0c5009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 >> 172.21.15.43:0/1967679502 conn(0x7f0c500b23d0 msgr2=0x7f0c5015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 shutdown_connections 2024-09-17T05:34:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.238+0000 7f0c55c00640 1 -- 172.21.15.43:0/1967679502 wait complete. 2024-09-17T05:34:06.244 INFO:tasks.ceph.mds.c.smithi043.stderr: -1879> 2024-09-17T05:33:57.142+0000 7f4096e00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-17T05:34:06.248 INFO:tasks.ceph.mds.c.smithi043.stderr: -315> 2024-09-17T05:34:05.130+0000 7f409be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:06.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:34:06.370 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:34:06.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 -- 172.21.15.43:0/3847493536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900b3920 msgr2=0x7f4a900ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 --2- 172.21.15.43:0/3847493536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900b3920 0x7f4a900ad0b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f4a900711f0 tx=0x7f4a8801ccc0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 -- 172.21.15.43:0/3847493536 shutdown_connections 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 --2- 172.21.15.43:0/3847493536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a900b3920 0x7f4a900ad0b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 --2- 172.21.15.43:0/3847493536 >> v2:172.21.15.62:3300/0 conn(0x7f4a9006f110 0x7f4a900b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 --2- 172.21.15.43:0/3847493536 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a90155d70 0x7f4a9015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 -- 172.21.15.43:0/3847493536 >> 172.21.15.43:0/3847493536 conn(0x7f4a900b23d0 msgr2=0x7f4a900d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 -- 172.21.15.43:0/3847493536 shutdown_connections 2024-09-17T05:34:06.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.426+0000 7f4a98000640 1 -- 172.21.15.43:0/3847493536 wait complete. 2024-09-17T05:34:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 Processor -- start 2024-09-17T05:34:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- start start 2024-09-17T05:34:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 0x7f4a9009bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 0x7f4a9015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9015f8d0 con 0x7f4a90155d70 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9015f750 con 0x7f4a900b3920 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9015fa50 con 0x7f4a9006f110 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a95600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 0x7f4a9009bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a95600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 0x7f4a9009bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:46430/0 (socket says 172.21.15.43:46430) 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49126/0 (socket says 172.21.15.43:49126) 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a95600640 1 -- 172.21.15.43:0/1594918070 learned_addr learned my addr 172.21.15.43:0/1594918070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a94c00640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 0x7f4a9015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 -- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 msgr2=0x7f4a9009bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 0x7f4a9009bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 -- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 msgr2=0x7f4a9015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 0x7f4a9015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 -- 172.21.15.43:0/1594918070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a900d0320 con 0x7f4a90155d70 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a94c00640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 0x7f4a9015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a96000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ac000 tx=0x7f4a88009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880b2040 con 0x7f4a90155d70 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a88004d40 con 0x7f4a90155d70 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88004220 con 0x7f4a90155d70 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a900cd7b0 con 0x7f4a90155d70 2024-09-17T05:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.430+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a901664c0 con 0x7f4a90155d70 2024-09-17T05:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9015ff20 con 0x7f4a90155d70 2024-09-17T05:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a880be060 con 0x7f4a90155d70 2024-09-17T05:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a8d600640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 0x7f4a70075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8891+0+0 (secure 0 0 0) 0x7f4a8812ee90 con 0x7f4a90155d70 2024-09-17T05:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a95600640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 0x7f4a70075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.434+0000 7f4a95600640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 0x7f4a70075740 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400bb80 tx=0x7f4a8400b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.438+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4a88016970 con 0x7f4a90155d70 2024-09-17T05:34:06.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:06.678+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4a900ce1a0 con 0x7f4a90155d70 2024-09-17T05:34:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.238+0000 7f4a8d600640 1 -- 172.21.15.43:0/1594918070 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v317) ==== 107+0+0 (secure 0 0 0) 0x7f4a880f9210 con 0x7f4a90155d70 2024-09-17T05:34:07.241 INFO:teuthology.orchestra.run.smithi043.stderr:nothing to do: rank is not damaged 2024-09-17T05:34:07.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 msgr2=0x7f4a70075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 0x7f4a70075740 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400bb80 tx=0x7f4a8400b390 comp rx=0 tx=0).stop 2024-09-17T05:34:07.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 msgr2=0x7f4a9015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ac000 tx=0x7f4a88009350 comp rx=0 tx=0).stop 2024-09-17T05:34:07.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 shutdown_connections 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f4a70073230 0x7f4a70075740 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a900b3920 0x7f4a9015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a90155d70 0x7f4a9015ac50 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 --2- 172.21.15.43:0/1594918070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f4a9006f110 0x7f4a9009bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 >> 172.21.15.43:0/1594918070 conn(0x7f4a900b23d0 msgr2=0x7f4a900cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 shutdown_connections 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.242+0000 7f4a98000640 1 -- 172.21.15.43:0/1594918070 wait complete. 2024-09-17T05:34:07.267 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:34:07.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 Processor -- start 2024-09-17T05:34:07.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 -- start start 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x55923292c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 0x55923292caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559232897f40 con 0x5592328ee4d0 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5592328a4d00 con 0x5592329a1e40 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f0873f0e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559232839a00 con 0x55923291acb0 2024-09-17T05:34:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f086d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f086d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49134/0 (socket says 172.21.15.43:49134) 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.322+0000 7f086d000640 1 -- 172.21.15.43:0/3186339660 learned_addr learned my addr 172.21.15.43:0/3186339660 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 -- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 msgr2=0x55923292c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0867e00640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x55923292c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086da00640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 0x55923292caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x55923292c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 -- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 msgr2=0x55923292caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 0x55923292caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 -- 172.21.15.43:0/3186339660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592329345b0 con 0x5592328ee4d0 2024-09-17T05:34:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0867e00640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x55923292c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086da00640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 0x55923292caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f086d000640 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f086000b250 tx=0x7f086000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9511151ad4bf4f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0866a00640 1 -- 172.21.15.43:0/3186339660 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086000a040 con 0x5592328ee4d0 2024-09-17T05:34:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0866a00640 1 -- 172.21.15.43:0/3186339660 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f086000f620 con 0x5592328ee4d0 2024-09-17T05:34:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0866a00640 1 -- 172.21.15.43:0/3186339660 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086000fa90 con 0x5592328ee4d0 2024-09-17T05:34:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- 172.21.15.43:0/3186339660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 msgr2=0x55923291a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f086000b250 tx=0x7f086000b710 comp rx=0 tx=0).stop 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- 172.21.15.43:0/3186339660 shutdown_connections 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592329a1e40 0x55923292caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x55923292c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 --2- 172.21.15.43:0/3186339660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592328ee4d0 0x55923291a3c0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- 172.21.15.43:0/3186339660 >> 172.21.15.43:0/3186339660 conn(0x5592328d7d80 msgr2=0x5592328e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- 172.21.15.43:0/3186339660 shutdown_connections 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- 172.21.15.43:0/3186339660 wait complete. 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 Processor -- start 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.326+0000 7f0873f0e080 1 -- start start 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592328ee4d0 0x5592329bd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x559232a4f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 0x5592328e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559232a495f0 con 0x5592329a1e40 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5592329be340 con 0x5592328ee4d0 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559232931070 con 0x55923291acb0 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 0x5592328e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 0x5592328e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49140/0 (socket says 172.21.15.43:49140) 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 learned_addr learned my addr 172.21.15.43:0/322759633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592328ee4d0 0x5592329bd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x559232a4f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 msgr2=0x559232a4f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x559232a4f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592328ee4d0 msgr2=0x5592329bd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592328ee4d0 0x5592329bd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592329303f0 con 0x5592329a1e40 2024-09-17T05:34:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55923291acb0 0x559232a4f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f086da00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 0x5592328e9890 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0868006f70 tx=0x7f086800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0865600640 1 -- 172.21.15.43:0/322759633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868015030 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0865600640 1 -- 172.21.15.43:0/322759633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08680113f0 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0865600640 1 -- 172.21.15.43:0/322759633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868011860 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559232930690 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592329bd740 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.330+0000 7f0865600640 1 -- 172.21.15.43:0/322759633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0868018070 con 0x5592329a1e40 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0865600640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f084c071be0 0x7f084c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0865600640 1 -- 172.21.15.43:0/322759633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8891+0+0 (secure 0 0 0) 0x7f0868093bc0 con 0x5592329a1e40 2024-09-17T05:34:07.337 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559232a4f550 0x559232a47f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a48470 con 0x559232a4f550 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f084c071be0 0x7f084c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559232a4f550 0x559232a47f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f084c071be0 0x7f084c0740f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f086000b250 tx=0x7f086000e000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.334+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559232a4f550 0x559232a47f30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v126'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5592328e9dd0 con 0x559232a4f550 2024-09-17T05:34:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v126'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a626e0 con 0x559232a4f550 2024-09-17T05:34:07.343 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:34:07.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v126'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.338+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a63a90 con 0x559232a4f550 2024-09-17T05:34:07.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.342+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.342+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a63e30 con 0x559232a4f550 2024-09-17T05:34:07.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.342+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.342+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62a80 con 0x559232a4f550 2024-09-17T05:34:07.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e70 con 0x559232a4f550 2024-09-17T05:34:07.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v126'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a62e70 con 0x559232a4f550 2024-09-17T05:34:07.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v126'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.346+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.352 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:34:07.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v126'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.350+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.354+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.354+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.354+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.354+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.361 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:34:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.358+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.362+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.362+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.362+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.362+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.366+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.366+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.366+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v126'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.366+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v126'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.373 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:34:07.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v126'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.370+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.374+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.374+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.374+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.374+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v126'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v126'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.382 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:34:07.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v126'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.378+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.382+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.382+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.382+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.382+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a62e20 con 0x559232a4f550 2024-09-17T05:34:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.382+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x559232a641d0 0x559232a665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a66b30 con 0x559232a641d0 2024-09-17T05:34:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x559232a641d0 0x559232a665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x559232a641d0 0x559232a665f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'823 uv802 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.386+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a66ed0 con 0x559232a641d0 2024-09-17T05:34:07.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'824 uv824 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a67270 con 0x559232a641d0 2024-09-17T05:34:07.393 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:34:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a67610 con 0x559232a641d0 2024-09-17T05:34:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.390+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a679b0 con 0x559232a641d0 2024-09-17T05:34:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a67d50 con 0x559232a641d0 2024-09-17T05:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v126'828 uv821 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a680f0 con 0x559232a641d0 2024-09-17T05:34:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v126'829 uv829 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.394+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a680f0 con 0x559232a641d0 2024-09-17T05:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.398+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v126'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.401 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.398+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.398+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.398+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v126'833 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v126'834 uv834 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.402+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.406+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v126'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.408 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.406+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.406+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v126'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.406+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v126'838 uv818 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v126'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.410+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v126'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.414+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a488f0 con 0x559232a641d0 2024-09-17T05:34:07.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a641d0 2024-09-17T05:34:07.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x559232a68ab0 0x559232a68ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6b640 con 0x559232a68ab0 2024-09-17T05:34:07.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x559232a68ab0 0x559232a68ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x559232a68ab0 0x559232a68ec0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v126'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.418+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a6b640 con 0x559232a68ab0 2024-09-17T05:34:07.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v126'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6b640 con 0x559232a68ab0 2024-09-17T05:34:07.425 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:34:07.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v126'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ca40 con 0x559232a68ab0 2024-09-17T05:34:07.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.422+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ca40 con 0x559232a68ab0 2024-09-17T05:34:07.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ba30 con 0x559232a68ab0 2024-09-17T05:34:07.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ba30 con 0x559232a4f550 2024-09-17T05:34:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'160 uv159 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.426+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a6ba30 con 0x559232a4f550 2024-09-17T05:34:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.430+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'161 uv161 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.430+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ba30 con 0x559232a4f550 2024-09-17T05:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.430+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'162 uv162 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.434 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.430+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6ba30 con 0x559232a4f550 2024-09-17T05:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.430+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'163 uv163 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x559232a6bf40 0x559232a6e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6e9c0 con 0x559232a6bf40 2024-09-17T05:34:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f086da00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x559232a6bf40 0x559232a6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f086da00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x559232a6bf40 0x559232a6e360 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v126'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a6ee70 con 0x559232a6bf40 2024-09-17T05:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v126'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.434+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f320 con 0x559232a6bf40 2024-09-17T05:34:07.440 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:34:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v126'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a681f0 con 0x559232a6bf40 2024-09-17T05:34:07.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a70cb0 con 0x559232a6bf40 2024-09-17T05:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.438+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6fbc0 con 0x559232a6bf40 2024-09-17T05:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x559232a70680 0x559232a73110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a736e0 con 0x559232a70680 2024-09-17T05:34:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x559232a70680 0x559232a73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f086d000640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x559232a70680 0x559232a73110 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v126'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.442+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a73720 con 0x559232a70680 2024-09-17T05:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v126'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.449 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v126'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a74b50 con 0x559232a70680 2024-09-17T05:34:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.446+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a74b50 con 0x559232a70680 2024-09-17T05:34:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v126'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.450+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.454+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v126'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.454+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.458 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:34:07.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.454+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v126'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.454+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a73b40 con 0x559232a70680 2024-09-17T05:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f086d000640 1 -- 172.21.15.43:0/322759633 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086000d040 con 0x559232a70680 2024-09-17T05:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x559232a741c0 0x559232a765e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a76c00 con 0x559232a741c0 2024-09-17T05:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x559232a741c0 0x559232a765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.458+0000 7f0867e00640 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x559232a741c0 0x559232a765e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v126'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a76c00 con 0x559232a741c0 2024-09-17T05:34:07.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v126'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a741c0 2024-09-17T05:34:07.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v126'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.466 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:34:07.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.462+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a741c0 2024-09-17T05:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a741c0 2024-09-17T05:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a4f550 2024-09-17T05:34:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v126'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.466+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a4f550 2024-09-17T05:34:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v126'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a4f550 2024-09-17T05:34:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v126'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.473 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:34:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a78030 con 0x559232a4f550 2024-09-17T05:34:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.470+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a78030 con 0x559232a4f550 2024-09-17T05:34:07.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a4f550 2024-09-17T05:34:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a4f550 2024-09-17T05:34:07.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a68ab0 2024-09-17T05:34:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v126'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.474+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a68ab0 2024-09-17T05:34:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.478+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v126'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.478+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a68ab0 2024-09-17T05:34:07.481 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:34:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.478+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v126'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.478+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v126'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.482+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.486+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v126'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.486+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.490 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:34:07.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.486+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v126'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.486+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a6f7b0 con 0x559232a68ab0 2024-09-17T05:34:07.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a79ed0 con 0x559232a68ab0 2024-09-17T05:34:07.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a68ab0 2024-09-17T05:34:07.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77020 con 0x559232a6bf40 2024-09-17T05:34:07.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v126'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.490+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a77410 con 0x559232a6bf40 2024-09-17T05:34:07.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.494+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.494+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77800 con 0x559232a6bf40 2024-09-17T05:34:07.498 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:34:07.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.494+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v126'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.494+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a77800 con 0x559232a6bf40 2024-09-17T05:34:07.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a6bf40 2024-09-17T05:34:07.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v126'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a6bf40 2024-09-17T05:34:07.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v126'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.498+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a6bf40 2024-09-17T05:34:07.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v126'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.504 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:34:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a787c0 con 0x559232a6bf40 2024-09-17T05:34:07.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a6bf40 2024-09-17T05:34:07.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.502+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a6bf40 2024-09-17T05:34:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f086807f050 con 0x559232a6bf40 2024-09-17T05:34:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v126'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v126'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.506+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v126'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.512 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:34:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.510+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v126'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v126'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v126'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.519 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:34:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.514+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.518+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.518+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.518+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.518+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.522+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v126'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.522+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.522+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v126'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.522+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.527 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:34:07.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v126'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.526+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a777b0 con 0x559232a741c0 2024-09-17T05:34:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x559232a78050 con 0x559232a741c0 2024-09-17T05:34:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0867e00640 1 -- 172.21.15.43:0/322759633 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f08580a7ab0 con 0x559232a741c0 2024-09-17T05:34:07.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x559232a641d0 msgr2=0x559232a665f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x559232a641d0 0x559232a665f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x559232a70680 msgr2=0x559232a73110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x559232a70680 0x559232a73110 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x559232a741c0 msgr2=0x559232a765e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x559232a741c0 0x559232a765e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x559232a68ab0 msgr2=0x559232a68ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 reap_dead start 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x559232a68ab0 0x559232a68ec0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559232a4f550 msgr2=0x559232a47f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x559232a4f550 0x559232a47f30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x559232a6bf40 msgr2=0x559232a6e360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x559232a6bf40 0x559232a6e360 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f084c071be0 msgr2=0x7f084c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f084c071be0 0x7f084c0740f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f086000b250 tx=0x7f086000e000 comp rx=0 tx=0).stop 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 msgr2=0x5592328e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 --2- 172.21.15.43:0/322759633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5592329a1e40 0x5592328e9890 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0868006f70 tx=0x7f086800c5e0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f086da00640 1 -- 172.21.15.43:0/322759633 reap_dead start 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 shutdown_connections 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 >> 172.21.15.43:0/322759633 conn(0x5592328d7d80 msgr2=0x55923291a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 shutdown_connections 2024-09-17T05:34:07.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.530+0000 7f0873f0e080 1 -- 172.21.15.43:0/322759633 wait complete. 2024-09-17T05:34:07.539 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:34:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.638+0000 7fdc864cc080 1 Processor -- start 2024-09-17T05:34:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.638+0000 7fdc864cc080 1 -- start start 2024-09-17T05:34:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 0x556c2b2d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b2bae40 0x556c2b202890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c2b152a00 con 0x556c2b233cb0 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556c2b1b0f40 con 0x556c2b204d70 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556c2b2bbb80 con 0x556c2b2bae40 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49158/0 (socket says 172.21.15.43:49158) 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 -- 172.21.15.43:0/679890644 learned_addr learned my addr 172.21.15.43:0/679890644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 0x556c2b2d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc80000640 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b2bae40 0x556c2b202890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 -- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b2bae40 msgr2=0x556c2b202890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b2bae40 0x556c2b202890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 -- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 msgr2=0x556c2b2d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 0x556c2b2d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 -- 172.21.15.43:0/679890644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2b12bc10 con 0x556c2b233cb0 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7f600640 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 0x556c2b2d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004f60 tx=0x7fdc6c01a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1767544023fbb5ce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7d800640 1 -- 172.21.15.43:0/679890644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0a7810 con 0x556c2b233cb0 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7d800640 1 -- 172.21.15.43:0/679890644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6c0a7b10 con 0x556c2b233cb0 2024-09-17T05:34:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc7d800640 1 -- 172.21.15.43:0/679890644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0b0470 con 0x556c2b233cb0 2024-09-17T05:34:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- 172.21.15.43:0/679890644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 msgr2=0x556c2b2d7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c004f60 tx=0x7fdc6c01a870 comp rx=0 tx=0).stop 2024-09-17T05:34:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- 172.21.15.43:0/679890644 shutdown_connections 2024-09-17T05:34:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b2bae40 0x556c2b202890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b2d7fc0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 --2- 172.21.15.43:0/679890644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b204d70 0x556c2b2d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- 172.21.15.43:0/679890644 >> 172.21.15.43:0/679890644 conn(0x556c2b1f0d80 msgr2=0x556c2b24cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- 172.21.15.43:0/679890644 shutdown_connections 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.642+0000 7fdc864cc080 1 -- 172.21.15.43:0/679890644 wait complete. 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 Processor -- start 2024-09-17T05:34:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- start start 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b204d70 0x556c2b371270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b3723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b2bae40 0x556c2b2cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556c2b2d9e20 con 0x556c2b233cb0 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556c2b2da600 con 0x556c2b2bae40 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x556c2b2da0c0 con 0x556c2b204d70 2024-09-17T05:34:07.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b3723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7f600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b204d70 0x556c2b371270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b3723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49162/0 (socket says 172.21.15.43:49162) 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 -- 172.21.15.43:0/1681326813 learned_addr learned my addr 172.21.15.43:0/1681326813 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc80000640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b2bae40 0x556c2b2cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 -- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b204d70 msgr2=0x556c2b371270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b204d70 0x556c2b371270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 -- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b2bae40 msgr2=0x556c2b2cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c2b2bae40 0x556c2b2cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 -- 172.21.15.43:0/1681326813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2b203780 con 0x556c2b233cb0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7f600640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x556c2b204d70 0x556c2b371270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc7ec00640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b3723a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c002010 tx=0x7fdc6c0a7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc77e00640 1 -- 172.21.15.43:0/1681326813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0b1790 con 0x556c2b233cb0 2024-09-17T05:34:07.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc77e00640 1 -- 172.21.15.43:0/1681326813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6c0b1a40 con 0x556c2b233cb0 2024-09-17T05:34:07.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c2b3729d0 con 0x556c2b233cb0 2024-09-17T05:34:07.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc77e00640 1 -- 172.21.15.43:0/1681326813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0bc410 con 0x556c2b233cb0 2024-09-17T05:34:07.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.646+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2b372f10 con 0x556c2b233cb0 2024-09-17T05:34:07.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc77e00640 1 -- 172.21.15.43:0/1681326813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc6c0ba030 con 0x556c2b233cb0 2024-09-17T05:34:07.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc77e00640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc60071be0 0x7fdc600740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc77e00640 1 -- 172.21.15.43:0/1681326813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8891+0+0 (secure 0 0 0) 0x7fdc6c1322a0 con 0x556c2b233cb0 2024-09-17T05:34:07.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc7f600640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc60071be0 0x7fdc600740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc864cc080 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556c2b230480 0x556c2b3738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x556c2b373e30 con 0x556c2b230480 2024-09-17T05:34:07.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc80000640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556c2b230480 0x556c2b3738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.650+0000 7fdc7f600640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc60071be0 0x7fdc600740f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000ce40 tx=0x7fdc7000f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc80000640 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556c2b230480 0x556c2b3738f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc80000640 1 -- 172.21.15.43:0/1681326813 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v126'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc780026a0 con 0x556c2b230480 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556c2b230480 msgr2=0x556c2b3738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x556c2b230480 0x556c2b3738f0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc60071be0 msgr2=0x7fdc600740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc60071be0 0x7fdc600740f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000ce40 tx=0x7fdc7000f040 comp rx=0 tx=0).stop 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 msgr2=0x556c2b3723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 --2- 172.21.15.43:0/1681326813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556c2b233cb0 0x556c2b3723a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c002010 tx=0x7fdc6c0a7920 comp rx=0 tx=0).stop 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc80000640 1 -- 172.21.15.43:0/1681326813 reap_dead start 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 shutdown_connections 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 >> 172.21.15.43:0/1681326813 conn(0x556c2b1f0d80 msgr2=0x556c2b24cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 shutdown_connections 2024-09-17T05:34:07.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.654+0000 7fdc864cc080 1 -- 172.21.15.43:0/1681326813 wait complete. 2024-09-17T05:34:07.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- 172.21.15.43:0/4269593611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 msgr2=0x7fe6780ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 --2- 172.21.15.43:0/4269593611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe6780ad0b0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fe6780af0a0 tx=0x7fe67001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- 172.21.15.43:0/4269593611 shutdown_connections 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 --2- 172.21.15.43:0/4269593611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe6780ad0b0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 --2- 172.21.15.43:0/4269593611 >> v2:172.21.15.62:3300/0 conn(0x7fe6780b3af0 0x7fe67815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- 172.21.15.43:0/4269593611 >> 172.21.15.43:0/4269593611 conn(0x7fe6780b23d0 msgr2=0x7fe6780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- 172.21.15.43:0/4269593611 shutdown_connections 2024-09-17T05:34:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- 172.21.15.43:0/4269593611 wait complete. 2024-09-17T05:34:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 Processor -- start 2024-09-17T05:34:07.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.822+0000 7fe67f800640 1 -- start start 2024-09-17T05:34:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe678155d70 0x7fe6780ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6780ce640 0x7fe6780cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe678168c30 con 0x7fe67806f110 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe678168ab0 con 0x7fe678155d70 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe678168db0 con 0x7fe6780ce640 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe677e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe678155d70 0x7fe6780ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49176/0 (socket says 172.21.15.43:49176) 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 -- 172.21.15.43:0/78743482 learned_addr learned my addr 172.21.15.43:0/78743482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67d800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6780ce640 0x7fe6780cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 -- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6780ce640 msgr2=0x7fe6780cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6780ce640 0x7fe6780cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 -- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe678155d70 msgr2=0x7fe6780ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe678155d70 0x7fe6780ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 -- 172.21.15.43:0/78743482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6780ca980 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67ce00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c3f0 tx=0x7fe66c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017070 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c0053c0 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6780cfe90 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c003860 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67809bf60 con 0x7fe67806f110 2024-09-17T05:34:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.826+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6780ad0b0 con 0x7fe67806f110 2024-09-17T05:34:07.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.830+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe66c01a070 con 0x7fe67806f110 2024-09-17T05:34:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.830+0000 7fe674c00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 0x7fe658075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.830+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8891+0+0 (secure 0 0 0) 0x7fe66c0941f0 con 0x7fe67806f110 2024-09-17T05:34:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.830+0000 7fe677e00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 0x7fe658075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.830+0000 7fe677e00640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 0x7fe658075710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe678166130 tx=0x7fe66400b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:07.834+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe66c05dea0 con 0x7fe67806f110 2024-09-17T05:34:08.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.042+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe678165e80 con 0x7fe67806f110 2024-09-17T05:34:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.242+0000 7fe674c00640 1 -- 172.21.15.43:0/78743482 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v318) ==== 162+0+0 (secure 0 0 0) 0x7fe66c0628d0 con 0x7fe67806f110 2024-09-17T05:34:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:34:08.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 msgr2=0x7fe658075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 0x7fe658075710 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe678166130 tx=0x7fe66400b390 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 msgr2=0x7fe67809ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c3f0 tx=0x7fe66c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 shutdown_connections 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe658073200 0x7fe658075710 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe678155d70 0x7fe6780ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe6780ce640 0x7fe6780cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 --2- 172.21.15.43:0/78743482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe67806f110 0x7fe67809ba20 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 >> 172.21.15.43:0/78743482 conn(0x7fe6780b23d0 msgr2=0x7fe67809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 shutdown_connections 2024-09-17T05:34:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.246+0000 7fe67f800640 1 -- 172.21.15.43:0/78743482 wait complete. 2024-09-17T05:34:08.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- 172.21.15.43:0/2620575674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240d0d80 msgr2=0x7f0c240cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 --2- 172.21.15.43:0/2620575674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240d0d80 0x7f0c240cce70 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f0c24075050 tx=0x7f0c0c01c800 comp rx=0 tx=0).stop 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- 172.21.15.43:0/2620575674 shutdown_connections 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 --2- 172.21.15.43:0/2620575674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240d0d80 0x7f0c240cce70 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 --2- 172.21.15.43:0/2620575674 >> v2:172.21.15.62:3300/0 conn(0x7f0c240b7ce0 0x7f0c240d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- 172.21.15.43:0/2620575674 >> 172.21.15.43:0/2620575674 conn(0x7f0c240cee80 msgr2=0x7f0c240cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- 172.21.15.43:0/2620575674 shutdown_connections 2024-09-17T05:34:08.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- 172.21.15.43:0/2620575674 wait complete. 2024-09-17T05:34:08.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 Processor -- start 2024-09-17T05:34:08.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.430+0000 7f0c2b000640 1 -- start start 2024-09-17T05:34:08.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c240b7ce0 0x7f0c240cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 0x7f0c2415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24169e10 con 0x7f0c240cd3b0 2024-09-17T05:34:08.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24169c90 con 0x7f0c240b7ce0 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0c24169f90 con 0x7f0c240d0d80 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49202/0 (socket says 172.21.15.43:49202) 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 -- 172.21.15.43:0/477644160 learned_addr learned my addr 172.21.15.43:0/477644160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c23e00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c240b7ce0 0x7f0c240cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c29000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 0x7f0c2415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 -- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 msgr2=0x7f0c2415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 0x7f0c2415a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 -- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c240b7ce0 msgr2=0x7f0c240cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c240b7ce0 0x7f0c240cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 -- 172.21.15.43:0/477644160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c24157630 con 0x7f0c240cd3b0 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c29000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 0x7f0c2415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c1be00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c01aeb0 tx=0x7f0c0c0a6500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0be020 con 0x7f0c240cd3b0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c0c004040 con 0x7f0c240cd3b0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c24157910 con 0x7f0c240cd3b0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c004490 con 0x7f0c240cd3b0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c240cb8c0 con 0x7f0c240cd3b0 2024-09-17T05:34:08.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.434+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c241573a0 con 0x7f0c240cd3b0 2024-09-17T05:34:08.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.438+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c0c003600 con 0x7f0c240cd3b0 2024-09-17T05:34:08.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.438+0000 7f0c20c00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 0x7f0c00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.438+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8891+0+0 (secure 0 0 0) 0x7f0c0c12f4c0 con 0x7f0c240cd3b0 2024-09-17T05:34:08.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.438+0000 7f0c23e00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 0x7f0c00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.438+0000 7f0c23e00640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 0x7f0c00075710 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0c24071220 tx=0x7f0c14005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.442+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c0c0163f0 con 0x7f0c240cd3b0 2024-09-17T05:34:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.650+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c240d0840 con 0x7f0c240cd3b0 2024-09-17T05:34:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.650+0000 7f0c20c00640 1 -- 172.21.15.43:0/477644160 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+5221 (secure 0 0 0) 0x7f0c0c0f98e0 con 0x7f0c240cd3b0 2024-09-17T05:34:08.654 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:08.654 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":319,"btime":"2024-09-17T05:34:08:250265+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":319,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:08.250253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"c","rank":0,"incarnation":319,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6835/2308652168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2308652168},{"type":"v1","addr":"172.21.15.43:6835","nonce":2308652168}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 319 2024-09-17T05:34:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 msgr2=0x7f0c00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 0x7f0c00075710 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0c24071220 tx=0x7f0c14005f40 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 msgr2=0x7f0c2415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c01aeb0 tx=0x7f0c0c0a6500 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 shutdown_connections 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0c00073200 0x7f0c00075710 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0c240d0d80 0x7f0c2415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c240cd3b0 0x7f0c2415a160 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 --2- 172.21.15.43:0/477644160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c240b7ce0 0x7f0c240cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 >> 172.21.15.43:0/477644160 conn(0x7f0c240cee80 msgr2=0x7f0c240cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 shutdown_connections 2024-09-17T05:34:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:08.654+0000 7f0c2b000640 1 -- 172.21.15.43:0/477644160 wait complete. 2024-09-17T05:34:08.677 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:34:11.270 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:34:11.266+0000 7f9bd5000640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:11.271 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:11.270+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:34:11.271 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:11.270+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:34:12.250 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:34:12.250+0000 7f9bda000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:13.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 -- 172.21.15.43:0/3359308132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 msgr2=0x7f07240b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 --2- 172.21.15.43:0/3359308132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f07240b33e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f0724072b60 tx=0x7f071001c740 comp rx=0 tx=0).stop 2024-09-17T05:34:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 -- 172.21.15.43:0/3359308132 shutdown_connections 2024-09-17T05:34:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 --2- 172.21.15.43:0/3359308132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f07240b33e0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 --2- 172.21.15.43:0/3359308132 >> v2:172.21.15.62:3300/0 conn(0x7f0724155d70 0x7f072415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 -- 172.21.15.43:0/3359308132 >> 172.21.15.43:0/3359308132 conn(0x7f07240b23d0 msgr2=0x7f07240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 -- 172.21.15.43:0/3359308132 shutdown_connections 2024-09-17T05:34:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.846+0000 7f072c200640 1 -- 172.21.15.43:0/3359308132 wait complete. 2024-09-17T05:34:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 Processor -- start 2024-09-17T05:34:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 -- start start 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07240b3920 0x7f072415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0724155d70 0x7f0724164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07241689f0 con 0x7f072406f110 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0724168870 con 0x7f07240b3920 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072c200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0724168b70 con 0x7f0724155d70 2024-09-17T05:34:13.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f0729800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f0729800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36842/0 (socket says 172.21.15.43:36842) 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f0729800640 1 -- 172.21.15.43:0/355278080 learned_addr learned my addr 172.21.15.43:0/355278080 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f0728e00640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07240b3920 0x7f072415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.850+0000 7f072a200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0724155d70 0x7f0724164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 -- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0724155d70 msgr2=0x7f0724164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0724155d70 0x7f0724164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 -- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07240b3920 msgr2=0x7f072415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07240b3920 0x7f072415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 -- 172.21.15.43:0/355278080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07240cc8c0 con 0x7f072406f110 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0729800640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f071800d6d0 tx=0x7f071800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718017070 con 0x7f072406f110 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0718009b10 con 0x7f072406f110 2024-09-17T05:34:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718010460 con 0x7f072406f110 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07241fbff0 con 0x7f072406f110 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241fc870 con 0x7f072406f110 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07240b33e0 con 0x7f072406f110 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.854+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f071801a070 con 0x7f072406f110 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.858+0000 7f0721600640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 0x7f0704075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.858+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7f07180956f0 con 0x7f072406f110 2024-09-17T05:34:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.858+0000 7f0728e00640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 0x7f0704075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.858+0000 7f0728e00640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 0x7f0704075710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f07100af000 tx=0x7f071001af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:13.858+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f071805f3c0 con 0x7f072406f110 2024-09-17T05:34:14.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.070+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07240ad430 con 0x7f072406f110 2024-09-17T05:34:14.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f0721600640 1 -- 172.21.15.43:0/355278080 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 323 v323) ==== 95+0+4383 (secure 0 0 0) 0x7f0718063df0 con 0x7f072406f110 2024-09-17T05:34:14.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:14.076 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":323,"btime":"2024-09-17T05:34:11:272273+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":323,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:11.272267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:14.076 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 323 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 msgr2=0x7f0704075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 0x7f0704075710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f07100af000 tx=0x7f071001af80 comp rx=0 tx=0).stop 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 msgr2=0x7f072409d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f071800d6d0 tx=0x7f071800db90 comp rx=0 tx=0).stop 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 shutdown_connections 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0704073200 0x7f0704075710 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07240b3920 0x7f072415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0724155d70 0x7f0724164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 --2- 172.21.15.43:0/355278080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072406f110 0x7f072409d6d0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 >> 172.21.15.43:0/355278080 conn(0x7f07240b23d0 msgr2=0x7f07240b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 shutdown_connections 2024-09-17T05:34:14.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.074+0000 7f072c200640 1 -- 172.21.15.43:0/355278080 wait complete. 2024-09-17T05:34:14.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:34:14.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 -- 172.21.15.43:0/2691858077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9524004810 msgr2=0x7f9524004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 --2- 172.21.15.43:0/2691858077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9524004810 0x7f9524004c20 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f9540073060 tx=0x7f952801c730 comp rx=0 tx=0).stop 2024-09-17T05:34:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 -- 172.21.15.43:0/2691858077 shutdown_connections 2024-09-17T05:34:14.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 --2- 172.21.15.43:0/2691858077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9524004810 0x7f9524004c20 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 -- 172.21.15.43:0/2691858077 >> 172.21.15.43:0/2691858077 conn(0x7f95400b23d0 msgr2=0x7f95400d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 -- 172.21.15.43:0/2691858077 shutdown_connections 2024-09-17T05:34:14.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.254+0000 7f9546200640 1 -- 172.21.15.43:0/2691858077 wait complete. 2024-09-17T05:34:14.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 Processor -- start 2024-09-17T05:34:14.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 -- start start 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 0x7f95401f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 0x7f95400c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95400ad6f0 con 0x7f954009dd80 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95400ad570 con 0x7f95400c8840 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f95400ad870 con 0x7f9524004810 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36862/0 (socket says 172.21.15.43:36862) 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 -- 172.21.15.43:0/661578727 learned_addr learned my addr 172.21.15.43:0/661578727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f953fe00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 0x7f95400c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f953f400640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 0x7f95401f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 -- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 msgr2=0x7f95401f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 0x7f95401f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 -- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 msgr2=0x7f95400c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 0x7f95400c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 -- 172.21.15.43:0/661578727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95400943b0 con 0x7f954009dd80 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f953fe00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 0x7f95400c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f953f400640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 0x7f95401f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9536a00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f952801b000 tx=0x7f9528001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95280036c0 con 0x7f954009dd80 2024-09-17T05:34:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9528003950 con 0x7f954009dd80 2024-09-17T05:34:14.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528003d00 con 0x7f954009dd80 2024-09-17T05:34:14.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.258+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9540094690 con 0x7f954009dd80 2024-09-17T05:34:14.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9540094c50 con 0x7f954009dd80 2024-09-17T05:34:14.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95400addd0 con 0x7f954009dd80 2024-09-17T05:34:14.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f95280a5510 con 0x7f954009dd80 2024-09-17T05:34:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f9537e00640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 0x7f951c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7f952812e690 con 0x7f954009dd80 2024-09-17T05:34:14.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.262+0000 7f953f400640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 0x7f951c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.266+0000 7f953f400640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 0x7f951c075740 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f95400715c0 tx=0x7f953000b310 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.266+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f95280b3070 con 0x7f954009dd80 2024-09-17T05:34:14.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.478+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95400b0130 con 0x7f954009dd80 2024-09-17T05:34:14.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9537e00640 1 -- 172.21.15.43:0/661578727 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 323 v323) ==== 95+0+4383 (secure 0 0 0) 0x7f9528016490 con 0x7f954009dd80 2024-09-17T05:34:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:14.484 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":323,"btime":"2024-09-17T05:34:11:272273+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":323,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:11.272267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:14.484 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 323 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 msgr2=0x7f951c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 0x7f951c075740 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f95400715c0 tx=0x7f953000b310 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 msgr2=0x7f95400c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f952801b000 tx=0x7f9528001440 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 shutdown_connections 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f951c073230 0x7f951c075740 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400c8840 0x7f95400c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f954009dd80 0x7f95400c8300 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 --2- 172.21.15.43:0/661578727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9524004810 0x7f95401f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 >> 172.21.15.43:0/661578727 conn(0x7f95400b23d0 msgr2=0x7f95400b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 shutdown_connections 2024-09-17T05:34:14.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.482+0000 7f9546200640 1 -- 172.21.15.43:0/661578727 wait complete. 2024-09-17T05:34:14.506 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:34:14.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 -- 172.21.15.43:0/1051069112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbec004020 msgr2=0x7fbbec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 --2- 172.21.15.43:0/1051069112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbec004020 0x7fbbec004810 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009b30 tx=0x7fbbf001d240 comp rx=0 tx=0).stop 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc06800640 1 -- 172.21.15.43:0/1051069112 reap_dead start 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 -- 172.21.15.43:0/1051069112 shutdown_connections 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 -- 172.21.15.43:0/1051069112 >> 172.21.15.43:0/1051069112 conn(0x7fbc080b23d0 msgr2=0x7fbc080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 -- 172.21.15.43:0/1051069112 shutdown_connections 2024-09-17T05:34:14.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.662+0000 7fbc0d200640 1 -- 172.21.15.43:0/1051069112 wait complete. 2024-09-17T05:34:14.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 Processor -- start 2024-09-17T05:34:14.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- start start 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 0x7fbc080d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc080cf150 0x7fbc080cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081663b0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080ade00 con 0x7fbbec004020 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbc08166530 con 0x7fbc080cf150 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36892/0 (socket says 172.21.15.43:36892) 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 -- 172.21.15.43:0/2003384181 learned_addr learned my addr 172.21.15.43:0/2003384181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:14.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc05e00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 0x7fbc080d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc06800640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc080cf150 0x7fbc080cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 -- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc080cf150 msgr2=0x7fbc080cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc080cf150 0x7fbc080cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 -- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 msgr2=0x7fbc080d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 0x7fbc080d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 -- 172.21.15.43:0/2003384181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc08155070 con 0x7fbc080ca8e0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc05e00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 0x7fbc080d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbffe00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af040 tx=0x7fbbf0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00b1040 con 0x7fbc080ca8e0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc080cfaa0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf00bb070 con 0x7fbc080ca8e0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0004290 con 0x7fbc080ca8e0 2024-09-17T05:34:14.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.666+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc080cd440 con 0x7fbc080ca8e0 2024-09-17T05:34:14.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0815c7e0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbf00046d0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbbfe000640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 0x7fbbe4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7fbbf01300d0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbc05e00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 0x7fbbe4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.670+0000 7fbc05e00640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 0x7fbbe4075740 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8001cd0 tx=0x7fbbf8000ee0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.674+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc080cfaa0 con 0x7fbc080ca8e0 2024-09-17T05:34:14.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc080cdb30 con 0x7fbc080ca8e0 2024-09-17T05:34:14.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbbfe000640 1 -- 172.21.15.43:0/2003384181 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 323 v323) ==== 95+0+4383 (secure 0 0 0) 0x7fbbf0016f80 con 0x7fbc080ca8e0 2024-09-17T05:34:14.890 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:14.890 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":323,"btime":"2024-09-17T05:34:11:272273+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":323,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:11.272267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:14.890 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 323 2024-09-17T05:34:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 msgr2=0x7fbbe4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 0x7fbbe4075740 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8001cd0 tx=0x7fbbf8000ee0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 msgr2=0x7fbc08154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.886+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af040 tx=0x7fbbf0005590 comp rx=0 tx=0).stop 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 shutdown_connections 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbbe4073230 0x7fbbe4075740 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbc080cf150 0x7fbc080cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080ca8e0 0x7fbc08154b30 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 --2- 172.21.15.43:0/2003384181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbec004020 0x7fbc080d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 >> 172.21.15.43:0/2003384181 conn(0x7fbc080b23d0 msgr2=0x7fbc0809db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 shutdown_connections 2024-09-17T05:34:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:14.890+0000 7fbc0d200640 1 -- 172.21.15.43:0/2003384181 wait complete. 2024-09-17T05:34:14.912 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-09-17T05:34:14.912 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-09-17T05:34:14.913 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:14.913 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:14.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:14.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:34:14.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:14.924 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:34:14.979 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:34:14.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:34:14.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:34:15.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 -- 172.21.15.43:0/4004946018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc880b3af0 msgr2=0x7efc880ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 --2- 172.21.15.43:0/4004946018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc880b3af0 0x7efc880ad0b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7efc80009c80 tx=0x7efc8001cce0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 -- 172.21.15.43:0/4004946018 shutdown_connections 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 --2- 172.21.15.43:0/4004946018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc880b3af0 0x7efc880ad0b0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 --2- 172.21.15.43:0/4004946018 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc88155d70 0x7efc8815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 -- 172.21.15.43:0/4004946018 >> 172.21.15.43:0/4004946018 conn(0x7efc880b23d0 msgr2=0x7efc880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 -- 172.21.15.43:0/4004946018 shutdown_connections 2024-09-17T05:34:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.178+0000 7efc8f600640 1 -- 172.21.15.43:0/4004946018 wait complete. 2024-09-17T05:34:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 Processor -- start 2024-09-17T05:34:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 -- start start 2024-09-17T05:34:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc8806f110 0x7efc881f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 0x7efc881f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc88168c80 con 0x7efc881f8a20 2024-09-17T05:34:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc88168b00 con 0x7efc88155d70 2024-09-17T05:34:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8f600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efc88168e00 con 0x7efc8806f110 2024-09-17T05:34:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36916/0 (socket says 172.21.15.43:36916) 2024-09-17T05:34:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 -- 172.21.15.43:0/2180486370 learned_addr learned my addr 172.21.15.43:0/2180486370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8cc00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc8806f110 0x7efc881f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc87e00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 0x7efc881f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 -- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc8806f110 msgr2=0x7efc881f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc8806f110 0x7efc881f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 -- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 msgr2=0x7efc881f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 0x7efc881f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 -- 172.21.15.43:0/2180486370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc880cf530 con 0x7efc881f8a20 2024-09-17T05:34:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc87e00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 0x7efc881f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.182+0000 7efc8d600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7efc80009bd0 tx=0x7efc80005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800bb070 con 0x7efc881f8a20 2024-09-17T05:34:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc80004b40 con 0x7efc881f8a20 2024-09-17T05:34:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800b2040 con 0x7efc881f8a20 2024-09-17T05:34:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc880cf810 con 0x7efc881f8a20 2024-09-17T05:34:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc880cfdd0 con 0x7efc881f8a20 2024-09-17T05:34:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc880ad0b0 con 0x7efc881f8a20 2024-09-17T05:34:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc80004000 con 0x7efc881f8a20 2024-09-17T05:34:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 0x7efc68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.186+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7efc8012ed80 con 0x7efc881f8a20 2024-09-17T05:34:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.190+0000 7efc8cc00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 0x7efc68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.190+0000 7efc8cc00640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 0x7efc68075710 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7efc880715c0 tx=0x7efc7c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.190+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc80016990 con 0x7efc881f8a20 2024-09-17T05:34:15.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:15.406+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efc880cf280 con 0x7efc881f8a20 2024-09-17T05:34:16.025 INFO:tasks.ceph.mds.c.smithi043.stderr: -58> 2024-09-17T05:34:11.266+0000 7f9bd5000640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:16.025 INFO:tasks.ceph.mds.c.smithi043.stderr: -33> 2024-09-17T05:34:12.250+0000 7f9bda000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:16.146 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:34:16.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.294+0000 7efc84c00640 1 -- 172.21.15.43:0/2180486370 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v324) ==== 161+0+0 (secure 0 0 0) 0x7efc800f8af0 con 0x7efc881f8a20 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:34:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 msgr2=0x7efc68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 0x7efc68075710 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7efc880715c0 tx=0x7efc7c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 msgr2=0x7efc8809d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7efc80009bd0 tx=0x7efc80005b00 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 shutdown_connections 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efc68073200 0x7efc68075710 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc88155d70 0x7efc881f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc881f8a20 0x7efc8809d640 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 --2- 172.21.15.43:0/2180486370 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efc8806f110 0x7efc881f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 >> 172.21.15.43:0/2180486370 conn(0x7efc880b23d0 msgr2=0x7efc880afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 shutdown_connections 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.298+0000 7efc8f600640 1 -- 172.21.15.43:0/2180486370 wait complete. 2024-09-17T05:34:16.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:34:16.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 -- 172.21.15.43:0/2219378699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0910 msgr2=0x7f3a6006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 --2- 172.21.15.43:0/2219378699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0910 0x7f3a6006ce60 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f3a600d12a0 tx=0x7f3a4c01c770 comp rx=0 tx=0).stop 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 -- 172.21.15.43:0/2219378699 shutdown_connections 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 --2- 172.21.15.43:0/2219378699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0910 0x7f3a6006ce60 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 --2- 172.21.15.43:0/2219378699 >> v2:172.21.15.62:3300/0 conn(0x7f3a60070620 0x7f3a600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 -- 172.21.15.43:0/2219378699 >> 172.21.15.43:0/2219378699 conn(0x7f3a600b23d0 msgr2=0x7f3a600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 -- 172.21.15.43:0/2219378699 shutdown_connections 2024-09-17T05:34:16.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.478+0000 7f3a64a00640 1 -- 172.21.15.43:0/2219378699 wait complete. 2024-09-17T05:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 Processor -- start 2024-09-17T05:34:16.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 -- start start 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a60070620 0x7f3a6009c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 0x7f3a601f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3a600ce570 con 0x7f3a600d0d20 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3a600ce3f0 con 0x7f3a60070620 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a64a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3a600ce6f0 con 0x7f3a600d0910 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36958/0 (socket says 172.21.15.43:36958) 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 -- 172.21.15.43:0/3476840727 learned_addr learned my addr 172.21.15.43:0/3476840727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5de00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a60070620 0x7f3a6009c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5d400640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 0x7f3a601f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 -- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 msgr2=0x7f3a601f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 0x7f3a601f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 -- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a60070620 msgr2=0x7f3a6009c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a60070620 0x7f3a6009c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 -- 172.21.15.43:0/3476840727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a600aebf0 con 0x7f3a600d0d20 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5d400640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 0x7f3a601f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:16.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.482+0000 7f3a5e800640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c700 tx=0x7f3a5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a58017070 con 0x7f3a600d0d20 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a5800ce10 con 0x7f3a600d0d20 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a600aeed0 con 0x7f3a600d0d20 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a580052d0 con 0x7f3a600d0d20 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a601f9610 con 0x7f3a600d0d20 2024-09-17T05:34:16.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a600ae960 con 0x7f3a600d0d20 2024-09-17T05:34:16.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a5801a070 con 0x7f3a600d0d20 2024-09-17T05:34:16.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 0x7f3a40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7f3a580944f0 con 0x7f3a600d0d20 2024-09-17T05:34:16.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.486+0000 7f3a5de00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 0x7f3a40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.490+0000 7f3a5de00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 0x7f3a40075740 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000bb50 tx=0x7f3a5000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.494+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a5805e140 con 0x7f3a600d0d20 2024-09-17T05:34:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:16.734+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3a601f9120 con 0x7f3a600d0d20 2024-09-17T05:34:17.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a56000640 1 -- 172.21.15.43:0/3476840727 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v325) ==== 101+0+0 (secure 0 0 0) 0x7f3a58062b70 con 0x7f3a600d0d20 2024-09-17T05:34:17.301 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 msgr2=0x7f3a40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 0x7f3a40075740 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000bb50 tx=0x7f3a5000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 msgr2=0x7f3a600ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800c700 tx=0x7f3a5800cbc0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 shutdown_connections 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3a40073230 0x7f3a40075740 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a600d0d20 0x7f3a600ad0b0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3a600d0910 0x7f3a601f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 --2- 172.21.15.43:0/3476840727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a60070620 0x7f3a6009c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 >> 172.21.15.43:0/3476840727 conn(0x7f3a600b23d0 msgr2=0x7f3a600ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 shutdown_connections 2024-09-17T05:34:17.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.298+0000 7f3a64a00640 1 -- 172.21.15.43:0/3476840727 wait complete. 2024-09-17T05:34:17.326 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 Processor -- start 2024-09-17T05:34:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 -- start start 2024-09-17T05:34:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c02d4d0 0x56241c1883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c191510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56241bf78a00 con 0x56241c0e0e40 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56241bfd6f40 con 0x56241c02d4d0 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdca598c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56241c0e1b30 con 0x56241c059cb0 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdc9f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdc9f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36964/0 (socket says 172.21.15.43:36964) 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdc9f600640 1 -- 172.21.15.43:0/750967326 learned_addr learned my addr 172.21.15.43:0/750967326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c02d4d0 0x56241c1883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.382+0000 7fdc9e200640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c191510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 -- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 msgr2=0x56241c191510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c191510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 -- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c02d4d0 msgr2=0x56241c1883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c02d4d0 0x56241c1883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 -- 172.21.15.43:0/750967326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241bf51c10 con 0x56241c0e0e40 2024-09-17T05:34:17.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9e200640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c191510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9f600640 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdc9801a920 tx=0x7fdc9801aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef75fdec9929df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9ce00640 1 -- 172.21.15.43:0/750967326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98004020 con 0x56241c0e0e40 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9ce00640 1 -- 172.21.15.43:0/750967326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc980042b0 con 0x56241c0e0e40 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdc9ce00640 1 -- 172.21.15.43:0/750967326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980bb6d0 con 0x56241c0e0e40 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- 172.21.15.43:0/750967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 msgr2=0x56241c191a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fdc9801a920 tx=0x7fdc9801aab0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- 172.21.15.43:0/750967326 shutdown_connections 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c0e0e40 0x56241c191a50 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c191510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 --2- 172.21.15.43:0/750967326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c02d4d0 0x56241c1883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- 172.21.15.43:0/750967326 >> 172.21.15.43:0/750967326 conn(0x56241c016d80 msgr2=0x56241c028890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- 172.21.15.43:0/750967326 shutdown_connections 2024-09-17T05:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- 172.21.15.43:0/750967326 wait complete. 2024-09-17T05:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 Processor -- start 2024-09-17T05:34:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.386+0000 7fdca598c080 1 -- start start 2024-09-17T05:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c0726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c0e0e40 0x56241c188b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56241c059b30 con 0x56241c02d4d0 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56241c073420 con 0x56241c0e0e40 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56241c0fb470 con 0x56241c059cb0 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36972/0 (socket says 172.21.15.43:36972) 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 learned_addr learned my addr 172.21.15.43:0/18283697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c0726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c0e0e40 0x56241c188b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 msgr2=0x56241c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56241c059cb0 0x56241c0726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c0e0e40 msgr2=0x56241c188b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c0e0e40 0x56241c188b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241c071a20 con 0x56241c02d4d0 2024-09-17T05:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56241c0e0e40 0x56241c188b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000ef20 tx=0x7fdc9000c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc97400640 1 -- 172.21.15.43:0/18283697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90016030 con 0x56241c02d4d0 2024-09-17T05:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc97400640 1 -- 172.21.15.43:0/18283697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc90001980 con 0x56241c02d4d0 2024-09-17T05:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdc97400640 1 -- 172.21.15.43:0/18283697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000a040 con 0x56241c02d4d0 2024-09-17T05:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56241c071cc0 con 0x56241c02d4d0 2024-09-17T05:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.390+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241c072200 con 0x56241c02d4d0 2024-09-17T05:34:17.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc97400640 1 -- 172.21.15.43:0/18283697 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc90019070 con 0x56241c02d4d0 2024-09-17T05:34:17.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc97400640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 0x7fdc800740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc97400640 1 -- 172.21.15.43:0/18283697 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7fdc90093ab0 con 0x56241c02d4d0 2024-09-17T05:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 0x7fdc800740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.397 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:34:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 0x56241c1906a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c187b40 con 0x56241c072be0 2024-09-17T05:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 0x56241c1906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 0x7fdc800740f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c004fd0 tx=0x7fdc8c004890 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.394+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 0x56241c1906a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.398+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v127'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.398+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c190be0 con 0x56241c072be0 2024-09-17T05:34:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.398+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v127'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.398+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a3ba0 con 0x56241c072be0 2024-09-17T05:34:17.404 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:34:17.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v127'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4f50 con 0x56241c072be0 2024-09-17T05:34:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a52f0 con 0x56241c072be0 2024-09-17T05:34:17.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.402+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a3f40 con 0x56241c072be0 2024-09-17T05:34:17.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.406+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.406+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a42e0 con 0x56241c072be0 2024-09-17T05:34:17.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.406+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v127'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.406+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v127'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.413 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:34:17.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v127'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.410+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.414+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.414+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.414+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.414+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.422 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:34:17.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.418+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.422+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.422+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.422+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.422+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4680 con 0x56241c072be0 2024-09-17T05:34:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4ab0 con 0x56241c072be0 2024-09-17T05:34:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v127'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.426+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.430+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v127'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.430+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.433 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:34:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.430+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v127'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.430+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.434+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v127'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v127'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.442 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:34:17.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v127'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.438+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c072be0 2024-09-17T05:34:17.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56241c1a5ba0 0x56241c1a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.442+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a4900 con 0x56241c1a5ba0 2024-09-17T05:34:17.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.446+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56241c1a5ba0 0x56241c1a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.446+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56241c1a5ba0 0x56241c1a7fc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.446+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'845 uv827 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.446+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a8500 con 0x56241c1a5ba0 2024-09-17T05:34:17.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.450+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'846 uv846 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.450+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a88a0 con 0x56241c1a5ba0 2024-09-17T05:34:17.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.450+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.454 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:34:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.450+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a90f0 con 0x56241c1a5ba0 2024-09-17T05:34:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9490 con 0x56241c1a5ba0 2024-09-17T05:34:17.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9830 con 0x56241c1a5ba0 2024-09-17T05:34:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v127'850 uv832 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.454+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a9bd0 con 0x56241c1a5ba0 2024-09-17T05:34:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.458+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v127'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.458+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.462 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:34:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.458+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v127'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.458+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v127'855 uv837 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.462+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.466+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v127'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.466+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.470 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:34:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.466+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v127'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.466+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae140 con 0x56241c1a5ba0 2024-09-17T05:34:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.470+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v127'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.470+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae140 con 0x56241c1a5ba0 2024-09-17T05:34:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.470+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.470+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v127'860 uv843 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v127'861 uv861 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v127'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.479 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:34:17.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.474+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.478+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.478+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.478+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.478+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1a9f70 con 0x56241c1a5ba0 2024-09-17T05:34:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.478+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1a5ba0 2024-09-17T05:34:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56241c1aaa00 0x56241c1aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ad3e0 con 0x56241c1aaa00 2024-09-17T05:34:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56241c1aaa00 0x56241c1aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56241c1aaa00 0x56241c1aae10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v127'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1ad780 con 0x56241c1aaa00 2024-09-17T05:34:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v127'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.482+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1adb60 con 0x56241c1aaa00 2024-09-17T05:34:17.488 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:34:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v127'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1af440 con 0x56241c1aaa00 2024-09-17T05:34:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1af7e0 con 0x56241c1aaa00 2024-09-17T05:34:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.486+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae430 con 0x56241c1aaa00 2024-09-17T05:34:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.490+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.490+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae850 con 0x56241c072be0 2024-09-17T05:34:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.490+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'164 uv163 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.490+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1ae850 con 0x56241c072be0 2024-09-17T05:34:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.494+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'165 uv165 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.494+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae850 con 0x56241c072be0 2024-09-17T05:34:17.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.494+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'166 uv166 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.498 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:34:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.494+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae850 con 0x56241c072be0 2024-09-17T05:34:17.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.494+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'167 uv167 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 0x56241c1b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b24e0 con 0x56241c1afb80 2024-09-17T05:34:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 0x56241c1b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdc9f600640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 0x56241c1b1fa0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v127'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b24e0 con 0x56241c1afb80 2024-09-17T05:34:17.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v127'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.498+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b24e0 con 0x56241c1afb80 2024-09-17T05:34:17.504 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:34:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.502+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v127'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.502+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b34f0 con 0x56241c1afb80 2024-09-17T05:34:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.502+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.502+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b34f0 con 0x56241c1afb80 2024-09-17T05:34:17.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.502+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b24e0 con 0x56241c1afb80 2024-09-17T05:34:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56241c1b29f0 0x56241c1b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5510 con 0x56241c1b29f0 2024-09-17T05:34:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56241c1b29f0 0x56241c1b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdc9ec00640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56241c1b29f0 0x56241c1b4e10 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v127'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.506+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v127'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.513 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:34:17.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v127'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b6940 con 0x56241c1b29f0 2024-09-17T05:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.510+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b6940 con 0x56241c1b29f0 2024-09-17T05:34:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v127'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.514+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v127'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.521 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:34:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v127'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.518+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b5930 con 0x56241c1b29f0 2024-09-17T05:34:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdc9ec00640 1 -- 172.21.15.43:0/18283697 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc9000d040 con 0x56241c1b29f0 2024-09-17T05:34:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56241c1b5fb0 0x56241c1b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b8a80 con 0x56241c1b5fb0 2024-09-17T05:34:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56241c1b5fb0 0x56241c1b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdc9e200640 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56241c1b5fb0 0x56241c1b83d0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v127'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.522+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b8e50 con 0x56241c1b5fb0 2024-09-17T05:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v127'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1b5fb0 2024-09-17T05:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v127'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.529 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:34:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1b5fb0 2024-09-17T05:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.526+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1b5fb0 2024-09-17T05:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c072be0 2024-09-17T05:34:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v127'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c072be0 2024-09-17T05:34:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v127'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.530+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c072be0 2024-09-17T05:34:17.536 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:34:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v127'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ba280 con 0x56241c072be0 2024-09-17T05:34:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ba280 con 0x56241c072be0 2024-09-17T05:34:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.534+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c072be0 2024-09-17T05:34:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.538+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c072be0 2024-09-17T05:34:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.538+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1aaa00 2024-09-17T05:34:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.538+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v127'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.538+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1aaa00 2024-09-17T05:34:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v127'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9270 con 0x56241c1aaa00 2024-09-17T05:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v127'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.545 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:34:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.542+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.546+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.546+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.546+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v127'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.546+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v127'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v127'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.553 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:34:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ae020 con 0x56241c1aaa00 2024-09-17T05:34:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.550+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1bc080 con 0x56241c1aaa00 2024-09-17T05:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.554+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1aaa00 2024-09-17T05:34:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.554+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.554+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v127'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.554+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v127'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.561 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:34:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.558+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.562+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v127'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.562+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.562+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v127'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.562+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.567 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:34:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v127'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.566+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1afb80 2024-09-17T05:34:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc98002050 con 0x56241c1afb80 2024-09-17T05:34:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v127'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v127'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.570+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v127'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.576 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:34:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.574+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.578+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v127'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.578+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.578+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v127'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.578+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v127'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.583 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.582+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v127'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v127'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v127'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.590 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:34:17.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.586+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ba730 con 0x56241c1b5fb0 2024-09-17T05:34:17.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1ba730 con 0x56241c1b5fb0 2024-09-17T05:34:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9720 con 0x56241c1b5fb0 2024-09-17T05:34:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.590+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x56241c1b9990 con 0x56241c1b5fb0 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdc9e200640 1 -- 172.21.15.43:0/18283697 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdc8c00ab80 con 0x56241c1b5fb0 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56241c1a5ba0 msgr2=0x56241c1a7fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x56241c1a5ba0 0x56241c1a7fc0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56241c1b29f0 msgr2=0x56241c1b4e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x56241c1b29f0 0x56241c1b4e10 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56241c1b5fb0 msgr2=0x56241c1b83d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x56241c1b5fb0 0x56241c1b83d0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56241c1aaa00 msgr2=0x56241c1aae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x56241c1aaa00 0x56241c1aae10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 reap_dead start 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 msgr2=0x56241c1906a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 0x56241c1906a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdc9f600640 1 -- 172.21.15.43:0/18283697 reap_dead start 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 msgr2=0x56241c1b1fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 0x56241c1b1fa0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 msgr2=0x7fdc800740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 0x7fdc800740f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c004fd0 tx=0x7fdc8c004890 comp rx=0 tx=0).stop 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 msgr2=0x56241c0f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000ef20 tx=0x7fdc9000c6f0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 shutdown_connections 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x56241c1afb80 0x56241c1b1fa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdc80071be0 0x7fdc800740f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x56241c072be0 0x56241c1906a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 --2- 172.21.15.43:0/18283697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56241c02d4d0 0x56241c0f80c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 >> 172.21.15.43:0/18283697 conn(0x56241c016d80 msgr2=0x56241c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 shutdown_connections 2024-09-17T05:34:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.594+0000 7fdca598c080 1 -- 172.21.15.43:0/18283697 wait complete. 2024-09-17T05:34:17.602 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:34:17.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.698+0000 7f2ffd847080 1 Processor -- start 2024-09-17T05:34:17.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.698+0000 7f2ffd847080 1 -- start start 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868099e0 0x55d186808f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1868784d0 0x55d186896f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d18670da00 con 0x55d1868099e0 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d18676bf40 con 0x55d1867bf360 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d18687ad10 con 0x55d1868784d0 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868099e0 0x55d186808f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:36558/0 (socket says 172.21.15.43:36558) 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 -- 172.21.15.43:0/4086343943 learned_addr learned my addr 172.21.15.43:0/4086343943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff6800640 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1868784d0 0x55d186896f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 -- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1868784d0 msgr2=0x55d186896f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1868784d0 0x55d186896f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 -- 172.21.15.43:0/4086343943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868099e0 msgr2=0x55d186808f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868099e0 0x55d186808f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 -- 172.21.15.43:0/4086343943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1866e6c10 con 0x55d1867bf360 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004f60 tx=0x7f2fe8018080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df54e95801f6f102 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2fefe00640 1 -- 172.21.15.43:0/4086343943 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80ab4b0 con 0x55d1867bf360 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2fefe00640 1 -- 172.21.15.43:0/4086343943 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe80ab980 con 0x55d1867bf360 2024-09-17T05:34:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2fefe00640 1 -- 172.21.15.43:0/4086343943 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80abb30 con 0x55d1867bf360 2024-09-17T05:34:17.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 msgr2=0x55d186875cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004f60 tx=0x7f2fe8018080 comp rx=0 tx=0).stop 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- 172.21.15.43:0/4086343943 shutdown_connections 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1868784d0 0x55d186896f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868099e0 0x55d186808f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 --2- 172.21.15.43:0/4086343943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1867bf360 0x55d186875cf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- 172.21.15.43:0/4086343943 >> 172.21.15.43:0/4086343943 conn(0x55d1867abd80 msgr2=0x55d1868778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- 172.21.15.43:0/4086343943 shutdown_connections 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- 172.21.15.43:0/4086343943 wait complete. 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 Processor -- start 2024-09-17T05:34:17.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.702+0000 7f2ffd847080 1 -- start start 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 0x55d18692f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 0x55d186897eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 0x55d186927040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d186876520 con 0x55d1868784d0 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d186808290 con 0x55d1868099e0 2024-09-17T05:34:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55d186896550 con 0x55d1867bf360 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 0x55d186927040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 0x55d186897eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 0x55d18692f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 0x55d18692f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48428/0 (socket says 172.21.15.43:48428) 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 0x55d186897eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:36564/0 (socket says 172.21.15.43:36564) 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 0x55d186927040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36988/0 (socket says 172.21.15.43:36988) 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5e00640 1 -- 172.21.15.43:0/2970875114 learned_addr learned my addr 172.21.15.43:0/2970875114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 -- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 msgr2=0x55d18692f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 0x55d18692f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 -- 172.21.15.43:0/2970875114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 msgr2=0x55d186927040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 0x55d186927040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 -- 172.21.15.43:0/2970875114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1868967f0 con 0x55d1868099e0 2024-09-17T05:34:17.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55d1867bf360 0x55d18692f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff6800640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d1868784d0 0x55d186927040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ff5400640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 0x55d186897eb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400a6c0 tx=0x7f2fe400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2feea00640 1 -- 172.21.15.43:0/2970875114 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401c070 con 0x55d1868099e0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2feea00640 1 -- 172.21.15.43:0/2970875114 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe4008db0 con 0x55d1868099e0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d186896a90 con 0x55d1868099e0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2feea00640 1 -- 172.21.15.43:0/2970875114 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40176c0 con 0x55d1868099e0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.706+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d186898780 con 0x55d1868099e0 2024-09-17T05:34:17.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2feea00640 1 -- 172.21.15.43:0/2970875114 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2fe401f070 con 0x55d1868099e0 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2feea00640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fd8071be0 0x7f2fd80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2feea00640 1 -- 172.21.15.43:0/2970875114 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7f2fe4093e90 con 0x55d1868099e0 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fd8071be0 0x7f2fd80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ffd847080 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d1868976e0 0x55d186874fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55d186927580 con 0x55d1868976e0 2024-09-17T05:34:17.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ff6800640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d1868976e0 0x55d186874fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ff5e00640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fd8071be0 0x7f2fd80740f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004240 tx=0x7f2fe8002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.710+0000 7f2ff6800640 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d1868976e0 0x55d186874fb0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ff6800640 1 -- 172.21.15.43:0/2970875114 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v127'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2ff0002a70 con 0x55d1868976e0 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d1868976e0 msgr2=0x55d186874fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55d1868976e0 0x55d186874fb0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fd8071be0 msgr2=0x7f2fd80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2fd8071be0 0x7f2fd80740f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004240 tx=0x7f2fe8002040 comp rx=0 tx=0).stop 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 msgr2=0x55d186897eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 --2- 172.21.15.43:0/2970875114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1868099e0 0x55d186897eb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400a6c0 tx=0x7f2fe400bf90 comp rx=0 tx=0).stop 2024-09-17T05:34:17.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ff6800640 1 -- 172.21.15.43:0/2970875114 reap_dead start 2024-09-17T05:34:17.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 shutdown_connections 2024-09-17T05:34:17.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 >> 172.21.15.43:0/2970875114 conn(0x55d1867abd80 msgr2=0x55d186877910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 shutdown_connections 2024-09-17T05:34:17.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.714+0000 7f2ffd847080 1 -- 172.21.15.43:0/2970875114 wait complete. 2024-09-17T05:34:17.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.878+0000 7fc4c7a00640 1 -- 172.21.15.43:0/2261600110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 msgr2=0x7fc4c00b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.878+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/2261600110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c00b3130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009840 tx=0x7fc4b401c820 comp rx=0 tx=0).stop 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- 172.21.15.43:0/2261600110 shutdown_connections 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/2261600110 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 0x7fc4c00cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/2261600110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c00b3130 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- 172.21.15.43:0/2261600110 >> 172.21.15.43:0/2261600110 conn(0x7fc4c00cee60 msgr2=0x7fc4c00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- 172.21.15.43:0/2261600110 shutdown_connections 2024-09-17T05:34:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- 172.21.15.43:0/2261600110 wait complete. 2024-09-17T05:34:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 Processor -- start 2024-09-17T05:34:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- start start 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 0x7fc4c0156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00b2ae0 con 0x7fc4c00d0650 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00b2960 con 0x7fc4c009ba60 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.882+0000 7fc4c7a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c0169e30 con 0x7fc4c00d0240 2024-09-17T05:34:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36996/0 (socket says 172.21.15.43:36996) 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4bfe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 0x7fc4c0156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:36582/0 (socket says 172.21.15.43:36582) 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5a00640 1 -- 172.21.15.43:0/604864424 learned_addr learned my addr 172.21.15.43:0/604864424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 -- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 msgr2=0x7fc4c0156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 0x7fc4c0156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 -- 172.21.15.43:0/604864424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 msgr2=0x7fc4c0157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 -- 172.21.15.43:0/604864424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c015b6e0 con 0x7fc4c009ba60 2024-09-17T05:34:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c5000640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40ac000 tx=0x7fc4b4005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40a8a90 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b40a8c30 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4005040 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c015b9c0 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c015bf80 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c00b3130 con 0x7fc4c009ba60 2024-09-17T05:34:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.886+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4b40c0060 con 0x7fc4c009ba60 2024-09-17T05:34:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.890+0000 7fc4bcc00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 0x7fc4a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.890+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8977+0+0 (secure 0 0 0) 0x7fc4b412f4f0 con 0x7fc4c009ba60 2024-09-17T05:34:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.890+0000 7fc4bfe00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 0x7fc4a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.890+0000 7fc4bfe00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 0x7fc4a0075740 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0071650 tx=0x7fc4ac0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:17.894+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4b40ac030 con 0x7fc4c009ba60 2024-09-17T05:34:18.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.102+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc4c0071960 con 0x7fc4c009ba60 2024-09-17T05:34:18.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.266+0000 7fc4bcc00640 1 -- 172.21.15.43:0/604864424 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v326) ==== 162+0+0 (secure 0 0 0) 0x7fc4b4016410 con 0x7fc4c009ba60 2024-09-17T05:34:18.270 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:34:18.271 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:34:18.270+0000 7f01c7c00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 msgr2=0x7fc4a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 0x7fc4a0075740 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0071650 tx=0x7fc4ac0091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 msgr2=0x7fc4c0152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40ac000 tx=0x7fc4b4005db0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 shutdown_connections 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc4a0073230 0x7fc4a0075740 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c00d0650 0x7fc4c0157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.273 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc4c00d0240 0x7fc4c0156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 --2- 172.21.15.43:0/604864424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c009ba60 0x7fc4c0152390 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 >> 172.21.15.43:0/604864424 conn(0x7fc4c00cee60 msgr2=0x7fc4c0071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.274 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:34:18.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 shutdown_connections 2024-09-17T05:34:18.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.270+0000 7fc4c7a00640 1 -- 172.21.15.43:0/604864424 wait complete. 2024-09-17T05:34:18.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.450+0000 7f3ef7600640 1 -- 172.21.15.43:0/917212293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 msgr2=0x7f3ef00cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.450+0000 7f3ef7600640 1 --2- 172.21.15.43:0/917212293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef00cd040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009900 tx=0x7f3ed801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 -- 172.21.15.43:0/917212293 shutdown_connections 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 --2- 172.21.15.43:0/917212293 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 0x7f3ef00cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 --2- 172.21.15.43:0/917212293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef00cd040 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 -- 172.21.15.43:0/917212293 >> 172.21.15.43:0/917212293 conn(0x7f3ef00cee60 msgr2=0x7f3ef00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 -- 172.21.15.43:0/917212293 shutdown_connections 2024-09-17T05:34:18.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 -- 172.21.15.43:0/917212293 wait complete. 2024-09-17T05:34:18.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 Processor -- start 2024-09-17T05:34:18.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 -- start start 2024-09-17T05:34:18.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.454+0000 7f3ef7600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 0x7f3ef01f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef00b2b80 con 0x7f3ef009ba60 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef00b2a00 con 0x7f3ef00d0240 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef0169e30 con 0x7f3ef00d0650 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37008/0 (socket says 172.21.15.43:37008) 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 -- 172.21.15.43:0/1422277514 learned_addr learned my addr 172.21.15.43:0/1422277514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ee7e00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef5600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 0x7f3ef01f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 -- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 msgr2=0x7f3ef01f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 0x7f3ef01f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 -- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 msgr2=0x7f3ef015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 -- 172.21.15.43:0/1422277514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef01662e0 con 0x7f3ef009ba60 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef4c00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c3f0 tx=0x7f3ee000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00052d0 con 0x7f3ef009ba60 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ee0007020 con 0x7f3ef009ba60 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0007470 con 0x7f3ef009ba60 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef01665c0 con 0x7f3ef009ba60 2024-09-17T05:34:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01f89d0 con 0x7f3ef009ba60 2024-09-17T05:34:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef00b3130 con 0x7f3ef009ba60 2024-09-17T05:34:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.458+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ee00067e0 con 0x7f3ef009ba60 2024-09-17T05:34:18.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.462+0000 7f3eed600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 0x7f3ecc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.462+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7f3ee0094480 con 0x7f3ef009ba60 2024-09-17T05:34:18.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.462+0000 7f3ee7e00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 0x7f3ecc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.462+0000 7f3ee7e00640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 0x7f3ecc075740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed80a8040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.466+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ee0001090 con 0x7f3ef009ba60 2024-09-17T05:34:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.674+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ef0073090 con 0x7f3ef009ba60 2024-09-17T05:34:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.674+0000 7f3eed600640 1 -- 172.21.15.43:0/1422277514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 327 v327) ==== 95+0+4383 (secure 0 0 0) 0x7f3ee005e000 con 0x7f3ef009ba60 2024-09-17T05:34:18.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:18.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":327,"btime":"2024-09-17T05:34:18:273267+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:18.273173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 327 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 msgr2=0x7f3ecc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 0x7f3ecc075740 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed80a8040 comp rx=0 tx=0).stop 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 msgr2=0x7f3ef01fc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c3f0 tx=0x7f3ee000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 shutdown_connections 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ecc073230 0x7f3ecc075740 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef00d0240 0x7f3ef015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ef00d0650 0x7f3ef01f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 --2- 172.21.15.43:0/1422277514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef009ba60 0x7f3ef01fc8c0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 >> 172.21.15.43:0/1422277514 conn(0x7f3ef00cee60 msgr2=0x7f3ef006df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 shutdown_connections 2024-09-17T05:34:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.678+0000 7f3ef7600640 1 -- 172.21.15.43:0/1422277514 wait complete. 2024-09-17T05:34:18.701 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:34:18.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 -- 172.21.15.43:0/765863469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad740b6840 msgr2=0x7fad5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 --2- 172.21.15.43:0/765863469 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad740b6840 0x7fad5c004810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fad7409f4b0 tx=0x7fad6001d210 comp rx=0 tx=0).stop 2024-09-17T05:34:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad79e00640 1 -- 172.21.15.43:0/765863469 reap_dead start 2024-09-17T05:34:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 -- 172.21.15.43:0/765863469 shutdown_connections 2024-09-17T05:34:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 -- 172.21.15.43:0/765863469 >> 172.21.15.43:0/765863469 conn(0x7fad740b2e80 msgr2=0x7fad740b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 -- 172.21.15.43:0/765863469 shutdown_connections 2024-09-17T05:34:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 -- 172.21.15.43:0/765863469 wait complete. 2024-09-17T05:34:18.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.858+0000 7fad7be00640 1 Processor -- start 2024-09-17T05:34:18.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- start start 2024-09-17T05:34:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 0x7fad741f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad741f84e0 0x7fad741fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad74166ca0 con 0x7fad740b6840 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad74166b20 con 0x7fad74159950 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fad74166e20 con 0x7fad741f84e0 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37026/0 (socket says 172.21.15.43:37026) 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 -- 172.21.15.43:0/4188010403 learned_addr learned my addr 172.21.15.43:0/4188010403 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad78a00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 0x7fad741f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79e00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad741f84e0 0x7fad741fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 -- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad741f84e0 msgr2=0x7fad741fa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad741f84e0 0x7fad741fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 -- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 msgr2=0x7fad741f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 0x7fad741f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 -- 172.21.15.43:0/4188010403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7415a080 con 0x7fad740b6840 2024-09-17T05:34:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad78a00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 0x7fad741f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad79400640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fad68006600 tx=0x7fad680039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68015070 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad68011920 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7415a360 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68011d70 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7415a920 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.862+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad740cd8f0 con 0x7fad740b6840 2024-09-17T05:34:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.866+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad68018070 con 0x7fad740b6840 2024-09-17T05:34:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.866+0000 7fad71600640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 0x7fad54075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.866+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7fad68095dd0 con 0x7fad740b6840 2024-09-17T05:34:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.866+0000 7fad78a00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 0x7fad54075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.866+0000 7fad78a00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 0x7fad54075740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fad600ae000 tx=0x7fad60019fb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:18.870+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad6805ffa0 con 0x7fad740b6840 2024-09-17T05:34:19.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.078+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad740cb600 con 0x7fad740b6840 2024-09-17T05:34:19.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.078+0000 7fad71600640 1 -- 172.21.15.43:0/4188010403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 327 v327) ==== 95+0+4383 (secure 0 0 0) 0x7fad6805f8a0 con 0x7fad740b6840 2024-09-17T05:34:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:19.083 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":327,"btime":"2024-09-17T05:34:18:273267+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:18.273173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:19.084 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 327 2024-09-17T05:34:19.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 msgr2=0x7fad54075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 0x7fad54075740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fad600ae000 tx=0x7fad60019fb0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 msgr2=0x7fad740ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fad68006600 tx=0x7fad680039d0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 shutdown_connections 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad54073230 0x7fad54075740 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad74159950 0x7fad741f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad741f84e0 0x7fad741fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 --2- 172.21.15.43:0/4188010403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740b6840 0x7fad740ca860 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 >> 172.21.15.43:0/4188010403 conn(0x7fad740b2e80 msgr2=0x7fad74165f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 shutdown_connections 2024-09-17T05:34:19.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.082+0000 7fad7be00640 1 -- 172.21.15.43:0/4188010403 wait complete. 2024-09-17T05:34:19.106 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:34:19.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- 172.21.15.43:0/915425345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c0b3920 msgr2=0x7ff49c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/915425345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c0b3920 0x7ff49c0ad0b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0711f0 tx=0x7ff49401ccc0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- 172.21.15.43:0/915425345 shutdown_connections 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/915425345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c0b3920 0x7ff49c0ad0b0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/915425345 >> v2:172.21.15.62:3300/0 conn(0x7ff49c06f110 0x7ff49c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/915425345 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c155d70 0x7ff49c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- 172.21.15.43:0/915425345 >> 172.21.15.43:0/915425345 conn(0x7ff49c0b23d0 msgr2=0x7ff49c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- 172.21.15.43:0/915425345 shutdown_connections 2024-09-17T05:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- 172.21.15.43:0/915425345 wait complete. 2024-09-17T05:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 Processor -- start 2024-09-17T05:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.262+0000 7ff4a3e00640 1 -- start start 2024-09-17T05:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 0x7ff49c096a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c0b3920 0x7ff49c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c15f8d0 con 0x7ff49c155d70 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c15f750 con 0x7ff49c06f110 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c15fa50 con 0x7ff49c0b3920 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37058/0 (socket says 172.21.15.43:37058) 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 -- 172.21.15.43:0/4195703205 learned_addr learned my addr 172.21.15.43:0/4195703205 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1400640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 0x7ff49c096a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a0a00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c0b3920 0x7ff49c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 -- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c0b3920 msgr2=0x7ff49c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c0b3920 0x7ff49c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 -- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 msgr2=0x7ff49c096a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 0x7ff49c096a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 -- 172.21.15.43:0/4195703205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c096fd0 con 0x7ff49c155d70 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1400640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 0x7ff49c096a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a1e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff49401b370 tx=0x7ff494005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940bb070 con 0x7ff49c155d70 2024-09-17T05:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff494004d40 con 0x7ff49c155d70 2024-09-17T05:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940b2040 con 0x7ff49c155d70 2024-09-17T05:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c0974c0 con 0x7ff49c155d70 2024-09-17T05:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c097a80 con 0x7ff49c155d70 2024-09-17T05:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.266+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c15ff20 con 0x7ff49c155d70 2024-09-17T05:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.270+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff4940be070 con 0x7ff49c155d70 2024-09-17T05:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.270+0000 7ff499600640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 0x7ff47c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.270+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7ff49412ed40 con 0x7ff49c155d70 2024-09-17T05:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.270+0000 7ff4a1400640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 0x7ff47c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.270+0000 7ff4a1400640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 0x7ff47c075740 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff490005d90 tx=0x7ff490005d20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.274+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff494016970 con 0x7ff49c155d70 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.482+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff49c0751b0 con 0x7ff49c155d70 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.482+0000 7ff499600640 1 -- 172.21.15.43:0/4195703205 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 327 v327) ==== 95+0+4383 (secure 0 0 0) 0x7ff4940f8f90 con 0x7ff49c155d70 2024-09-17T05:34:19.487 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:19.487 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":327,"btime":"2024-09-17T05:34:18:273267+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9079,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":276}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:18.273173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:19.487 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 327 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 msgr2=0x7ff47c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 0x7ff47c075740 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff490005d90 tx=0x7ff490005d20 comp rx=0 tx=0).stop 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 msgr2=0x7ff49c1fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff49401b370 tx=0x7ff494005b00 comp rx=0 tx=0).stop 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 shutdown_connections 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff47c073230 0x7ff47c075740 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49c155d70 0x7ff49c1fa010 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff49c0b3920 0x7ff49c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 --2- 172.21.15.43:0/4195703205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff49c06f110 0x7ff49c096a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 >> 172.21.15.43:0/4195703205 conn(0x7ff49c0b23d0 msgr2=0x7ff49c09d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 shutdown_connections 2024-09-17T05:34:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.486+0000 7ff4a3e00640 1 -- 172.21.15.43:0/4195703205 wait complete. 2024-09-17T05:34:19.509 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-09-17T05:34:19.509 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-09-17T05:34:19.510 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:19.510 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:19.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:19.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:34:19.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:19.519 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:34:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T05:34:19.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:34:19.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.770+0000 7f12c6e00640 1 -- 172.21.15.43:0/162388874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 msgr2=0x7f12c015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.770+0000 7f12c6e00640 1 --2- 172.21.15.43:0/162388874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015b310 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009840 tx=0x7f12b401c790 comp rx=0 tx=0).stop 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.770+0000 7f12c6e00640 1 -- 172.21.15.43:0/162388874 shutdown_connections 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.770+0000 7f12c6e00640 1 --2- 172.21.15.43:0/162388874 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c00ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.770+0000 7f12c6e00640 1 --2- 172.21.15.43:0/162388874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015b310 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- 172.21.15.43:0/162388874 >> 172.21.15.43:0/162388874 conn(0x7f12c00d0010 msgr2=0x7f12c00d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- 172.21.15.43:0/162388874 shutdown_connections 2024-09-17T05:34:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- 172.21.15.43:0/162388874 wait complete. 2024-09-17T05:34:19.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 Processor -- start 2024-09-17T05:34:19.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- start start 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c006fe40 0x7f12c01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c014db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0168f80 con 0x7f12c00b2f60 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0168e00 con 0x7f12c006fe40 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.774+0000 7f12c6e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f12c0169100 con 0x7f12c00adf80 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37076/0 (socket says 172.21.15.43:37076) 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 -- 172.21.15.43:0/4136110891 learned_addr learned my addr 172.21.15.43:0/4136110891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c5a00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c006fe40 0x7f12c01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c5000640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c014db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 -- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 msgr2=0x7f12c014db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c014db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 -- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c006fe40 msgr2=0x7f12c01524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c006fe40 0x7f12c01524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 -- 172.21.15.43:0/4136110891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c014e0a0 con 0x7f12c00b2f60 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c5000640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c014db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6400640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f12b800cb40 tx=0x7f12b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8016020 con 0x7f12c00b2f60 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b800ab60 con 0x7f12c00b2f60 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8012420 con 0x7f12c00b2f60 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c015a2c0 con 0x7f12c00b2f60 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c015cee0 con 0x7f12c00b2f60 2024-09-17T05:34:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c00ae390 con 0x7f12c00b2f60 2024-09-17T05:34:19.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.778+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f12b8019020 con 0x7f12c00b2f60 2024-09-17T05:34:19.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.782+0000 7f12bd600640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 0x7f12a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.782+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7f12b8095160 con 0x7f12c00b2f60 2024-09-17T05:34:19.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.782+0000 7f12c5a00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 0x7f12a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.782+0000 7f12c5a00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 0x7f12a0075740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f12b40ad000 tx=0x7f12b401afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.786+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12b8020020 con 0x7f12c00b2f60 2024-09-17T05:34:19.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:19.994+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f12c0071030 con 0x7f12c00b2f60 2024-09-17T05:34:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.314+0000 7f12bd600640 1 -- 172.21.15.43:0/4136110891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v328) ==== 161+0+0 (secure 0 0 0) 0x7f12b805f3b0 con 0x7f12c00b2f60 2024-09-17T05:34:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:34:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 msgr2=0x7f12a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 0x7f12a0075740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f12b40ad000 tx=0x7f12b401afd0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 msgr2=0x7f12c015d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f12b800cb40 tx=0x7f12b800cc30 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 shutdown_connections 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073230 0x7f12a0075740 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c00b2f60 0x7f12c015d9c0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c00adf80 0x7f12c014db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 --2- 172.21.15.43:0/4136110891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c006fe40 0x7f12c01524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 >> 172.21.15.43:0/4136110891 conn(0x7f12c00d0010 msgr2=0x7f12c00afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 shutdown_connections 2024-09-17T05:34:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.318+0000 7f12c6e00640 1 -- 172.21.15.43:0/4136110891 wait complete. 2024-09-17T05:34:20.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-17T05:34:20.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 -- 172.21.15.43:0/3886382542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f70004f20 msgr2=0x7f5f7000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 --2- 172.21.15.43:0/3886382542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f70004f20 0x7f5f7000b9d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009e90 tx=0x7f5f8001d5d0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f8e000640 1 -- 172.21.15.43:0/3886382542 reap_dead start 2024-09-17T05:34:20.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 -- 172.21.15.43:0/3886382542 shutdown_connections 2024-09-17T05:34:20.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 -- 172.21.15.43:0/3886382542 >> 172.21.15.43:0/3886382542 conn(0x7f5f880b2810 msgr2=0x7f5f880b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 -- 172.21.15.43:0/3886382542 shutdown_connections 2024-09-17T05:34:20.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.502+0000 7f5f90000640 1 -- 172.21.15.43:0/3886382542 wait complete. 2024-09-17T05:34:20.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 Processor -- start 2024-09-17T05:34:20.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- start start 2024-09-17T05:34:20.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f70004f20 0x7f5f881fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f88151c90 0x7f5f881520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88167990 con 0x7f5f881f6dc0 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88167810 con 0x7f5f88151c90 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f88167b10 con 0x7f5f70004f20 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37094/0 (socket says 172.21.15.43:37094) 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 -- 172.21.15.43:0/2716586210 learned_addr learned my addr 172.21.15.43:0/2716586210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8d600640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f70004f20 0x7f5f881fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8e000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f88151c90 0x7f5f881520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 -- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f70004f20 msgr2=0x7f5f881fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f70004f20 0x7f5f881fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 -- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f88151c90 msgr2=0x7f5f881520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f88151c90 0x7f5f881520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 -- 172.21.15.43:0/2716586210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881525e0 con 0x7f5f881f6dc0 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f8cc00640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f5f740014b0 tx=0x7f5f740039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74015070 con 0x7f5f881f6dc0 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f74011860 con 0x7f5f881f6dc0 2024-09-17T05:34:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f88159360 con 0x7f5f881f6dc0 2024-09-17T05:34:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74011cb0 con 0x7f5f881f6dc0 2024-09-17T05:34:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.506+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f88159080 con 0x7f5f881f6dc0 2024-09-17T05:34:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8806fce0 con 0x7f5f881f6dc0 2024-09-17T05:34:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f74018070 con 0x7f5f881f6dc0 2024-09-17T05:34:20.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f85600640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 0x7f5f68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7f5f74095b70 con 0x7f5f881f6dc0 2024-09-17T05:34:20.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f8d600640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 0x7f5f68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.510+0000 7f5f8d600640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 0x7f5f68075710 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c10 tx=0x7f5f7c006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.514+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5f881525e0 con 0x7f5f881f6dc0 2024-09-17T05:34:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:20.758+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5f88158df0 con 0x7f5f881f6dc0 2024-09-17T05:34:21.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.322+0000 7f5f85600640 1 -- 172.21.15.43:0/2716586210 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v329) ==== 101+0+0 (secure 0 0 0) 0x7f5f7405f890 con 0x7f5f881f6dc0 2024-09-17T05:34:21.326 INFO:teuthology.orchestra.run.smithi043.stderr:repaired: restoring rank 6:0 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 msgr2=0x7f5f68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 0x7f5f68075710 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c10 tx=0x7f5f7c006b20 comp rx=0 tx=0).stop 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 msgr2=0x7f5f8813e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f5f740014b0 tx=0x7f5f740039d0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 shutdown_connections 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5f68073200 0x7f5f68075710 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f88151c90 0x7f5f881520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f881f6dc0 0x7f5f8813e510 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 --2- 172.21.15.43:0/2716586210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f70004f20 0x7f5f881fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 >> 172.21.15.43:0/2716586210 conn(0x7f5f880b2810 msgr2=0x7f5f8814e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 shutdown_connections 2024-09-17T05:34:21.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.326+0000 7f5f90000640 1 -- 172.21.15.43:0/2716586210 wait complete. 2024-09-17T05:34:21.351 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-17T05:34:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 Processor -- start 2024-09-17T05:34:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 -- start start 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f1eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f157b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e91f0c1f40 con 0x7fb304002350 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e91f0ced00 con 0x55e91f157730 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb32392d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e91f063a00 con 0x55e91f1ea620 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37110/0 (socket says 172.21.15.43:37110) 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 -- 172.21.15.43:0/1271474913 learned_addr learned my addr 172.21.15.43:0/1271474913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb317e00640 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f1eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31d400640 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 -- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 msgr2=0x55e91f1eb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f1eb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 -- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 msgr2=0x55e91f157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 -- 172.21.15.43:0/1271474913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91f1e9700 con 0x7fb304002350 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb31ca00640 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb31001bdf0 tx=0x7fb304004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0e586873646cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb316a00640 1 -- 172.21.15.43:0/1271474913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310001500 con 0x7fb304002350 2024-09-17T05:34:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb316a00640 1 -- 172.21.15.43:0/1271474913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3100016a0 con 0x7fb304002350 2024-09-17T05:34:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.410+0000 7fb316a00640 1 -- 172.21.15.43:0/1271474913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3100a4960 con 0x7fb304002350 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- 172.21.15.43:0/1271474913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 msgr2=0x55e91f1eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb31001bdf0 tx=0x7fb304004130 comp rx=0 tx=0).stop 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- 172.21.15.43:0/1271474913 shutdown_connections 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f1eb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- 172.21.15.43:0/1271474913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb304002350 0x55e91f1eb360 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- 172.21.15.43:0/1271474913 >> 172.21.15.43:0/1271474913 conn(0x55e91f101d80 msgr2=0x55e91f154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- 172.21.15.43:0/1271474913 shutdown_connections 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- 172.21.15.43:0/1271474913 wait complete. 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 Processor -- start 2024-09-17T05:34:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- start start 2024-09-17T05:34:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f1e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 0x55e91f158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e91f15e4d0 con 0x55e91f286310 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e91f15e230 con 0x55e91f157730 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb32392d080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e91f1e8fc0 con 0x55e91f1ea620 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb31d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 0x55e91f158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb31ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f1e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb31d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 0x55e91f158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37114/0 (socket says 172.21.15.43:37114) 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 learned_addr learned my addr 172.21.15.43:0/3317135359 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.414+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 msgr2=0x55e91f157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e91f1ea620 0x55e91f157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 msgr2=0x55e91f1e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e91f157730 0x55e91f1e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91f13d5d0 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb31d400640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 0x55e91f158080 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x55e91f119870 tx=0x7fb31800f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb315600640 1 -- 172.21.15.43:0/3317135359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180137e0 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb315600640 1 -- 172.21.15.43:0/3317135359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb318019070 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e91f2856a0 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb315600640 1 -- 172.21.15.43:0/3317135359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318013be0 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91f285930 con 0x55e91f286310 2024-09-17T05:34:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb315600640 1 -- 172.21.15.43:0/3317135359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb318013d80 con 0x55e91f286310 2024-09-17T05:34:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.418+0000 7fb315600640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2fc071be0 0x7fb2fc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb315600640 1 -- 172.21.15.43:0/3317135359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7fb318093dd0 con 0x55e91f286310 2024-09-17T05:34:21.423 INFO:teuthology.orchestra.run.smithi043.stdout:Importing pool 2024-09-17T05:34:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2fc071be0 0x7fb2fc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e91f115360 0x55e91f115770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f115cb0 con 0x55e91f115360 2024-09-17T05:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e91f115360 0x55e91f115770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2fc071be0 0x7fb2fc0740f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b6000 tx=0x7fb3100092b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.422+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e91f115360 0x55e91f115770 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.426+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(1 601.00000000 [create] v128'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.426+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f291e30 con 0x55e91f115360 2024-09-17T05:34:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.426+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(2 601.00000000 [delete] v128'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.426+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2921d0 con 0x55e91f115360 2024-09-17T05:34:21.431 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-17T05:34:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(3 601.00000000 [create] v128'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f293580 con 0x55e91f115360 2024-09-17T05:34:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f293920 con 0x55e91f115360 2024-09-17T05:34:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.430+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292570 con 0x55e91f115360 2024-09-17T05:34:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.434+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.434+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292960 con 0x55e91f115360 2024-09-17T05:34:21.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.434+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 7 ==== osd_op_reply(7 602.00000000 [create] v128'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.434+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f292d50 con 0x55e91f115360 2024-09-17T05:34:21.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 8 ==== osd_op_reply(8 602.00000000 [delete] v128'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d50 con 0x55e91f115360 2024-09-17T05:34:21.441 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-17T05:34:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 9 ==== osd_op_reply(9 602.00000000 [create] v128'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.438+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.442+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.442+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.442+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.442+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.451 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-17T05:34:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.446+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.450+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.450+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.450+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.450+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292d00 con 0x55e91f115360 2024-09-17T05:34:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f293180 con 0x55e91f115360 2024-09-17T05:34:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 20 ==== osd_op_reply(20 600.00000000 [create] v128'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.454+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 21 ==== osd_op_reply(21 600.00000000 [delete] v128'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.461 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-17T05:34:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 22 ==== osd_op_reply(22 600.00000000 [create] v128'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.458+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.462+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.462+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.462+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.462+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 26 ==== osd_op_reply(26 603.00000000 [create] v128'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 27 ==== osd_op_reply(27 603.00000000 [delete] v128'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 28 ==== osd_op_reply(28 603.00000000 [create] v128'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.470 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-17T05:34:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.466+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f115360 2024-09-17T05:34:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55e91f2941d0 0x55e91f2965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.470+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f292f80 con 0x55e91f2941d0 2024-09-17T05:34:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.474+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55e91f2941d0 0x55e91f2965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.474+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55e91f2941d0 0x55e91f2965f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.474+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'866 uv849 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.474+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f296b30 con 0x55e91f2941d0 2024-09-17T05:34:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.478+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'867 uv867 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.478+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f296ed0 con 0x55e91f2941d0 2024-09-17T05:34:21.481 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-17T05:34:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.478+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.478+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f297720 con 0x55e91f2941d0 2024-09-17T05:34:21.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f297ac0 con 0x55e91f2941d0 2024-09-17T05:34:21.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f297e60 con 0x55e91f2941d0 2024-09-17T05:34:21.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 6 ==== osd_op_reply(37 200.00000000 [create] v128'871 uv854 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.482+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f298200 con 0x55e91f2941d0 2024-09-17T05:34:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.486+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 7 ==== osd_op_reply(38 200.00000000 [delete] v128'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.486+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.486+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 8 ==== osd_op_reply(39 200.00000000 [create] v128'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.490 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-17T05:34:21.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.486+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.490+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.490+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.490+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.490+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 11 ==== osd_op_reply(42 200.00000001 [create] v128'876 uv859 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 12 ==== osd_op_reply(43 200.00000001 [delete] v128'877 uv877 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.498 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-17T05:34:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 13 ==== osd_op_reply(44 200.00000001 [create] v128'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.494+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29c770 con 0x55e91f2941d0 2024-09-17T05:34:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.498+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v128'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.498+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29c770 con 0x55e91f2941d0 2024-09-17T05:34:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.498+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.498+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 16 ==== osd_op_reply(47 606.00000000 [create] v128'881 uv865 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 17 ==== osd_op_reply(48 606.00000000 [delete] v128'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 18 ==== osd_op_reply(49 606.00000000 [create] v128'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.507 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-17T05:34:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.502+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2995b0 con 0x55e91f2941d0 2024-09-17T05:34:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2995b0 con 0x55e91f2941d0 2024-09-17T05:34:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2985a0 con 0x55e91f2941d0 2024-09-17T05:34:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.0 v2:172.21.15.43:6800/4272274264 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2941d0 2024-09-17T05:34:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55e91f298e70 0x55e91f299280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.506+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f298e70 2024-09-17T05:34:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55e91f298e70 0x55e91f299280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55e91f298e70 0x55e91f299280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(53 607.00000000 [create] v128'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f298e70 2024-09-17T05:34:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== osd_op_reply(54 607.00000000 [delete] v128'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.510+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f298e70 2024-09-17T05:34:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 3 ==== osd_op_reply(55 607.00000000 [create] v128'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.516 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-17T05:34:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29c9b0 con 0x55e91f298e70 2024-09-17T05:34:21.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29c9b0 con 0x55e91f298e70 2024-09-17T05:34:21.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.514+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f298e70 2024-09-17T05:34:21.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f115360 2024-09-17T05:34:21.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'168 uv167 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f115360 2024-09-17T05:34:21.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.518+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29b9a0 con 0x55e91f115360 2024-09-17T05:34:21.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'170 uv170 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.525 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-17T05:34:21.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29bd70 con 0x55e91f115360 2024-09-17T05:34:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'171 uv171 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55e91f29c6b0 0x55e91f29edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29bd70 con 0x55e91f29c6b0 2024-09-17T05:34:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb31d400640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55e91f29c6b0 0x55e91f29edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.522+0000 7fb31d400640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55e91f29c6b0 0x55e91f29edd0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.526+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(63 608.00000000 [create] v128'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.526+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f29f310 con 0x55e91f29c6b0 2024-09-17T05:34:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.526+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== osd_op_reply(64 608.00000000 [delete] v128'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.526+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29f6b0 con 0x55e91f29c6b0 2024-09-17T05:34:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 3 ==== osd_op_reply(65 608.00000000 [create] v128'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.532 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-17T05:34:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a0b00 con 0x55e91f29c6b0 2024-09-17T05:34:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a0f80 con 0x55e91f29c6b0 2024-09-17T05:34:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.530+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f29fa50 con 0x55e91f29c6b0 2024-09-17T05:34:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55e91f2a04c0 0x55e91f2a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3900 con 0x55e91f2a04c0 2024-09-17T05:34:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55e91f2a04c0 0x55e91f2a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb31ca00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55e91f2a04c0 0x55e91f2a3380 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(69 604.00000000 [create] v128'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.534+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(70 604.00000000 [delete] v128'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.541 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-17T05:34:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(71 604.00000000 [create] v128'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a4ce0 con 0x55e91f2a04c0 2024-09-17T05:34:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.538+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a4ce0 con 0x55e91f2a04c0 2024-09-17T05:34:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 7 ==== osd_op_reply(75 500.00000000 [create] v128'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.542+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 8 ==== osd_op_reply(76 500.00000000 [delete] v128'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.549 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-17T05:34:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 9 ==== osd_op_reply(77 500.00000000 [create] v128'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.546+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a3cd0 con 0x55e91f2a04c0 2024-09-17T05:34:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb31ca00640 1 -- 172.21.15.43:0/3317135359 <== osd.1 v2:172.21.15.43:6816/1939991414 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3100a47c0 con 0x55e91f2a04c0 2024-09-17T05:34:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e91f2a4350 0x55e91f2a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a6f10 con 0x55e91f2a4350 2024-09-17T05:34:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e91f2a4350 0x55e91f2a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb317e00640 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e91f2a4350 0x55e91f2a67f0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.550+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(80 mds_snaptable [create] v128'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.554+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a6f10 con 0x55e91f2a4350 2024-09-17T05:34:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.554+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== osd_op_reply(81 mds_snaptable [delete] v128'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.554+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f2a4350 2024-09-17T05:34:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.554+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 3 ==== osd_op_reply(82 mds_snaptable [create] v128'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.558 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-17T05:34:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.554+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f2a4350 2024-09-17T05:34:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f2a4350 2024-09-17T05:34:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f115360 2024-09-17T05:34:21.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 36 ==== osd_op_reply(85 605.00000000 [create] v128'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.558+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f115360 2024-09-17T05:34:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 37 ==== osd_op_reply(86 605.00000000 [delete] v128'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f115360 2024-09-17T05:34:21.565 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-17T05:34:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 38 ==== osd_op_reply(87 605.00000000 [create] v128'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a8310 con 0x55e91f115360 2024-09-17T05:34:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.562+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a8310 con 0x55e91f115360 2024-09-17T05:34:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f115360 2024-09-17T05:34:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.6 v2:172.21.15.62:6802/2388352512 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f115360 2024-09-17T05:34:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f298e70 2024-09-17T05:34:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 7 ==== osd_op_reply(91 mds0_inotable [create] v128'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.566+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f298e70 2024-09-17T05:34:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.570+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 8 ==== osd_op_reply(92 mds0_inotable [delete] v128'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.570+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7300 con 0x55e91f298e70 2024-09-17T05:34:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.570+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 9 ==== osd_op_reply(93 mds0_inotable [create] v128'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.574 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-17T05:34:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.570+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 12 ==== osd_op_reply(96 100.00000000 [create] v128'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.574+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.578+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 13 ==== osd_op_reply(97 100.00000000 [delete] v128'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.578+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.578+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 14 ==== osd_op_reply(98 100.00000000 [create] v128'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.582 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-17T05:34:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.578+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a73a0 con 0x55e91f298e70 2024-09-17T05:34:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2aa1f0 con 0x55e91f298e70 2024-09-17T05:34:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.5 v2:172.21.15.62:6826/3886236614 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f298e70 2024-09-17T05:34:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7850 con 0x55e91f29c6b0 2024-09-17T05:34:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 7 ==== osd_op_reply(101 mds0_sessionmap [create] v128'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.582+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.586+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.586+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.586+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 9 ==== osd_op_reply(103 mds0_sessionmap [create] v128'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.590 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-17T05:34:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.586+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.590+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.590+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.590+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 11 ==== osd_op_reply(105 609.00000000 [create] v128'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.590+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 12 ==== osd_op_reply(106 609.00000000 [delete] v128'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 13 ==== osd_op_reply(107 609.00000000 [create] v128'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.597 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-17T05:34:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a8c00 con 0x55e91f29c6b0 2024-09-17T05:34:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.594+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a8c00 con 0x55e91f29c6b0 2024-09-17T05:34:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f29c6b0 2024-09-17T05:34:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 <== osd.7 v2:172.21.15.62:6810/1426001971 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb318059dc0 con 0x55e91f29c6b0 2024-09-17T05:34:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 6 ==== osd_op_reply(111 400.00000000 [create] v128'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.598+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 7 ==== osd_op_reply(112 400.00000000 [delete] v128'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 8 ==== osd_op_reply(113 400.00000000 [create] v128'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.606 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-17T05:34:21.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.602+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.606+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.606+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.606+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 11 ==== osd_op_reply(116 100.00000000.inode [create] v128'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.606+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 12 ==== osd_op_reply(117 100.00000000.inode [delete] v128'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 13 ==== osd_op_reply(118 100.00000000.inode [create] v128'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-17T05:34:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.610+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 16 ==== osd_op_reply(121 1.00000000 [create] v128'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 17 ==== osd_op_reply(122 1.00000000 [delete] v128'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.614+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.618+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 18 ==== osd_op_reply(123 1.00000000 [create] v128'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.621 INFO:teuthology.orchestra.run.smithi043.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-17T05:34:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.618+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.618+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.618+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a7bf0 con 0x55e91f2a4350 2024-09-17T05:34:21.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e91f2a8000 con 0x55e91f2a4350 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb317e00640 1 -- 172.21.15.43:0/3317135359 <== osd.3 v2:172.21.15.43:6817/6899353 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb30800c120 con 0x55e91f2a4350 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55e91f2941d0 msgr2=0x55e91f2965f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x55e91f2941d0 0x55e91f2965f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55e91f2a04c0 msgr2=0x55e91f2a3380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x55e91f2a04c0 0x55e91f2a3380 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e91f2a4350 msgr2=0x55e91f2a67f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e91f2a4350 0x55e91f2a67f0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55e91f298e70 msgr2=0x55e91f299280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 reap_dead start 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x55e91f298e70 0x55e91f299280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e91f115360 msgr2=0x55e91f115770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x55e91f115360 0x55e91f115770 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55e91f29c6b0 msgr2=0x55e91f29edd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x55e91f29c6b0 0x55e91f29edd0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2fc071be0 msgr2=0x7fb2fc0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb2fc071be0 0x7fb2fc0740f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb3100b6000 tx=0x7fb3100092b0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 msgr2=0x55e91f158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 --2- 172.21.15.43:0/3317135359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e91f286310 0x55e91f158080 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x55e91f119870 tx=0x7fb31800f8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb31d400640 1 -- 172.21.15.43:0/3317135359 reap_dead start 2024-09-17T05:34:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 shutdown_connections 2024-09-17T05:34:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 >> 172.21.15.43:0/3317135359 conn(0x55e91f101d80 msgr2=0x55e91f154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 shutdown_connections 2024-09-17T05:34:21.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.622+0000 7fb32392d080 1 -- 172.21.15.43:0/3317135359 wait complete. 2024-09-17T05:34:21.632 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T05:34:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.730+0000 7fdbe15d5080 1 Processor -- start 2024-09-17T05:34:21.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.730+0000 7fdbe15d5080 1 -- start start 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc0002350 0x55e2167e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 0x55e2167e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2166eea00 con 0x55e21679efb0 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e21674cf40 con 0x55e2167cc8c0 2024-09-17T05:34:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e216857890 con 0x7fdbc0002350 2024-09-17T05:34:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbdb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 0x55e2167e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbda800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc0002350 0x55e2167e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:36728/0 (socket says 172.21.15.43:36728) 2024-09-17T05:34:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 -- 172.21.15.43:0/1345056252 learned_addr learned my addr 172.21.15.43:0/1345056252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 -- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc0002350 msgr2=0x55e2167e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc0002350 0x55e2167e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 -- 172.21.15.43:0/1345056252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 msgr2=0x55e2167e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 0x55e2167e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 -- 172.21.15.43:0/1345056252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2166c7c10 con 0x55e2167cc8c0 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbdb200640 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 0x55e2167e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd9e00640 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009de0 tx=0x7fdbc800e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971946a61c77c484 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd8a00640 1 -- 172.21.15.43:0/1345056252 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010420 con 0x55e2167cc8c0 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd8a00640 1 -- 172.21.15.43:0/1345056252 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbc8015020 con 0x55e2167cc8c0 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbd8a00640 1 -- 172.21.15.43:0/1345056252 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010710 con 0x55e2167cc8c0 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 msgr2=0x55e21679ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009de0 tx=0x7fdbc800e710 comp rx=0 tx=0).stop 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- 172.21.15.43:0/1345056252 shutdown_connections 2024-09-17T05:34:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2167cc8c0 0x55e21679ea70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e21679efb0 0x55e2167e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/1345056252 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbc0002350 0x55e2167e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- 172.21.15.43:0/1345056252 >> 172.21.15.43:0/1345056252 conn(0x55e21678cd80 msgr2=0x55e216875a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- 172.21.15.43:0/1345056252 shutdown_connections 2024-09-17T05:34:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 -- 172.21.15.43:0/1345056252 wait complete. 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.734+0000 7fdbe15d5080 1 Processor -- start 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- start start 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 0x55e2167c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2167cc8c0 0x55e2168730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0002350 0x55e21690da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e2167a2d70 con 0x55e2167cc8c0 2024-09-17T05:34:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2167e1a90 con 0x7fdbc0002350 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x55e2167e1d30 con 0x55e21679efb0 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2167cc8c0 0x55e2168730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 0x55e2167c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 0x55e2167c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3301/0 says I am v2:172.21.15.43:48634/0 (socket says 172.21.15.43:48634) 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 -- 172.21.15.43:0/247369768 learned_addr learned my addr 172.21.15.43:0/247369768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbd9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2167cc8c0 0x55e2168730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37146/0 (socket says 172.21.15.43:37146) 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbdb200640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0002350 0x55e21690da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 -- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0002350 msgr2=0x55e21690da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0002350 0x55e21690da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 -- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2167cc8c0 msgr2=0x55e2168730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e2167cc8c0 0x55e2168730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 -- 172.21.15.43:0/247369768 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21690e530 con 0x55e21679efb0 2024-09-17T05:34:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbda800640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 0x55e2167c65a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc002040 tx=0x7fdbcc0ad700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbd3400640 1 -- 172.21.15.43:0/247369768 <== mon.2 v2:172.21.15.62:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004e20 con 0x55e21679efb0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbd3400640 1 -- 172.21.15.43:0/247369768 <== mon.2 v2:172.21.15.62:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbcc0adbf0 con 0x55e21679efb0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbd3400640 1 -- 172.21.15.43:0/247369768 <== mon.2 v2:172.21.15.62:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0b8450 con 0x55e21679efb0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e21690e7d0 con 0x55e21679efb0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.738+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55e2167ca5a0 con 0x55e21679efb0 2024-09-17T05:34:21.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbd3400640 1 -- 172.21.15.43:0/247369768 <== mon.2 v2:172.21.15.62:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdbcc0b85f0 con 0x55e21679efb0 2024-09-17T05:34:21.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbd3400640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc071c10 0x7fdbbc074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbd3400640 1 -- 172.21.15.43:0/247369768 <== mon.2 v2:172.21.15.62:3301/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7fdbcc12e690 con 0x55e21679efb0 2024-09-17T05:34:21.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbd9e00640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc071c10 0x7fdbbc074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e2167cbef0 0x55e2167caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55e2167caf80 con 0x55e2167cbef0 2024-09-17T05:34:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbdb200640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e2167cbef0 0x55e2167caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbd9e00640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc071c10 0x7fdbbc074120 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009de0 tx=0x7fdbc8011000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.742+0000 7fdbdb200640 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e2167cbef0 0x55e2167caa40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbdb200640 1 -- 172.21.15.43:0/247369768 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v128'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbd4002040 con 0x55e2167cbef0 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e2167cbef0 msgr2=0x55e2167caa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x55e2167cbef0 0x55e2167caa40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc071c10 msgr2=0x7fdbbc074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc071c10 0x7fdbbc074120 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009de0 tx=0x7fdbc8011000 comp rx=0 tx=0).stop 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 msgr2=0x55e2167c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 --2- 172.21.15.43:0/247369768 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x55e21679efb0 0x55e2167c65a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc002040 tx=0x7fdbcc0ad700 comp rx=0 tx=0).stop 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbdb200640 1 -- 172.21.15.43:0/247369768 reap_dead start 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 shutdown_connections 2024-09-17T05:34:21.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 >> 172.21.15.43:0/247369768 conn(0x55e21678cd80 msgr2=0x55e216875a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 shutdown_connections 2024-09-17T05:34:21.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.746+0000 7fdbe15d5080 1 -- 172.21.15.43:0/247369768 wait complete. 2024-09-17T05:34:21.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- 172.21.15.43:0/1044951275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff70006f110 msgr2=0x7ff7000ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 --2- 172.21.15.43:0/1044951275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff70006f110 0x7ff7000ad0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009bc0 tx=0x7ff6f801cae0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- 172.21.15.43:0/1044951275 shutdown_connections 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 --2- 172.21.15.43:0/1044951275 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff70006f110 0x7ff7000ad0b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 --2- 172.21.15.43:0/1044951275 >> v2:172.21.15.62:3300/0 conn(0x7ff7000b3af0 0x7ff70015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- 172.21.15.43:0/1044951275 >> 172.21.15.43:0/1044951275 conn(0x7ff7000b23d0 msgr2=0x7ff7000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- 172.21.15.43:0/1044951275 shutdown_connections 2024-09-17T05:34:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- 172.21.15.43:0/1044951275 wait complete. 2024-09-17T05:34:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 Processor -- start 2024-09-17T05:34:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.914+0000 7ff707800640 1 -- start start 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 0x7ff7000cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 0x7ff700161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff700168c10 con 0x7ff70006f110 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff700168a90 con 0x7ff700160f30 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff700168d90 con 0x7ff700155d70 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6ffe00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 0x7ff7000cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37158/0 (socket says 172.21.15.43:37158) 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 -- 172.21.15.43:0/2197503973 learned_addr learned my addr 172.21.15.43:0/2197503973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff705800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 0x7ff700161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 -- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 msgr2=0x7ff7000cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 0x7ff7000cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 -- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 msgr2=0x7ff700161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 0x7ff700161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 -- 172.21.15.43:0/2197503973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7001fbf80 con 0x7ff70006f110 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff705800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 0x7ff700161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6ffe00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 0x7ff7000cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff704e00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c3d0 tx=0x7ff6f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4017070 con 0x7ff70006f110 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f4003600 con 0x7ff70006f110 2024-09-17T05:34:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4003a50 con 0x7ff70006f110 2024-09-17T05:34:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001fc260 con 0x7ff70006f110 2024-09-17T05:34:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001fc820 con 0x7ff70006f110 2024-09-17T05:34:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7000ad0b0 con 0x7ff70006f110 2024-09-17T05:34:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.918+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6f401a070 con 0x7ff70006f110 2024-09-17T05:34:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.922+0000 7ff6fcc00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 0x7ff6e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.922+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7ff6f40941b0 con 0x7ff70006f110 2024-09-17T05:34:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.922+0000 7ff6ffe00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 0x7ff6e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:21.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.922+0000 7ff6ffe00640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 0x7ff6e0075710 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff70015c8d0 tx=0x7ff6ec0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:21.926+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6f405de40 con 0x7ff70006f110 2024-09-17T05:34:22.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.134+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff7000af690 con 0x7ff70006f110 2024-09-17T05:34:22.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:22.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.326+0000 7ff6fcc00640 1 -- 172.21.15.43:0/2197503973 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v330) ==== 162+0+0 (secure 0 0 0) 0x7ff6f4062870 con 0x7ff70006f110 2024-09-17T05:34:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:34:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 msgr2=0x7ff6e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 0x7ff6e0075710 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff70015c8d0 tx=0x7ff6ec0091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 msgr2=0x7ff700160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c3d0 tx=0x7ff6f400c890 comp rx=0 tx=0).stop 2024-09-17T05:34:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 shutdown_connections 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff6e0073200 0x7ff6e0075710 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff700160f30 0x7ff700161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff700155d70 0x7ff7000cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 --2- 172.21.15.43:0/2197503973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff70006f110 0x7ff700160360 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 >> 172.21.15.43:0/2197503973 conn(0x7ff7000b23d0 msgr2=0x7ff70006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 shutdown_connections 2024-09-17T05:34:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.330+0000 7ff707800640 1 -- 172.21.15.43:0/2197503973 wait complete. 2024-09-17T05:34:22.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 -- 172.21.15.43:0/3880570432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d4070620 msgr2=0x7f26d40b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 --2- 172.21.15.43:0/3880570432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d4070620 0x7f26d40b30d0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009900 tx=0x7f26c801c850 comp rx=0 tx=0).stop 2024-09-17T05:34:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 -- 172.21.15.43:0/3880570432 shutdown_connections 2024-09-17T05:34:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 --2- 172.21.15.43:0/3880570432 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d40cb860 0x7f26d40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 --2- 172.21.15.43:0/3880570432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d4070620 0x7f26d40b30d0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 -- 172.21.15.43:0/3880570432 >> 172.21.15.43:0/3880570432 conn(0x7f26d40b23d0 msgr2=0x7f26d40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 -- 172.21.15.43:0/3880570432 shutdown_connections 2024-09-17T05:34:22.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.554+0000 7f26dc200640 1 -- 172.21.15.43:0/3880570432 wait complete. 2024-09-17T05:34:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 Processor -- start 2024-09-17T05:34:22.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- start start 2024-09-17T05:34:22.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:22.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 0x7f26d41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:22.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26d40cb860 0x7f26d41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40ce630 con 0x7f26d406ce60 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40ce4b0 con 0x7f26d40cb860 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40ce7b0 con 0x7f26d4070620 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37172/0 (socket says 172.21.15.43:37172) 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 -- 172.21.15.43:0/606969133 learned_addr learned my addr 172.21.15.43:0/606969133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26da200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26d40cb860 0x7f26d41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d8e00640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 0x7f26d41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 -- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 msgr2=0x7f26d41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 0x7f26d41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 -- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26d40cb860 msgr2=0x7f26d41527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26d40cb860 0x7f26d41527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 -- 172.21.15.43:0/606969133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d4074280 con 0x7f26d406ce60 2024-09-17T05:34:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d8e00640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 0x7f26d41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d9800640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f26c801a480 tx=0x7f26c80a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80ac070 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c8004030 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d4074560 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004480 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.558+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d4158e60 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.562+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d40ced80 con 0x7f26d406ce60 2024-09-17T05:34:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.562+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26c80af070 con 0x7f26d406ce60 2024-09-17T05:34:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.562+0000 7f26d1600640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 0x7f26b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.562+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9063+0+0 (secure 0 0 0) 0x7f26c812ed60 con 0x7f26d406ce60 2024-09-17T05:34:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.562+0000 7f26d8e00640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 0x7f26b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:22.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.566+0000 7f26d8e00640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 0x7f26b4075710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f26d4159730 tx=0x7f26c0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:22.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.566+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26c80bf020 con 0x7f26d406ce60 2024-09-17T05:34:22.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.778+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26d415a070 con 0x7f26d406ce60 2024-09-17T05:34:22.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.778+0000 7f26d1600640 1 -- 172.21.15.43:0/606969133 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4392 (secure 0 0 0) 0x7f26c80b6070 con 0x7f26d406ce60 2024-09-17T05:34:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:22.783 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":331,"btime":"2024-09-17T05:34:22:332684+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:22.332672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9079},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9079":{"gid":9079,"name":"d","rank":0,"incarnation":331,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6835/3935849352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3935849352},{"type":"v1","addr":"172.21.15.62:6835","nonce":3935849352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 331 2024-09-17T05:34:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 msgr2=0x7f26b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 0x7f26b4075710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f26d4159730 tx=0x7f26c0009040 comp rx=0 tx=0).stop 2024-09-17T05:34:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 msgr2=0x7f26d4158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f26c801a480 tx=0x7f26c80a65d0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 shutdown_connections 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26b4073200 0x7f26b4075710 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26d40cb860 0x7f26d41527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26d4070620 0x7f26d41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 --2- 172.21.15.43:0/606969133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d406ce60 0x7f26d4158920 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 >> 172.21.15.43:0/606969133 conn(0x7f26d40b23d0 msgr2=0x7f26d40ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 shutdown_connections 2024-09-17T05:34:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:22.782+0000 7f26dc200640 1 -- 172.21.15.43:0/606969133 wait complete. 2024-09-17T05:34:22.805 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:34:23.026 INFO:tasks.ceph.mds.c.smithi043.stderr: -51> 2024-09-17T05:34:18.270+0000 7f01c7c00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:23.145 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:34:25.358 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:34:25.354+0000 7fd004a00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:25.358 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:34:25.354+0000 7fd004a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-17T05:34:25.364 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:25.361+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:34:25.364 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:25.361+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-17T05:34:27.648 INFO:tasks.ceph.mds.d.smithi062.stderr:2024-09-17T05:34:27.650+0000 7fd009a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:27.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 -- 172.21.15.43:0/86955852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a80b5d20 msgr2=0x7fd3a80d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 --2- 172.21.15.43:0/86955852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a80b5d20 0x7fd3a80d0390 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8073870 tx=0x7fd39001c7a0 comp rx=0 tx=0).stop 2024-09-17T05:34:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 -- 172.21.15.43:0/86955852 shutdown_connections 2024-09-17T05:34:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 --2- 172.21.15.43:0/86955852 >> v2:172.21.15.62:3300/0 conn(0x7fd3a806fce0 0x7fd3a80700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 --2- 172.21.15.43:0/86955852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a80b5d20 0x7fd3a80d0390 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 -- 172.21.15.43:0/86955852 >> 172.21.15.43:0/86955852 conn(0x7fd3a80b2810 msgr2=0x7fd3a80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 -- 172.21.15.43:0/86955852 shutdown_connections 2024-09-17T05:34:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.973+0000 7fd3aee00640 1 -- 172.21.15.43:0/86955852 wait complete. 2024-09-17T05:34:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 Processor -- start 2024-09-17T05:34:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 -- start start 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 0x7fd3a80cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 0x7fd3a80cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81694c0 con 0x7fd3a806fce0 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8169340 con 0x7fd3a814a460 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a8169640 con 0x7fd3a815f470 2024-09-17T05:34:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44972/0 (socket says 172.21.15.43:44972) 2024-09-17T05:34:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 -- 172.21.15.43:0/1598446035 learned_addr learned my addr 172.21.15.43:0/1598446035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3ace00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 0x7fd3a80cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd39fe00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 0x7fd3a80cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 -- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 msgr2=0x7fd3a80cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 0x7fd3a80cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 -- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 msgr2=0x7fd3a80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 0x7fd3a80cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 -- 172.21.15.43:0/1598446035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a815dad0 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3ace00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 0x7fd3a80cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd39fe00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 0x7fd3a80cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a7e00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c3c0 tx=0x7fd39800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3980053c0 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398003860 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.977+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a815eed0 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a815dd70 con 0x7fd3a806fce0 2024-09-17T05:34:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a80700f0 con 0x7fd3a806fce0 2024-09-17T05:34:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd39801a070 con 0x7fd3a806fce0 2024-09-17T05:34:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd3a4c00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 0x7fd384075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fd398094690 con 0x7fd3a806fce0 2024-09-17T05:34:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.981+0000 7fd39fe00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 0x7fd384075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.985+0000 7fd39fe00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 0x7fd384075740 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd3900af000 tx=0x7fd39001afe0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:27.985+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd398021020 con 0x7fd3a806fce0 2024-09-17T05:34:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.197+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3a8075dd0 con 0x7fd3a806fce0 2024-09-17T05:34:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.197+0000 7fd3a4c00640 1 -- 172.21.15.43:0/1598446035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+4384 (secure 0 0 0) 0x7fd39805e9a0 con 0x7fd3a806fce0 2024-09-17T05:34:28.202 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:28.202 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":335,"btime":"2024-09-17T05:34:25:365323+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 335 2024-09-17T05:34:28.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 msgr2=0x7fd384075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 0x7fd384075740 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd3900af000 tx=0x7fd39001afe0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 msgr2=0x7fd3a815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c3c0 tx=0x7fd39800c880 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 shutdown_connections 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd384073230 0x7fd384075740 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3a814a460 0x7fd3a80cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd3a815f470 0x7fd3a80cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 --2- 172.21.15.43:0/1598446035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a806fce0 0x7fd3a815e290 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 >> 172.21.15.43:0/1598446035 conn(0x7fd3a80b2810 msgr2=0x7fd3a80738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 shutdown_connections 2024-09-17T05:34:28.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.201+0000 7fd3aee00640 1 -- 172.21.15.43:0/1598446035 wait complete. 2024-09-17T05:34:28.224 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:34:28.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- 172.21.15.43:0/1991144191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac09ba60 msgr2=0x7f61ac0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 --2- 172.21.15.43:0/1991144191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac09ba60 0x7f61ac0b3130 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f61ac076b20 tx=0x7f619c01c800 comp rx=0 tx=0).stop 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- 172.21.15.43:0/1991144191 shutdown_connections 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 --2- 172.21.15.43:0/1991144191 >> v2:172.21.15.62:3300/0 conn(0x7f61ac0d0240 0x7f61ac0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 --2- 172.21.15.43:0/1991144191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac09ba60 0x7f61ac0b3130 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- 172.21.15.43:0/1991144191 >> 172.21.15.43:0/1991144191 conn(0x7f61ac0cee60 msgr2=0x7f61ac0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- 172.21.15.43:0/1991144191 shutdown_connections 2024-09-17T05:34:28.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- 172.21.15.43:0/1991144191 wait complete. 2024-09-17T05:34:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 Processor -- start 2024-09-17T05:34:28.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.381+0000 7f61b0e00640 1 -- start start 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61ac09ba60 0x7f61ac1fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61ac0d0650 0x7f61ac14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0b2a50 con 0x7f61ac0d0240 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0b28d0 con 0x7f61ac09ba60 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f61ac0b2bd0 con 0x7f61ac0d0650 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44982/0 (socket says 172.21.15.43:44982) 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 -- 172.21.15.43:0/708791806 learned_addr learned my addr 172.21.15.43:0/708791806 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a9e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61ac09ba60 0x7f61ac1fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61aa800640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61ac0d0650 0x7f61ac14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 -- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61ac0d0650 msgr2=0x7f61ac14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61ac0d0650 0x7f61ac14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 -- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61ac09ba60 msgr2=0x7f61ac1fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61ac09ba60 0x7f61ac1fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 -- 172.21.15.43:0/708791806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac149950 con 0x7f61ac0d0240 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a1400640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f619400e770 tx=0x7f619400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619400cc60 con 0x7f61ac0d0240 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f619400ce00 con 0x7f61ac0d0240 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194004280 con 0x7f61ac0d0240 2024-09-17T05:34:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac1fc190 con 0x7f61ac0d0240 2024-09-17T05:34:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac149bc0 con 0x7f61ac0d0240 2024-09-17T05:34:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.385+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac0b3130 con 0x7f61ac0d0240 2024-09-17T05:34:28.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.389+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6194003600 con 0x7f61ac0d0240 2024-09-17T05:34:28.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.389+0000 7f61a2a00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 0x7f6188075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.389+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7f6194095790 con 0x7f61ac0d0240 2024-09-17T05:34:28.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.389+0000 7f61a9e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 0x7f6188075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.389+0000 7f61a9e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 0x7f6188075710 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f619c0af000 tx=0x7f619c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.393+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f619405f340 con 0x7f61ac0d0240 2024-09-17T05:34:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.605+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61ac1fbf20 con 0x7f61ac0d0240 2024-09-17T05:34:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61a2a00640 1 -- 172.21.15.43:0/708791806 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+4384 (secure 0 0 0) 0x7f6194063d70 con 0x7f61ac0d0240 2024-09-17T05:34:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:28.612 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":335,"btime":"2024-09-17T05:34:25:365323+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 335 2024-09-17T05:34:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 msgr2=0x7f6188075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 0x7f6188075710 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f619c0af000 tx=0x7f619c0b2040 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 msgr2=0x7f61ac146ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f619400e770 tx=0x7f619400ec30 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 shutdown_connections 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6188073200 0x7f6188075710 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f61ac0d0650 0x7f61ac14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61ac0d0240 0x7f61ac146ee0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 --2- 172.21.15.43:0/708791806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f61ac09ba60 0x7f61ac1fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 >> 172.21.15.43:0/708791806 conn(0x7f61ac0cee60 msgr2=0x7f61ac0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:28.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 shutdown_connections 2024-09-17T05:34:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.609+0000 7f61b0e00640 1 -- 172.21.15.43:0/708791806 wait complete. 2024-09-17T05:34:28.634 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:34:28.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 -- 172.21.15.43:0/1412588558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 msgr2=0x7fa5080b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 --2- 172.21.15.43:0/1412588558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa5080b30d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009900 tx=0x7fa4f801c850 comp rx=0 tx=0).stop 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 -- 172.21.15.43:0/1412588558 shutdown_connections 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 --2- 172.21.15.43:0/1412588558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 0x7fa5080cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 --2- 172.21.15.43:0/1412588558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa5080b30d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 -- 172.21.15.43:0/1412588558 >> 172.21.15.43:0/1412588558 conn(0x7fa5080b23d0 msgr2=0x7fa5080d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 -- 172.21.15.43:0/1412588558 shutdown_connections 2024-09-17T05:34:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.789+0000 7fa50d800640 1 -- 172.21.15.43:0/1412588558 wait complete. 2024-09-17T05:34:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 Processor -- start 2024-09-17T05:34:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- start start 2024-09-17T05:34:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa50815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 0x7fa50809cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080ce630 con 0x7fa50806ce60 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5080ce4b0 con 0x7fa508070620 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa5080ce7b0 con 0x7fa5080cb860 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44984/0 (socket says 172.21.15.43:44984) 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 -- 172.21.15.43:0/349025265 learned_addr learned my addr 172.21.15.43:0/349025265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa4fe000640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa50815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa507400640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 0x7fa50809cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 -- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 msgr2=0x7fa50809cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 0x7fa50809cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 -- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 msgr2=0x7fa50815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa50815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 -- 172.21.15.43:0/349025265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5080c41d0 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa506a00640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fa4f801a480 tx=0x7fa4f80a64a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80ac070 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4f80050c0 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5080c4490 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8005510 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.793+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5080c68e0 con 0x7fa50806ce60 2024-09-17T05:34:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.797+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5080ced80 con 0x7fa50806ce60 2024-09-17T05:34:28.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.797+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4f80af070 con 0x7fa50806ce60 2024-09-17T05:34:28.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.797+0000 7fa4ff400640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 0x7fa4e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:28.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.797+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fa4f812ec00 con 0x7fa50806ce60 2024-09-17T05:34:28.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.797+0000 7fa4fe000640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 0x7fa4e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:28.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.801+0000 7fa4fe000640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 0x7fa4e4075740 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa5080bfdd0 tx=0x7fa4f0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:28.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:28.801+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa4f80bf020 con 0x7fa50806ce60 2024-09-17T05:34:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa50806e070 con 0x7fa50806ce60 2024-09-17T05:34:29.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa4ff400640 1 -- 172.21.15.43:0/349025265 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+4384 (secure 0 0 0) 0x7fa4f80b6070 con 0x7fa50806ce60 2024-09-17T05:34:29.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:29.017 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":335,"btime":"2024-09-17T05:34:25:365323+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:29.017 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 335 2024-09-17T05:34:29.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 msgr2=0x7fa4e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 0x7fa4e4075740 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa5080bfdd0 tx=0x7fa4f0009040 comp rx=0 tx=0).stop 2024-09-17T05:34:29.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 msgr2=0x7fa508159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fa4f801a480 tx=0x7fa4f80a64a0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 shutdown_connections 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4e4073230 0x7fa4e4075740 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa508070620 0x7fa50815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa5080cb860 0x7fa50809cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 --2- 172.21.15.43:0/349025265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa50806ce60 0x7fa508159d40 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 >> 172.21.15.43:0/349025265 conn(0x7fa5080b23d0 msgr2=0x7fa5080ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 shutdown_connections 2024-09-17T05:34:29.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.013+0000 7fa50d800640 1 -- 172.21.15.43:0/349025265 wait complete. 2024-09-17T05:34:29.038 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-09-17T05:34:29.039 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-09-17T05:34:29.039 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:34:29.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:29.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:29.217 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:34:29.218 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:34:29.218 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:29.218 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:29.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:29.227 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:34:29.227 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:34:29.227 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:29.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:29.236 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:34:29.237 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:29.237 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:29.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:29.279 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:34:29.279 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:34:29.280 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:29.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:29.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- 172.21.15.43:0/4093814905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220004d50 msgr2=0x7f22200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 --2- 172.21.15.43:0/4093814905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2220004d50 0x7f22200051c0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f223400c610 tx=0x7f223401d2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f223be00640 1 -- 172.21.15.43:0/4093814905 reap_dead start 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- 172.21.15.43:0/4093814905 shutdown_connections 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- 172.21.15.43:0/4093814905 >> 172.21.15.43:0/4093814905 conn(0x7f223c0b23d0 msgr2=0x7f223c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- 172.21.15.43:0/4093814905 shutdown_connections 2024-09-17T05:34:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- 172.21.15.43:0/4093814905 wait complete. 2024-09-17T05:34:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 Processor -- start 2024-09-17T05:34:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.481+0000 7f2242800640 1 -- start start 2024-09-17T05:34:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2220004d50 0x7f223c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f223c0c7f50 0x7f223c1f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f223c167ec0 con 0x7f223c1fa4b0 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f223c167d40 con 0x7f223c0c7f50 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f223c168040 con 0x7f2220004d50 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45006/0 (socket says 172.21.15.43:45006) 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 -- 172.21.15.43:0/1187928637 learned_addr learned my addr 172.21.15.43:0/1187928637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223b400640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2220004d50 0x7f223c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2233e00640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f223c0c7f50 0x7f223c1f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 -- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2220004d50 msgr2=0x7f223c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2220004d50 0x7f223c0c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 -- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f223c0c7f50 msgr2=0x7f223c1f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f223c0c7f50 0x7f223c1f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 -- 172.21.15.43:0/1187928637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c0c3d90 con 0x7f223c1fa4b0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f223be00640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f223401bbb0 tx=0x7f22340012a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234002040 con 0x7f223c1fa4b0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2234003c70 con 0x7f223c1fa4b0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c0c4070 con 0x7f223c1fa4b0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340b9690 con 0x7f223c1fa4b0 2024-09-17T05:34:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c0c4630 con 0x7f223c1fa4b0 2024-09-17T05:34:29.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.485+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223c0700a0 con 0x7f223c1fa4b0 2024-09-17T05:34:29.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.489+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f22340b9830 con 0x7f223c1fa4b0 2024-09-17T05:34:29.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.489+0000 7f2233400640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 0x7f2218075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.489+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7f223412f650 con 0x7f223c1fa4b0 2024-09-17T05:34:29.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.489+0000 7f223b400640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 0x7f2218075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.489+0000 7f223b400640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 0x7f2218075710 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ba0 tx=0x7f222c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.493+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2234017090 con 0x7f223c1fa4b0 2024-09-17T05:34:29.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.705+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f223c1faa70 con 0x7f223c1fa4b0 2024-09-17T05:34:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.705+0000 7f2233400640 1 -- 172.21.15.43:0/1187928637 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+4384 (secure 0 0 0) 0x7f22340f9850 con 0x7f223c1fa4b0 2024-09-17T05:34:29.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:29.711 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":335,"btime":"2024-09-17T05:34:25:365323+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 335 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 msgr2=0x7f2218075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 0x7f2218075710 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ba0 tx=0x7f222c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 msgr2=0x7f223c0c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f223401bbb0 tx=0x7f22340012a0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 shutdown_connections 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2218073200 0x7f2218075710 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f223c0c7f50 0x7f223c1f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f223c1fa4b0 0x7f223c0c29c0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 --2- 172.21.15.43:0/1187928637 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2220004d50 0x7f223c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 >> 172.21.15.43:0/1187928637 conn(0x7f223c0b23d0 msgr2=0x7f223c0ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 shutdown_connections 2024-09-17T05:34:29.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.709+0000 7f2242800640 1 -- 172.21.15.43:0/1187928637 wait complete. 2024-09-17T05:34:29.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:29.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- 172.21.15.43:0/2037885622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380b3920 msgr2=0x7ff5380ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 --2- 172.21.15.43:0/2037885622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380b3920 0x7ff5380ad0b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff5380711f0 tx=0x7ff53001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- 172.21.15.43:0/2037885622 shutdown_connections 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 --2- 172.21.15.43:0/2037885622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380b3920 0x7ff5380ad0b0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 --2- 172.21.15.43:0/2037885622 >> v2:172.21.15.62:3300/0 conn(0x7ff53806f110 0x7ff5380b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 --2- 172.21.15.43:0/2037885622 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff538155d70 0x7ff53815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- 172.21.15.43:0/2037885622 >> 172.21.15.43:0/2037885622 conn(0x7ff5380b23d0 msgr2=0x7ff5380d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- 172.21.15.43:0/2037885622 shutdown_connections 2024-09-17T05:34:29.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- 172.21.15.43:0/2037885622 wait complete. 2024-09-17T05:34:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 Processor -- start 2024-09-17T05:34:29.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.889+0000 7ff53de00640 1 -- start start 2024-09-17T05:34:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff53806f110 0x7ff5380c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5380b3920 0x7ff5380cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff53815f8d0 con 0x7ff538155d70 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff53815f750 con 0x7ff5380b3920 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff53815fa50 con 0x7ff53806f110 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45014/0 (socket says 172.21.15.43:45014) 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 -- 172.21.15.43:0/3166043224 learned_addr learned my addr 172.21.15.43:0/3166043224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff536a00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff53806f110 0x7ff5380c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff52fe00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5380b3920 0x7ff5380cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 -- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff53806f110 msgr2=0x7ff5380c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff53806f110 0x7ff5380c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 -- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5380b3920 msgr2=0x7ff5380cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5380b3920 0x7ff5380cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 -- 172.21.15.43:0/3166043224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5380c3e70 con 0x7ff538155d70 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff537400640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7ff53001b370 tx=0x7ff530005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300bb070 con 0x7ff538155d70 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff530004d40 con 0x7ff538155d70 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300b2040 con 0x7ff538155d70 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5381fa160 con 0x7ff538155d70 2024-09-17T05:34:29.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5381fa6e0 con 0x7ff538155d70 2024-09-17T05:34:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.893+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53815ff20 con 0x7ff538155d70 2024-09-17T05:34:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.897+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff530004240 con 0x7ff538155d70 2024-09-17T05:34:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.897+0000 7ff52f400640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 0x7ff514075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.897+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7ff53012ef70 con 0x7ff538155d70 2024-09-17T05:34:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.897+0000 7ff536a00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 0x7ff514075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.897+0000 7ff536a00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 0x7ff514075740 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff5280058d0 tx=0x7ff528005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:29.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:29.901+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff530016970 con 0x7ff538155d70 2024-09-17T05:34:30.113 INFO:tasks.ceph.mds.d.smithi062.stderr: -56> 2024-09-17T05:34:25.354+0000 7fd004a00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-17T05:34:30.114 INFO:tasks.ceph.mds.d.smithi062.stderr: -53> 2024-09-17T05:34:25.354+0000 7fd004a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-17T05:34:30.114 INFO:tasks.ceph.mds.d.smithi062.stderr: -23> 2024-09-17T05:34:27.650+0000 7fd009a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:30.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.113+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5380ce570 con 0x7ff538155d70 2024-09-17T05:34:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.113+0000 7ff52f400640 1 -- 172.21.15.43:0/3166043224 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+4384 (secure 0 0 0) 0x7ff5300f9200 con 0x7ff538155d70 2024-09-17T05:34:30.118 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:30.118 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":335,"btime":"2024-09-17T05:34:25:365323+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 335 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 msgr2=0x7ff514075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 0x7ff514075740 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff5280058d0 tx=0x7ff528005860 comp rx=0 tx=0).stop 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 msgr2=0x7ff5380cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7ff53001b370 tx=0x7ff530005b00 comp rx=0 tx=0).stop 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 shutdown_connections 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff514073230 0x7ff514075740 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5380b3920 0x7ff5380cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff538155d70 0x7ff5380cfba0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 --2- 172.21.15.43:0/3166043224 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff53806f110 0x7ff5380c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 >> 172.21.15.43:0/3166043224 conn(0x7ff5380b23d0 msgr2=0x7ff5380b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:30.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 shutdown_connections 2024-09-17T05:34:30.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.117+0000 7ff53de00640 1 -- 172.21.15.43:0/3166043224 wait complete. 2024-09-17T05:34:30.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:30.237 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:34:30.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 -- 172.21.15.43:0/1816239202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba40d0240 msgr2=0x7f8ba40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/1816239202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba40d0240 0x7f8ba40cd040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009840 tx=0x7f8b8c01c820 comp rx=0 tx=0).stop 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 -- 172.21.15.43:0/1816239202 shutdown_connections 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/1816239202 >> v2:172.21.15.62:3300/0 conn(0x7f8ba40d0650 0x7f8ba40cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/1816239202 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba40d0240 0x7f8ba40cd040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 -- 172.21.15.43:0/1816239202 >> 172.21.15.43:0/1816239202 conn(0x7f8ba40cee60 msgr2=0x7f8ba40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.293+0000 7f8ba8c00640 1 -- 172.21.15.43:0/1816239202 shutdown_connections 2024-09-17T05:34:30.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 -- 172.21.15.43:0/1816239202 wait complete. 2024-09-17T05:34:30.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 Processor -- start 2024-09-17T05:34:30.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 -- start start 2024-09-17T05:34:30.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba409ba60 0x7f8ba4157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba40d0650 0x7f8ba40ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40b2a70 con 0x7f8ba40d0240 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40b28f0 con 0x7f8ba40d0650 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8ba8c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba4169e30 con 0x7f8ba409ba60 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8b9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8b9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45018/0 (socket says 172.21.15.43:45018) 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.297+0000 7f8b9be00640 1 -- 172.21.15.43:0/260323955 learned_addr learned my addr 172.21.15.43:0/260323955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8ba2800640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba40d0650 0x7f8ba40ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8ba1e00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba409ba60 0x7f8ba4157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 -- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba409ba60 msgr2=0x7f8ba4157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba409ba60 0x7f8ba4157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 -- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba40d0650 msgr2=0x7f8ba40ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba40d0650 0x7f8ba40ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 -- 172.21.15.43:0/260323955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba41584a0 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9be00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c098200 tx=0x7f8b8c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0be070 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b8c0a8a90 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba4158780 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c005040 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba4158d40 con 0x7f8ba40d0240 2024-09-17T05:34:30.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba40b3130 con 0x7f8ba40d0240 2024-09-17T05:34:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.301+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b8c0a6040 con 0x7f8ba40d0240 2024-09-17T05:34:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.305+0000 7f8b9a000640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 0x7f8b80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.305+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7f8b8c12f0c0 con 0x7f8ba40d0240 2024-09-17T05:34:30.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.305+0000 7f8ba1e00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 0x7f8b80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.305+0000 7f8ba1e00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 0x7f8b80075710 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8b94001870 tx=0x7f8b94001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.309+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b8c0ac030 con 0x7f8ba40d0240 2024-09-17T05:34:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.485+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ba4158230 con 0x7f8ba40d0240 2024-09-17T05:34:30.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.485+0000 7f8b9a000640 1 -- 172.21.15.43:0/260323955 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+21858 (secure 0 0 0) 0x7f8b8c016410 con 0x7f8ba40d0240 2024-09-17T05:34:30.491 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:30.491 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:25.365065+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:29:40.518875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:29:41.731305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6833/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6835/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6835/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6833/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6835/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.43:6835/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6833/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6835/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.43:6833/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.62:6834/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.43:6835/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.62:6834/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6834/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.43:6833/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.43:6833/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.62:6834/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.62:6834/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6835/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.62:6834/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.43:6835/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.62:6834/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6833/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.43:6833/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6833/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.43:6833/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6835/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6833/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6833/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.43:6833/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.43:6835/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.62:6835/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.43:6835/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.43:6835/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.62:6834/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6835/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.43:6835/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6833/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6835/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6833/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6834/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.62:6835/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6835/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.62:6835/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.62:6834/3412634924":"2024-09-18T05:32:49.498519+0000","172.21.15.43:6833/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.62:6835/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.62:6834/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.43:6833/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.62:6835/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.62:6835/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6835/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.62:6835/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.62:6835/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.43:6833/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6835/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.43:6833/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.43:6833/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.62:6834/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.43:6833/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.43:6835/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.62:6835/3412634924":"2024-09-18T05:32:49.498519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 msgr2=0x7f8b80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 0x7f8b80075710 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8b94001870 tx=0x7f8b94001380 comp rx=0 tx=0).stop 2024-09-17T05:34:30.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 msgr2=0x7f8ba40af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c098200 tx=0x7f8b8c005db0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 shutdown_connections 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8b80073200 0x7f8b80075710 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba40d0650 0x7f8ba40ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ba40d0240 0x7f8ba40af140 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 --2- 172.21.15.43:0/260323955 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ba409ba60 0x7f8ba4157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 >> 172.21.15.43:0/260323955 conn(0x7f8ba40cee60 msgr2=0x7f8ba40b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 shutdown_connections 2024-09-17T05:34:30.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.489+0000 7f8ba8c00640 1 -- 172.21.15.43:0/260323955 wait complete. 2024-09-17T05:34:30.513 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:34:30.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- 172.21.15.43:0/2749104169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 msgr2=0x7fdf140ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 --2- 172.21.15.43:0/2749104169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf140ad0b0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009bc0 tx=0x7fdf0c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- 172.21.15.43:0/2749104169 shutdown_connections 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 --2- 172.21.15.43:0/2749104169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf140ad0b0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 --2- 172.21.15.43:0/2749104169 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf140b3af0 0x7fdf1415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- 172.21.15.43:0/2749104169 >> 172.21.15.43:0/2749104169 conn(0x7fdf140b23d0 msgr2=0x7fdf140d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- 172.21.15.43:0/2749104169 shutdown_connections 2024-09-17T05:34:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- 172.21.15.43:0/2749104169 wait complete. 2024-09-17T05:34:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 Processor -- start 2024-09-17T05:34:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.669+0000 7fdf1a000640 1 -- start start 2024-09-17T05:34:30.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf14155d70 0x7fdf1409d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf14156cf0 0x7fdf14164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14168be0 con 0x7fdf1406f110 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14168a60 con 0x7fdf14156cf0 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdf14168d60 con 0x7fdf14155d70 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45026/0 (socket says 172.21.15.43:45026) 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 -- 172.21.15.43:0/3884585019 learned_addr learned my addr 172.21.15.43:0/3884585019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf0aa00640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf14155d70 0x7fdf1409d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13e00640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf14156cf0 0x7fdf14164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 -- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf14155d70 msgr2=0x7fdf1409d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf14155d70 0x7fdf1409d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 -- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf14156cf0 msgr2=0x7fdf14164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf14156cf0 0x7fdf14164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 -- 172.21.15.43:0/3884585019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf14073090 con 0x7fdf1406f110 2024-09-17T05:34:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf13400640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c3a0 tx=0x7fdf0400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04017070 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf14073370 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf04003600 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003a50 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf14073930 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.673+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf140ad0b0 con 0x7fdf1406f110 2024-09-17T05:34:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.677+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf0401a070 con 0x7fdf1406f110 2024-09-17T05:34:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.677+0000 7fdf0be00640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 0x7fdef0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.677+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fdf040942d0 con 0x7fdf1406f110 2024-09-17T05:34:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.677+0000 7fdf0aa00640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 0x7fdef0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.677+0000 7fdf0aa00640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 0x7fdef0075710 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b3cc0 tx=0x7fdefc00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.681+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf0405de60 con 0x7fdf1406f110 2024-09-17T05:34:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.889+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf1415ea50 con 0x7fdf1406f110 2024-09-17T05:34:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.889+0000 7fdf0be00640 1 -- 172.21.15.43:0/3884585019 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) ==== 95+0+5210 (secure 0 0 0) 0x7fdf04062890 con 0x7fdf1406f110 2024-09-17T05:34:30.895 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:30.895 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":336,"btime":"2024-09-17T05:34:30:378522+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1090667792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1090667792},{"type":"v1","addr":"172.21.15.62:6835","nonce":1090667792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 336 2024-09-17T05:34:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 msgr2=0x7fdef0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 0x7fdef0075710 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b3cc0 tx=0x7fdefc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 msgr2=0x7fdf141fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c3a0 tx=0x7fdf0400c860 comp rx=0 tx=0).stop 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 shutdown_connections 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef0073200 0x7fdef0075710 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf14156cf0 0x7fdf14164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf14155d70 0x7fdf1409d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 --2- 172.21.15.43:0/3884585019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1406f110 0x7fdf141fc900 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 >> 172.21.15.43:0/3884585019 conn(0x7fdf140b23d0 msgr2=0x7fdf140b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 shutdown_connections 2024-09-17T05:34:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:30.893+0000 7fdf1a000640 1 -- 172.21.15.43:0/3884585019 wait complete. 2024-09-17T05:34:30.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- 172.21.15.43:0/3019617632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 msgr2=0x7fbb240b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3019617632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240b33e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c01c820 comp rx=0 tx=0).stop 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- 172.21.15.43:0/3019617632 shutdown_connections 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3019617632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240b33e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3019617632 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 0x7fbb2415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- 172.21.15.43:0/3019617632 >> 172.21.15.43:0/3019617632 conn(0x7fbb240b23d0 msgr2=0x7fbb240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- 172.21.15.43:0/3019617632 shutdown_connections 2024-09-17T05:34:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- 172.21.15.43:0/3019617632 wait complete. 2024-09-17T05:34:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 Processor -- start 2024-09-17T05:34:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.073+0000 7fbb28c00640 1 -- start start 2024-09-17T05:34:31.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 0x7fbb240c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24168a60 con 0x7fbb240b3920 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241688e0 con 0x7fbb2406f110 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbb24168be0 con 0x7fbb24155d70 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45036/0 (socket says 172.21.15.43:45036) 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 -- 172.21.15.43:0/3577491750 learned_addr learned my addr 172.21.15.43:0/3577491750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb21e00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb22800640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 0x7fbb240c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 -- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 msgr2=0x7fbb240c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 0x7fbb240c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 -- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 msgr2=0x7fbb240c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 -- 172.21.15.43:0/3577491750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240cadc0 con 0x7fbb240b3920 2024-09-17T05:34:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb21e00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb19400640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b0040 tx=0x7fbb0c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0b2040 con 0x7fbb240b3920 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb0c0bd070 con 0x7fbb240b3920 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb240cb0a0 con 0x7fbb240b3920 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003600 con 0x7fbb240b3920 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb240cb660 con 0x7fbb240b3920 2024-09-17T05:34:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.077+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb240b33e0 con 0x7fbb240b3920 2024-09-17T05:34:31.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.081+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb0c0037a0 con 0x7fbb240b3920 2024-09-17T05:34:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.081+0000 7fbb1aa00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 0x7fbb00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.081+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fbb0c12f8c0 con 0x7fbb240b3920 2024-09-17T05:34:31.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.081+0000 7fbb21e00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 0x7fbb00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.081+0000 7fbb21e00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 0x7fbb00075740 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbb240715c0 tx=0x7fbb140091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.085+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb0c0c7020 con 0x7fbb240b3920 2024-09-17T05:34:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.293+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb2406dd50 con 0x7fbb240b3920 2024-09-17T05:34:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.293+0000 7fbb1aa00640 1 -- 172.21.15.43:0/3577491750 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) ==== 95+0+5210 (secure 0 0 0) 0x7fbb0c0164d0 con 0x7fbb240b3920 2024-09-17T05:34:31.299 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:31.299 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":336,"btime":"2024-09-17T05:34:30:378522+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1090667792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1090667792},{"type":"v1","addr":"172.21.15.62:6835","nonce":1090667792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":335,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:29:42.776833+0000","modified":"2024-09-17T05:34:25.365318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T05:34:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 336 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 msgr2=0x7fbb00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 0x7fbb00075740 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbb240715c0 tx=0x7fbb140091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 msgr2=0x7fbb2409d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0b0040 tx=0x7fbb0c005e70 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 shutdown_connections 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbb00073230 0x7fbb00075740 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbb24155d70 0x7fbb240c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb240b3920 0x7fbb2409d920 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 --2- 172.21.15.43:0/3577491750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb2406f110 0x7fbb240c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 >> 172.21.15.43:0/3577491750 conn(0x7fbb240b23d0 msgr2=0x7fbb24165b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 shutdown_connections 2024-09-17T05:34:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.297+0000 7fbb28c00640 1 -- 172.21.15.43:0/3577491750 wait complete. 2024-09-17T05:34:31.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- 172.21.15.43:0/3700000527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 msgr2=0x7fa4fc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 --2- 172.21.15.43:0/3700000527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc15c7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009900 tx=0x7fa4f001c820 comp rx=0 tx=0).stop 2024-09-17T05:34:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- 172.21.15.43:0/3700000527 shutdown_connections 2024-09-17T05:34:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 --2- 172.21.15.43:0/3700000527 >> v2:172.21.15.62:3300/0 conn(0x7fa4fc0b3af0 0x7fa4fc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 --2- 172.21.15.43:0/3700000527 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc15c7e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- 172.21.15.43:0/3700000527 >> 172.21.15.43:0/3700000527 conn(0x7fa4fc0b23d0 msgr2=0x7fa4fc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:31.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- 172.21.15.43:0/3700000527 shutdown_connections 2024-09-17T05:34:31.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- 172.21.15.43:0/3700000527 wait complete. 2024-09-17T05:34:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 Processor -- start 2024-09-17T05:34:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.481+0000 7fa503400640 1 -- start start 2024-09-17T05:34:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 0x7fa4fc164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc168c80 con 0x7fa4fc09ca80 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc168b00 con 0x7fa4fc06f110 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc168e00 con 0x7fa4fc155d70 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45050/0 (socket says 172.21.15.43:45050) 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 -- 172.21.15.43:0/78147971 learned_addr learned my addr 172.21.15.43:0/78147971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa500a00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 0x7fa4fc164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa4fbe00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 -- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 msgr2=0x7fa4fc164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 -- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 msgr2=0x7fa4fc164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 0x7fa4fc164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 -- 172.21.15.43:0/78147971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc0733e0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa500a00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 0x7fa4fc164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa501400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400ec30 tx=0x7fa4f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4003b80 con 0x7fa4fc09ca80 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4f4003e10 con 0x7fa4fc09ca80 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4004280 con 0x7fa4fc09ca80 2024-09-17T05:34:31.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc0736c0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc073c80 con 0x7fa4fc09ca80 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.485+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc0ad0b0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.489+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4f4004420 con 0x7fa4fc09ca80 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.489+0000 7fa4f8c00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 0x7fa4dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.489+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fa4f4095be0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.489+0000 7fa500a00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 0x7fa4dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.489+0000 7fa500a00640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 0x7fa4dc075740 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00af000 tx=0x7fa4f001b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.493+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa4f405f8b0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.665+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4fc0af7a0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa4f8c00640 1 -- 172.21.15.43:0/78147971 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+21858 (secure 0 0 0) 0x7fa4f40642e0 con 0x7fa4fc09ca80 2024-09-17T05:34:31.673 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:31.673 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:25.365065+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:29:40.518875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:29:41.731305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6833/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6835/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6835/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6833/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6835/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.43:6835/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6833/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6835/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.43:6833/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.62:6834/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.43:6835/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.62:6834/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6834/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.43:6833/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.43:6833/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.62:6834/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.62:6834/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6835/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.62:6834/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.43:6835/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.62:6834/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6833/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.43:6833/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6833/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.43:6833/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6835/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6833/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6833/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.43:6833/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.43:6835/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.62:6835/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.43:6835/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.43:6835/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.62:6834/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6835/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.43:6835/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6833/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6835/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6833/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6834/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.62:6835/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6835/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.62:6835/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.62:6834/3412634924":"2024-09-18T05:32:49.498519+0000","172.21.15.43:6833/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.62:6835/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.62:6834/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.43:6833/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.62:6835/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.62:6835/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6835/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.62:6835/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.62:6835/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.43:6833/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6835/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.43:6833/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.43:6833/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.62:6834/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.43:6833/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.43:6835/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.62:6835/3412634924":"2024-09-18T05:32:49.498519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 msgr2=0x7fa4dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 0x7fa4dc075740 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00af000 tx=0x7fa4f001b060 comp rx=0 tx=0).stop 2024-09-17T05:34:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 msgr2=0x7fa4fc0cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400ec30 tx=0x7fa4f400c6a0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 shutdown_connections 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4dc073230 0x7fa4dc075740 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4fc09ca80 0x7fa4fc0cb930 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4fc155d70 0x7fa4fc164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 --2- 172.21.15.43:0/78147971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4fc06f110 0x7fa4fc164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 >> 172.21.15.43:0/78147971 conn(0x7fa4fc0b23d0 msgr2=0x7fa4fc0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 shutdown_connections 2024-09-17T05:34:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.669+0000 7fa503400640 1 -- 172.21.15.43:0/78147971 wait complete. 2024-09-17T05:34:31.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- 172.21.15.43:0/1900398675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 msgr2=0x7f7ff40b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 --2- 172.21.15.43:0/1900398675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff40b33e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4072b60 tx=0x7f7fe001c740 comp rx=0 tx=0).stop 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- 172.21.15.43:0/1900398675 shutdown_connections 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 --2- 172.21.15.43:0/1900398675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff40b33e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 --2- 172.21.15.43:0/1900398675 >> v2:172.21.15.62:3300/0 conn(0x7f7ff4155d70 0x7f7ff415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- 172.21.15.43:0/1900398675 >> 172.21.15.43:0/1900398675 conn(0x7f7ff40b23d0 msgr2=0x7f7ff40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- 172.21.15.43:0/1900398675 shutdown_connections 2024-09-17T05:34:31.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- 172.21.15.43:0/1900398675 wait complete. 2024-09-17T05:34:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 Processor -- start 2024-09-17T05:34:31.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.853+0000 7f7ffba00640 1 -- start start 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 0x7f7ff409da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4155d70 0x7f7ff4164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff41689d0 con 0x7f7ff406f110 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4168850 con 0x7f7ff40b3920 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff4168b50 con 0x7f7ff4155d70 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45076/0 (socket says 172.21.15.43:45076) 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 -- 172.21.15.43:0/691363757 learned_addr learned my addr 172.21.15.43:0/691363757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 -- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4155d70 msgr2=0x7f7ff4164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4155d70 0x7f7ff4164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 -- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 msgr2=0x7f7ff409da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff3e00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 0x7f7ff409da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 0x7f7ff409da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 -- 172.21.15.43:0/691363757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff409df80 con 0x7f7ff406f110 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff3e00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 0x7f7ff409da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff9000640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d750 tx=0x7f7fe800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8017070 con 0x7f7ff406f110 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fe8004110 con 0x7f7ff406f110 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80099d0 con 0x7f7ff406f110 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff41f9950 con 0x7f7ff406f110 2024-09-17T05:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff40cb160 con 0x7f7ff406f110 2024-09-17T05:34:31.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff40cb3c0 con 0x7f7ff406f110 2024-09-17T05:34:31.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.857+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7fe8009b70 con 0x7f7ff406f110 2024-09-17T05:34:31.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.861+0000 7f7ff0c00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 0x7f7fd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:31.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.861+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7f7fe8095100 con 0x7f7ff406f110 2024-09-17T05:34:31.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.861+0000 7f7ff3e00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 0x7f7fd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:31.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.861+0000 7f7ff3e00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 0x7f7fd4075740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00af000 tx=0x7f7fe001af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:31.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:31.865+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7fe805f300 con 0x7f7ff406f110 2024-09-17T05:34:32.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.073+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7ff40b33e0 con 0x7f7ff406f110 2024-09-17T05:34:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ff0c00640 1 -- 172.21.15.43:0/691363757 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v337) ==== 161+0+0 (secure 0 0 0) 0x7f7fe805ec00 con 0x7f7ff406f110 2024-09-17T05:34:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:34:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 msgr2=0x7f7fd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 0x7f7fd4075740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00af000 tx=0x7f7fe001af80 comp rx=0 tx=0).stop 2024-09-17T05:34:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 msgr2=0x7f7ff4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d750 tx=0x7f7fe800dc10 comp rx=0 tx=0).stop 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 shutdown_connections 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd4073230 0x7f7fd4075740 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff40b3920 0x7f7ff409da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4155d70 0x7f7ff4164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 --2- 172.21.15.43:0/691363757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff406f110 0x7f7ff4156300 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 >> 172.21.15.43:0/691363757 conn(0x7f7ff40b23d0 msgr2=0x7f7ff41655d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 shutdown_connections 2024-09-17T05:34:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.389+0000 7f7ffba00640 1 -- 172.21.15.43:0/691363757 wait complete. 2024-09-17T05:34:32.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:34:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 -- 172.21.15.43:0/345106579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c06f110 msgr2=0x7fae7c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 --2- 172.21.15.43:0/345106579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c06f110 0x7fae7c0ad0b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0af0a0 tx=0x7fae7401cac0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 -- 172.21.15.43:0/345106579 shutdown_connections 2024-09-17T05:34:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 --2- 172.21.15.43:0/345106579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c06f110 0x7fae7c0ad0b0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 --2- 172.21.15.43:0/345106579 >> v2:172.21.15.62:3300/0 conn(0x7fae7c0b3af0 0x7fae7c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 -- 172.21.15.43:0/345106579 >> 172.21.15.43:0/345106579 conn(0x7fae7c0b23d0 msgr2=0x7fae7c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 -- 172.21.15.43:0/345106579 shutdown_connections 2024-09-17T05:34:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.573+0000 7fae82c00640 1 -- 172.21.15.43:0/345106579 wait complete. 2024-09-17T05:34:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 Processor -- start 2024-09-17T05:34:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 -- start start 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c06f110 0x7fae7c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 0x7fae7c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c168b90 con 0x7fae7c155d70 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c168a10 con 0x7fae7c06f110 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae82c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c168d10 con 0x7fae7c15de30 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45088/0 (socket says 172.21.15.43:45088) 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae73e00640 1 -- 172.21.15.43:0/2659542916 learned_addr learned my addr 172.21.15.43:0/2659542916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae7be00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c06f110 0x7fae7c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.577+0000 7fae80c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 0x7fae7c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 -- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 msgr2=0x7fae7c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 0x7fae7c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 -- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c06f110 msgr2=0x7fae7c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c06f110 0x7fae7c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 -- 172.21.15.43:0/2659542916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c15e780 con 0x7fae7c155d70 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae80c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 0x7fae7c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae73e00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e7c0 tx=0x7fae6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400cc60 con 0x7fae7c155d70 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae6400ce00 con 0x7fae7c155d70 2024-09-17T05:34:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c0ce100 con 0x7fae7c155d70 2024-09-17T05:34:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64005280 con 0x7fae7c155d70 2024-09-17T05:34:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c0c5150 con 0x7fae7c155d70 2024-09-17T05:34:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c0c4ec0 con 0x7fae7c155d70 2024-09-17T05:34:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae64005420 con 0x7fae7c155d70 2024-09-17T05:34:32.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.581+0000 7fae78c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 0x7fae58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:32.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.585+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7fae64095a60 con 0x7fae7c155d70 2024-09-17T05:34:32.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.585+0000 7fae7be00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 0x7fae58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:32.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.585+0000 7fae7be00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 0x7fae58075710 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0715c0 tx=0x7fae6c001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:32.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.589+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fae7c15e780 con 0x7fae7c155d70 2024-09-17T05:34:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:32.797+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fae7c0ad0b0 con 0x7fae7c155d70 2024-09-17T05:34:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.389+0000 7fae78c00640 1 -- 172.21.15.43:0/2659542916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v338) ==== 104+0+0 (secure 0 0 0) 0x7fae6405fbe0 con 0x7fae7c155d70 2024-09-17T05:34:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 msgr2=0x7fae58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 0x7fae58075710 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0715c0 tx=0x7fae6c001380 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 msgr2=0x7fae7c0c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e7c0 tx=0x7fae6400ec80 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 shutdown_connections 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fae58073200 0x7fae58075710 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fae7c15de30 0x7fae7c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae7c155d70 0x7fae7c0c4700 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 --2- 172.21.15.43:0/2659542916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae7c06f110 0x7fae7c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 >> 172.21.15.43:0/2659542916 conn(0x7fae7c0b23d0 msgr2=0x7fae7c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 shutdown_connections 2024-09-17T05:34:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.393+0000 7fae82c00640 1 -- 172.21.15.43:0/2659542916 wait complete. 2024-09-17T05:34:33.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- 172.21.15.43:0/1346016895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80cf980 msgr2=0x7f16c80731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 --2- 172.21.15.43:0/1346016895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80cf980 0x7f16c80731d0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009c80 tx=0x7f16c001cda0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- 172.21.15.43:0/1346016895 shutdown_connections 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 --2- 172.21.15.43:0/1346016895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80cf980 0x7f16c80731d0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 --2- 172.21.15.43:0/1346016895 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c8074c60 0x7f16c80b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- 172.21.15.43:0/1346016895 >> 172.21.15.43:0/1346016895 conn(0x7f16c806ce60 msgr2=0x7f16c806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:33.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- 172.21.15.43:0/1346016895 shutdown_connections 2024-09-17T05:34:33.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- 172.21.15.43:0/1346016895 wait complete. 2024-09-17T05:34:33.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 Processor -- start 2024-09-17T05:34:33.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.589+0000 7f16d0600640 1 -- start start 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c815b7e0 0x7f16c8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80ad150 0x7f16c8156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16c8168fa0 con 0x7f16c8074c60 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16c8168e20 con 0x7f16c80ad150 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f16c8169120 con 0x7f16c815b7e0 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45112/0 (socket says 172.21.15.43:45112) 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 -- 172.21.15.43:0/2298623532 learned_addr learned my addr 172.21.15.43:0/2298623532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16ce600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80ad150 0x7f16c8156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cd200640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c815b7e0 0x7f16c8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 -- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c815b7e0 msgr2=0x7f16c8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c815b7e0 0x7f16c8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 -- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80ad150 msgr2=0x7f16c8156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80ad150 0x7f16c8156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 -- 172.21.15.43:0/2298623532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c815e180 con 0x7f16c8074c60 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16cdc00640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d720 tx=0x7f16bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017070 con 0x7f16c8074c60 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16bc009cc0 con 0x7f16c8074c60 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c80afff0 con 0x7f16c8074c60 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc005280 con 0x7f16c8074c60 2024-09-17T05:34:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c80b05b0 con 0x7f16c8074c60 2024-09-17T05:34:33.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c80706a0 con 0x7f16c8074c60 2024-09-17T05:34:33.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.593+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f16bc005420 con 0x7f16c8074c60 2024-09-17T05:34:33.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.597+0000 7f16c5600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 0x7f16a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:33.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.597+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9149+0+0 (secure 0 0 0) 0x7f16bc0954c0 con 0x7f16c8074c60 2024-09-17T05:34:33.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.597+0000 7f16cd200640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 0x7f16a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:33.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.597+0000 7f16cd200640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 0x7f16a8075710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f16c8070390 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:33.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.601+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f16bc05f0f0 con 0x7f16c8074c60 2024-09-17T05:34:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:33.777+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16c80703c0 con 0x7f16c8074c60 2024-09-17T05:34:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.401+0000 7f16c5600640 1 -- 172.21.15.43:0/2298623532 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) ==== 181+0+0 (secure 0 0 0) 0x7f16bc063b20 con 0x7f16c8074c60 2024-09-17T05:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:34:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.405+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 msgr2=0x7f16a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.405+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 0x7f16a8075710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f16c8070390 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2024-09-17T05:34:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.405+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 msgr2=0x7f16c814d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.405+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d720 tx=0x7f16bc00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 shutdown_connections 2024-09-17T05:34:34.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f16a8073200 0x7f16a8075710 secure :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f16c8070390 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2024-09-17T05:34:34.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c80ad150 0x7f16c8156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f16c815b7e0 0x7f16c8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 --2- 172.21.15.43:0/2298623532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c8074c60 0x7f16c814d490 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 >> 172.21.15.43:0/2298623532 conn(0x7f16c806ce60 msgr2=0x7f16c80cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:34.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 shutdown_connections 2024-09-17T05:34:34.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.409+0000 7f16d0600640 1 -- 172.21.15.43:0/2298623532 wait complete. 2024-09-17T05:34:34.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- 172.21.15.43:0/3149647852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c0cf980 msgr2=0x7f574c0b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 --2- 172.21.15.43:0/3149647852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c0cf980 0x7f574c0b3b80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f5738009840 tx=0x7f573801c820 comp rx=0 tx=0).stop 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- 172.21.15.43:0/3149647852 shutdown_connections 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 --2- 172.21.15.43:0/3149647852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c0cf980 0x7f574c0b3b80 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- 172.21.15.43:0/3149647852 >> 172.21.15.43:0/3149647852 conn(0x7f574c06ce60 msgr2=0x7f574c06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- 172.21.15.43:0/3149647852 shutdown_connections 2024-09-17T05:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- 172.21.15.43:0/3149647852 wait complete. 2024-09-17T05:34:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 Processor -- start 2024-09-17T05:34:34.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.605+0000 7f5754000640 1 -- start start 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c15b7e0 0x7f574c14bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 0x7f574c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f574c169fa0 con 0x7f574c074c60 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f574c169e20 con 0x7f574c15b7e0 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f574c16a120 con 0x7f574c14c470 2024-09-17T05:34:34.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48720/0 (socket says 172.21.15.43:48720) 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 -- 172.21.15.43:0/873890615 learned_addr learned my addr 172.21.15.43:0/873890615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5750c00640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c15b7e0 0x7f574c14bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5752000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 0x7f574c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 -- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 msgr2=0x7f574c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 0x7f574c15f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 -- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c15b7e0 msgr2=0x7f574c14bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c15b7e0 0x7f574c14bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 -- 172.21.15.43:0/873890615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c0aee60 con 0x7f574c074c60 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5752000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 0x7f574c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5751600640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f574000c450 tx=0x7f574000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400052d0 con 0x7f574c074c60 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5740005470 con 0x7f574c074c60 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c0af140 con 0x7f574c074c60 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740012870 con 0x7f574c074c60 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c166a60 con 0x7f574c074c60 2024-09-17T05:34:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c15af80 con 0x7f574c074c60 2024-09-17T05:34:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.609+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5740003600 con 0x7f574c074c60 2024-09-17T05:34:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.613+0000 7f5749600640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 0x7f572c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.613+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8711+0+0 (secure 0 0 0) 0x7f57400948b0 con 0x7f574c074c60 2024-09-17T05:34:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.613+0000 7f5750c00640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 0x7f572c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:34.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.613+0000 7f5750c00640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 0x7f572c075710 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57380af000 tx=0x7f573801b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.617+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f574005ecc0 con 0x7f574c074c60 2024-09-17T05:34:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:34.789+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f574c0af310 con 0x7f574c074c60 2024-09-17T05:34:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.405+0000 7f5749600640 1 -- 172.21.15.43:0/873890615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) ==== 169+0+0 (secure 0 0 0) 0x7f574005e5c0 con 0x7f574c074c60 2024-09-17T05:34:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 msgr2=0x7f572c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 0x7f572c075710 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57380af000 tx=0x7f573801b060 comp rx=0 tx=0).stop 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 msgr2=0x7f574c0b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f574000c450 tx=0x7f574000c910 comp rx=0 tx=0).stop 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 shutdown_connections 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f572c073200 0x7f572c075710 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c15b7e0 0x7f574c14bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f574c14c470 0x7f574c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 --2- 172.21.15.43:0/873890615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f574c074c60 0x7f574c0b1620 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.409+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 >> 172.21.15.43:0/873890615 conn(0x7f574c06ce60 msgr2=0x7f574c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.413+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 shutdown_connections 2024-09-17T05:34:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.413+0000 7f5754000640 1 -- 172.21.15.43:0/873890615 wait complete. 2024-09-17T05:34:35.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:34:35.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 -- 172.21.15.43:0/451111816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 msgr2=0x7f13540b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 --2- 172.21.15.43:0/451111816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540b33e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1340009840 tx=0x7f134001c760 comp rx=0 tx=0).stop 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 -- 172.21.15.43:0/451111816 shutdown_connections 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 --2- 172.21.15.43:0/451111816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540b33e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 --2- 172.21.15.43:0/451111816 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1354155d70 0x7f135415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 -- 172.21.15.43:0/451111816 >> 172.21.15.43:0/451111816 conn(0x7f13540b23d0 msgr2=0x7f13540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 -- 172.21.15.43:0/451111816 shutdown_connections 2024-09-17T05:34:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.601+0000 7f135b400640 1 -- 172.21.15.43:0/451111816 wait complete. 2024-09-17T05:34:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 Processor -- start 2024-09-17T05:34:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 -- start start 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13540b3920 0x7f13540cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1354168b10 con 0x7f1354155d70 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1354168990 con 0x7f135406f110 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f135b400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1354168c90 con 0x7f13540b3920 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48722/0 (socket says 172.21.15.43:48722) 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 -- 172.21.15.43:0/1161862988 learned_addr learned my addr 172.21.15.43:0/1161862988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1358a00640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1353e00640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13540b3920 0x7f13540cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 -- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13540b3920 msgr2=0x7f13540cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13540b3920 0x7f13540cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 -- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 msgr2=0x7f13540ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.605+0000 7f1359400640 1 -- 172.21.15.43:0/1161862988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541f9030 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1359400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c760 tx=0x7f134c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c003b80 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f134c003d20 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c00aab0 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541f9310 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13540cd000 con 0x7f1354155d70 2024-09-17T05:34:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13541f8da0 con 0x7f1354155d70 2024-09-17T05:34:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f134c00ac50 con 0x7f1354155d70 2024-09-17T05:34:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 0x7f1334075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.609+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7f134c0949a0 con 0x7f1354155d70 2024-09-17T05:34:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.613+0000 7f1358a00640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 0x7f1334075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.613+0000 7f1358a00640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 0x7f1334075710 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f13480096c0 tx=0x7f1348009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:35.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.617+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13540b3d30 con 0x7f1354155d70 2024-09-17T05:34:35.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.789+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f13540b33e0 con 0x7f1354155d70 2024-09-17T05:34:35.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.793+0000 7f1350c00640 1 -- 172.21.15.43:0/1161862988 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f134c05e900 con 0x7f1354155d70 2024-09-17T05:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 msgr2=0x7f1334075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 0x7f1334075710 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f13480096c0 tx=0x7f1348009260 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 msgr2=0x7f1354154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c760 tx=0x7f134c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 shutdown_connections 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1334073200 0x7f1334075710 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1354155d70 0x7f1354154d60 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f13540b3920 0x7f13540cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 --2- 172.21.15.43:0/1161862988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f135406f110 0x7f13540ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 >> 172.21.15.43:0/1161862988 conn(0x7f13540b23d0 msgr2=0x7f135406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 shutdown_connections 2024-09-17T05:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.797+0000 7f135b400640 1 -- 172.21.15.43:0/1161862988 wait complete. 2024-09-17T05:34:35.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-17T05:34:35.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.973+0000 7ffbe2000640 1 -- 172.21.15.43:0/2585483220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc0b3af0 msgr2=0x7ffbdc15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.973+0000 7ffbe2000640 1 --2- 172.21.15.43:0/2585483220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc0b3af0 0x7ffbdc15c7e0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009840 tx=0x7ffbc401c790 comp rx=0 tx=0).stop 2024-09-17T05:34:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.973+0000 7ffbe2000640 1 -- 172.21.15.43:0/2585483220 shutdown_connections 2024-09-17T05:34:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.973+0000 7ffbe2000640 1 --2- 172.21.15.43:0/2585483220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc0b3af0 0x7ffbdc15c7e0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.973+0000 7ffbe2000640 1 -- 172.21.15.43:0/2585483220 >> 172.21.15.43:0/2585483220 conn(0x7ffbdc0b23d0 msgr2=0x7ffbdc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- 172.21.15.43:0/2585483220 shutdown_connections 2024-09-17T05:34:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- 172.21.15.43:0/2585483220 wait complete. 2024-09-17T05:34:35.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 Processor -- start 2024-09-17T05:34:35.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- start start 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc06f110 0x7ffbdc166e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc0c7b60 0x7ffbdc074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc169d70 con 0x7ffbdc155d70 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc169bf0 con 0x7ffbdc0c7b60 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbe2000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc169ef0 con 0x7ffbdc06f110 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.977+0000 7ffbd2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48736/0 (socket says 172.21.15.43:48736) 2024-09-17T05:34:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 -- 172.21.15.43:0/1931613163 learned_addr learned my addr 172.21.15.43:0/1931613163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbdbe00640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc0c7b60 0x7ffbdc074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbdb400640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc06f110 0x7ffbdc166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 -- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc06f110 msgr2=0x7ffbdc166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc06f110 0x7ffbdc166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 -- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc0c7b60 msgr2=0x7ffbdc074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc0c7b60 0x7ffbdc074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 -- 172.21.15.43:0/1931613163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc0c3d30 con 0x7ffbdc155d70 2024-09-17T05:34:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd2a00640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ffbc401cc70 tx=0x7ffbc4005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40b2040 con 0x7ffbdc155d70 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbc4004040 con 0x7ffbdc155d70 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc0cbb00 con 0x7ffbdc155d70 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4004410 con 0x7ffbdc155d70 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc0cc0c0 con 0x7ffbdc155d70 2024-09-17T05:34:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc15c7e0 con 0x7ffbdc155d70 2024-09-17T05:34:35.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.981+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffbc40bd030 con 0x7ffbdc155d70 2024-09-17T05:34:35.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.985+0000 7ffbd3e00640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 0x7ffbb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:35.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.985+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7ffbc412ef30 con 0x7ffbdc155d70 2024-09-17T05:34:35.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.985+0000 7ffbdb400640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 0x7ffbb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:35.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.985+0000 7ffbdb400640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 0x7ffbb8075710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0715c0 tx=0x7ffbcc001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:35.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:35.985+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffbc415ddb0 con 0x7ffbdc155d70 2024-09-17T05:34:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.225+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7ffbdc0c8060 con 0x7ffbdc155d70 2024-09-17T05:34:36.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbd3e00640 1 -- 172.21.15.43:0/1931613163 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v582) ==== 135+0+0 (secure 0 0 0) 0x7ffbc40c4060 con 0x7ffbdc155d70 2024-09-17T05:34:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 msgr2=0x7ffbb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 0x7ffbb8075710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0715c0 tx=0x7ffbcc001380 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 msgr2=0x7ffbdc073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ffbc401cc70 tx=0x7ffbc4005ca0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 shutdown_connections 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffbb8073200 0x7ffbb8075710 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbdc0c7b60 0x7ffbdc074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbdc155d70 0x7ffbdc073f50 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 --2- 172.21.15.43:0/1931613163 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffbdc06f110 0x7ffbdc166e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 >> 172.21.15.43:0/1931613163 conn(0x7ffbdc0b23d0 msgr2=0x7ffbdc06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 shutdown_connections 2024-09-17T05:34:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.417+0000 7ffbe2000640 1 -- 172.21.15.43:0/1931613163 wait complete. 2024-09-17T05:34:36.443 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-09-17T05:34:36.444 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-09-17T05:34:36.444 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-17T05:34:36.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- 172.21.15.43:0/2187570878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad400b3b10 msgr2=0x7fad400b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 --2- 172.21.15.43:0/2187570878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad400b3b10 0x7fad400b3f20 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fad400752a0 tx=0x7fad3801cc60 comp rx=0 tx=0).stop 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- 172.21.15.43:0/2187570878 shutdown_connections 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 --2- 172.21.15.43:0/2187570878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad400b3b10 0x7fad400b3f20 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 --2- 172.21.15.43:0/2187570878 >> v2:172.21.15.62:3300/0 conn(0x7fad400b5610 0x7fad400b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 --2- 172.21.15.43:0/2187570878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad4015f470 0x7fad400d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- 172.21.15.43:0/2187570878 >> 172.21.15.43:0/2187570878 conn(0x7fad400b2810 msgr2=0x7fad400b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- 172.21.15.43:0/2187570878 shutdown_connections 2024-09-17T05:34:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- 172.21.15.43:0/2187570878 wait complete. 2024-09-17T05:34:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 Processor -- start 2024-09-17T05:34:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.609+0000 7fad45c00640 1 -- start start 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad400b3b10 0x7fad40143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad400b5610 0x7fad40152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fad40070940 con 0x7fad4015f470 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad400707c0 con 0x7fad400b3b10 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fad40070ac0 con 0x7fad400b5610 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad37e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad400b5610 0x7fad40152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48758/0 (socket says 172.21.15.43:48758) 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 -- 172.21.15.43:0/3927416373 learned_addr learned my addr 172.21.15.43:0/3927416373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3ea00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad400b3b10 0x7fad40143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 -- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad400b5610 msgr2=0x7fad40152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad400b5610 0x7fad40152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 -- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad400b3b10 msgr2=0x7fad40143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad400b3b10 0x7fad40143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 -- 172.21.15.43:0/3927416373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401f9490 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad3f400640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fad3801b310 tx=0x7fad380a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad380bb070 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad38004d40 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad380b2040 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401f9770 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401437c0 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.613+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad40070f50 con 0x7fad4015f470 2024-09-17T05:34:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.617+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad38004320 con 0x7fad4015f470 2024-09-17T05:34:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.617+0000 7fad36a00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 0x7fad1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.617+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7fad3812ed30 con 0x7fad4015f470 2024-09-17T05:34:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.617+0000 7fad3ea00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 0x7fad1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.617+0000 7fad3ea00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 0x7fad1c075710 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fad30005fd0 tx=0x7fad3000a2d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.621+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad38016970 con 0x7fad4015f470 2024-09-17T05:34:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.797+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad400765d0 con 0x7fad4015f470 2024-09-17T05:34:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.797+0000 7fad36a00640 1 -- 172.21.15.43:0/3927416373 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+18030 (secure 0 0 0) 0x7fad380f8d50 con 0x7fad4015f470 2024-09-17T05:34:36.801 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:36.802 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":131,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:35.407232+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6833/1613623362":"2024-09-18T05:34:18.272911+0000","172.21.15.43:6835/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6835/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6833/2696141215":"2024-09-18T05:33:44.705848+0000","172.21.15.43:6835/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.43:6835/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6833/3762350799":"2024-09-18T05:33:16.473578+0000","172.21.15.43:6835/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.43:6833/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.62:6834/647815137":"2024-09-18T05:32:57.599675+0000","172.21.15.43:6835/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.62:6834/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6834/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.43:6833/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.43:6833/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.62:6834/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.62:6834/3644279875":"2024-09-18T05:30:58.496775+0000","172.21.15.62:6835/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.62:6834/2775198885":"2024-09-18T05:30:52.367787+0000","172.21.15.43:6835/4105315924":"2024-09-18T05:34:06.237412+0000","172.21.15.62:6834/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6833/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.43:6833/279176341":"2024-09-18T05:31:05.541294+0000","172.21.15.62:6835/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6833/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.43:6833/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6835/273818866":"2024-09-18T05:30:08.113534+0000","172.21.15.43:6833/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6833/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.43:6833/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.43:6835/3640890143":"2024-09-18T05:30:28.229649+0000","172.21.15.62:6835/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.43:6835/1587862715":"2024-09-18T05:31:49.459141+0000","172.21.15.43:6835/1981693955":"2024-09-18T05:30:13.152555+0000","172.21.15.62:6834/3277842592":"2024-09-18T05:30:41.396242+0000","172.21.15.43:6835/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.43:6835/1578591246":"2024-09-18T05:31:27.260942+0000","172.21.15.43:6835/352682704":"2024-09-18T05:30:54.467138+0000","172.21.15.43:6833/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6833/2023604015":"2024-09-18T05:31:41.375774+0000","172.21.15.43:6835/274766899":"2024-09-18T05:29:52.620716+0000","172.21.15.43:6833/2308652168":"2024-09-18T05:34:11.271966+0000","172.21.15.43:6833/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6834/1590794613":"2024-09-18T05:30:24.225848+0000","172.21.15.62:6835/1077468406":"2024-09-18T05:31:34.313578+0000","172.21.15.43:6835/3611976391":"2024-09-18T05:30:20.206882+0000","172.21.15.62:6835/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.62:6834/3412634924":"2024-09-18T05:32:49.498519+0000","172.21.15.43:6833/1507988053":"2024-09-18T05:33:33.376586+0000","172.21.15.62:6835/2584621842":"2024-09-18T05:31:45.414409+0000","172.21.15.43:6835/3742224456":"2024-09-18T05:32:06.064785+0000","172.21.15.62:6834/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.43:6833/2992252190":"2024-09-18T05:31:12.052972+0000","172.21.15.62:6835/1306620592":"2024-09-18T05:32:10.088635+0000","172.21.15.62:6835/3935849352":"2024-09-18T05:34:25.365036+0000","172.21.15.43:6835/324676924":"2024-09-18T05:32:14.126639+0000","172.21.15.43:6835/3094176220":"2024-09-18T05:33:50.112138+0000","172.21.15.62:6835/2196052200":"2024-09-18T05:32:03.861884+0000","172.21.15.62:6834/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.62:6835/2994345600":"2024-09-18T05:31:25.236711+0000","172.21.15.62:6835/2407777034":"2024-09-18T05:32:18.163569+0000","172.21.15.43:6833/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/3572479007":"2024-09-18T05:32:22.223422+0000","172.21.15.43:6835/1404316783":"2024-09-18T05:32:30.299996+0000","172.21.15.62:6835/3432953722":"2024-09-18T05:32:26.259793+0000","172.21.15.43:6833/1653179583":"2024-09-18T05:32:53.543785+0000","172.21.15.43:6833/351071204":"2024-09-18T05:33:11.430207+0000","172.21.15.62:6834/2354591248":"2024-09-18T05:32:43.445225+0000","172.21.15.43:6833/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.43:6835/3812598362":"2024-09-18T05:32:45.466679+0000","172.21.15.62:6835/3412634924":"2024-09-18T05:32:49.498519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:36.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 msgr2=0x7fad1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 0x7fad1c075710 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fad30005fd0 tx=0x7fad3000a2d0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 msgr2=0x7fad401f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fad3801b310 tx=0x7fad380a5a00 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 shutdown_connections 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fad1c073200 0x7fad1c075710 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4015f470 0x7fad401f8940 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fad400b5610 0x7fad40152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 --2- 172.21.15.43:0/3927416373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad400b3b10 0x7fad40143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 >> 172.21.15.43:0/3927416373 conn(0x7fad400b2810 msgr2=0x7fad400b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 shutdown_connections 2024-09-17T05:34:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.801+0000 7fad45c00640 1 -- 172.21.15.43:0/3927416373 wait complete. 2024-09-17T05:34:36.824 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:23:13.868734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:34:36.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 -- 172.21.15.43:0/3994090466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53780b3920 msgr2=0x7f53780ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 --2- 172.21.15.43:0/3994090466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53780b3920 0x7f53780ad0b0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f53780711f0 tx=0x7f537001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 -- 172.21.15.43:0/3994090466 shutdown_connections 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 --2- 172.21.15.43:0/3994090466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53780b3920 0x7f53780ad0b0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 --2- 172.21.15.43:0/3994090466 >> v2:172.21.15.62:3300/0 conn(0x7f537806f110 0x7f53780b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 --2- 172.21.15.43:0/3994090466 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5378155d70 0x7f537815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 -- 172.21.15.43:0/3994090466 >> 172.21.15.43:0/3994090466 conn(0x7f53780b23d0 msgr2=0x7f53780d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 -- 172.21.15.43:0/3994090466 shutdown_connections 2024-09-17T05:34:36.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.981+0000 7f537d400640 1 -- 172.21.15.43:0/3994090466 wait complete. 2024-09-17T05:34:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 Processor -- start 2024-09-17T05:34:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 -- start start 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 0x7f53780ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 0x7f537809ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f537815f8d0 con 0x7f5378155d70 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f537815f750 con 0x7f53780b3920 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f537815fa50 con 0x7f537806f110 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48780/0 (socket says 172.21.15.43:48780) 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 -- 172.21.15.43:0/844385920 learned_addr learned my addr 172.21.15.43:0/844385920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5376800640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 0x7f53780ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 -- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 msgr2=0x7f53780ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f536de00640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 0x7f537809ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 0x7f53780ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 -- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 msgr2=0x7f537809ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 0x7f537809ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 -- 172.21.15.43:0/844385920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537809af60 con 0x7f5378155d70 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5376800640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 0x7f53780ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f536de00640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 0x7f537809ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f5377200640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f5370009bd0 tx=0x7f5370003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53700bb070 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5370004b40 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53700b2040 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.985+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537809a860 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53780cc590 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537815ff20 con 0x7f5378155d70 2024-09-17T05:34:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5370003fd0 con 0x7f5378155d70 2024-09-17T05:34:36.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f536f400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 0x7f5354075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:36.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7f537012ed90 con 0x7f5378155d70 2024-09-17T05:34:36.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.989+0000 7f5376800640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 0x7f5354075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:36.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.993+0000 7f5376800640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 0x7f5354075740 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53780715c0 tx=0x7f53680091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:36.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:36.993+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53700c4020 con 0x7f5378155d70 2024-09-17T05:34:37.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.169+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53780af7a0 con 0x7f5378155d70 2024-09-17T05:34:37.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f536f400640 1 -- 172.21.15.43:0/844385920 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) ==== 93+0+10 (secure 0 0 0) 0x7f5370016970 con 0x7f5378155d70 2024-09-17T05:34:37.176 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 msgr2=0x7f5354075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 0x7f5354075740 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53780715c0 tx=0x7f53680091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 msgr2=0x7f5378154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f5370009bd0 tx=0x7f5370003f60 comp rx=0 tx=0).stop 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 shutdown_connections 2024-09-17T05:34:37.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5354073230 0x7f5354075740 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53780b3920 0x7f537809ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5378155d70 0x7f5378154eb0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 --2- 172.21.15.43:0/844385920 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f537806f110 0x7f53780ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 >> 172.21.15.43:0/844385920 conn(0x7f53780b23d0 msgr2=0x7f537809d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 shutdown_connections 2024-09-17T05:34:37.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.173+0000 7f537d400640 1 -- 172.21.15.43:0/844385920 wait complete. 2024-09-17T05:34:37.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 -- 172.21.15.43:0/3084060320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa006fc90 msgr2=0x7fefa00700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 --2- 172.21.15.43:0/3084060320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa006fc90 0x7fefa00700a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009900 tx=0x7fef8c01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 -- 172.21.15.43:0/3084060320 shutdown_connections 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 --2- 172.21.15.43:0/3084060320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa006fc90 0x7fefa00700a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 --2- 172.21.15.43:0/3084060320 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa015fe10 0x7fefa00b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 -- 172.21.15.43:0/3084060320 >> 172.21.15.43:0/3084060320 conn(0x7fefa00b23d0 msgr2=0x7fefa00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 -- 172.21.15.43:0/3084060320 shutdown_connections 2024-09-17T05:34:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.353+0000 7fefa8200640 1 -- 172.21.15.43:0/3084060320 wait complete. 2024-09-17T05:34:37.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 Processor -- start 2024-09-17T05:34:37.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 -- start start 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa006fc90 0x7fefa015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 0x7fefa01f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fefa016a020 con 0x7fefa00730b0 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefa0169ea0 con 0x7fefa015fe10 2024-09-17T05:34:37.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa8200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fefa016a1a0 con 0x7fefa006fc90 2024-09-17T05:34:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48786/0 (socket says 172.21.15.43:48786) 2024-09-17T05:34:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 -- 172.21.15.43:0/3640528993 learned_addr learned my addr 172.21.15.43:0/3640528993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa5800640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa006fc90 0x7fefa015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa6200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 0x7fefa01f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 -- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa006fc90 msgr2=0x7fefa015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa006fc90 0x7fefa015a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 -- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 msgr2=0x7fefa01f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 0x7fefa01f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 -- 172.21.15.43:0/3640528993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00764f0 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa6200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 0x7fefa01f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.357+0000 7fefa4e00640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009850 tx=0x7fef8c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0b2040 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef8c004330 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c005290 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa0153a50 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa015e230 con 0x7fefa00730b0 2024-09-17T05:34:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa00700a0 con 0x7fefa00730b0 2024-09-17T05:34:37.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fef8c0bd060 con 0x7fefa00730b0 2024-09-17T05:34:37.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 0x7fef80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:37.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7fef8c12e4c0 con 0x7fefa00730b0 2024-09-17T05:34:37.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.361+0000 7fefa5800640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 0x7fef80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:37.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.365+0000 7fefa5800640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 0x7fef80075710 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef94009a10 tx=0x7fef940091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:37.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.365+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef8c0164d0 con 0x7fefa00730b0 2024-09-17T05:34:37.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:37.605+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7fefa00cc5e0 con 0x7fefa00730b0 2024-09-17T05:34:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.425+0000 7fef9d600640 1 -- 172.21.15.43:0/3640528993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v584) ==== 138+0+0 (secure 0 0 0) 0x7fef8c0f8a10 con 0x7fefa00730b0 2024-09-17T05:34:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.425+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 msgr2=0x7fef80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.425+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 0x7fef80075710 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef94009a10 tx=0x7fef940091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.425+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 msgr2=0x7fefa00997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.425+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009850 tx=0x7fef8c005d60 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 shutdown_connections 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fef80073200 0x7fef80075710 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa015fe10 0x7fefa01f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefa00730b0 0x7fefa00997d0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 --2- 172.21.15.43:0/3640528993 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fefa006fc90 0x7fefa015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 >> 172.21.15.43:0/3640528993 conn(0x7fefa00b23d0 msgr2=0x7fefa0074170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 shutdown_connections 2024-09-17T05:34:38.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.429+0000 7fefa8200640 1 -- 172.21.15.43:0/3640528993 wait complete. 2024-09-17T05:34:38.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:34:38.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2552657427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b48155d70 msgr2=0x7f1b4815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2552657427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b48155d70 0x7f1b4815c7e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009840 tx=0x7f1b3801c790 comp rx=0 tx=0).stop 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2552657427 shutdown_connections 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2552657427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b48155d70 0x7f1b4815c7e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2552657427 >> 172.21.15.43:0/2552657427 conn(0x7f1b480b23d0 msgr2=0x7f1b480d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2552657427 shutdown_connections 2024-09-17T05:34:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2552657427 wait complete. 2024-09-17T05:34:38.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 Processor -- start 2024-09-17T05:34:38.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.617+0000 7f1b4ec00640 1 -- start start 2024-09-17T05:34:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 0x7f1b4809c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 0x7f1b481faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48169d70 con 0x7f1b48155d70 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48169bf0 con 0x7f1b4806f110 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1b48169ef0 con 0x7f1b480cd7d0 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48802/0 (socket says 172.21.15.43:48802) 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 -- 172.21.15.43:0/2679629403 learned_addr learned my addr 172.21.15.43:0/2679629403 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b47e00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 0x7f1b4809c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4cc00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 0x7f1b481faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 -- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 msgr2=0x7f1b481faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 0x7f1b481faa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 -- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 msgr2=0x7f1b4809c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 0x7f1b4809c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 -- 172.21.15.43:0/2679629403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b481fb110 con 0x7f1b48155d70 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4cc00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 0x7f1b481faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b47e00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 0x7f1b4809c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b3f400640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c3c0 tx=0x7f1b3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30011790 con 0x7f1b48155d70 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4809cde0 con 0x7f1b48155d70 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b30017070 con 0x7f1b48155d70 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30011cc0 con 0x7f1b48155d70 2024-09-17T05:34:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b480cb8a0 con 0x7f1b48155d70 2024-09-17T05:34:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.621+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4815c7e0 con 0x7f1b48155d70 2024-09-17T05:34:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.625+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1b3001c460 con 0x7f1b48155d70 2024-09-17T05:34:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.625+0000 7f1b44c00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 0x7f1b24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.625+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7f1b30094ab0 con 0x7f1b48155d70 2024-09-17T05:34:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.625+0000 7f1b47e00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 0x7f1b24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.625+0000 7f1b47e00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 0x7f1b24075710 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1b380af000 tx=0x7f1b3801afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:38.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.629+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1b3005f0a0 con 0x7f1b48155d70 2024-09-17T05:34:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.805+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1b4815ba60 con 0x7f1b48155d70 2024-09-17T05:34:38.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.805+0000 7f1b44c00640 1 -- 172.21.15.43:0/2679629403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7f1b3005e9a0 con 0x7f1b48155d70 2024-09-17T05:34:38.810 INFO:teuthology.orchestra.run.smithi043.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:34:38.810 INFO:teuthology.orchestra.run.smithi043.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:34:38.810 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:34:38.810 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_client 20/20 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3321.045932 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3468.163391 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3218.669447 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3236.289304 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4289.724541 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4170.587616 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5232.119320 2024-09-17T05:34:38.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4223.597904 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds 20/20 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced debug_ms 2/2 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_frag true 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds dev mds_verify_scatter true 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced client_mount_timeout 600 2024-09-17T05:34:38.812 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_client 20/20 2024-09-17T05:34:38.813 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced debug_ms 1/1 2024-09-17T05:34:38.813 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:34:38.813 INFO:teuthology.orchestra.run.smithi043.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:34:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 msgr2=0x7f1b24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 0x7f1b24075710 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1b380af000 tx=0x7f1b3801afd0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 msgr2=0x7f1b481fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c3c0 tx=0x7f1b3000c880 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 shutdown_connections 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1b24073200 0x7f1b24075710 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1b480cd7d0 0x7f1b481faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b48155d70 0x7f1b481fa540 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 --2- 172.21.15.43:0/2679629403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b4806f110 0x7f1b4809c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 >> 172.21.15.43:0/2679629403 conn(0x7f1b480b23d0 msgr2=0x7f1b480b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 shutdown_connections 2024-09-17T05:34:38.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.809+0000 7f1b4ec00640 1 -- 172.21.15.43:0/2679629403 wait complete. 2024-09-17T05:34:38.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 -- 172.21.15.43:0/192285478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 msgr2=0x7f858806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 --2- 172.21.15.43:0/192285478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f858806f110 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f8580009bc0 tx=0x7f858001cb10 comp rx=0 tx=0).stop 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 -- 172.21.15.43:0/192285478 shutdown_connections 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 --2- 172.21.15.43:0/192285478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f858806f110 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 -- 172.21.15.43:0/192285478 >> 172.21.15.43:0/192285478 conn(0x7f85880b23d0 msgr2=0x7f85880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 -- 172.21.15.43:0/192285478 shutdown_connections 2024-09-17T05:34:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.989+0000 7f858cc00640 1 -- 172.21.15.43:0/192285478 wait complete. 2024-09-17T05:34:38.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 Processor -- start 2024-09-17T05:34:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 -- start start 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8588155d70 0x7f8588153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 0x7f8588154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8588169e60 con 0x7f858815c7e0 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8588169ce0 con 0x7f858809db10 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f858cc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8588169fe0 con 0x7f8588155d70 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48812/0 (socket says 172.21.15.43:48812) 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 -- 172.21.15.43:0/3725340364 learned_addr learned my addr 172.21.15.43:0/3725340364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:39.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f8586800640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 0x7f8588154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f8585e00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8588155d70 0x7f8588153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 -- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8588155d70 msgr2=0x7f8588153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8588155d70 0x7f8588153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 -- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 msgr2=0x7f8588154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 0x7f8588154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f857d400640 1 -- 172.21.15.43:0/3725340364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880cd840 con 0x7f858815c7e0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.993+0000 7f8586800640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 0x7f8588154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857d400640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f857000c3c0 tx=0x7f857000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570011790 con 0x7f858815c7e0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8570017070 con 0x7f858815c7e0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85880731c0 con 0x7f858815c7e0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570011ce0 con 0x7f858815c7e0 2024-09-17T05:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881fc900 con 0x7f858815c7e0 2024-09-17T05:34:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85881fcbe0 con 0x7f858815c7e0 2024-09-17T05:34:39.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f857001c460 con 0x7f858815c7e0 2024-09-17T05:34:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 0x7f8564075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:38.997+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7f85700949f0 con 0x7f858815c7e0 2024-09-17T05:34:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.001+0000 7f8585e00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 0x7f8564075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.001+0000 7f8585e00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 0x7f8564075740 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f85880715c0 tx=0x7f8578008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:39.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.001+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f857005efd0 con 0x7f858815c7e0 2024-09-17T05:34:39.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.177+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f858806f110 con 0x7f858815c7e0 2024-09-17T05:34:39.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.181+0000 7f857ea00640 1 -- 172.21.15.43:0/3725340364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f857005e8d0 con 0x7f858815c7e0 2024-09-17T05:34:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 msgr2=0x7f8564075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 0x7f8564075740 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f85880715c0 tx=0x7f8578008040 comp rx=0 tx=0).stop 2024-09-17T05:34:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 msgr2=0x7f8588154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f857000c3c0 tx=0x7f857000c880 comp rx=0 tx=0).stop 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 shutdown_connections 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8564073230 0x7f8564075740 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f858809db10 0x7f8588154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858815c7e0 0x7f8588154410 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 --2- 172.21.15.43:0/3725340364 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8588155d70 0x7f8588153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 >> 172.21.15.43:0/3725340364 conn(0x7f85880b23d0 msgr2=0x7f858815f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 shutdown_connections 2024-09-17T05:34:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.185+0000 7f858cc00640 1 -- 172.21.15.43:0/3725340364 wait complete. 2024-09-17T05:34:39.208 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:34:39.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:39.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:39.398 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:34:39.398 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:39.398 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:39.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:39.407 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:34:39.407 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:39.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:39.417 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:39.417 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:39.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:39.459 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:34:39.459 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:39.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:39.469 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:34:39.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 -- 172.21.15.43:0/4008152480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 msgr2=0x7f04d40cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 --2- 172.21.15.43:0/4008152480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d40cd750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009bc0 tx=0x7f04cc01cba0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 -- 172.21.15.43:0/4008152480 shutdown_connections 2024-09-17T05:34:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 --2- 172.21.15.43:0/4008152480 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d40cd750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 --2- 172.21.15.43:0/4008152480 >> v2:172.21.15.62:3300/0 conn(0x7f04d409ba60 0x7f04d40b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 -- 172.21.15.43:0/4008152480 >> 172.21.15.43:0/4008152480 conn(0x7f04d40cee60 msgr2=0x7f04d40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 -- 172.21.15.43:0/4008152480 shutdown_connections 2024-09-17T05:34:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.657+0000 7f04d9800640 1 -- 172.21.15.43:0/4008152480 wait complete. 2024-09-17T05:34:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 Processor -- start 2024-09-17T05:34:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- start start 2024-09-17T05:34:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 0x7f04d415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04d40b2bb0 con 0x7f04d409ba60 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04d40b2a30 con 0x7f04d40d0240 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f04d4169e30 con 0x7f04d40d0650 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48822/0 (socket says 172.21.15.43:48822) 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 -- 172.21.15.43:0/1697382905 learned_addr learned my addr 172.21.15.43:0/1697382905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04ca000640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 0x7f04d415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d3400640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 -- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 msgr2=0x7f04d41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d41555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 -- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 msgr2=0x7f04d415f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 0x7f04d415f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 -- 172.21.15.43:0/1697382905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d40b06a0 con 0x7f04d409ba60 2024-09-17T05:34:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04ca000640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 0x7f04d415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d2a00640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9d0 tx=0x7f04c400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c400cde0 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04c4004020 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d40b0980 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004470 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.661+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41507e0 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.665+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d40b3130 con 0x7f04d409ba60 2024-09-17T05:34:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.665+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04c4003600 con 0x7f04d409ba60 2024-09-17T05:34:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.665+0000 7f04cb400640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 0x7f04b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.665+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7f04c4095520 con 0x7f04d409ba60 2024-09-17T05:34:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.665+0000 7f04ca000640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 0x7f04b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.669+0000 7f04ca000640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 0x7f04b0075740 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b02d0 tx=0x7f04bc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.669+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04c405fb10 con 0x7f04d409ba60 2024-09-17T05:34:39.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04d41576d0 con 0x7f04d409ba60 2024-09-17T05:34:39.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04cb400640 1 -- 172.21.15.43:0/1697382905 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 338 v338) ==== 95+0+3909 (secure 0 0 0) 0x7f04c405f410 con 0x7f04d409ba60 2024-09-17T05:34:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:39.884 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":338,"btime":"2024-09-17T05:34:33:392669+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/32397943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":32397943},{"type":"v1","addr":"172.21.15.62:6837","nonce":32397943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2554782422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2554782422},{"type":"v1","addr":"172.21.15.43:6834","nonce":2554782422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1090667792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1090667792},{"type":"v1","addr":"172.21.15.62:6835","nonce":1090667792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":10326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4060420802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4060420802},{"type":"v1","addr":"172.21.15.43:6835","nonce":4060420802}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[]} 2024-09-17T05:34:39.884 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 338 2024-09-17T05:34:39.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 msgr2=0x7f04b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 0x7f04b0075740 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b02d0 tx=0x7f04bc009040 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 msgr2=0x7f04d414f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f04c400e9d0 tx=0x7f04c400ee90 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 shutdown_connections 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f04b0073230 0x7f04b0075740 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04d40d0240 0x7f04d415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f04d40d0650 0x7f04d41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 --2- 172.21.15.43:0/1697382905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d409ba60 0x7f04d414f080 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 >> 172.21.15.43:0/1697382905 conn(0x7f04d40cee60 msgr2=0x7f04d40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 shutdown_connections 2024-09-17T05:34:39.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:39.881+0000 7f04d9800640 1 -- 172.21.15.43:0/1697382905 wait complete. 2024-09-17T05:34:39.907 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:34:39.907 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:34:39.907 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:34:39.907 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:39.908 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:34:39.908 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:34:39.908 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:34:39.908 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:39.908 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:34:39.908 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:34:39.908 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:34:39.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:39.909 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:34:39.909 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:34:39.909 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:34:39.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:39.919 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:34:39.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:34:39.923 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:34:39.923 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:34:39.930 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:34:39.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:34:39.933 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:34:39.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:34:39.977 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:34:39.978 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:39.980 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:34:39.980 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:39.981 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:34:39.982 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:39.984 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:34:39.984 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:39.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:34:40.077 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:34:40.079 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-09-17T05:34:40.092 INFO:tasks.ceph.mds.b.smithi062.stdout:starting mds.b at 2024-09-17T05:34:40.092 INFO:tasks.ceph.mds.d.smithi062.stdout:starting mds.d at 2024-09-17T05:34:40.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T05:34:40.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T05:34:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- 172.21.15.43:0/2057330985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 msgr2=0x7fab2c167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 --2- 172.21.15.43:0/2057330985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c167c80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fab24009bc0 tx=0x7fab2401cba0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- 172.21.15.43:0/2057330985 shutdown_connections 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 --2- 172.21.15.43:0/2057330985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c167c80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- 172.21.15.43:0/2057330985 >> 172.21.15.43:0/2057330985 conn(0x7fab2c0cee80 msgr2=0x7fab2c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- 172.21.15.43:0/2057330985 shutdown_connections 2024-09-17T05:34:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- 172.21.15.43:0/2057330985 wait complete. 2024-09-17T05:34:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 Processor -- start 2024-09-17T05:34:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.149+0000 7fab33a00640 1 -- start start 2024-09-17T05:34:40.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2c0d0f50 0x7fab2c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c16af80 con 0x7fab2c0b7ce0 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c16ae00 con 0x7fab2c0cd580 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fab2c16b100 con 0x7fab2c0d0f50 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab31000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab31000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48862/0 (socket says 172.21.15.43:48862) 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab31000640 1 -- 172.21.15.43:0/963512010 learned_addr learned my addr 172.21.15.43:0/963512010 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:57262/0 (socket says 172.21.15.43:57262) 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab31a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2c0d0f50 0x7fab2c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 -- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2c0d0f50 msgr2=0x7fab2c154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2c0d0f50 0x7fab2c154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 -- 172.21.15.43:0/963512010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 msgr2=0x7fab2c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 -- 172.21.15.43:0/963512010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c159d80 con 0x7fab2c0cd580 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab31000640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab2be00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fab2c074250 tx=0x7fab1800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18011790 con 0x7fab2c0cd580 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab18017070 con 0x7fab2c0cd580 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c15ebf0 con 0x7fab2c0cd580 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18011ce0 con 0x7fab2c0cd580 2024-09-17T05:34:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c15f1b0 con 0x7fab2c0cd580 2024-09-17T05:34:40.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.153+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c0d0840 con 0x7fab2c0cd580 2024-09-17T05:34:40.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.157+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fab1801c460 con 0x7fab2c0cd580 2024-09-17T05:34:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.157+0000 7fab28c00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 0x7fab0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.157+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8297+0+0 (secure 0 0 0) 0x7fab18094820 con 0x7fab2c0cd580 2024-09-17T05:34:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.157+0000 7fab31000640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 0x7fab0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.157+0000 7fab31000640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 0x7fab0c075710 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fab20000c10 tx=0x7fab20009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:40.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.161+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fab1805e730 con 0x7fab2c0cd580 2024-09-17T05:34:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.349+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fab2c0aff50 con 0x7fab2c0cd580 2024-09-17T05:34:40.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.441+0000 7fab28c00640 1 -- 172.21.15.43:0/963512010 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) ==== 97+0+0 (secure 0 0 0) 0x7fab18063160 con 0x7fab2c0cd580 2024-09-17T05:34:40.446 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 msgr2=0x7fab0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 0x7fab0c075710 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fab20000c10 tx=0x7fab20009040 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 msgr2=0x7fab2c14f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fab2c074250 tx=0x7fab1800c820 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 shutdown_connections 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fab0c073200 0x7fab0c075710 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab2c0cd580 0x7fab2c14f150 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fab2c0d0f50 0x7fab2c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 --2- 172.21.15.43:0/963512010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c0b7ce0 0x7fab2c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 >> 172.21.15.43:0/963512010 conn(0x7fab2c0cee80 msgr2=0x7fab2c0b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 shutdown_connections 2024-09-17T05:34:40.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.445+0000 7fab33a00640 1 -- 172.21.15.43:0/963512010 wait complete. 2024-09-17T05:34:40.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:34:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 -- 172.21.15.43:0/3758592620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25406f110 msgr2=0x7ff2540ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 --2- 172.21.15.43:0/3758592620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25406f110 0x7ff2540ad0b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009c80 tx=0x7ff24c01cce0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 -- 172.21.15.43:0/3758592620 shutdown_connections 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 --2- 172.21.15.43:0/3758592620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff25406f110 0x7ff2540ad0b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 --2- 172.21.15.43:0/3758592620 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff2540b3af0 0x7ff25415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 -- 172.21.15.43:0/3758592620 >> 172.21.15.43:0/3758592620 conn(0x7ff2540b23d0 msgr2=0x7ff2540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 -- 172.21.15.43:0/3758592620 shutdown_connections 2024-09-17T05:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.625+0000 7ff258c00640 1 -- 172.21.15.43:0/3758592620 wait complete. 2024-09-17T05:34:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 Processor -- start 2024-09-17T05:34:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- start start 2024-09-17T05:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 0x7ff2540cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25409d760 0x7ff25409db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff254168c80 con 0x7ff254155d70 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff254168b00 con 0x7ff25406f110 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff254168e00 con 0x7ff25409d760 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48890/0 (socket says 172.21.15.43:48890) 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 -- 172.21.15.43:0/646165495 learned_addr learned my addr 172.21.15.43:0/646165495 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff252800640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25409d760 0x7ff25409db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251e00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 0x7ff2540cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 -- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25409d760 msgr2=0x7ff25409db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25409d760 0x7ff25409db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 -- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 msgr2=0x7ff2540cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 0x7ff2540cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 -- 172.21.15.43:0/646165495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25409e0b0 con 0x7ff254155d70 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251e00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 0x7ff2540cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff251400640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d980 tx=0x7ff24000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240017070 con 0x7ff254155d70 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff240004310 con 0x7ff254155d70 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2540ca920 con 0x7ff254155d70 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240009c30 con 0x7ff254155d70 2024-09-17T05:34:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.629+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25409d1d0 con 0x7ff254155d70 2024-09-17T05:34:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2540ad0b0 con 0x7ff254155d70 2024-09-17T05:34:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff240005020 con 0x7ff254155d70 2024-09-17T05:34:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff24a000640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 0x7ff234075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4857+0+0 (secure 0 0 0) 0x7ff2400942e0 con 0x7ff254155d70 2024-09-17T05:34:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff251e00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 0x7ff234075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.633+0000 7ff251e00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 0x7ff234075740 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff2540715c0 tx=0x7ff244009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.637+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff240020020 con 0x7ff254155d70 2024-09-17T05:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff2540ca630 con 0x7ff254155d70 2024-09-17T05:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff24a000640 1 -- 172.21.15.43:0/646165495 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5304 (secure 0 0 0) 0x7ff25409e0b0 con 0x7ff254155d70 2024-09-17T05:34:40.876 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmzsslMhAAmzb818jGPnjPkNa53Dzg7Q==", 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmLgF9BBAAFMLh1ARmhIfwMiR1hYkjsQ==", 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.878 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmP+k/OhAA5OQBQFPyuaTsEq9C9N91BQ==", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmGzRmDBAANsLOF5Rk/vNo7D17TjcM0g==", 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.879 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAQEulm/nGEKxAAmzWXD9pVddVyy3qxwE/g2Q==", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAUEulm+DHEBhAAUN2lfOJENFfc2bnEREmdSA==", 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-09-17T05:34:40.881 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAXEulmTRILLRAAXMRY6c2ZOZi7sQkQHS/Brw==", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbEulmM7sGGBAAEid+smjs0jbXF1B3ja2IBg==", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAhEulmM2xtGhAAxaQv4XPlVI1GAtTm2dGB4A==", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAkEulmWXGgLRAAYjO4br/UlNu0kh79zKjMbA==", 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.883 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoEulmfKyyBhAAexVlCa5kiXc4ZGDniYIeyg==", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArEulmfkn/KRAAt18aWkYhb4HSntV4gXvPkA==", 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-09-17T05:34:40.885 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw==", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulmf26UGRAA0AH3eJ2aN9Qd9/cml9Jygg==", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:34:40.886 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm5OZvFRAAvIB0GwsAAZUqEtqklJFDMQ==", 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPEulm3JnAHRAAqsq0OMNl3VzSjh8Jr0v9Xw==", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmY+2HExAA+wZx3TbpDw2jdMFEj1KKPg==", 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.888 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulm96VcLRAAvWENTG146FzkBvWcagCTNQ==", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAOEulmQca2KBAAtP1LyVAE3iisZQRJPM8aGg==", 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-17T05:34:40.890 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:34:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 msgr2=0x7ff234075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 0x7ff234075740 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff2540715c0 tx=0x7ff244009040 comp rx=0 tx=0).stop 2024-09-17T05:34:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 msgr2=0x7ff254164e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d980 tx=0x7ff24000de40 comp rx=0 tx=0).stop 2024-09-17T05:34:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 shutdown_connections 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff234073230 0x7ff234075740 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff25409d760 0x7ff25409db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff254155d70 0x7ff254164e10 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 --2- 172.21.15.43:0/646165495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff25406f110 0x7ff2540cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 >> 172.21.15.43:0/646165495 conn(0x7ff2540b23d0 msgr2=0x7ff2540afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 shutdown_connections 2024-09-17T05:34:40.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:40.873+0000 7ff258c00640 1 -- 172.21.15.43:0/646165495 wait complete. 2024-09-17T05:34:40.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- 172.21.15.43:0/3964612495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c0b3af0 msgr2=0x7f3c8c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 --2- 172.21.15.43:0/3964612495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c0b3af0 0x7f3c8c0ad0b0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009bc0 tx=0x7f3c8401cae0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- 172.21.15.43:0/3964612495 shutdown_connections 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 --2- 172.21.15.43:0/3964612495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c0b3af0 0x7f3c8c0ad0b0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 --2- 172.21.15.43:0/3964612495 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 0x7f3c8c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- 172.21.15.43:0/3964612495 >> 172.21.15.43:0/3964612495 conn(0x7f3c8c0b23d0 msgr2=0x7f3c8c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:41.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- 172.21.15.43:0/3964612495 shutdown_connections 2024-09-17T05:34:41.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- 172.21.15.43:0/3964612495 wait complete. 2024-09-17T05:34:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 Processor -- start 2024-09-17T05:34:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.057+0000 7f3c93600640 1 -- start start 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c8c06f110 0x7f3c8c1648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 0x7f3c8c1fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c168be0 con 0x7f3c8c1fc170 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c168a60 con 0x7f3c8c06f110 2024-09-17T05:34:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c168d60 con 0x7f3c8c155d70 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48908/0 (socket says 172.21.15.43:48908) 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 -- 172.21.15.43:0/515070968 learned_addr learned my addr 172.21.15.43:0/515070968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c90c00640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c8c06f110 0x7f3c8c1648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c8be00640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 0x7f3c8c1fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 -- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 msgr2=0x7f3c8c1fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 0x7f3c8c1fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 -- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c8c06f110 msgr2=0x7f3c8c1648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c8c06f110 0x7f3c8c1648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 -- 172.21.15.43:0/515070968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c1fc580 con 0x7f3c8c1fc170 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c91600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401c550 tx=0x7f3c84003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b2040 con 0x7f3c8c1fc170 2024-09-17T05:34:41.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c840bb070 con 0x7f3c8c1fc170 2024-09-17T05:34:41.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84004370 con 0x7f3c8c1fc170 2024-09-17T05:34:41.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c1f9880 con 0x7f3c8c1fc170 2024-09-17T05:34:41.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c1f9e40 con 0x7f3c8c1fc170 2024-09-17T05:34:41.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.061+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c0ad0b0 con 0x7f3c8c1fc170 2024-09-17T05:34:41.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.065+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c84004510 con 0x7f3c8c1fc170 2024-09-17T05:34:41.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.065+0000 7f3c88c00640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 0x7f3c6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.065+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4857+0+0 (secure 0 0 0) 0x7f3c8412f270 con 0x7f3c8c1fc170 2024-09-17T05:34:41.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.065+0000 7f3c90c00640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 0x7f3c6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.065+0000 7f3c90c00640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 0x7f3c6c075710 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0715c0 tx=0x7f3c800091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:41.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.069+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3c8c1fc580 con 0x7f3c8c1fc170 2024-09-17T05:34:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.249+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3c8c0cfff0 con 0x7f3c8c1fc170 2024-09-17T05:34:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.249+0000 7f3c88c00640 1 -- 172.21.15.43:0/515070968 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) ==== 84+0+65 (secure 0 0 0) 0x7f3c84016790 con 0x7f3c8c1fc170 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:34:41.255 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 msgr2=0x7f3c6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 0x7f3c6c075710 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0715c0 tx=0x7f3c800091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 msgr2=0x7f3c8c1fa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f3c8401c550 tx=0x7f3c84003f40 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 shutdown_connections 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3c6c073200 0x7f3c6c075710 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c8c1fc170 0x7f3c8c1fa920 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3c8c155d70 0x7f3c8c1fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 --2- 172.21.15.43:0/515070968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c8c06f110 0x7f3c8c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 >> 172.21.15.43:0/515070968 conn(0x7f3c8c0b23d0 msgr2=0x7f3c8c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 shutdown_connections 2024-09-17T05:34:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.253+0000 7f3c93600640 1 -- 172.21.15.43:0/515070968 wait complete. 2024-09-17T05:34:41.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:34:41.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- 172.21.15.43:0/3870753460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2015d120 msgr2=0x7f5a2015d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 --2- 172.21.15.43:0/3870753460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2015d120 0x7f5a2015d530 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009900 tx=0x7f5a0801c850 comp rx=0 tx=0).stop 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- 172.21.15.43:0/3870753460 shutdown_connections 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 --2- 172.21.15.43:0/3870753460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a2015d120 0x7f5a2015d530 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 --2- 172.21.15.43:0/3870753460 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a20160470 0x7f5a20160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- 172.21.15.43:0/3870753460 >> 172.21.15.43:0/3870753460 conn(0x7f5a2006ce60 msgr2=0x7f5a2006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- 172.21.15.43:0/3870753460 shutdown_connections 2024-09-17T05:34:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- 172.21.15.43:0/3870753460 wait complete. 2024-09-17T05:34:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 Processor -- start 2024-09-17T05:34:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.441+0000 7f5a25c00640 1 -- start start 2024-09-17T05:34:41.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a2006f0d0 0x7f5a201f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a2015d120 0x7f5a20153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2016a790 con 0x7f5a20160470 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2016a610 con 0x7f5a2006f0d0 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2016a910 con 0x7f5a2015d120 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48918/0 (socket says 172.21.15.43:48918) 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 -- 172.21.15.43:0/4276933597 learned_addr learned my addr 172.21.15.43:0/4276933597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1ea00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a2006f0d0 0x7f5a201f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a17e00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a2015d120 0x7f5a20153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 -- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a2015d120 msgr2=0x7f5a20153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a2015d120 0x7f5a20153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 -- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a2006f0d0 msgr2=0x7f5a201f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a2006f0d0 0x7f5a201f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 -- 172.21.15.43:0/4276933597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a20151cc0 con 0x7f5a20160470 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a1f400640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c6b0 tx=0x7f5a1800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003b80 con 0x7f5a20160470 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a20151fa0 con 0x7f5a20160470 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a18003d20 con 0x7f5a20160470 2024-09-17T05:34:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18005330 con 0x7f5a20160470 2024-09-17T05:34:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.445+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a20152560 con 0x7f5a20160470 2024-09-17T05:34:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a20151a30 con 0x7f5a20160470 2024-09-17T05:34:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a180054d0 con 0x7f5a20160470 2024-09-17T05:34:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a16a00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 0x7f59fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4857+0+0 (secure 0 0 0) 0x7f5a18093c10 con 0x7f5a20160470 2024-09-17T05:34:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a1ea00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 0x7f59fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.449+0000 7f5a1ea00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 0x7f59fc075710 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000bb50 tx=0x7f5a1000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.453+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a20160dc0 con 0x7f5a20160470 2024-09-17T05:34:41.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:41.629+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a200b0e90 con 0x7f5a20160470 2024-09-17T05:34:42.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.449+0000 7f5a16a00640 1 -- 172.21.15.43:0/4276933597 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) ==== 138+0+0 (secure 0 0 0) 0x7f5a1805e8a0 con 0x7f5a20160470 2024-09-17T05:34:42.454 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-17T05:34:42.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 msgr2=0x7f59fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:42.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 0x7f59fc075710 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000bb50 tx=0x7f5a1000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 msgr2=0x7f5a200ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c6b0 tx=0x7f5a1800cb70 comp rx=0 tx=0).stop 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 shutdown_connections 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f59fc073200 0x7f59fc075710 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a20160470 0x7f5a200ae1a0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5a2015d120 0x7f5a20153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 --2- 172.21.15.43:0/4276933597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a2006f0d0 0x7f5a201f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 >> 172.21.15.43:0/4276933597 conn(0x7f5a2006ce60 msgr2=0x7f5a200992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:42.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 shutdown_connections 2024-09-17T05:34:42.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.453+0000 7f5a25c00640 1 -- 172.21.15.43:0/4276933597 wait complete. 2024-09-17T05:34:42.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 -- 172.21.15.43:0/2917191932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 msgr2=0x7f8ef40b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 --2- 172.21.15.43:0/2917191932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef40b5480 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009840 tx=0x7f8edc01c700 comp rx=0 tx=0).stop 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 -- 172.21.15.43:0/2917191932 shutdown_connections 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 --2- 172.21.15.43:0/2917191932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef40b5480 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 -- 172.21.15.43:0/2917191932 >> 172.21.15.43:0/2917191932 conn(0x7f8ef40691b0 msgr2=0x7f8ef40695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 -- 172.21.15.43:0/2917191932 shutdown_connections 2024-09-17T05:34:42.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.653+0000 7f8ef3200640 1 -- 172.21.15.43:0/2917191932 wait complete. 2024-09-17T05:34:42.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 Processor -- start 2024-09-17T05:34:42.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 -- start start 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ef40d08b0 0x7f8ef415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 0x7f8ef415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef416aa40 con 0x7f8ef40b5070 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef416abc0 con 0x7f8ef415c4e0 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef416ad40 con 0x7f8ef40d08b0 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48944/0 (socket says 172.21.15.43:48944) 2024-09-17T05:34:42.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 -- 172.21.15.43:0/3839889547 learned_addr learned my addr 172.21.15.43:0/3839889547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ee9400640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ef40d08b0 0x7f8ef415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef2800640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 0x7f8ef415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 -- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ef40d08b0 msgr2=0x7f8ef415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ef40d08b0 0x7f8ef415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 -- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 msgr2=0x7f8ef415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 0x7f8ef415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 -- 172.21.15.43:0/3839889547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4155b50 con 0x7f8ef40b5070 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef2800640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 0x7f8ef415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef1e00640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c7c0 tx=0x7f8ee400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4003b80 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ee4003d20 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef4155e30 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.657+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400aa90 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef4158e00 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef40b5480 con 0x7f8ef40b5070 2024-09-17T05:34:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ee400ac30 con 0x7f8ef40b5070 2024-09-17T05:34:42.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8eeaa00640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 0x7f8ed0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:42.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5227+0+0 (secure 0 0 0) 0x7f8ee4093eb0 con 0x7f8ef40b5070 2024-09-17T05:34:42.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.661+0000 7f8ee9400640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 0x7f8ed0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:42.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.665+0000 7f8ee9400640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 0x7f8ed0075740 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc01af40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:42.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.669+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ee405f0a0 con 0x7f8ef40b5070 2024-09-17T05:34:42.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:42.845+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8ef406a1e0 con 0x7f8ef40b5070 2024-09-17T05:34:43.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.453+0000 7f8eeaa00640 1 -- 172.21.15.43:0/3839889547 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) ==== 170+0+0 (secure 0 0 0) 0x7f8ee405e9a0 con 0x7f8ef40b5070 2024-09-17T05:34:43.457 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-17T05:34:43.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 msgr2=0x7f8ed0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:43.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 0x7f8ed0075740 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc01af40 comp rx=0 tx=0).stop 2024-09-17T05:34:43.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 msgr2=0x7f8ef4151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:43.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c7c0 tx=0x7f8ee400cc80 comp rx=0 tx=0).stop 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 shutdown_connections 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ed0073230 0x7f8ed0075740 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef415c4e0 0x7f8ef415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ef40d08b0 0x7f8ef415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 --2- 172.21.15.43:0/3839889547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ef40b5070 0x7f8ef4151180 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 >> 172.21.15.43:0/3839889547 conn(0x7f8ef40691b0 msgr2=0x7f8ef40b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 shutdown_connections 2024-09-17T05:34:43.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.457+0000 7f8ef3200640 1 -- 172.21.15.43:0/3839889547 wait complete. 2024-09-17T05:34:43.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:34:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 -- 172.21.15.43:0/298523942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db80d07c0 msgr2=0x7f5db80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 --2- 172.21.15.43:0/298523942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db80d07c0 0x7f5db80d0bd0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009bc0 tx=0x7f5db001cb10 comp rx=0 tx=0).stop 2024-09-17T05:34:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 -- 172.21.15.43:0/298523942 shutdown_connections 2024-09-17T05:34:43.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 --2- 172.21.15.43:0/298523942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db80d07c0 0x7f5db80d0bd0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 -- 172.21.15.43:0/298523942 >> 172.21.15.43:0/298523942 conn(0x7f5db80691b0 msgr2=0x7f5db80695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:43.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 -- 172.21.15.43:0/298523942 shutdown_connections 2024-09-17T05:34:43.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 -- 172.21.15.43:0/298523942 wait complete. 2024-09-17T05:34:43.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.661+0000 7f5dbea00640 1 Processor -- start 2024-09-17T05:34:43.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- start start 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db80d07c0 0x7f5db8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5db80b6c40 0x7f5db806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5db816abe0 con 0x7f5db815c610 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5db816aa60 con 0x7f5db80d07c0 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5db816ad60 con 0x7f5db80b6c40 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 -- 172.21.15.43:0/1812926052 learned_addr learned my addr 172.21.15.43:0/1812926052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbd600640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db80d07c0 0x7f5db8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbe000640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5db80b6c40 0x7f5db806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 -- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5db80b6c40 msgr2=0x7f5db806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5db80b6c40 0x7f5db806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 -- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db80d07c0 msgr2=0x7f5db8155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db80d07c0 0x7f5db8155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 -- 172.21.15.43:0/1812926052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db80c8960 con 0x7f5db815c610 2024-09-17T05:34:43.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbcc00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c420 tx=0x7f5da400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4017070 con 0x7f5db815c610 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5da40053c0 con 0x7f5db815c610 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4003860 con 0x7f5db815c610 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db80c5fe0 con 0x7f5db815c610 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.665+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81583d0 con 0x7f5db815c610 2024-09-17T05:34:43.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.669+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db80d0bd0 con 0x7f5db815c610 2024-09-17T05:34:43.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.669+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5da401a070 con 0x7f5db815c610 2024-09-17T05:34:43.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.669+0000 7f5db5600640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 0x7f5d98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:43.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.669+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5609+0+0 (secure 0 0 0) 0x7f5da40937a0 con 0x7f5db815c610 2024-09-17T05:34:43.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.669+0000 7f5dbd600640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 0x7f5d98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:43.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.673+0000 7f5dbd600640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 0x7f5d98075740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5dac001180 tx=0x7f5dac001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:43.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.673+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5da4021080 con 0x7f5db815c610 2024-09-17T05:34:43.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:43.881+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5db806a1e0 con 0x7f5db815c610 2024-09-17T05:34:44.496 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 14 and data pool 15 2024-09-17T05:34:44.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.489+0000 7f5db5600640 1 -- 172.21.15.43:0/1812926052 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v341) ==== 174+0+0 (secure 0 0 0) 0x7f5da4091e20 con 0x7f5db815c610 2024-09-17T05:34:44.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 msgr2=0x7f5d98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 0x7f5d98075740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5dac001180 tx=0x7f5dac001110 comp rx=0 tx=0).stop 2024-09-17T05:34:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 msgr2=0x7f5db806df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c420 tx=0x7f5da400c8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 shutdown_connections 2024-09-17T05:34:44.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d98073230 0x7f5d98075740 secure :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5dac001180 tx=0x7f5dac001110 comp rx=0 tx=0).stop 2024-09-17T05:34:44.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5db80b6c40 0x7f5db806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5db815c610 0x7f5db806df70 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 --2- 172.21.15.43:0/1812926052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db80d07c0 0x7f5db8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.497+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 >> 172.21.15.43:0/1812926052 conn(0x7f5db80691b0 msgr2=0x7f5db815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:44.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.501+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 shutdown_connections 2024-09-17T05:34:44.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.501+0000 7f5dbea00640 1 -- 172.21.15.43:0/1812926052 wait complete. 2024-09-17T05:34:44.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 -- 172.21.15.43:0/1181100451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 msgr2=0x7fded40cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 --2- 172.21.15.43:0/1181100451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded40cd370 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009840 tx=0x7fdec401c790 comp rx=0 tx=0).stop 2024-09-17T05:34:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 -- 172.21.15.43:0/1181100451 shutdown_connections 2024-09-17T05:34:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 --2- 172.21.15.43:0/1181100451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded40cd370 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 -- 172.21.15.43:0/1181100451 >> 172.21.15.43:0/1181100451 conn(0x7fded40b2810 msgr2=0x7fded40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 -- 172.21.15.43:0/1181100451 shutdown_connections 2024-09-17T05:34:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.693+0000 7fded9600640 1 -- 172.21.15.43:0/1181100451 wait complete. 2024-09-17T05:34:44.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 Processor -- start 2024-09-17T05:34:44.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 -- start start 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fded40cd8b0 0x7fded40adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 0x7fded41fc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fded416af50 con 0x7fded40cdfb0 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fded416add0 con 0x7fded40ce3c0 2024-09-17T05:34:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded9600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fded416b0d0 con 0x7fded40cd8b0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54076/0 (socket says 172.21.15.43:54076) 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 -- 172.21.15.43:0/572306185 learned_addr learned my addr 172.21.15.43:0/572306185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded3400640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 0x7fded41fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2a00640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fded40cd8b0 0x7fded40adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 -- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fded40cd8b0 msgr2=0x7fded40adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fded40cd8b0 0x7fded40adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 -- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 msgr2=0x7fded41fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 0x7fded41fc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 -- 172.21.15.43:0/572306185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded40af0f0 con 0x7fded40cdfb0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded3400640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 0x7fded41fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.697+0000 7fded2000640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b410 tx=0x7fdec000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0015030 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdec0004020 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00044b0 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded40af3d0 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded40b3d90 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded40cd370 con 0x7fded40cdfb0 2024-09-17T05:34:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdec0018070 con 0x7fded40cdfb0 2024-09-17T05:34:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 0x7fdeb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fdec0093e10 con 0x7fded40cdfb0 2024-09-17T05:34:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.701+0000 7fded2a00640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 0x7fdeb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.705+0000 7fded2a00640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 0x7fdeb4075710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdec40af000 tx=0x7fdec401afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:44.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.705+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdec0097030 con 0x7fded40cdfb0 2024-09-17T05:34:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.885+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fded4153340 con 0x7fded40cdfb0 2024-09-17T05:34:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.885+0000 7fdecaa00640 1 -- 172.21.15.43:0/572306185 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16632 (secure 0 0 0) 0x7fdec005ee20 con 0x7fded40cdfb0 2024-09-17T05:34:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:44.891 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":138,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:43.887818+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:41.643835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:42.859239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 msgr2=0x7fdeb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 0x7fdeb4075710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdec40af000 tx=0x7fdec401afd0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 msgr2=0x7fded414b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b410 tx=0x7fdec000b8d0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 shutdown_connections 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdeb4073200 0x7fdeb4075710 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded40ce3c0 0x7fded41fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded40cdfb0 0x7fded414b290 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 --2- 172.21.15.43:0/572306185 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fded40cd8b0 0x7fded40adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 >> 172.21.15.43:0/572306185 conn(0x7fded40b2810 msgr2=0x7fded414e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 shutdown_connections 2024-09-17T05:34:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:44.889+0000 7fded9600640 1 -- 172.21.15.43:0/572306185 wait complete. 2024-09-17T05:34:44.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- 172.21.15.43:0/4194052628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9006f110 msgr2=0x7f2f900b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 --2- 172.21.15.43:0/4194052628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9006f110 0x7f2f900b33e0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f2f90072b60 tx=0x7f2f7801c800 comp rx=0 tx=0).stop 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- 172.21.15.43:0/4194052628 shutdown_connections 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 --2- 172.21.15.43:0/4194052628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9006f110 0x7f2f900b33e0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 --2- 172.21.15.43:0/4194052628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f900b3920 0x7f2f900ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 --2- 172.21.15.43:0/4194052628 >> v2:172.21.15.62:3300/0 conn(0x7f2f90155d70 0x7f2f9015c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- 172.21.15.43:0/4194052628 >> 172.21.15.43:0/4194052628 conn(0x7f2f900b23d0 msgr2=0x7f2f900d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- 172.21.15.43:0/4194052628 shutdown_connections 2024-09-17T05:34:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- 172.21.15.43:0/4194052628 wait complete. 2024-09-17T05:34:45.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 Processor -- start 2024-09-17T05:34:45.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.069+0000 7f2f97400640 1 -- start start 2024-09-17T05:34:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9006f110 0x7f2f901fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f90155d70 0x7f2f9009c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015f8c0 con 0x7f2f900b3920 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015f740 con 0x7f2f9006f110 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9015fa40 con 0x7f2f90155d70 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54102/0 (socket says 172.21.15.43:54102) 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 -- 172.21.15.43:0/4120313194 learned_addr learned my addr 172.21.15.43:0/4120313194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f94a00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9006f110 0x7f2f901fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f95400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f90155d70 0x7f2f9009c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 -- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f90155d70 msgr2=0x7f2f9009c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f90155d70 0x7f2f9009c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 -- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9006f110 msgr2=0x7f2f901fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9006f110 0x7f2f901fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 -- 172.21.15.43:0/4120313194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f900d02f0 con 0x7f2f900b3920 2024-09-17T05:34:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f87e00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b0040 tx=0x7f2f78005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780b2040 con 0x7f2f900b3920 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f78004330 con 0x7f2f900b3920 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78005290 con 0x7f2f900b3920 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f900d05d0 con 0x7f2f900b3920 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901fc370 con 0x7f2f900b3920 2024-09-17T05:34:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.073+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9015fef0 con 0x7f2f900b3920 2024-09-17T05:34:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.077+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2f780bd060 con 0x7f2f900b3920 2024-09-17T05:34:45.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.077+0000 7f2f8d600640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 0x7f2f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.077+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7812d600 con 0x7f2f900b3920 2024-09-17T05:34:45.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.077+0000 7f2f94a00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 0x7f2f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.077+0000 7f2f94a00640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 0x7f2f6c075710 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2f900715c0 tx=0x7f2f800091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:45.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.081+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2f780164b0 con 0x7f2f900b3920 2024-09-17T05:34:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f900b00c0 con 0x7f2f900b3920 2024-09-17T05:34:45.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f8d600640 1 -- 172.21.15.43:0/4120313194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16632 (secure 0 0 0) 0x7f2f780f80d0 con 0x7f2f900b3920 2024-09-17T05:34:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:45.261 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":138,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:43.887818+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:41.643835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:42.859239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 msgr2=0x7f2f6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 0x7f2f6c075710 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2f900715c0 tx=0x7f2f800091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 msgr2=0x7f2f900cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b0040 tx=0x7f2f78005e70 comp rx=0 tx=0).stop 2024-09-17T05:34:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 shutdown_connections 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2f6c073200 0x7f2f6c075710 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2f90155d70 0x7f2f9009c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f900b3920 0x7f2f900cf230 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 --2- 172.21.15.43:0/4120313194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f9006f110 0x7f2f901fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 >> 172.21.15.43:0/4120313194 conn(0x7f2f900b23d0 msgr2=0x7f2f900b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 shutdown_connections 2024-09-17T05:34:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.257+0000 7f2f97400640 1 -- 172.21.15.43:0/4120313194 wait complete. 2024-09-17T05:34:45.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:34:45.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- 172.21.15.43:0/3949329473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 msgr2=0x7fc44c0ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 --2- 172.21.15.43:0/3949329473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c0ad0b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fc444009c80 tx=0x7fc44401cce0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- 172.21.15.43:0/3949329473 shutdown_connections 2024-09-17T05:34:45.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 --2- 172.21.15.43:0/3949329473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c0ad0b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 --2- 172.21.15.43:0/3949329473 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c0b3af0 0x7fc44c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- 172.21.15.43:0/3949329473 >> 172.21.15.43:0/3949329473 conn(0x7fc44c0b23d0 msgr2=0x7fc44c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:45.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- 172.21.15.43:0/3949329473 shutdown_connections 2024-09-17T05:34:45.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- 172.21.15.43:0/3949329473 wait complete. 2024-09-17T05:34:45.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 Processor -- start 2024-09-17T05:34:45.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.441+0000 7fc451200640 1 -- start start 2024-09-17T05:34:45.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c155d70 0x7fc44c160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc44c1fc910 0x7fc44c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c168c80 con 0x7fc44c06f110 2024-09-17T05:34:45.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c168b00 con 0x7fc44c1fc910 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc44c168e00 con 0x7fc44c155d70 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54118/0 (socket says 172.21.15.43:54118) 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 -- 172.21.15.43:0/1216390458 learned_addr learned my addr 172.21.15.43:0/1216390458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc44a800640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc44c1fc910 0x7fc44c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc443e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c155d70 0x7fc44c160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 -- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c155d70 msgr2=0x7fc44c160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c155d70 0x7fc44c160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 -- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc44c1fc910 msgr2=0x7fc44c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc44c1fc910 0x7fc44c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 -- 172.21.15.43:0/1216390458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c1f8860 con 0x7fc44c06f110 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc449e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c3f0 tx=0x7fc43c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c017070 con 0x7fc44c06f110 2024-09-17T05:34:45.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc43c0053c0 con 0x7fc44c06f110 2024-09-17T05:34:45.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003860 con 0x7fc44c06f110 2024-09-17T05:34:45.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c1f8b40 con 0x7fc44c06f110 2024-09-17T05:34:45.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c1f9100 con 0x7fc44c06f110 2024-09-17T05:34:45.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.445+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44c0ad0b0 con 0x7fc44c06f110 2024-09-17T05:34:45.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.449+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc43c01a070 con 0x7fc44c06f110 2024-09-17T05:34:45.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.449+0000 7fc442000640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 0x7fc428075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:45.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.449+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fc43c093500 con 0x7fc44c06f110 2024-09-17T05:34:45.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.449+0000 7fc443e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 0x7fc428075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:45.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.449+0000 7fc443e00640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 0x7fc428075710 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1554f0 tx=0x7fc43400b390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:45.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.453+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc43c05dec0 con 0x7fc44c06f110 2024-09-17T05:34:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:45.669+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc44c0b1940 con 0x7fc44c06f110 2024-09-17T05:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.505+0000 7fc442000640 1 -- 172.21.15.43:0/1216390458 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v344) ==== 118+0+0 (secure 0 0 0) 0x7fc43c0628f0 con 0x7fc44c06f110 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 msgr2=0x7fc428075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 0x7fc428075710 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1554f0 tx=0x7fc43400b390 comp rx=0 tx=0).stop 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 msgr2=0x7fc44c154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c3f0 tx=0x7fc43c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 shutdown_connections 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc428073200 0x7fc428075710 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc44c1fc910 0x7fc44c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc44c155d70 0x7fc44c160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 --2- 172.21.15.43:0/1216390458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c06f110 0x7fc44c154b30 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 >> 172.21.15.43:0/1216390458 conn(0x7fc44c0b23d0 msgr2=0x7fc44c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 shutdown_connections 2024-09-17T05:34:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.509+0000 7fc451200640 1 -- 172.21.15.43:0/1216390458 wait complete. 2024-09-17T05:34:46.537 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:34:46.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:34:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 -- 172.21.15.43:0/801852082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c15d8d0 msgr2=0x7fbf9c0d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 --2- 172.21.15.43:0/801852082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c0d26e0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0cfd40 tx=0x7fbf9401cd50 comp rx=0 tx=0).stop 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 -- 172.21.15.43:0/801852082 shutdown_connections 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 --2- 172.21.15.43:0/801852082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c0d26e0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 --2- 172.21.15.43:0/801852082 >> v2:172.21.15.62:3300/0 conn(0x7fbf9c15d4c0 0x7fbf9c0d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 --2- 172.21.15.43:0/801852082 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c0ccf00 0x7fbf9c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 -- 172.21.15.43:0/801852082 >> 172.21.15.43:0/801852082 conn(0x7fbf9c0691b0 msgr2=0x7fbf9c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 -- 172.21.15.43:0/801852082 shutdown_connections 2024-09-17T05:34:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.701+0000 7fbfa1800640 1 -- 172.21.15.43:0/801852082 wait complete. 2024-09-17T05:34:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 Processor -- start 2024-09-17T05:34:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- start start 2024-09-17T05:34:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c15d4c0 0x7fbf9c14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c06f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c153160 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c152fe0 con 0x7fbf9c15d8d0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9c1532e0 con 0x7fbf9c15d4c0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54134/0 (socket says 172.21.15.43:54134) 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 -- 172.21.15.43:0/2563867191 learned_addr learned my addr 172.21.15.43:0/2563867191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf93e00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c15d4c0 0x7fbf9c14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa0e00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c06f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 -- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c15d4c0 msgr2=0x7fbf9c14e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c15d4c0 0x7fbf9c14e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 -- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf9c15d8d0 msgr2=0x7fbf9c06f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c06f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 -- 172.21.15.43:0/2563867191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c14a810 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf9be00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d690 tx=0x7fbf8c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c017070 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf8c005310 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c06eb90 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0097c0 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c165b90 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.705+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9c06e980 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.709+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf8c009960 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.709+0000 7fbf98c00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 0x7fbf78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:46.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.709+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fbf8c0945e0 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.709+0000 7fbf93e00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 0x7fbf78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:46.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.709+0000 7fbf93e00640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 0x7fbf78075740 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c1fb8c0 tx=0x7fbf8400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.713+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf8c05f020 con 0x7fbf9c0ccf00 2024-09-17T05:34:46.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T05:34:46.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T05:34:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:46.921+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbf9c1601a0 con 0x7fbf9c0ccf00 2024-09-17T05:34:47.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.513+0000 7fbf98c00640 1 -- 172.21.15.43:0/2563867191 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v345) ==== 122+0+0 (secure 0 0 0) 0x7fbf8c063a50 con 0x7fbf9c0ccf00 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 msgr2=0x7fbf78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 0x7fbf78075740 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c1fb8c0 tx=0x7fbf8400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 msgr2=0x7fbf9c14dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d690 tx=0x7fbf8c00db50 comp rx=0 tx=0).stop 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 shutdown_connections 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fbf78073230 0x7fbf78075740 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf9c15d8d0 0x7fbf9c06f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fbf9c15d4c0 0x7fbf9c14e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 --2- 172.21.15.43:0/2563867191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf9c0ccf00 0x7fbf9c14dc00 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 >> 172.21.15.43:0/2563867191 conn(0x7fbf9c0691b0 msgr2=0x7fbf9c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 shutdown_connections 2024-09-17T05:34:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.517+0000 7fbfa1800640 1 -- 172.21.15.43:0/2563867191 wait complete. 2024-09-17T05:34:47.777 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:34:47.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:47.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 -- 172.21.15.43:0/227397902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 msgr2=0x7f14e00b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 --2- 172.21.15.43:0/227397902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e00b5a20 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009bc0 tx=0x7f14c801ca80 comp rx=0 tx=0).stop 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 -- 172.21.15.43:0/227397902 shutdown_connections 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 --2- 172.21.15.43:0/227397902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e00b5a20 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 --2- 172.21.15.43:0/227397902 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 0x7f14e00d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 -- 172.21.15.43:0/227397902 >> 172.21.15.43:0/227397902 conn(0x7f14e00b2810 msgr2=0x7f14e00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 -- 172.21.15.43:0/227397902 shutdown_connections 2024-09-17T05:34:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.945+0000 7f14e5e00640 1 -- 172.21.15.43:0/227397902 wait complete. 2024-09-17T05:34:47.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 Processor -- start 2024-09-17T05:34:47.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- start start 2024-09-17T05:34:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14e00b3b10 0x7f14e00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 0x7f14e00b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01694d0 con 0x7f14e00b5610 2024-09-17T05:34:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0169350 con 0x7f14e00b3b10 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f14e0169650 con 0x7f14e015f470 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54158/0 (socket says 172.21.15.43:54158) 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 -- 172.21.15.43:0/395814321 learned_addr learned my addr 172.21.15.43:0/395814321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14dea00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14e00b3b10 0x7f14e00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14df400640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 0x7f14e00b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 -- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 msgr2=0x7f14e00b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 0x7f14e00b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 -- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14e00b3b10 msgr2=0x7f14e00afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14e00b3b10 0x7f14e00afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 -- 172.21.15.43:0/395814321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e01579d0 con 0x7f14e00b5610 2024-09-17T05:34:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d6000640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009b10 tx=0x7f14c8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:47.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b2040 con 0x7f14e00b5610 2024-09-17T05:34:47.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14c8004130 con 0x7f14e00b5610 2024-09-17T05:34:47.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e01fce10 con 0x7f14e00b5610 2024-09-17T05:34:47.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004560 con 0x7f14e00b5610 2024-09-17T05:34:47.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.949+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01fd3d0 con 0x7f14e00b5610 2024-09-17T05:34:47.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.953+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e00b5a20 con 0x7f14e00b5610 2024-09-17T05:34:47.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.953+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f14c80bd060 con 0x7f14e00b5610 2024-09-17T05:34:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.953+0000 7f14d7e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 0x7f14bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.953+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f14c812dd80 con 0x7f14e00b5610 2024-09-17T05:34:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.953+0000 7f14dea00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 0x7f14bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.957+0000 7f14dea00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 0x7f14bc075710 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f14e0069610 tx=0x7f14d00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:47.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:47.957+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14c8016790 con 0x7f14e00b5610 2024-09-17T05:34:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.169+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14e00cfb20 con 0x7f14e00b5610 2024-09-17T05:34:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14d7e00640 1 -- 172.21.15.43:0/395814321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) ==== 95+0+5233 (secure 0 0 0) 0x7f14c80f87d0 con 0x7f14e00b5610 2024-09-17T05:34:48.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:48.176 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":345,"btime":"2024-09-17T05:34:47:511187+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:34:46.927801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10464":{"gid":10464,"name":"c","rank":0,"incarnation":341,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/4038368577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4038368577},{"type":"v1","addr":"172.21.15.43:6834","nonce":4038368577}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10464,"qdb_cluster":[10464]},"id":7}]} 2024-09-17T05:34:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 345 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 msgr2=0x7f14bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 0x7f14bc075710 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f14e0069610 tx=0x7f14d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 msgr2=0x7f14e0158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009b10 tx=0x7f14c8005a60 comp rx=0 tx=0).stop 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 shutdown_connections 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f14bc073200 0x7f14bc075710 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f14e015f470 0x7f14e00b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14e00b5610 0x7f14e0158e80 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 --2- 172.21.15.43:0/395814321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14e00b3b10 0x7f14e00afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 >> 172.21.15.43:0/395814321 conn(0x7f14e00b2810 msgr2=0x7f14e0164cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 shutdown_connections 2024-09-17T05:34:48.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.173+0000 7f14e5e00640 1 -- 172.21.15.43:0/395814321 wait complete. 2024-09-17T05:34:48.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:48.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 -- 172.21.15.43:0/3785003761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff180b3af0 msgr2=0x7eff180ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 --2- 172.21.15.43:0/3785003761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff180b3af0 0x7eff180ad0b0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7eff18072b60 tx=0x7eff1001cac0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 -- 172.21.15.43:0/3785003761 shutdown_connections 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 --2- 172.21.15.43:0/3785003761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff180b3af0 0x7eff180ad0b0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 --2- 172.21.15.43:0/3785003761 >> v2:172.21.15.62:3300/0 conn(0x7eff18155d70 0x7eff1815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 -- 172.21.15.43:0/3785003761 >> 172.21.15.43:0/3785003761 conn(0x7eff180b23d0 msgr2=0x7eff180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 -- 172.21.15.43:0/3785003761 shutdown_connections 2024-09-17T05:34:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.353+0000 7eff1f400640 1 -- 172.21.15.43:0/3785003761 wait complete. 2024-09-17T05:34:48.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 Processor -- start 2024-09-17T05:34:48.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- start start 2024-09-17T05:34:48.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff1806f110 0x7eff18165d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 0x7eff1809d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff18168b90 con 0x7eff1815b560 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff18168a10 con 0x7eff18155d70 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7eff18168d10 con 0x7eff1806f110 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54182/0 (socket says 172.21.15.43:54182) 2024-09-17T05:34:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 -- 172.21.15.43:0/2304253348 learned_addr learned my addr 172.21.15.43:0/2304253348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff17e00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 0x7eff1809d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1ca00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff1806f110 0x7eff18165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 -- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff1806f110 msgr2=0x7eff18165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff1806f110 0x7eff18165d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 -- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 msgr2=0x7eff1809d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 0x7eff1809d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 -- 172.21.15.43:0/2304253348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff180ca190 con 0x7eff1815b560 2024-09-17T05:34:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff17e00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 0x7eff1809d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1d400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7eff100986b0 tx=0x7eff10003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff100b2040 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff100bb070 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10004330 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff180cf8c0 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.357+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff18165810 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.361+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff180ad0b0 con 0x7eff1815b560 2024-09-17T05:34:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.361+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7eff100044d0 con 0x7eff1815b560 2024-09-17T05:34:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.361+0000 7eff14c00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 0x7efef8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.361+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7eff1012ecb0 con 0x7eff1815b560 2024-09-17T05:34:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.361+0000 7eff1ca00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 0x7efef8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.365+0000 7eff1ca00640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 0x7efef8075710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7eff180715c0 tx=0x7eff0c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.365+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7eff10016770 con 0x7eff1815b560 2024-09-17T05:34:48.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.541+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff180af720 con 0x7eff1815b560 2024-09-17T05:34:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.541+0000 7eff14c00640 1 -- 172.21.15.43:0/2304253348 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16619 (secure 0 0 0) 0x7eff100f9cc0 con 0x7eff1815b560 2024-09-17T05:34:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:48.547 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:34:45.471249+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:41.643835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:42.859239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 msgr2=0x7efef8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 0x7efef8075710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7eff180715c0 tx=0x7eff0c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 msgr2=0x7eff1815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7eff100986b0 tx=0x7eff10003f40 comp rx=0 tx=0).stop 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 shutdown_connections 2024-09-17T05:34:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efef8073200 0x7efef8075710 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff18155d70 0x7eff1809d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff1815b560 0x7eff1815a710 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 --2- 172.21.15.43:0/2304253348 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7eff1806f110 0x7eff18165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 >> 172.21.15.43:0/2304253348 conn(0x7eff180b23d0 msgr2=0x7eff1815f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 shutdown_connections 2024-09-17T05:34:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.545+0000 7eff1f400640 1 -- 172.21.15.43:0/2304253348 wait complete. 2024-09-17T05:34:48.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:34:48.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 -- 172.21.15.43:0/3381978208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f937c003b40 msgr2=0x7f937c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 --2- 172.21.15.43:0/3381978208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f937c003b40 0x7f937c003fb0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f9388009d00 tx=0x7f938801d3d0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 -- 172.21.15.43:0/3381978208 shutdown_connections 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 --2- 172.21.15.43:0/3381978208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f937c003b40 0x7f937c003fb0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 -- 172.21.15.43:0/3381978208 >> 172.21.15.43:0/3381978208 conn(0x7f93940b23d0 msgr2=0x7f93940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 -- 172.21.15.43:0/3381978208 shutdown_connections 2024-09-17T05:34:48.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.725+0000 7f939ba00640 1 -- 172.21.15.43:0/3381978208 wait complete. 2024-09-17T05:34:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 Processor -- start 2024-09-17T05:34:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 -- start start 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f937c003b40 0x7f93941f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93941fc750 0x7f93940ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93940ad600 con 0x7f9394098ca0 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93940ad480 con 0x7f937c003b40 2024-09-17T05:34:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f93940ad780 con 0x7f93941fc750 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54188/0 (socket says 172.21.15.43:54188) 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 -- 172.21.15.43:0/3246572699 learned_addr learned my addr 172.21.15.43:0/3246572699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9399000640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f937c003b40 0x7f93941f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9399a00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93941fc750 0x7f93940ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 -- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93941fc750 msgr2=0x7f93940ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93941fc750 0x7f93940ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 -- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f937c003b40 msgr2=0x7f93941f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f937c003b40 0x7f93941f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 -- 172.21.15.43:0/3246572699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940c4ac0 con 0x7f9394098ca0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9393e00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f9380003e80 tx=0x7f9380001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380015070 con 0x7f9394098ca0 2024-09-17T05:34:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9380011840 con 0x7f9394098ca0 2024-09-17T05:34:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93940c4da0 con 0x7f9394098ca0 2024-09-17T05:34:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.729+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380011c70 con 0x7f9394098ca0 2024-09-17T05:34:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.733+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93940c5360 con 0x7f9394098ca0 2024-09-17T05:34:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.733+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93941fcfb0 con 0x7f9394098ca0 2024-09-17T05:34:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.733+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9380018070 con 0x7f9394098ca0 2024-09-17T05:34:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.733+0000 7f9390c00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 0x7f9374075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.733+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f9380094d40 con 0x7f9394098ca0 2024-09-17T05:34:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.737+0000 7f9399000640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 0x7f9374075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.737+0000 7f9399000640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 0x7f9374075710 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93880ad000 tx=0x7f938801a170 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.741+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f938005f710 con 0x7f9394098ca0 2024-09-17T05:34:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.913+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93940adce0 con 0x7f9374073200 2024-09-17T05:34:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.917+0000 7f9390c00640 1 -- 172.21.15.43:0/3246572699 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248108 (secure 0 0 0) 0x7f93940adce0 con 0x7f9374073200 2024-09-17T05:34:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:34:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 msgr2=0x7f9374075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 0x7f9374075710 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f93880ad000 tx=0x7f938801a170 comp rx=0 tx=0).stop 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 msgr2=0x7f93941fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f9380003e80 tx=0x7f9380001640 comp rx=0 tx=0).stop 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 shutdown_connections 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9374073200 0x7f9374075710 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f93941fc750 0x7f93940ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9394098ca0 0x7f93941fc210 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 --2- 172.21.15.43:0/3246572699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f937c003b40 0x7f93941f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 >> 172.21.15.43:0/3246572699 conn(0x7f93940b23d0 msgr2=0x7f93940cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 shutdown_connections 2024-09-17T05:34:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:48.921+0000 7f939ba00640 1 -- 172.21.15.43:0/3246572699 wait complete. 2024-09-17T05:34:48.947 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T05:34:48.030474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":301688,"kb_used_data":14736,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754673032,"statfs":{"total":773094113280,"available":772785184768,"internally_reserved":0,"allocated":15089664,"data_stored":7845047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.571531"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167073+0000","last_change":"2024-09-17T05:34:44.497120+0000","last_active":"2024-09-17T05:34:46.167073+0000","last_peered":"2024-09-17T05:34:46.167073+0000","last_clean":"2024-09-17T05:34:46.167073+0000","last_became_active":"2024-09-17T05:34:44.496734+0000","last_became_peered":"2024-09-17T05:34:44.496734+0000","last_unstale":"2024-09-17T05:34:46.167073+0000","last_undegraded":"2024-09-17T05:34:46.167073+0000","last_fullsized":"2024-09-17T05:34:46.167073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:55.766746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166945+0000","last_change":"2024-09-17T05:34:44.504496+0000","last_active":"2024-09-17T05:34:46.166945+0000","last_peered":"2024-09-17T05:34:46.166945+0000","last_clean":"2024-09-17T05:34:46.166945+0000","last_became_active":"2024-09-17T05:34:44.504097+0000","last_became_peered":"2024-09-17T05:34:44.504097+0000","last_unstale":"2024-09-17T05:34:46.166945+0000","last_undegraded":"2024-09-17T05:34:46.166945+0000","last_fullsized":"2024-09-17T05:34:46.166945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:31.928876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466484+0000","last_change":"2024-09-17T05:34:43.466484+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466484+0000","last_undegraded":"2024-09-17T05:34:43.466484+0000","last_fullsized":"2024-09-17T05:34:43.466484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166881+0000","last_change":"2024-09-17T05:34:44.511242+0000","last_active":"2024-09-17T05:34:46.166881+0000","last_peered":"2024-09-17T05:34:46.166881+0000","last_clean":"2024-09-17T05:34:46.166881+0000","last_became_active":"2024-09-17T05:34:44.510544+0000","last_became_peered":"2024-09-17T05:34:44.510544+0000","last_unstale":"2024-09-17T05:34:46.166881+0000","last_undegraded":"2024-09-17T05:34:46.166881+0000","last_fullsized":"2024-09-17T05:34:46.166881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:16.675435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.469446+0000","last_change":"2024-09-17T05:34:43.469446+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.469446+0000","last_undegraded":"2024-09-17T05:34:43.469446+0000","last_fullsized":"2024-09-17T05:34:43.469446+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467679+0000","last_change":"2024-09-17T05:34:43.467679+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467679+0000","last_undegraded":"2024-09-17T05:34:43.467679+0000","last_fullsized":"2024-09-17T05:34:43.467679+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468846+0000","last_change":"2024-09-17T05:34:43.468846+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468846+0000","last_undegraded":"2024-09-17T05:34:43.468846+0000","last_fullsized":"2024-09-17T05:34:43.468846+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466911+0000","last_change":"2024-09-17T05:34:43.466911+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466911+0000","last_undegraded":"2024-09-17T05:34:43.466911+0000","last_fullsized":"2024-09-17T05:34:43.466911+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471899+0000","last_change":"2024-09-17T05:34:43.471899+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471899+0000","last_undegraded":"2024-09-17T05:34:43.471899+0000","last_fullsized":"2024-09-17T05:34:43.471899+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465962+0000","last_change":"2024-09-17T05:34:43.465962+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465962+0000","last_undegraded":"2024-09-17T05:34:43.465962+0000","last_fullsized":"2024-09-17T05:34:43.465962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471985+0000","last_change":"2024-09-17T05:34:43.471985+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471985+0000","last_undegraded":"2024-09-17T05:34:43.471985+0000","last_fullsized":"2024-09-17T05:34:43.471985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467723+0000","last_change":"2024-09-17T05:34:43.467723+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467723+0000","last_undegraded":"2024-09-17T05:34:43.467723+0000","last_fullsized":"2024-09-17T05:34:43.467723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474063+0000","last_change":"2024-09-17T05:34:43.474063+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474063+0000","last_undegraded":"2024-09-17T05:34:43.474063+0000","last_fullsized":"2024-09-17T05:34:43.474063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166241+0000","last_change":"2024-09-17T05:34:44.498246+0000","last_active":"2024-09-17T05:34:46.166241+0000","last_peered":"2024-09-17T05:34:46.166241+0000","last_clean":"2024-09-17T05:34:46.166241+0000","last_became_active":"2024-09-17T05:34:44.495932+0000","last_became_peered":"2024-09-17T05:34:44.495932+0000","last_unstale":"2024-09-17T05:34:46.166241+0000","last_undegraded":"2024-09-17T05:34:46.166241+0000","last_fullsized":"2024-09-17T05:34:46.166241+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:11.396023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166253+0000","last_change":"2024-09-17T05:34:44.503954+0000","last_active":"2024-09-17T05:34:46.166253+0000","last_peered":"2024-09-17T05:34:46.166253+0000","last_clean":"2024-09-17T05:34:46.166253+0000","last_became_active":"2024-09-17T05:34:44.502592+0000","last_became_peered":"2024-09-17T05:34:44.502592+0000","last_unstale":"2024-09-17T05:34:46.166253+0000","last_undegraded":"2024-09-17T05:34:46.166253+0000","last_fullsized":"2024-09-17T05:34:46.166253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:48.068029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468782+0000","last_change":"2024-09-17T05:34:43.468782+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468782+0000","last_undegraded":"2024-09-17T05:34:43.468782+0000","last_fullsized":"2024-09-17T05:34:43.468782+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475494+0000","last_change":"2024-09-17T05:34:43.475494+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475494+0000","last_undegraded":"2024-09-17T05:34:43.475494+0000","last_fullsized":"2024-09-17T05:34:43.475494+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167005+0000","last_change":"2024-09-17T05:34:44.501872+0000","last_active":"2024-09-17T05:34:46.167005+0000","last_peered":"2024-09-17T05:34:46.167005+0000","last_clean":"2024-09-17T05:34:46.167005+0000","last_became_active":"2024-09-17T05:34:44.498268+0000","last_became_peered":"2024-09-17T05:34:44.498268+0000","last_unstale":"2024-09-17T05:34:46.167005+0000","last_undegraded":"2024-09-17T05:34:46.167005+0000","last_fullsized":"2024-09-17T05:34:46.167005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:27.861661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473976+0000","last_change":"2024-09-17T05:34:43.473976+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473976+0000","last_undegraded":"2024-09-17T05:34:43.473976+0000","last_fullsized":"2024-09-17T05:34:43.473976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465620+0000","last_change":"2024-09-17T05:34:43.465620+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465620+0000","last_undegraded":"2024-09-17T05:34:43.465620+0000","last_fullsized":"2024-09-17T05:34:43.465620+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468129+0000","last_change":"2024-09-17T05:34:43.468129+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468129+0000","last_undegraded":"2024-09-17T05:34:43.468129+0000","last_fullsized":"2024-09-17T05:34:43.468129+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467304+0000","last_change":"2024-09-17T05:34:43.467304+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467304+0000","last_undegraded":"2024-09-17T05:34:43.467304+0000","last_fullsized":"2024-09-17T05:34:43.467304+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166332+0000","last_change":"2024-09-17T05:34:44.495822+0000","last_active":"2024-09-17T05:34:46.166332+0000","last_peered":"2024-09-17T05:34:46.166332+0000","last_clean":"2024-09-17T05:34:46.166332+0000","last_became_active":"2024-09-17T05:34:44.495403+0000","last_became_peered":"2024-09-17T05:34:44.495403+0000","last_unstale":"2024-09-17T05:34:46.166332+0000","last_undegraded":"2024-09-17T05:34:46.166332+0000","last_fullsized":"2024-09-17T05:34:46.166332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:15.566322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472479+0000","last_change":"2024-09-17T05:34:43.472479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472479+0000","last_undegraded":"2024-09-17T05:34:43.472479+0000","last_fullsized":"2024-09-17T05:34:43.472479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465223+0000","last_change":"2024-09-17T05:34:43.465223+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465223+0000","last_undegraded":"2024-09-17T05:34:43.465223+0000","last_fullsized":"2024-09-17T05:34:43.465223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475574+0000","last_change":"2024-09-17T05:34:43.475574+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475574+0000","last_undegraded":"2024-09-17T05:34:43.475574+0000","last_fullsized":"2024-09-17T05:34:43.475574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471952+0000","last_change":"2024-09-17T05:34:43.471952+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471952+0000","last_undegraded":"2024-09-17T05:34:43.471952+0000","last_fullsized":"2024-09-17T05:34:43.471952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476158+0000","last_change":"2024-09-17T05:34:43.476158+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476158+0000","last_undegraded":"2024-09-17T05:34:43.476158+0000","last_fullsized":"2024-09-17T05:34:43.476158+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473479+0000","last_change":"2024-09-17T05:34:43.473479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473479+0000","last_undegraded":"2024-09-17T05:34:43.473479+0000","last_fullsized":"2024-09-17T05:34:43.473479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473815+0000","last_change":"2024-09-17T05:34:43.473815+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473815+0000","last_undegraded":"2024-09-17T05:34:43.473815+0000","last_fullsized":"2024-09-17T05:34:43.473815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467581+0000","last_change":"2024-09-17T05:34:43.467581+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467581+0000","last_undegraded":"2024-09-17T05:34:43.467581+0000","last_fullsized":"2024-09-17T05:34:43.467581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166227+0000","last_change":"2024-09-17T05:34:44.515160+0000","last_active":"2024-09-17T05:34:46.166227+0000","last_peered":"2024-09-17T05:34:46.166227+0000","last_clean":"2024-09-17T05:34:46.166227+0000","last_became_active":"2024-09-17T05:34:44.514775+0000","last_became_peered":"2024-09-17T05:34:44.514775+0000","last_unstale":"2024-09-17T05:34:46.166227+0000","last_undegraded":"2024-09-17T05:34:46.166227+0000","last_fullsized":"2024-09-17T05:34:46.166227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:18.490230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.478231+0000","last_change":"2024-09-17T05:34:43.478231+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.478231+0000","last_undegraded":"2024-09-17T05:34:43.478231+0000","last_fullsized":"2024-09-17T05:34:43.478231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498701+0000","last_change":"2024-09-17T05:34:43.492447+0000","last_active":"2024-09-17T05:34:43.498701+0000","last_peered":"2024-09-17T05:34:43.498701+0000","last_clean":"2024-09-17T05:34:43.498701+0000","last_became_active":"2024-09-17T05:34:43.492128+0000","last_became_peered":"2024-09-17T05:34:43.492128+0000","last_unstale":"2024-09-17T05:34:43.498701+0000","last_undegraded":"2024-09-17T05:34:43.498701+0000","last_fullsized":"2024-09-17T05:34:43.498701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:37.605990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476119+0000","last_change":"2024-09-17T05:34:43.476119+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476119+0000","last_undegraded":"2024-09-17T05:34:43.476119+0000","last_fullsized":"2024-09-17T05:34:43.476119+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488969+0000","last_change":"2024-09-17T05:34:43.488048+0000","last_active":"2024-09-17T05:34:43.488969+0000","last_peered":"2024-09-17T05:34:43.488969+0000","last_clean":"2024-09-17T05:34:43.488969+0000","last_became_active":"2024-09-17T05:34:43.487637+0000","last_became_peered":"2024-09-17T05:34:43.487637+0000","last_unstale":"2024-09-17T05:34:43.488969+0000","last_undegraded":"2024-09-17T05:34:43.488969+0000","last_fullsized":"2024-09-17T05:34:43.488969+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:57.771444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166891+0000","last_change":"2024-09-17T05:34:44.497033+0000","last_active":"2024-09-17T05:34:46.166891+0000","last_peered":"2024-09-17T05:34:46.166891+0000","last_clean":"2024-09-17T05:34:46.166891+0000","last_became_active":"2024-09-17T05:34:44.496645+0000","last_became_peered":"2024-09-17T05:34:44.496645+0000","last_unstale":"2024-09-17T05:34:46.166891+0000","last_undegraded":"2024-09-17T05:34:46.166891+0000","last_fullsized":"2024-09-17T05:34:46.166891+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:37.177571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496221+0000","last_change":"2024-09-17T05:34:43.490759+0000","last_active":"2024-09-17T05:34:43.496221+0000","last_peered":"2024-09-17T05:34:43.496221+0000","last_clean":"2024-09-17T05:34:43.496221+0000","last_became_active":"2024-09-17T05:34:43.489512+0000","last_became_peered":"2024-09-17T05:34:43.489512+0000","last_unstale":"2024-09-17T05:34:43.496221+0000","last_undegraded":"2024-09-17T05:34:43.496221+0000","last_fullsized":"2024-09-17T05:34:43.496221+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465255+0000","last_change":"2024-09-17T05:34:43.465255+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465255+0000","last_undegraded":"2024-09-17T05:34:43.465255+0000","last_fullsized":"2024-09-17T05:34:43.465255+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166985+0000","last_change":"2024-09-17T05:34:43.497073+0000","last_active":"2024-09-17T05:34:46.166985+0000","last_peered":"2024-09-17T05:34:46.166985+0000","last_clean":"2024-09-17T05:34:46.166985+0000","last_became_active":"2024-09-17T05:34:43.496723+0000","last_became_peered":"2024-09-17T05:34:43.496723+0000","last_unstale":"2024-09-17T05:34:46.166985+0000","last_undegraded":"2024-09-17T05:34:46.166985+0000","last_fullsized":"2024-09-17T05:34:46.166985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:36.943277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464326+0000","last_change":"2024-09-17T05:34:43.464326+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464326+0000","last_undegraded":"2024-09-17T05:34:43.464326+0000","last_fullsized":"2024-09-17T05:34:43.464326+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166073+0000","last_change":"2024-09-17T05:34:43.484224+0000","last_active":"2024-09-17T05:34:46.166073+0000","last_peered":"2024-09-17T05:34:46.166073+0000","last_clean":"2024-09-17T05:34:46.166073+0000","last_became_active":"2024-09-17T05:34:43.483865+0000","last_became_peered":"2024-09-17T05:34:43.483865+0000","last_unstale":"2024-09-17T05:34:46.166073+0000","last_undegraded":"2024-09-17T05:34:46.166073+0000","last_fullsized":"2024-09-17T05:34:46.166073+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.065970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468869+0000","last_change":"2024-09-17T05:34:43.468869+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468869+0000","last_undegraded":"2024-09-17T05:34:43.468869+0000","last_fullsized":"2024-09-17T05:34:43.468869+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485165+0000","last_change":"2024-09-17T05:34:43.479766+0000","last_active":"2024-09-17T05:34:43.485165+0000","last_peered":"2024-09-17T05:34:43.485165+0000","last_clean":"2024-09-17T05:34:43.485165+0000","last_became_active":"2024-09-17T05:34:43.479411+0000","last_became_peered":"2024-09-17T05:34:43.479411+0000","last_unstale":"2024-09-17T05:34:43.485165+0000","last_undegraded":"2024-09-17T05:34:43.485165+0000","last_fullsized":"2024-09-17T05:34:43.485165+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474995+0000","last_change":"2024-09-17T05:34:43.474995+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474995+0000","last_undegraded":"2024-09-17T05:34:43.474995+0000","last_fullsized":"2024-09-17T05:34:43.474995+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486688+0000","last_change":"2024-09-17T05:34:43.485518+0000","last_active":"2024-09-17T05:34:43.486688+0000","last_peered":"2024-09-17T05:34:43.486688+0000","last_clean":"2024-09-17T05:34:43.486688+0000","last_became_active":"2024-09-17T05:34:43.485155+0000","last_became_peered":"2024-09-17T05:34:43.485155+0000","last_unstale":"2024-09-17T05:34:43.486688+0000","last_undegraded":"2024-09-17T05:34:43.486688+0000","last_fullsized":"2024-09-17T05:34:43.486688+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:15.459388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465390+0000","last_change":"2024-09-17T05:34:43.465390+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465390+0000","last_undegraded":"2024-09-17T05:34:43.465390+0000","last_fullsized":"2024-09-17T05:34:43.465390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496365+0000","last_change":"2024-09-17T05:34:43.492218+0000","last_active":"2024-09-17T05:34:43.496365+0000","last_peered":"2024-09-17T05:34:43.496365+0000","last_clean":"2024-09-17T05:34:43.496365+0000","last_became_active":"2024-09-17T05:34:43.491883+0000","last_became_peered":"2024-09-17T05:34:43.491883+0000","last_unstale":"2024-09-17T05:34:43.496365+0000","last_undegraded":"2024-09-17T05:34:43.496365+0000","last_fullsized":"2024-09-17T05:34:43.496365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:14.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167150+0000","last_change":"2024-09-17T05:34:44.510566+0000","last_active":"2024-09-17T05:34:46.167150+0000","last_peered":"2024-09-17T05:34:46.167150+0000","last_clean":"2024-09-17T05:34:46.167150+0000","last_became_active":"2024-09-17T05:34:44.510105+0000","last_became_peered":"2024-09-17T05:34:44.510105+0000","last_unstale":"2024-09-17T05:34:46.167150+0000","last_undegraded":"2024-09-17T05:34:46.167150+0000","last_fullsized":"2024-09-17T05:34:46.167150+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.973701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494247+0000","last_change":"2024-09-17T05:34:43.486372+0000","last_active":"2024-09-17T05:34:43.494247+0000","last_peered":"2024-09-17T05:34:43.494247+0000","last_clean":"2024-09-17T05:34:43.494247+0000","last_became_active":"2024-09-17T05:34:43.485972+0000","last_became_peered":"2024-09-17T05:34:43.485972+0000","last_unstale":"2024-09-17T05:34:43.494247+0000","last_undegraded":"2024-09-17T05:34:43.494247+0000","last_fullsized":"2024-09-17T05:34:43.494247+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.479109+0000","last_change":"2024-09-17T05:34:43.479109+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.479109+0000","last_undegraded":"2024-09-17T05:34:43.479109+0000","last_fullsized":"2024-09-17T05:34:43.479109+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167064+0000","last_change":"2024-09-17T05:34:43.488432+0000","last_active":"2024-09-17T05:34:46.167064+0000","last_peered":"2024-09-17T05:34:46.167064+0000","last_clean":"2024-09-17T05:34:46.167064+0000","last_became_active":"2024-09-17T05:34:43.487723+0000","last_became_peered":"2024-09-17T05:34:43.487723+0000","last_unstale":"2024-09-17T05:34:46.167064+0000","last_undegraded":"2024-09-17T05:34:46.167064+0000","last_fullsized":"2024-09-17T05:34:46.167064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:03.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.470033+0000","last_change":"2024-09-17T05:34:43.470033+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.470033+0000","last_undegraded":"2024-09-17T05:34:43.470033+0000","last_fullsized":"2024-09-17T05:34:43.470033+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484380+0000","last_change":"2024-09-17T05:34:43.482270+0000","last_active":"2024-09-17T05:34:43.484380+0000","last_peered":"2024-09-17T05:34:43.484380+0000","last_clean":"2024-09-17T05:34:43.484380+0000","last_became_active":"2024-09-17T05:34:43.481870+0000","last_became_peered":"2024-09-17T05:34:43.481870+0000","last_unstale":"2024-09-17T05:34:43.484380+0000","last_undegraded":"2024-09-17T05:34:43.484380+0000","last_fullsized":"2024-09-17T05:34:43.484380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.851530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473951+0000","last_change":"2024-09-17T05:34:43.473951+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473951+0000","last_undegraded":"2024-09-17T05:34:43.473951+0000","last_fullsized":"2024-09-17T05:34:43.473951+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485353+0000","last_change":"2024-09-17T05:34:43.483794+0000","last_active":"2024-09-17T05:34:43.485353+0000","last_peered":"2024-09-17T05:34:43.485353+0000","last_clean":"2024-09-17T05:34:43.485353+0000","last_became_active":"2024-09-17T05:34:43.483453+0000","last_became_peered":"2024-09-17T05:34:43.483453+0000","last_unstale":"2024-09-17T05:34:43.485353+0000","last_undegraded":"2024-09-17T05:34:43.485353+0000","last_fullsized":"2024-09-17T05:34:43.485353+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166308+0000","last_change":"2024-09-17T05:34:44.497676+0000","last_active":"2024-09-17T05:34:46.166308+0000","last_peered":"2024-09-17T05:34:46.166308+0000","last_clean":"2024-09-17T05:34:46.166308+0000","last_became_active":"2024-09-17T05:34:44.496021+0000","last_became_peered":"2024-09-17T05:34:44.496021+0000","last_unstale":"2024-09-17T05:34:46.166308+0000","last_undegraded":"2024-09-17T05:34:46.166308+0000","last_fullsized":"2024-09-17T05:34:46.166308+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.816080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491701+0000","last_change":"2024-09-17T05:34:43.486891+0000","last_active":"2024-09-17T05:34:43.491701+0000","last_peered":"2024-09-17T05:34:43.491701+0000","last_clean":"2024-09-17T05:34:43.491701+0000","last_became_active":"2024-09-17T05:34:43.485568+0000","last_became_peered":"2024-09-17T05:34:43.485568+0000","last_unstale":"2024-09-17T05:34:43.491701+0000","last_undegraded":"2024-09-17T05:34:43.491701+0000","last_fullsized":"2024-09-17T05:34:43.491701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:48.239606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467555+0000","last_change":"2024-09-17T05:34:43.467555+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467555+0000","last_undegraded":"2024-09-17T05:34:43.467555+0000","last_fullsized":"2024-09-17T05:34:43.467555+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166350+0000","last_change":"2024-09-17T05:34:43.490087+0000","last_active":"2024-09-17T05:34:46.166350+0000","last_peered":"2024-09-17T05:34:46.166350+0000","last_clean":"2024-09-17T05:34:46.166350+0000","last_became_active":"2024-09-17T05:34:43.488206+0000","last_became_peered":"2024-09-17T05:34:43.488206+0000","last_unstale":"2024-09-17T05:34:46.166350+0000","last_undegraded":"2024-09-17T05:34:46.166350+0000","last_fullsized":"2024-09-17T05:34:46.166350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:09.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467792+0000","last_change":"2024-09-17T05:34:43.467792+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467792+0000","last_undegraded":"2024-09-17T05:34:43.467792+0000","last_fullsized":"2024-09-17T05:34:43.467792+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482792+0000","last_change":"2024-09-17T05:34:43.480480+0000","last_active":"2024-09-17T05:34:43.482792+0000","last_peered":"2024-09-17T05:34:43.482792+0000","last_clean":"2024-09-17T05:34:43.482792+0000","last_became_active":"2024-09-17T05:34:43.480158+0000","last_became_peered":"2024-09-17T05:34:43.480158+0000","last_unstale":"2024-09-17T05:34:43.482792+0000","last_undegraded":"2024-09-17T05:34:43.482792+0000","last_fullsized":"2024-09-17T05:34:43.482792+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466318+0000","last_change":"2024-09-17T05:34:43.466318+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466318+0000","last_undegraded":"2024-09-17T05:34:43.466318+0000","last_fullsized":"2024-09-17T05:34:43.466318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"138'8","reported_seq":23,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166302+0000","last_change":"2024-09-17T05:34:43.480821+0000","last_active":"2024-09-17T05:34:46.166302+0000","last_peered":"2024-09-17T05:34:46.166302+0000","last_clean":"2024-09-17T05:34:46.166302+0000","last_became_active":"2024-09-17T05:34:43.480465+0000","last_became_peered":"2024-09-17T05:34:43.480465+0000","last_unstale":"2024-09-17T05:34:46.166302+0000","last_undegraded":"2024-09-17T05:34:46.166302+0000","last_fullsized":"2024-09-17T05:34:46.166302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:55.811669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477704+0000","last_change":"2024-09-17T05:34:43.477704+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477704+0000","last_undegraded":"2024-09-17T05:34:43.477704+0000","last_fullsized":"2024-09-17T05:34:43.477704+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.490985+0000","last_change":"2024-09-17T05:34:43.489754+0000","last_active":"2024-09-17T05:34:43.490985+0000","last_peered":"2024-09-17T05:34:43.490985+0000","last_clean":"2024-09-17T05:34:43.490985+0000","last_became_active":"2024-09-17T05:34:43.489338+0000","last_became_peered":"2024-09-17T05:34:43.489338+0000","last_unstale":"2024-09-17T05:34:43.490985+0000","last_undegraded":"2024-09-17T05:34:43.490985+0000","last_fullsized":"2024-09-17T05:34:43.490985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472867+0000","last_change":"2024-09-17T05:34:43.472867+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472867+0000","last_undegraded":"2024-09-17T05:34:43.472867+0000","last_fullsized":"2024-09-17T05:34:43.472867+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485935+0000","last_change":"2024-09-17T05:34:43.484097+0000","last_active":"2024-09-17T05:34:43.485935+0000","last_peered":"2024-09-17T05:34:43.485935+0000","last_clean":"2024-09-17T05:34:43.485935+0000","last_became_active":"2024-09-17T05:34:43.483720+0000","last_became_peered":"2024-09-17T05:34:43.483720+0000","last_unstale":"2024-09-17T05:34:43.485935+0000","last_undegraded":"2024-09-17T05:34:43.485935+0000","last_fullsized":"2024-09-17T05:34:43.485935+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:54.550132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464690+0000","last_change":"2024-09-17T05:34:43.464690+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464690+0000","last_undegraded":"2024-09-17T05:34:43.464690+0000","last_fullsized":"2024-09-17T05:34:43.464690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491259+0000","last_change":"2024-09-17T05:34:43.484124+0000","last_active":"2024-09-17T05:34:43.491259+0000","last_peered":"2024-09-17T05:34:43.491259+0000","last_clean":"2024-09-17T05:34:43.491259+0000","last_became_active":"2024-09-17T05:34:43.483814+0000","last_became_peered":"2024-09-17T05:34:43.483814+0000","last_unstale":"2024-09-17T05:34:43.491259+0000","last_undegraded":"2024-09-17T05:34:43.491259+0000","last_fullsized":"2024-09-17T05:34:43.491259+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:24.723614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472433+0000","last_change":"2024-09-17T05:34:43.472433+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472433+0000","last_undegraded":"2024-09-17T05:34:43.472433+0000","last_fullsized":"2024-09-17T05:34:43.472433+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166410+0000","last_change":"2024-09-17T05:34:43.488724+0000","last_active":"2024-09-17T05:34:46.166410+0000","last_peered":"2024-09-17T05:34:46.166410+0000","last_clean":"2024-09-17T05:34:46.166410+0000","last_became_active":"2024-09-17T05:34:43.488321+0000","last_became_peered":"2024-09-17T05:34:43.488321+0000","last_unstale":"2024-09-17T05:34:46.166410+0000","last_undegraded":"2024-09-17T05:34:46.166410+0000","last_fullsized":"2024-09-17T05:34:46.166410+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:31.408633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473697+0000","last_change":"2024-09-17T05:34:43.473697+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473697+0000","last_undegraded":"2024-09-17T05:34:43.473697+0000","last_fullsized":"2024-09-17T05:34:43.473697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488270+0000","last_change":"2024-09-17T05:34:43.486138+0000","last_active":"2024-09-17T05:34:43.488270+0000","last_peered":"2024-09-17T05:34:43.488270+0000","last_clean":"2024-09-17T05:34:43.488270+0000","last_became_active":"2024-09-17T05:34:43.485752+0000","last_became_peered":"2024-09-17T05:34:43.485752+0000","last_unstale":"2024-09-17T05:34:43.488270+0000","last_undegraded":"2024-09-17T05:34:43.488270+0000","last_fullsized":"2024-09-17T05:34:43.488270+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:25.830977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166207+0000","last_change":"2024-09-17T05:34:44.498312+0000","last_active":"2024-09-17T05:34:46.166207+0000","last_peered":"2024-09-17T05:34:46.166207+0000","last_clean":"2024-09-17T05:34:46.166207+0000","last_became_active":"2024-09-17T05:34:44.495324+0000","last_became_peered":"2024-09-17T05:34:44.495324+0000","last_unstale":"2024-09-17T05:34:46.166207+0000","last_undegraded":"2024-09-17T05:34:46.166207+0000","last_fullsized":"2024-09-17T05:34:46.166207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.787871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482748+0000","last_change":"2024-09-17T05:34:43.477604+0000","last_active":"2024-09-17T05:34:43.482748+0000","last_peered":"2024-09-17T05:34:43.482748+0000","last_clean":"2024-09-17T05:34:43.482748+0000","last_became_active":"2024-09-17T05:34:43.477206+0000","last_became_peered":"2024-09-17T05:34:43.477206+0000","last_unstale":"2024-09-17T05:34:43.482748+0000","last_undegraded":"2024-09-17T05:34:43.482748+0000","last_fullsized":"2024-09-17T05:34:43.482748+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:55.300288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472859+0000","last_change":"2024-09-17T05:34:43.472859+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472859+0000","last_undegraded":"2024-09-17T05:34:43.472859+0000","last_fullsized":"2024-09-17T05:34:43.472859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494923+0000","last_change":"2024-09-17T05:34:43.487120+0000","last_active":"2024-09-17T05:34:43.494923+0000","last_peered":"2024-09-17T05:34:43.494923+0000","last_clean":"2024-09-17T05:34:43.494923+0000","last_became_active":"2024-09-17T05:34:43.486767+0000","last_became_peered":"2024-09-17T05:34:43.486767+0000","last_unstale":"2024-09-17T05:34:43.494923+0000","last_undegraded":"2024-09-17T05:34:43.494923+0000","last_fullsized":"2024-09-17T05:34:43.494923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:43.570302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477665+0000","last_change":"2024-09-17T05:34:43.477665+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477665+0000","last_undegraded":"2024-09-17T05:34:43.477665+0000","last_fullsized":"2024-09-17T05:34:43.477665+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484272+0000","last_change":"2024-09-17T05:34:43.482392+0000","last_active":"2024-09-17T05:34:43.484272+0000","last_peered":"2024-09-17T05:34:43.484272+0000","last_clean":"2024-09-17T05:34:43.484272+0000","last_became_active":"2024-09-17T05:34:43.482075+0000","last_became_peered":"2024-09-17T05:34:43.482075+0000","last_unstale":"2024-09-17T05:34:43.484272+0000","last_undegraded":"2024-09-17T05:34:43.484272+0000","last_fullsized":"2024-09-17T05:34:43.484272+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474970+0000","last_change":"2024-09-17T05:34:43.474970+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474970+0000","last_undegraded":"2024-09-17T05:34:43.474970+0000","last_fullsized":"2024-09-17T05:34:43.474970+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498847+0000","last_change":"2024-09-17T05:34:43.493846+0000","last_active":"2024-09-17T05:34:43.498847+0000","last_peered":"2024-09-17T05:34:43.498847+0000","last_clean":"2024-09-17T05:34:43.498847+0000","last_became_active":"2024-09-17T05:34:43.493493+0000","last_became_peered":"2024-09-17T05:34:43.493493+0000","last_unstale":"2024-09-17T05:34:43.498847+0000","last_undegraded":"2024-09-17T05:34:43.498847+0000","last_fullsized":"2024-09-17T05:34:43.498847+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:46.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466430+0000","last_change":"2024-09-17T05:34:43.466430+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466430+0000","last_undegraded":"2024-09-17T05:34:43.466430+0000","last_fullsized":"2024-09-17T05:34:43.466430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491753+0000","last_change":"2024-09-17T05:34:43.487845+0000","last_active":"2024-09-17T05:34:43.491753+0000","last_peered":"2024-09-17T05:34:43.491753+0000","last_clean":"2024-09-17T05:34:43.491753+0000","last_became_active":"2024-09-17T05:34:43.487537+0000","last_became_peered":"2024-09-17T05:34:43.487537+0000","last_unstale":"2024-09-17T05:34:43.491753+0000","last_undegraded":"2024-09-17T05:34:43.491753+0000","last_fullsized":"2024-09-17T05:34:43.491753+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:10.323580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471452+0000","last_change":"2024-09-17T05:34:43.471452+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471452+0000","last_undegraded":"2024-09-17T05:34:43.471452+0000","last_fullsized":"2024-09-17T05:34:43.471452+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488918+0000","last_change":"2024-09-17T05:34:43.487549+0000","last_active":"2024-09-17T05:34:43.488918+0000","last_peered":"2024-09-17T05:34:43.488918+0000","last_clean":"2024-09-17T05:34:43.488918+0000","last_became_active":"2024-09-17T05:34:43.487089+0000","last_became_peered":"2024-09-17T05:34:43.487089+0000","last_unstale":"2024-09-17T05:34:43.488918+0000","last_undegraded":"2024-09-17T05:34:43.488918+0000","last_fullsized":"2024-09-17T05:34:43.488918+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:33.360373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471776+0000","last_change":"2024-09-17T05:34:43.471776+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471776+0000","last_undegraded":"2024-09-17T05:34:43.471776+0000","last_fullsized":"2024-09-17T05:34:43.471776+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498988+0000","last_change":"2024-09-17T05:34:43.495216+0000","last_active":"2024-09-17T05:34:43.498988+0000","last_peered":"2024-09-17T05:34:43.498988+0000","last_clean":"2024-09-17T05:34:43.498988+0000","last_became_active":"2024-09-17T05:34:43.494800+0000","last_became_peered":"2024-09-17T05:34:43.494800+0000","last_unstale":"2024-09-17T05:34:43.498988+0000","last_undegraded":"2024-09-17T05:34:43.498988+0000","last_fullsized":"2024-09-17T05:34:43.498988+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:03.650153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468214+0000","last_change":"2024-09-17T05:34:43.468214+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468214+0000","last_undegraded":"2024-09-17T05:34:43.468214+0000","last_fullsized":"2024-09-17T05:34:43.468214+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.492048+0000","last_change":"2024-09-17T05:34:43.491113+0000","last_active":"2024-09-17T05:34:43.492048+0000","last_peered":"2024-09-17T05:34:43.492048+0000","last_clean":"2024-09-17T05:34:43.492048+0000","last_became_active":"2024-09-17T05:34:43.490776+0000","last_became_peered":"2024-09-17T05:34:43.490776+0000","last_unstale":"2024-09-17T05:34:43.492048+0000","last_undegraded":"2024-09-17T05:34:43.492048+0000","last_fullsized":"2024-09-17T05:34:43.492048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:22.282026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476081+0000","last_change":"2024-09-17T05:34:43.476081+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476081+0000","last_undegraded":"2024-09-17T05:34:43.476081+0000","last_fullsized":"2024-09-17T05:34:43.476081+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496135+0000","last_change":"2024-09-17T05:34:43.490904+0000","last_active":"2024-09-17T05:34:43.496135+0000","last_peered":"2024-09-17T05:34:43.496135+0000","last_clean":"2024-09-17T05:34:43.496135+0000","last_became_active":"2024-09-17T05:34:43.490596+0000","last_became_peered":"2024-09-17T05:34:43.490596+0000","last_unstale":"2024-09-17T05:34:43.496135+0000","last_undegraded":"2024-09-17T05:34:43.496135+0000","last_fullsized":"2024-09-17T05:34:43.496135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:45.332471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464313+0000","last_change":"2024-09-17T05:34:43.464313+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464313+0000","last_undegraded":"2024-09-17T05:34:43.464313+0000","last_fullsized":"2024-09-17T05:34:43.464313+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494892+0000","last_change":"2024-09-17T05:34:43.485943+0000","last_active":"2024-09-17T05:34:43.494892+0000","last_peered":"2024-09-17T05:34:43.494892+0000","last_clean":"2024-09-17T05:34:43.494892+0000","last_became_active":"2024-09-17T05:34:43.485578+0000","last_became_peered":"2024-09-17T05:34:43.485578+0000","last_unstale":"2024-09-17T05:34:43.494892+0000","last_undegraded":"2024-09-17T05:34:43.494892+0000","last_fullsized":"2024-09-17T05:34:43.494892+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467211+0000","last_change":"2024-09-17T05:34:43.467211+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467211+0000","last_undegraded":"2024-09-17T05:34:43.467211+0000","last_fullsized":"2024-09-17T05:34:43.467211+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486113+0000","last_change":"2024-09-17T05:34:43.483784+0000","last_active":"2024-09-17T05:34:43.486113+0000","last_peered":"2024-09-17T05:34:43.486113+0000","last_clean":"2024-09-17T05:34:43.486113+0000","last_became_active":"2024-09-17T05:34:43.483417+0000","last_became_peered":"2024-09-17T05:34:43.483417+0000","last_unstale":"2024-09-17T05:34:43.486113+0000","last_undegraded":"2024-09-17T05:34:43.486113+0000","last_fullsized":"2024-09-17T05:34:43.486113+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:39.573633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":337,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.470110+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:34:43.470110+0000","last_peered":"2024-09-17T05:34:43.470110+0000","last_clean":"2024-09-17T05:34:43.470110+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:34:43.470110+0000","last_undegraded":"2024-09-17T05:34:43.470110+0000","last_fullsized":"2024-09-17T05:34:43.470110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509582,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":834542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45356,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326484,"statfs":{"total":96636764160,"available":96590319616,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41232,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330608,"statfs":{"total":96636764160,"available":96594542592,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:48.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- 172.21.15.43:0/2456738936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed64155d70 msgr2=0x7fed6415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 --2- 172.21.15.43:0/2456738936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed64155d70 0x7fed6415c7e0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fed640af0a0 tx=0x7fed5401c620 comp rx=0 tx=0).stop 2024-09-17T05:34:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- 172.21.15.43:0/2456738936 shutdown_connections 2024-09-17T05:34:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 --2- 172.21.15.43:0/2456738936 >> v2:172.21.15.62:3300/0 conn(0x7fed6406f110 0x7fed640b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 --2- 172.21.15.43:0/2456738936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed64155d70 0x7fed6415c7e0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- 172.21.15.43:0/2456738936 >> 172.21.15.43:0/2456738936 conn(0x7fed640b23d0 msgr2=0x7fed640d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- 172.21.15.43:0/2456738936 shutdown_connections 2024-09-17T05:34:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- 172.21.15.43:0/2456738936 wait complete. 2024-09-17T05:34:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 Processor -- start 2024-09-17T05:34:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.149+0000 7fed69600640 1 -- start start 2024-09-17T05:34:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed640b3920 0x7fed641f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed64155d70 0x7fed641fa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed64168a20 con 0x7fed6406f110 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed641688a0 con 0x7fed640b3920 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fed64168ba0 con 0x7fed64155d70 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54206/0 (socket says 172.21.15.43:54206) 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 -- 172.21.15.43:0/39829700 learned_addr learned my addr 172.21.15.43:0/39829700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed5a000640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed640b3920 0x7fed641f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed63400640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed64155d70 0x7fed641fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 -- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed64155d70 msgr2=0x7fed641fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed64155d70 0x7fed641fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 -- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed640b3920 msgr2=0x7fed641f9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed640b3920 0x7fed641f9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 -- 172.21.15.43:0/39829700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed64155440 con 0x7fed6406f110 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed62a00640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fed54098180 tx=0x7fed540a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed540ac070 con 0x7fed6406f110 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed54004320 con 0x7fed6406f110 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed640b00b0 con 0x7fed6406f110 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54005320 con 0x7fed6406f110 2024-09-17T05:34:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed641fabd0 con 0x7fed6406f110 2024-09-17T05:34:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.153+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed640b33e0 con 0x7fed6406f110 2024-09-17T05:34:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.157+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fed540af070 con 0x7fed6406f110 2024-09-17T05:34:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.157+0000 7fed5b400640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 0x7fed40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.157+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fed5412e160 con 0x7fed6406f110 2024-09-17T05:34:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.157+0000 7fed5a000640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 0x7fed40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.157+0000 7fed5a000640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 0x7fed40075740 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fed640c9bf0 tx=0x7fed4c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.161+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fed640b3d30 con 0x7fed6406f110 2024-09-17T05:34:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.333+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed640afe60 con 0x7fed40073230 2024-09-17T05:34:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.337+0000 7fed5b400640 1 -- 172.21.15.43:0/39829700 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248108 (secure 0 0 0) 0x7fed640afe60 con 0x7fed40073230 2024-09-17T05:34:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:49.346 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:34:49.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 msgr2=0x7fed40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 0x7fed40075740 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fed640c9bf0 tx=0x7fed4c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 msgr2=0x7fed64154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fed54098180 tx=0x7fed540a64f0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 shutdown_connections 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed40073230 0x7fed40075740 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed640b3920 0x7fed641f9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fed64155d70 0x7fed641fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 --2- 172.21.15.43:0/39829700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6406f110 0x7fed64154df0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 >> 172.21.15.43:0/39829700 conn(0x7fed640b23d0 msgr2=0x7fed6409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 shutdown_connections 2024-09-17T05:34:49.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.341+0000 7fed69600640 1 -- 172.21.15.43:0/39829700 wait complete. 2024-09-17T05:34:49.365 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T05:34:48.030474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":301688,"kb_used_data":14736,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754673032,"statfs":{"total":773094113280,"available":772785184768,"internally_reserved":0,"allocated":15089664,"data_stored":7845047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.571531"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167073+0000","last_change":"2024-09-17T05:34:44.497120+0000","last_active":"2024-09-17T05:34:46.167073+0000","last_peered":"2024-09-17T05:34:46.167073+0000","last_clean":"2024-09-17T05:34:46.167073+0000","last_became_active":"2024-09-17T05:34:44.496734+0000","last_became_peered":"2024-09-17T05:34:44.496734+0000","last_unstale":"2024-09-17T05:34:46.167073+0000","last_undegraded":"2024-09-17T05:34:46.167073+0000","last_fullsized":"2024-09-17T05:34:46.167073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:55.766746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166945+0000","last_change":"2024-09-17T05:34:44.504496+0000","last_active":"2024-09-17T05:34:46.166945+0000","last_peered":"2024-09-17T05:34:46.166945+0000","last_clean":"2024-09-17T05:34:46.166945+0000","last_became_active":"2024-09-17T05:34:44.504097+0000","last_became_peered":"2024-09-17T05:34:44.504097+0000","last_unstale":"2024-09-17T05:34:46.166945+0000","last_undegraded":"2024-09-17T05:34:46.166945+0000","last_fullsized":"2024-09-17T05:34:46.166945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:31.928876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466484+0000","last_change":"2024-09-17T05:34:43.466484+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466484+0000","last_undegraded":"2024-09-17T05:34:43.466484+0000","last_fullsized":"2024-09-17T05:34:43.466484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166881+0000","last_change":"2024-09-17T05:34:44.511242+0000","last_active":"2024-09-17T05:34:46.166881+0000","last_peered":"2024-09-17T05:34:46.166881+0000","last_clean":"2024-09-17T05:34:46.166881+0000","last_became_active":"2024-09-17T05:34:44.510544+0000","last_became_peered":"2024-09-17T05:34:44.510544+0000","last_unstale":"2024-09-17T05:34:46.166881+0000","last_undegraded":"2024-09-17T05:34:46.166881+0000","last_fullsized":"2024-09-17T05:34:46.166881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:16.675435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.469446+0000","last_change":"2024-09-17T05:34:43.469446+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.469446+0000","last_undegraded":"2024-09-17T05:34:43.469446+0000","last_fullsized":"2024-09-17T05:34:43.469446+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467679+0000","last_change":"2024-09-17T05:34:43.467679+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467679+0000","last_undegraded":"2024-09-17T05:34:43.467679+0000","last_fullsized":"2024-09-17T05:34:43.467679+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468846+0000","last_change":"2024-09-17T05:34:43.468846+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468846+0000","last_undegraded":"2024-09-17T05:34:43.468846+0000","last_fullsized":"2024-09-17T05:34:43.468846+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466911+0000","last_change":"2024-09-17T05:34:43.466911+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466911+0000","last_undegraded":"2024-09-17T05:34:43.466911+0000","last_fullsized":"2024-09-17T05:34:43.466911+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471899+0000","last_change":"2024-09-17T05:34:43.471899+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471899+0000","last_undegraded":"2024-09-17T05:34:43.471899+0000","last_fullsized":"2024-09-17T05:34:43.471899+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465962+0000","last_change":"2024-09-17T05:34:43.465962+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465962+0000","last_undegraded":"2024-09-17T05:34:43.465962+0000","last_fullsized":"2024-09-17T05:34:43.465962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471985+0000","last_change":"2024-09-17T05:34:43.471985+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471985+0000","last_undegraded":"2024-09-17T05:34:43.471985+0000","last_fullsized":"2024-09-17T05:34:43.471985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467723+0000","last_change":"2024-09-17T05:34:43.467723+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467723+0000","last_undegraded":"2024-09-17T05:34:43.467723+0000","last_fullsized":"2024-09-17T05:34:43.467723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474063+0000","last_change":"2024-09-17T05:34:43.474063+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474063+0000","last_undegraded":"2024-09-17T05:34:43.474063+0000","last_fullsized":"2024-09-17T05:34:43.474063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166241+0000","last_change":"2024-09-17T05:34:44.498246+0000","last_active":"2024-09-17T05:34:46.166241+0000","last_peered":"2024-09-17T05:34:46.166241+0000","last_clean":"2024-09-17T05:34:46.166241+0000","last_became_active":"2024-09-17T05:34:44.495932+0000","last_became_peered":"2024-09-17T05:34:44.495932+0000","last_unstale":"2024-09-17T05:34:46.166241+0000","last_undegraded":"2024-09-17T05:34:46.166241+0000","last_fullsized":"2024-09-17T05:34:46.166241+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:11.396023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166253+0000","last_change":"2024-09-17T05:34:44.503954+0000","last_active":"2024-09-17T05:34:46.166253+0000","last_peered":"2024-09-17T05:34:46.166253+0000","last_clean":"2024-09-17T05:34:46.166253+0000","last_became_active":"2024-09-17T05:34:44.502592+0000","last_became_peered":"2024-09-17T05:34:44.502592+0000","last_unstale":"2024-09-17T05:34:46.166253+0000","last_undegraded":"2024-09-17T05:34:46.166253+0000","last_fullsized":"2024-09-17T05:34:46.166253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:48.068029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468782+0000","last_change":"2024-09-17T05:34:43.468782+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468782+0000","last_undegraded":"2024-09-17T05:34:43.468782+0000","last_fullsized":"2024-09-17T05:34:43.468782+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475494+0000","last_change":"2024-09-17T05:34:43.475494+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475494+0000","last_undegraded":"2024-09-17T05:34:43.475494+0000","last_fullsized":"2024-09-17T05:34:43.475494+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167005+0000","last_change":"2024-09-17T05:34:44.501872+0000","last_active":"2024-09-17T05:34:46.167005+0000","last_peered":"2024-09-17T05:34:46.167005+0000","last_clean":"2024-09-17T05:34:46.167005+0000","last_became_active":"2024-09-17T05:34:44.498268+0000","last_became_peered":"2024-09-17T05:34:44.498268+0000","last_unstale":"2024-09-17T05:34:46.167005+0000","last_undegraded":"2024-09-17T05:34:46.167005+0000","last_fullsized":"2024-09-17T05:34:46.167005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:27.861661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473976+0000","last_change":"2024-09-17T05:34:43.473976+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473976+0000","last_undegraded":"2024-09-17T05:34:43.473976+0000","last_fullsized":"2024-09-17T05:34:43.473976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465620+0000","last_change":"2024-09-17T05:34:43.465620+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465620+0000","last_undegraded":"2024-09-17T05:34:43.465620+0000","last_fullsized":"2024-09-17T05:34:43.465620+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468129+0000","last_change":"2024-09-17T05:34:43.468129+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468129+0000","last_undegraded":"2024-09-17T05:34:43.468129+0000","last_fullsized":"2024-09-17T05:34:43.468129+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467304+0000","last_change":"2024-09-17T05:34:43.467304+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467304+0000","last_undegraded":"2024-09-17T05:34:43.467304+0000","last_fullsized":"2024-09-17T05:34:43.467304+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166332+0000","last_change":"2024-09-17T05:34:44.495822+0000","last_active":"2024-09-17T05:34:46.166332+0000","last_peered":"2024-09-17T05:34:46.166332+0000","last_clean":"2024-09-17T05:34:46.166332+0000","last_became_active":"2024-09-17T05:34:44.495403+0000","last_became_peered":"2024-09-17T05:34:44.495403+0000","last_unstale":"2024-09-17T05:34:46.166332+0000","last_undegraded":"2024-09-17T05:34:46.166332+0000","last_fullsized":"2024-09-17T05:34:46.166332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:15.566322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472479+0000","last_change":"2024-09-17T05:34:43.472479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472479+0000","last_undegraded":"2024-09-17T05:34:43.472479+0000","last_fullsized":"2024-09-17T05:34:43.472479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465223+0000","last_change":"2024-09-17T05:34:43.465223+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465223+0000","last_undegraded":"2024-09-17T05:34:43.465223+0000","last_fullsized":"2024-09-17T05:34:43.465223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475574+0000","last_change":"2024-09-17T05:34:43.475574+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475574+0000","last_undegraded":"2024-09-17T05:34:43.475574+0000","last_fullsized":"2024-09-17T05:34:43.475574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471952+0000","last_change":"2024-09-17T05:34:43.471952+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471952+0000","last_undegraded":"2024-09-17T05:34:43.471952+0000","last_fullsized":"2024-09-17T05:34:43.471952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476158+0000","last_change":"2024-09-17T05:34:43.476158+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476158+0000","last_undegraded":"2024-09-17T05:34:43.476158+0000","last_fullsized":"2024-09-17T05:34:43.476158+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473479+0000","last_change":"2024-09-17T05:34:43.473479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473479+0000","last_undegraded":"2024-09-17T05:34:43.473479+0000","last_fullsized":"2024-09-17T05:34:43.473479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473815+0000","last_change":"2024-09-17T05:34:43.473815+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473815+0000","last_undegraded":"2024-09-17T05:34:43.473815+0000","last_fullsized":"2024-09-17T05:34:43.473815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467581+0000","last_change":"2024-09-17T05:34:43.467581+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467581+0000","last_undegraded":"2024-09-17T05:34:43.467581+0000","last_fullsized":"2024-09-17T05:34:43.467581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166227+0000","last_change":"2024-09-17T05:34:44.515160+0000","last_active":"2024-09-17T05:34:46.166227+0000","last_peered":"2024-09-17T05:34:46.166227+0000","last_clean":"2024-09-17T05:34:46.166227+0000","last_became_active":"2024-09-17T05:34:44.514775+0000","last_became_peered":"2024-09-17T05:34:44.514775+0000","last_unstale":"2024-09-17T05:34:46.166227+0000","last_undegraded":"2024-09-17T05:34:46.166227+0000","last_fullsized":"2024-09-17T05:34:46.166227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:18.490230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.478231+0000","last_change":"2024-09-17T05:34:43.478231+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.478231+0000","last_undegraded":"2024-09-17T05:34:43.478231+0000","last_fullsized":"2024-09-17T05:34:43.478231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498701+0000","last_change":"2024-09-17T05:34:43.492447+0000","last_active":"2024-09-17T05:34:43.498701+0000","last_peered":"2024-09-17T05:34:43.498701+0000","last_clean":"2024-09-17T05:34:43.498701+0000","last_became_active":"2024-09-17T05:34:43.492128+0000","last_became_peered":"2024-09-17T05:34:43.492128+0000","last_unstale":"2024-09-17T05:34:43.498701+0000","last_undegraded":"2024-09-17T05:34:43.498701+0000","last_fullsized":"2024-09-17T05:34:43.498701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:37.605990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476119+0000","last_change":"2024-09-17T05:34:43.476119+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476119+0000","last_undegraded":"2024-09-17T05:34:43.476119+0000","last_fullsized":"2024-09-17T05:34:43.476119+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488969+0000","last_change":"2024-09-17T05:34:43.488048+0000","last_active":"2024-09-17T05:34:43.488969+0000","last_peered":"2024-09-17T05:34:43.488969+0000","last_clean":"2024-09-17T05:34:43.488969+0000","last_became_active":"2024-09-17T05:34:43.487637+0000","last_became_peered":"2024-09-17T05:34:43.487637+0000","last_unstale":"2024-09-17T05:34:43.488969+0000","last_undegraded":"2024-09-17T05:34:43.488969+0000","last_fullsized":"2024-09-17T05:34:43.488969+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:57.771444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166891+0000","last_change":"2024-09-17T05:34:44.497033+0000","last_active":"2024-09-17T05:34:46.166891+0000","last_peered":"2024-09-17T05:34:46.166891+0000","last_clean":"2024-09-17T05:34:46.166891+0000","last_became_active":"2024-09-17T05:34:44.496645+0000","last_became_peered":"2024-09-17T05:34:44.496645+0000","last_unstale":"2024-09-17T05:34:46.166891+0000","last_undegraded":"2024-09-17T05:34:46.166891+0000","last_fullsized":"2024-09-17T05:34:46.166891+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:37.177571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496221+0000","last_change":"2024-09-17T05:34:43.490759+0000","last_active":"2024-09-17T05:34:43.496221+0000","last_peered":"2024-09-17T05:34:43.496221+0000","last_clean":"2024-09-17T05:34:43.496221+0000","last_became_active":"2024-09-17T05:34:43.489512+0000","last_became_peered":"2024-09-17T05:34:43.489512+0000","last_unstale":"2024-09-17T05:34:43.496221+0000","last_undegraded":"2024-09-17T05:34:43.496221+0000","last_fullsized":"2024-09-17T05:34:43.496221+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465255+0000","last_change":"2024-09-17T05:34:43.465255+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465255+0000","last_undegraded":"2024-09-17T05:34:43.465255+0000","last_fullsized":"2024-09-17T05:34:43.465255+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166985+0000","last_change":"2024-09-17T05:34:43.497073+0000","last_active":"2024-09-17T05:34:46.166985+0000","last_peered":"2024-09-17T05:34:46.166985+0000","last_clean":"2024-09-17T05:34:46.166985+0000","last_became_active":"2024-09-17T05:34:43.496723+0000","last_became_peered":"2024-09-17T05:34:43.496723+0000","last_unstale":"2024-09-17T05:34:46.166985+0000","last_undegraded":"2024-09-17T05:34:46.166985+0000","last_fullsized":"2024-09-17T05:34:46.166985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:36.943277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464326+0000","last_change":"2024-09-17T05:34:43.464326+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464326+0000","last_undegraded":"2024-09-17T05:34:43.464326+0000","last_fullsized":"2024-09-17T05:34:43.464326+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166073+0000","last_change":"2024-09-17T05:34:43.484224+0000","last_active":"2024-09-17T05:34:46.166073+0000","last_peered":"2024-09-17T05:34:46.166073+0000","last_clean":"2024-09-17T05:34:46.166073+0000","last_became_active":"2024-09-17T05:34:43.483865+0000","last_became_peered":"2024-09-17T05:34:43.483865+0000","last_unstale":"2024-09-17T05:34:46.166073+0000","last_undegraded":"2024-09-17T05:34:46.166073+0000","last_fullsized":"2024-09-17T05:34:46.166073+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.065970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468869+0000","last_change":"2024-09-17T05:34:43.468869+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468869+0000","last_undegraded":"2024-09-17T05:34:43.468869+0000","last_fullsized":"2024-09-17T05:34:43.468869+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485165+0000","last_change":"2024-09-17T05:34:43.479766+0000","last_active":"2024-09-17T05:34:43.485165+0000","last_peered":"2024-09-17T05:34:43.485165+0000","last_clean":"2024-09-17T05:34:43.485165+0000","last_became_active":"2024-09-17T05:34:43.479411+0000","last_became_peered":"2024-09-17T05:34:43.479411+0000","last_unstale":"2024-09-17T05:34:43.485165+0000","last_undegraded":"2024-09-17T05:34:43.485165+0000","last_fullsized":"2024-09-17T05:34:43.485165+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474995+0000","last_change":"2024-09-17T05:34:43.474995+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474995+0000","last_undegraded":"2024-09-17T05:34:43.474995+0000","last_fullsized":"2024-09-17T05:34:43.474995+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486688+0000","last_change":"2024-09-17T05:34:43.485518+0000","last_active":"2024-09-17T05:34:43.486688+0000","last_peered":"2024-09-17T05:34:43.486688+0000","last_clean":"2024-09-17T05:34:43.486688+0000","last_became_active":"2024-09-17T05:34:43.485155+0000","last_became_peered":"2024-09-17T05:34:43.485155+0000","last_unstale":"2024-09-17T05:34:43.486688+0000","last_undegraded":"2024-09-17T05:34:43.486688+0000","last_fullsized":"2024-09-17T05:34:43.486688+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:15.459388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465390+0000","last_change":"2024-09-17T05:34:43.465390+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465390+0000","last_undegraded":"2024-09-17T05:34:43.465390+0000","last_fullsized":"2024-09-17T05:34:43.465390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496365+0000","last_change":"2024-09-17T05:34:43.492218+0000","last_active":"2024-09-17T05:34:43.496365+0000","last_peered":"2024-09-17T05:34:43.496365+0000","last_clean":"2024-09-17T05:34:43.496365+0000","last_became_active":"2024-09-17T05:34:43.491883+0000","last_became_peered":"2024-09-17T05:34:43.491883+0000","last_unstale":"2024-09-17T05:34:43.496365+0000","last_undegraded":"2024-09-17T05:34:43.496365+0000","last_fullsized":"2024-09-17T05:34:43.496365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:14.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167150+0000","last_change":"2024-09-17T05:34:44.510566+0000","last_active":"2024-09-17T05:34:46.167150+0000","last_peered":"2024-09-17T05:34:46.167150+0000","last_clean":"2024-09-17T05:34:46.167150+0000","last_became_active":"2024-09-17T05:34:44.510105+0000","last_became_peered":"2024-09-17T05:34:44.510105+0000","last_unstale":"2024-09-17T05:34:46.167150+0000","last_undegraded":"2024-09-17T05:34:46.167150+0000","last_fullsized":"2024-09-17T05:34:46.167150+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.973701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494247+0000","last_change":"2024-09-17T05:34:43.486372+0000","last_active":"2024-09-17T05:34:43.494247+0000","last_peered":"2024-09-17T05:34:43.494247+0000","last_clean":"2024-09-17T05:34:43.494247+0000","last_became_active":"2024-09-17T05:34:43.485972+0000","last_became_peered":"2024-09-17T05:34:43.485972+0000","last_unstale":"2024-09-17T05:34:43.494247+0000","last_undegraded":"2024-09-17T05:34:43.494247+0000","last_fullsized":"2024-09-17T05:34:43.494247+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.479109+0000","last_change":"2024-09-17T05:34:43.479109+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.479109+0000","last_undegraded":"2024-09-17T05:34:43.479109+0000","last_fullsized":"2024-09-17T05:34:43.479109+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167064+0000","last_change":"2024-09-17T05:34:43.488432+0000","last_active":"2024-09-17T05:34:46.167064+0000","last_peered":"2024-09-17T05:34:46.167064+0000","last_clean":"2024-09-17T05:34:46.167064+0000","last_became_active":"2024-09-17T05:34:43.487723+0000","last_became_peered":"2024-09-17T05:34:43.487723+0000","last_unstale":"2024-09-17T05:34:46.167064+0000","last_undegraded":"2024-09-17T05:34:46.167064+0000","last_fullsized":"2024-09-17T05:34:46.167064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:03.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.470033+0000","last_change":"2024-09-17T05:34:43.470033+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.470033+0000","last_undegraded":"2024-09-17T05:34:43.470033+0000","last_fullsized":"2024-09-17T05:34:43.470033+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484380+0000","last_change":"2024-09-17T05:34:43.482270+0000","last_active":"2024-09-17T05:34:43.484380+0000","last_peered":"2024-09-17T05:34:43.484380+0000","last_clean":"2024-09-17T05:34:43.484380+0000","last_became_active":"2024-09-17T05:34:43.481870+0000","last_became_peered":"2024-09-17T05:34:43.481870+0000","last_unstale":"2024-09-17T05:34:43.484380+0000","last_undegraded":"2024-09-17T05:34:43.484380+0000","last_fullsized":"2024-09-17T05:34:43.484380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.851530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473951+0000","last_change":"2024-09-17T05:34:43.473951+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473951+0000","last_undegraded":"2024-09-17T05:34:43.473951+0000","last_fullsized":"2024-09-17T05:34:43.473951+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485353+0000","last_change":"2024-09-17T05:34:43.483794+0000","last_active":"2024-09-17T05:34:43.485353+0000","last_peered":"2024-09-17T05:34:43.485353+0000","last_clean":"2024-09-17T05:34:43.485353+0000","last_became_active":"2024-09-17T05:34:43.483453+0000","last_became_peered":"2024-09-17T05:34:43.483453+0000","last_unstale":"2024-09-17T05:34:43.485353+0000","last_undegraded":"2024-09-17T05:34:43.485353+0000","last_fullsized":"2024-09-17T05:34:43.485353+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166308+0000","last_change":"2024-09-17T05:34:44.497676+0000","last_active":"2024-09-17T05:34:46.166308+0000","last_peered":"2024-09-17T05:34:46.166308+0000","last_clean":"2024-09-17T05:34:46.166308+0000","last_became_active":"2024-09-17T05:34:44.496021+0000","last_became_peered":"2024-09-17T05:34:44.496021+0000","last_unstale":"2024-09-17T05:34:46.166308+0000","last_undegraded":"2024-09-17T05:34:46.166308+0000","last_fullsized":"2024-09-17T05:34:46.166308+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.816080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491701+0000","last_change":"2024-09-17T05:34:43.486891+0000","last_active":"2024-09-17T05:34:43.491701+0000","last_peered":"2024-09-17T05:34:43.491701+0000","last_clean":"2024-09-17T05:34:43.491701+0000","last_became_active":"2024-09-17T05:34:43.485568+0000","last_became_peered":"2024-09-17T05:34:43.485568+0000","last_unstale":"2024-09-17T05:34:43.491701+0000","last_undegraded":"2024-09-17T05:34:43.491701+0000","last_fullsized":"2024-09-17T05:34:43.491701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:48.239606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467555+0000","last_change":"2024-09-17T05:34:43.467555+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467555+0000","last_undegraded":"2024-09-17T05:34:43.467555+0000","last_fullsized":"2024-09-17T05:34:43.467555+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166350+0000","last_change":"2024-09-17T05:34:43.490087+0000","last_active":"2024-09-17T05:34:46.166350+0000","last_peered":"2024-09-17T05:34:46.166350+0000","last_clean":"2024-09-17T05:34:46.166350+0000","last_became_active":"2024-09-17T05:34:43.488206+0000","last_became_peered":"2024-09-17T05:34:43.488206+0000","last_unstale":"2024-09-17T05:34:46.166350+0000","last_undegraded":"2024-09-17T05:34:46.166350+0000","last_fullsized":"2024-09-17T05:34:46.166350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:09.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467792+0000","last_change":"2024-09-17T05:34:43.467792+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467792+0000","last_undegraded":"2024-09-17T05:34:43.467792+0000","last_fullsized":"2024-09-17T05:34:43.467792+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482792+0000","last_change":"2024-09-17T05:34:43.480480+0000","last_active":"2024-09-17T05:34:43.482792+0000","last_peered":"2024-09-17T05:34:43.482792+0000","last_clean":"2024-09-17T05:34:43.482792+0000","last_became_active":"2024-09-17T05:34:43.480158+0000","last_became_peered":"2024-09-17T05:34:43.480158+0000","last_unstale":"2024-09-17T05:34:43.482792+0000","last_undegraded":"2024-09-17T05:34:43.482792+0000","last_fullsized":"2024-09-17T05:34:43.482792+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466318+0000","last_change":"2024-09-17T05:34:43.466318+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466318+0000","last_undegraded":"2024-09-17T05:34:43.466318+0000","last_fullsized":"2024-09-17T05:34:43.466318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"138'8","reported_seq":23,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166302+0000","last_change":"2024-09-17T05:34:43.480821+0000","last_active":"2024-09-17T05:34:46.166302+0000","last_peered":"2024-09-17T05:34:46.166302+0000","last_clean":"2024-09-17T05:34:46.166302+0000","last_became_active":"2024-09-17T05:34:43.480465+0000","last_became_peered":"2024-09-17T05:34:43.480465+0000","last_unstale":"2024-09-17T05:34:46.166302+0000","last_undegraded":"2024-09-17T05:34:46.166302+0000","last_fullsized":"2024-09-17T05:34:46.166302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:55.811669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477704+0000","last_change":"2024-09-17T05:34:43.477704+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477704+0000","last_undegraded":"2024-09-17T05:34:43.477704+0000","last_fullsized":"2024-09-17T05:34:43.477704+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.490985+0000","last_change":"2024-09-17T05:34:43.489754+0000","last_active":"2024-09-17T05:34:43.490985+0000","last_peered":"2024-09-17T05:34:43.490985+0000","last_clean":"2024-09-17T05:34:43.490985+0000","last_became_active":"2024-09-17T05:34:43.489338+0000","last_became_peered":"2024-09-17T05:34:43.489338+0000","last_unstale":"2024-09-17T05:34:43.490985+0000","last_undegraded":"2024-09-17T05:34:43.490985+0000","last_fullsized":"2024-09-17T05:34:43.490985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472867+0000","last_change":"2024-09-17T05:34:43.472867+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472867+0000","last_undegraded":"2024-09-17T05:34:43.472867+0000","last_fullsized":"2024-09-17T05:34:43.472867+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485935+0000","last_change":"2024-09-17T05:34:43.484097+0000","last_active":"2024-09-17T05:34:43.485935+0000","last_peered":"2024-09-17T05:34:43.485935+0000","last_clean":"2024-09-17T05:34:43.485935+0000","last_became_active":"2024-09-17T05:34:43.483720+0000","last_became_peered":"2024-09-17T05:34:43.483720+0000","last_unstale":"2024-09-17T05:34:43.485935+0000","last_undegraded":"2024-09-17T05:34:43.485935+0000","last_fullsized":"2024-09-17T05:34:43.485935+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:54.550132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464690+0000","last_change":"2024-09-17T05:34:43.464690+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464690+0000","last_undegraded":"2024-09-17T05:34:43.464690+0000","last_fullsized":"2024-09-17T05:34:43.464690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491259+0000","last_change":"2024-09-17T05:34:43.484124+0000","last_active":"2024-09-17T05:34:43.491259+0000","last_peered":"2024-09-17T05:34:43.491259+0000","last_clean":"2024-09-17T05:34:43.491259+0000","last_became_active":"2024-09-17T05:34:43.483814+0000","last_became_peered":"2024-09-17T05:34:43.483814+0000","last_unstale":"2024-09-17T05:34:43.491259+0000","last_undegraded":"2024-09-17T05:34:43.491259+0000","last_fullsized":"2024-09-17T05:34:43.491259+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:24.723614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472433+0000","last_change":"2024-09-17T05:34:43.472433+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472433+0000","last_undegraded":"2024-09-17T05:34:43.472433+0000","last_fullsized":"2024-09-17T05:34:43.472433+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166410+0000","last_change":"2024-09-17T05:34:43.488724+0000","last_active":"2024-09-17T05:34:46.166410+0000","last_peered":"2024-09-17T05:34:46.166410+0000","last_clean":"2024-09-17T05:34:46.166410+0000","last_became_active":"2024-09-17T05:34:43.488321+0000","last_became_peered":"2024-09-17T05:34:43.488321+0000","last_unstale":"2024-09-17T05:34:46.166410+0000","last_undegraded":"2024-09-17T05:34:46.166410+0000","last_fullsized":"2024-09-17T05:34:46.166410+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:31.408633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473697+0000","last_change":"2024-09-17T05:34:43.473697+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473697+0000","last_undegraded":"2024-09-17T05:34:43.473697+0000","last_fullsized":"2024-09-17T05:34:43.473697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488270+0000","last_change":"2024-09-17T05:34:43.486138+0000","last_active":"2024-09-17T05:34:43.488270+0000","last_peered":"2024-09-17T05:34:43.488270+0000","last_clean":"2024-09-17T05:34:43.488270+0000","last_became_active":"2024-09-17T05:34:43.485752+0000","last_became_peered":"2024-09-17T05:34:43.485752+0000","last_unstale":"2024-09-17T05:34:43.488270+0000","last_undegraded":"2024-09-17T05:34:43.488270+0000","last_fullsized":"2024-09-17T05:34:43.488270+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:25.830977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166207+0000","last_change":"2024-09-17T05:34:44.498312+0000","last_active":"2024-09-17T05:34:46.166207+0000","last_peered":"2024-09-17T05:34:46.166207+0000","last_clean":"2024-09-17T05:34:46.166207+0000","last_became_active":"2024-09-17T05:34:44.495324+0000","last_became_peered":"2024-09-17T05:34:44.495324+0000","last_unstale":"2024-09-17T05:34:46.166207+0000","last_undegraded":"2024-09-17T05:34:46.166207+0000","last_fullsized":"2024-09-17T05:34:46.166207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.787871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482748+0000","last_change":"2024-09-17T05:34:43.477604+0000","last_active":"2024-09-17T05:34:43.482748+0000","last_peered":"2024-09-17T05:34:43.482748+0000","last_clean":"2024-09-17T05:34:43.482748+0000","last_became_active":"2024-09-17T05:34:43.477206+0000","last_became_peered":"2024-09-17T05:34:43.477206+0000","last_unstale":"2024-09-17T05:34:43.482748+0000","last_undegraded":"2024-09-17T05:34:43.482748+0000","last_fullsized":"2024-09-17T05:34:43.482748+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:55.300288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472859+0000","last_change":"2024-09-17T05:34:43.472859+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472859+0000","last_undegraded":"2024-09-17T05:34:43.472859+0000","last_fullsized":"2024-09-17T05:34:43.472859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494923+0000","last_change":"2024-09-17T05:34:43.487120+0000","last_active":"2024-09-17T05:34:43.494923+0000","last_peered":"2024-09-17T05:34:43.494923+0000","last_clean":"2024-09-17T05:34:43.494923+0000","last_became_active":"2024-09-17T05:34:43.486767+0000","last_became_peered":"2024-09-17T05:34:43.486767+0000","last_unstale":"2024-09-17T05:34:43.494923+0000","last_undegraded":"2024-09-17T05:34:43.494923+0000","last_fullsized":"2024-09-17T05:34:43.494923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:43.570302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477665+0000","last_change":"2024-09-17T05:34:43.477665+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477665+0000","last_undegraded":"2024-09-17T05:34:43.477665+0000","last_fullsized":"2024-09-17T05:34:43.477665+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484272+0000","last_change":"2024-09-17T05:34:43.482392+0000","last_active":"2024-09-17T05:34:43.484272+0000","last_peered":"2024-09-17T05:34:43.484272+0000","last_clean":"2024-09-17T05:34:43.484272+0000","last_became_active":"2024-09-17T05:34:43.482075+0000","last_became_peered":"2024-09-17T05:34:43.482075+0000","last_unstale":"2024-09-17T05:34:43.484272+0000","last_undegraded":"2024-09-17T05:34:43.484272+0000","last_fullsized":"2024-09-17T05:34:43.484272+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474970+0000","last_change":"2024-09-17T05:34:43.474970+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474970+0000","last_undegraded":"2024-09-17T05:34:43.474970+0000","last_fullsized":"2024-09-17T05:34:43.474970+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498847+0000","last_change":"2024-09-17T05:34:43.493846+0000","last_active":"2024-09-17T05:34:43.498847+0000","last_peered":"2024-09-17T05:34:43.498847+0000","last_clean":"2024-09-17T05:34:43.498847+0000","last_became_active":"2024-09-17T05:34:43.493493+0000","last_became_peered":"2024-09-17T05:34:43.493493+0000","last_unstale":"2024-09-17T05:34:43.498847+0000","last_undegraded":"2024-09-17T05:34:43.498847+0000","last_fullsized":"2024-09-17T05:34:43.498847+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:46.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466430+0000","last_change":"2024-09-17T05:34:43.466430+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466430+0000","last_undegraded":"2024-09-17T05:34:43.466430+0000","last_fullsized":"2024-09-17T05:34:43.466430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491753+0000","last_change":"2024-09-17T05:34:43.487845+0000","last_active":"2024-09-17T05:34:43.491753+0000","last_peered":"2024-09-17T05:34:43.491753+0000","last_clean":"2024-09-17T05:34:43.491753+0000","last_became_active":"2024-09-17T05:34:43.487537+0000","last_became_peered":"2024-09-17T05:34:43.487537+0000","last_unstale":"2024-09-17T05:34:43.491753+0000","last_undegraded":"2024-09-17T05:34:43.491753+0000","last_fullsized":"2024-09-17T05:34:43.491753+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:10.323580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471452+0000","last_change":"2024-09-17T05:34:43.471452+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471452+0000","last_undegraded":"2024-09-17T05:34:43.471452+0000","last_fullsized":"2024-09-17T05:34:43.471452+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488918+0000","last_change":"2024-09-17T05:34:43.487549+0000","last_active":"2024-09-17T05:34:43.488918+0000","last_peered":"2024-09-17T05:34:43.488918+0000","last_clean":"2024-09-17T05:34:43.488918+0000","last_became_active":"2024-09-17T05:34:43.487089+0000","last_became_peered":"2024-09-17T05:34:43.487089+0000","last_unstale":"2024-09-17T05:34:43.488918+0000","last_undegraded":"2024-09-17T05:34:43.488918+0000","last_fullsized":"2024-09-17T05:34:43.488918+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:33.360373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471776+0000","last_change":"2024-09-17T05:34:43.471776+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471776+0000","last_undegraded":"2024-09-17T05:34:43.471776+0000","last_fullsized":"2024-09-17T05:34:43.471776+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498988+0000","last_change":"2024-09-17T05:34:43.495216+0000","last_active":"2024-09-17T05:34:43.498988+0000","last_peered":"2024-09-17T05:34:43.498988+0000","last_clean":"2024-09-17T05:34:43.498988+0000","last_became_active":"2024-09-17T05:34:43.494800+0000","last_became_peered":"2024-09-17T05:34:43.494800+0000","last_unstale":"2024-09-17T05:34:43.498988+0000","last_undegraded":"2024-09-17T05:34:43.498988+0000","last_fullsized":"2024-09-17T05:34:43.498988+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:03.650153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468214+0000","last_change":"2024-09-17T05:34:43.468214+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468214+0000","last_undegraded":"2024-09-17T05:34:43.468214+0000","last_fullsized":"2024-09-17T05:34:43.468214+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.492048+0000","last_change":"2024-09-17T05:34:43.491113+0000","last_active":"2024-09-17T05:34:43.492048+0000","last_peered":"2024-09-17T05:34:43.492048+0000","last_clean":"2024-09-17T05:34:43.492048+0000","last_became_active":"2024-09-17T05:34:43.490776+0000","last_became_peered":"2024-09-17T05:34:43.490776+0000","last_unstale":"2024-09-17T05:34:43.492048+0000","last_undegraded":"2024-09-17T05:34:43.492048+0000","last_fullsized":"2024-09-17T05:34:43.492048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:22.282026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476081+0000","last_change":"2024-09-17T05:34:43.476081+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476081+0000","last_undegraded":"2024-09-17T05:34:43.476081+0000","last_fullsized":"2024-09-17T05:34:43.476081+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496135+0000","last_change":"2024-09-17T05:34:43.490904+0000","last_active":"2024-09-17T05:34:43.496135+0000","last_peered":"2024-09-17T05:34:43.496135+0000","last_clean":"2024-09-17T05:34:43.496135+0000","last_became_active":"2024-09-17T05:34:43.490596+0000","last_became_peered":"2024-09-17T05:34:43.490596+0000","last_unstale":"2024-09-17T05:34:43.496135+0000","last_undegraded":"2024-09-17T05:34:43.496135+0000","last_fullsized":"2024-09-17T05:34:43.496135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:45.332471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464313+0000","last_change":"2024-09-17T05:34:43.464313+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464313+0000","last_undegraded":"2024-09-17T05:34:43.464313+0000","last_fullsized":"2024-09-17T05:34:43.464313+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494892+0000","last_change":"2024-09-17T05:34:43.485943+0000","last_active":"2024-09-17T05:34:43.494892+0000","last_peered":"2024-09-17T05:34:43.494892+0000","last_clean":"2024-09-17T05:34:43.494892+0000","last_became_active":"2024-09-17T05:34:43.485578+0000","last_became_peered":"2024-09-17T05:34:43.485578+0000","last_unstale":"2024-09-17T05:34:43.494892+0000","last_undegraded":"2024-09-17T05:34:43.494892+0000","last_fullsized":"2024-09-17T05:34:43.494892+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467211+0000","last_change":"2024-09-17T05:34:43.467211+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467211+0000","last_undegraded":"2024-09-17T05:34:43.467211+0000","last_fullsized":"2024-09-17T05:34:43.467211+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486113+0000","last_change":"2024-09-17T05:34:43.483784+0000","last_active":"2024-09-17T05:34:43.486113+0000","last_peered":"2024-09-17T05:34:43.486113+0000","last_clean":"2024-09-17T05:34:43.486113+0000","last_became_active":"2024-09-17T05:34:43.483417+0000","last_became_peered":"2024-09-17T05:34:43.483417+0000","last_unstale":"2024-09-17T05:34:43.486113+0000","last_undegraded":"2024-09-17T05:34:43.486113+0000","last_fullsized":"2024-09-17T05:34:43.486113+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:39.573633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":337,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.470110+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:34:43.470110+0000","last_peered":"2024-09-17T05:34:43.470110+0000","last_clean":"2024-09-17T05:34:43.470110+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:34:43.470110+0000","last_undegraded":"2024-09-17T05:34:43.470110+0000","last_fullsized":"2024-09-17T05:34:43.470110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509582,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":834542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45356,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326484,"statfs":{"total":96636764160,"available":96590319616,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41232,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330608,"statfs":{"total":96636764160,"available":96594542592,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:49.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:49.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- 172.21.15.43:0/891047171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 msgr2=0x7f5da4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 --2- 172.21.15.43:0/891047171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5da4004c20 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f5da80016c0 tx=0x7f5da801c770 comp rx=0 tx=0).stop 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- 172.21.15.43:0/891047171 shutdown_connections 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 --2- 172.21.15.43:0/891047171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5da4004c20 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- 172.21.15.43:0/891047171 >> 172.21.15.43:0/891047171 conn(0x7f5dc00b23d0 msgr2=0x7f5dc00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- 172.21.15.43:0/891047171 shutdown_connections 2024-09-17T05:34:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- 172.21.15.43:0/891047171 wait complete. 2024-09-17T05:34:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 Processor -- start 2024-09-17T05:34:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.569+0000 7f5dc6800640 1 -- start start 2024-09-17T05:34:49.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dc009d730 0x7f5dc0093730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dc0093c70 0x7f5dc0094080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00ad650 con 0x7f5da4004810 2024-09-17T05:34:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc00ad4d0 con 0x7f5dc0093c70 2024-09-17T05:34:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00ad7d0 con 0x7f5dc009d730 2024-09-17T05:34:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54214/0 (socket says 172.21.15.43:54214) 2024-09-17T05:34:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 -- 172.21.15.43:0/334907297 learned_addr learned my addr 172.21.15.43:0/334907297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 -- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dc009d730 msgr2=0x7f5dc0093730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dc009d730 0x7f5dc0093730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 -- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dc0093c70 msgr2=0x7f5dc0094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dc0093c70 0x7f5dc0094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 -- 172.21.15.43:0/334907297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00c6fc0 con 0x7f5da4004810 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dbf400640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f5db0011a10 tx=0x7f5db0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db001e070 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5db0004460 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000fb80 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc00c6ce0 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc0091a20 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.573+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc00add30 con 0x7f5da4004810 2024-09-17T05:34:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.577+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5db0021070 con 0x7f5da4004810 2024-09-17T05:34:49.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.577+0000 7f5db7400640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 0x7f5d9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.577+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f5db0094c20 con 0x7f5da4004810 2024-09-17T05:34:49.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.577+0000 7f5db7e00640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 0x7f5d9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.577+0000 7f5db7e00640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 0x7f5d9c075710 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0073060 tx=0x7f5da4005840 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:49.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.581+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5db005f550 con 0x7f5da4004810 2024-09-17T05:34:49.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.753+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5dc0075350 con 0x7f5d9c073200 2024-09-17T05:34:49.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.757+0000 7f5db7400640 1 -- 172.21.15.43:0/334907297 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248108 (secure 0 0 0) 0x7f5dc0075350 con 0x7f5d9c073200 2024-09-17T05:34:49.766 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:49.767 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 msgr2=0x7f5d9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 0x7f5d9c075710 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0073060 tx=0x7f5da4005840 comp rx=0 tx=0).stop 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 msgr2=0x7f5dc015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f5db0011a10 tx=0x7f5db0011ed0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 shutdown_connections 2024-09-17T05:34:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d9c073200 0x7f5d9c075710 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5dc0093c70 0x7f5dc0094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5dc009d730 0x7f5dc0093730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 --2- 172.21.15.43:0/334907297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5da4004810 0x7f5dc015b000 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 >> 172.21.15.43:0/334907297 conn(0x7f5dc00b23d0 msgr2=0x7f5dc00ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 shutdown_connections 2024-09-17T05:34:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.761+0000 7f5dc6800640 1 -- 172.21.15.43:0/334907297 wait complete. 2024-09-17T05:34:49.787 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-17T05:34:48.030474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":301688,"kb_used_data":14736,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754673032,"statfs":{"total":773094113280,"available":772785184768,"internally_reserved":0,"allocated":15089664,"data_stored":7845047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.571531"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167073+0000","last_change":"2024-09-17T05:34:44.497120+0000","last_active":"2024-09-17T05:34:46.167073+0000","last_peered":"2024-09-17T05:34:46.167073+0000","last_clean":"2024-09-17T05:34:46.167073+0000","last_became_active":"2024-09-17T05:34:44.496734+0000","last_became_peered":"2024-09-17T05:34:44.496734+0000","last_unstale":"2024-09-17T05:34:46.167073+0000","last_undegraded":"2024-09-17T05:34:46.167073+0000","last_fullsized":"2024-09-17T05:34:46.167073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:55.766746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166945+0000","last_change":"2024-09-17T05:34:44.504496+0000","last_active":"2024-09-17T05:34:46.166945+0000","last_peered":"2024-09-17T05:34:46.166945+0000","last_clean":"2024-09-17T05:34:46.166945+0000","last_became_active":"2024-09-17T05:34:44.504097+0000","last_became_peered":"2024-09-17T05:34:44.504097+0000","last_unstale":"2024-09-17T05:34:46.166945+0000","last_undegraded":"2024-09-17T05:34:46.166945+0000","last_fullsized":"2024-09-17T05:34:46.166945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:31.928876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466484+0000","last_change":"2024-09-17T05:34:43.466484+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466484+0000","last_undegraded":"2024-09-17T05:34:43.466484+0000","last_fullsized":"2024-09-17T05:34:43.466484+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166881+0000","last_change":"2024-09-17T05:34:44.511242+0000","last_active":"2024-09-17T05:34:46.166881+0000","last_peered":"2024-09-17T05:34:46.166881+0000","last_clean":"2024-09-17T05:34:46.166881+0000","last_became_active":"2024-09-17T05:34:44.510544+0000","last_became_peered":"2024-09-17T05:34:44.510544+0000","last_unstale":"2024-09-17T05:34:46.166881+0000","last_undegraded":"2024-09-17T05:34:46.166881+0000","last_fullsized":"2024-09-17T05:34:46.166881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:16.675435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.469446+0000","last_change":"2024-09-17T05:34:43.469446+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.469446+0000","last_undegraded":"2024-09-17T05:34:43.469446+0000","last_fullsized":"2024-09-17T05:34:43.469446+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467679+0000","last_change":"2024-09-17T05:34:43.467679+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467679+0000","last_undegraded":"2024-09-17T05:34:43.467679+0000","last_fullsized":"2024-09-17T05:34:43.467679+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468846+0000","last_change":"2024-09-17T05:34:43.468846+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468846+0000","last_undegraded":"2024-09-17T05:34:43.468846+0000","last_fullsized":"2024-09-17T05:34:43.468846+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466911+0000","last_change":"2024-09-17T05:34:43.466911+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466911+0000","last_undegraded":"2024-09-17T05:34:43.466911+0000","last_fullsized":"2024-09-17T05:34:43.466911+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471899+0000","last_change":"2024-09-17T05:34:43.471899+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471899+0000","last_undegraded":"2024-09-17T05:34:43.471899+0000","last_fullsized":"2024-09-17T05:34:43.471899+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465962+0000","last_change":"2024-09-17T05:34:43.465962+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465962+0000","last_undegraded":"2024-09-17T05:34:43.465962+0000","last_fullsized":"2024-09-17T05:34:43.465962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471985+0000","last_change":"2024-09-17T05:34:43.471985+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471985+0000","last_undegraded":"2024-09-17T05:34:43.471985+0000","last_fullsized":"2024-09-17T05:34:43.471985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467723+0000","last_change":"2024-09-17T05:34:43.467723+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467723+0000","last_undegraded":"2024-09-17T05:34:43.467723+0000","last_fullsized":"2024-09-17T05:34:43.467723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474063+0000","last_change":"2024-09-17T05:34:43.474063+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474063+0000","last_undegraded":"2024-09-17T05:34:43.474063+0000","last_fullsized":"2024-09-17T05:34:43.474063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166241+0000","last_change":"2024-09-17T05:34:44.498246+0000","last_active":"2024-09-17T05:34:46.166241+0000","last_peered":"2024-09-17T05:34:46.166241+0000","last_clean":"2024-09-17T05:34:46.166241+0000","last_became_active":"2024-09-17T05:34:44.495932+0000","last_became_peered":"2024-09-17T05:34:44.495932+0000","last_unstale":"2024-09-17T05:34:46.166241+0000","last_undegraded":"2024-09-17T05:34:46.166241+0000","last_fullsized":"2024-09-17T05:34:46.166241+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:11.396023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166253+0000","last_change":"2024-09-17T05:34:44.503954+0000","last_active":"2024-09-17T05:34:46.166253+0000","last_peered":"2024-09-17T05:34:46.166253+0000","last_clean":"2024-09-17T05:34:46.166253+0000","last_became_active":"2024-09-17T05:34:44.502592+0000","last_became_peered":"2024-09-17T05:34:44.502592+0000","last_unstale":"2024-09-17T05:34:46.166253+0000","last_undegraded":"2024-09-17T05:34:46.166253+0000","last_fullsized":"2024-09-17T05:34:46.166253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:48.068029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468782+0000","last_change":"2024-09-17T05:34:43.468782+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468782+0000","last_undegraded":"2024-09-17T05:34:43.468782+0000","last_fullsized":"2024-09-17T05:34:43.468782+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475494+0000","last_change":"2024-09-17T05:34:43.475494+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475494+0000","last_undegraded":"2024-09-17T05:34:43.475494+0000","last_fullsized":"2024-09-17T05:34:43.475494+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167005+0000","last_change":"2024-09-17T05:34:44.501872+0000","last_active":"2024-09-17T05:34:46.167005+0000","last_peered":"2024-09-17T05:34:46.167005+0000","last_clean":"2024-09-17T05:34:46.167005+0000","last_became_active":"2024-09-17T05:34:44.498268+0000","last_became_peered":"2024-09-17T05:34:44.498268+0000","last_unstale":"2024-09-17T05:34:46.167005+0000","last_undegraded":"2024-09-17T05:34:46.167005+0000","last_fullsized":"2024-09-17T05:34:46.167005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:27.861661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473976+0000","last_change":"2024-09-17T05:34:43.473976+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473976+0000","last_undegraded":"2024-09-17T05:34:43.473976+0000","last_fullsized":"2024-09-17T05:34:43.473976+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465620+0000","last_change":"2024-09-17T05:34:43.465620+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465620+0000","last_undegraded":"2024-09-17T05:34:43.465620+0000","last_fullsized":"2024-09-17T05:34:43.465620+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468129+0000","last_change":"2024-09-17T05:34:43.468129+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468129+0000","last_undegraded":"2024-09-17T05:34:43.468129+0000","last_fullsized":"2024-09-17T05:34:43.468129+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467304+0000","last_change":"2024-09-17T05:34:43.467304+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467304+0000","last_undegraded":"2024-09-17T05:34:43.467304+0000","last_fullsized":"2024-09-17T05:34:43.467304+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166332+0000","last_change":"2024-09-17T05:34:44.495822+0000","last_active":"2024-09-17T05:34:46.166332+0000","last_peered":"2024-09-17T05:34:46.166332+0000","last_clean":"2024-09-17T05:34:46.166332+0000","last_became_active":"2024-09-17T05:34:44.495403+0000","last_became_peered":"2024-09-17T05:34:44.495403+0000","last_unstale":"2024-09-17T05:34:46.166332+0000","last_undegraded":"2024-09-17T05:34:46.166332+0000","last_fullsized":"2024-09-17T05:34:46.166332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:15.566322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472479+0000","last_change":"2024-09-17T05:34:43.472479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472479+0000","last_undegraded":"2024-09-17T05:34:43.472479+0000","last_fullsized":"2024-09-17T05:34:43.472479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465223+0000","last_change":"2024-09-17T05:34:43.465223+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465223+0000","last_undegraded":"2024-09-17T05:34:43.465223+0000","last_fullsized":"2024-09-17T05:34:43.465223+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.475574+0000","last_change":"2024-09-17T05:34:43.475574+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.475574+0000","last_undegraded":"2024-09-17T05:34:43.475574+0000","last_fullsized":"2024-09-17T05:34:43.475574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471952+0000","last_change":"2024-09-17T05:34:43.471952+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471952+0000","last_undegraded":"2024-09-17T05:34:43.471952+0000","last_fullsized":"2024-09-17T05:34:43.471952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476158+0000","last_change":"2024-09-17T05:34:43.476158+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476158+0000","last_undegraded":"2024-09-17T05:34:43.476158+0000","last_fullsized":"2024-09-17T05:34:43.476158+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473479+0000","last_change":"2024-09-17T05:34:43.473479+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473479+0000","last_undegraded":"2024-09-17T05:34:43.473479+0000","last_fullsized":"2024-09-17T05:34:43.473479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473815+0000","last_change":"2024-09-17T05:34:43.473815+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473815+0000","last_undegraded":"2024-09-17T05:34:43.473815+0000","last_fullsized":"2024-09-17T05:34:43.473815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467581+0000","last_change":"2024-09-17T05:34:43.467581+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467581+0000","last_undegraded":"2024-09-17T05:34:43.467581+0000","last_fullsized":"2024-09-17T05:34:43.467581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166227+0000","last_change":"2024-09-17T05:34:44.515160+0000","last_active":"2024-09-17T05:34:46.166227+0000","last_peered":"2024-09-17T05:34:46.166227+0000","last_clean":"2024-09-17T05:34:46.166227+0000","last_became_active":"2024-09-17T05:34:44.514775+0000","last_became_peered":"2024-09-17T05:34:44.514775+0000","last_unstale":"2024-09-17T05:34:46.166227+0000","last_undegraded":"2024-09-17T05:34:46.166227+0000","last_fullsized":"2024-09-17T05:34:46.166227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:18.490230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.478231+0000","last_change":"2024-09-17T05:34:43.478231+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.478231+0000","last_undegraded":"2024-09-17T05:34:43.478231+0000","last_fullsized":"2024-09-17T05:34:43.478231+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498701+0000","last_change":"2024-09-17T05:34:43.492447+0000","last_active":"2024-09-17T05:34:43.498701+0000","last_peered":"2024-09-17T05:34:43.498701+0000","last_clean":"2024-09-17T05:34:43.498701+0000","last_became_active":"2024-09-17T05:34:43.492128+0000","last_became_peered":"2024-09-17T05:34:43.492128+0000","last_unstale":"2024-09-17T05:34:43.498701+0000","last_undegraded":"2024-09-17T05:34:43.498701+0000","last_fullsized":"2024-09-17T05:34:43.498701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:37.605990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476119+0000","last_change":"2024-09-17T05:34:43.476119+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476119+0000","last_undegraded":"2024-09-17T05:34:43.476119+0000","last_fullsized":"2024-09-17T05:34:43.476119+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488969+0000","last_change":"2024-09-17T05:34:43.488048+0000","last_active":"2024-09-17T05:34:43.488969+0000","last_peered":"2024-09-17T05:34:43.488969+0000","last_clean":"2024-09-17T05:34:43.488969+0000","last_became_active":"2024-09-17T05:34:43.487637+0000","last_became_peered":"2024-09-17T05:34:43.487637+0000","last_unstale":"2024-09-17T05:34:43.488969+0000","last_undegraded":"2024-09-17T05:34:43.488969+0000","last_fullsized":"2024-09-17T05:34:43.488969+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:57.771444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166891+0000","last_change":"2024-09-17T05:34:44.497033+0000","last_active":"2024-09-17T05:34:46.166891+0000","last_peered":"2024-09-17T05:34:46.166891+0000","last_clean":"2024-09-17T05:34:46.166891+0000","last_became_active":"2024-09-17T05:34:44.496645+0000","last_became_peered":"2024-09-17T05:34:44.496645+0000","last_unstale":"2024-09-17T05:34:46.166891+0000","last_undegraded":"2024-09-17T05:34:46.166891+0000","last_fullsized":"2024-09-17T05:34:46.166891+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:37.177571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496221+0000","last_change":"2024-09-17T05:34:43.490759+0000","last_active":"2024-09-17T05:34:43.496221+0000","last_peered":"2024-09-17T05:34:43.496221+0000","last_clean":"2024-09-17T05:34:43.496221+0000","last_became_active":"2024-09-17T05:34:43.489512+0000","last_became_peered":"2024-09-17T05:34:43.489512+0000","last_unstale":"2024-09-17T05:34:43.496221+0000","last_undegraded":"2024-09-17T05:34:43.496221+0000","last_fullsized":"2024-09-17T05:34:43.496221+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465255+0000","last_change":"2024-09-17T05:34:43.465255+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465255+0000","last_undegraded":"2024-09-17T05:34:43.465255+0000","last_fullsized":"2024-09-17T05:34:43.465255+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166985+0000","last_change":"2024-09-17T05:34:43.497073+0000","last_active":"2024-09-17T05:34:46.166985+0000","last_peered":"2024-09-17T05:34:46.166985+0000","last_clean":"2024-09-17T05:34:46.166985+0000","last_became_active":"2024-09-17T05:34:43.496723+0000","last_became_peered":"2024-09-17T05:34:43.496723+0000","last_unstale":"2024-09-17T05:34:46.166985+0000","last_undegraded":"2024-09-17T05:34:46.166985+0000","last_fullsized":"2024-09-17T05:34:46.166985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:36.943277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464326+0000","last_change":"2024-09-17T05:34:43.464326+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464326+0000","last_undegraded":"2024-09-17T05:34:43.464326+0000","last_fullsized":"2024-09-17T05:34:43.464326+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166073+0000","last_change":"2024-09-17T05:34:43.484224+0000","last_active":"2024-09-17T05:34:46.166073+0000","last_peered":"2024-09-17T05:34:46.166073+0000","last_clean":"2024-09-17T05:34:46.166073+0000","last_became_active":"2024-09-17T05:34:43.483865+0000","last_became_peered":"2024-09-17T05:34:43.483865+0000","last_unstale":"2024-09-17T05:34:46.166073+0000","last_undegraded":"2024-09-17T05:34:46.166073+0000","last_fullsized":"2024-09-17T05:34:46.166073+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.065970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468869+0000","last_change":"2024-09-17T05:34:43.468869+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468869+0000","last_undegraded":"2024-09-17T05:34:43.468869+0000","last_fullsized":"2024-09-17T05:34:43.468869+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485165+0000","last_change":"2024-09-17T05:34:43.479766+0000","last_active":"2024-09-17T05:34:43.485165+0000","last_peered":"2024-09-17T05:34:43.485165+0000","last_clean":"2024-09-17T05:34:43.485165+0000","last_became_active":"2024-09-17T05:34:43.479411+0000","last_became_peered":"2024-09-17T05:34:43.479411+0000","last_unstale":"2024-09-17T05:34:43.485165+0000","last_undegraded":"2024-09-17T05:34:43.485165+0000","last_fullsized":"2024-09-17T05:34:43.485165+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474995+0000","last_change":"2024-09-17T05:34:43.474995+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474995+0000","last_undegraded":"2024-09-17T05:34:43.474995+0000","last_fullsized":"2024-09-17T05:34:43.474995+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486688+0000","last_change":"2024-09-17T05:34:43.485518+0000","last_active":"2024-09-17T05:34:43.486688+0000","last_peered":"2024-09-17T05:34:43.486688+0000","last_clean":"2024-09-17T05:34:43.486688+0000","last_became_active":"2024-09-17T05:34:43.485155+0000","last_became_peered":"2024-09-17T05:34:43.485155+0000","last_unstale":"2024-09-17T05:34:43.486688+0000","last_undegraded":"2024-09-17T05:34:43.486688+0000","last_fullsized":"2024-09-17T05:34:43.486688+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:15.459388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.465390+0000","last_change":"2024-09-17T05:34:43.465390+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.465390+0000","last_undegraded":"2024-09-17T05:34:43.465390+0000","last_fullsized":"2024-09-17T05:34:43.465390+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496365+0000","last_change":"2024-09-17T05:34:43.492218+0000","last_active":"2024-09-17T05:34:43.496365+0000","last_peered":"2024-09-17T05:34:43.496365+0000","last_clean":"2024-09-17T05:34:43.496365+0000","last_became_active":"2024-09-17T05:34:43.491883+0000","last_became_peered":"2024-09-17T05:34:43.491883+0000","last_unstale":"2024-09-17T05:34:43.496365+0000","last_undegraded":"2024-09-17T05:34:43.496365+0000","last_fullsized":"2024-09-17T05:34:43.496365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:14.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167150+0000","last_change":"2024-09-17T05:34:44.510566+0000","last_active":"2024-09-17T05:34:46.167150+0000","last_peered":"2024-09-17T05:34:46.167150+0000","last_clean":"2024-09-17T05:34:46.167150+0000","last_became_active":"2024-09-17T05:34:44.510105+0000","last_became_peered":"2024-09-17T05:34:44.510105+0000","last_unstale":"2024-09-17T05:34:46.167150+0000","last_undegraded":"2024-09-17T05:34:46.167150+0000","last_fullsized":"2024-09-17T05:34:46.167150+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.973701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494247+0000","last_change":"2024-09-17T05:34:43.486372+0000","last_active":"2024-09-17T05:34:43.494247+0000","last_peered":"2024-09-17T05:34:43.494247+0000","last_clean":"2024-09-17T05:34:43.494247+0000","last_became_active":"2024-09-17T05:34:43.485972+0000","last_became_peered":"2024-09-17T05:34:43.485972+0000","last_unstale":"2024-09-17T05:34:43.494247+0000","last_undegraded":"2024-09-17T05:34:43.494247+0000","last_fullsized":"2024-09-17T05:34:43.494247+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.479109+0000","last_change":"2024-09-17T05:34:43.479109+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.479109+0000","last_undegraded":"2024-09-17T05:34:43.479109+0000","last_fullsized":"2024-09-17T05:34:43.479109+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167064+0000","last_change":"2024-09-17T05:34:43.488432+0000","last_active":"2024-09-17T05:34:46.167064+0000","last_peered":"2024-09-17T05:34:46.167064+0000","last_clean":"2024-09-17T05:34:46.167064+0000","last_became_active":"2024-09-17T05:34:43.487723+0000","last_became_peered":"2024-09-17T05:34:43.487723+0000","last_unstale":"2024-09-17T05:34:46.167064+0000","last_undegraded":"2024-09-17T05:34:46.167064+0000","last_fullsized":"2024-09-17T05:34:46.167064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:03.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.470033+0000","last_change":"2024-09-17T05:34:43.470033+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.470033+0000","last_undegraded":"2024-09-17T05:34:43.470033+0000","last_fullsized":"2024-09-17T05:34:43.470033+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484380+0000","last_change":"2024-09-17T05:34:43.482270+0000","last_active":"2024-09-17T05:34:43.484380+0000","last_peered":"2024-09-17T05:34:43.484380+0000","last_clean":"2024-09-17T05:34:43.484380+0000","last_became_active":"2024-09-17T05:34:43.481870+0000","last_became_peered":"2024-09-17T05:34:43.481870+0000","last_unstale":"2024-09-17T05:34:43.484380+0000","last_undegraded":"2024-09-17T05:34:43.484380+0000","last_fullsized":"2024-09-17T05:34:43.484380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.851530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473951+0000","last_change":"2024-09-17T05:34:43.473951+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473951+0000","last_undegraded":"2024-09-17T05:34:43.473951+0000","last_fullsized":"2024-09-17T05:34:43.473951+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485353+0000","last_change":"2024-09-17T05:34:43.483794+0000","last_active":"2024-09-17T05:34:43.485353+0000","last_peered":"2024-09-17T05:34:43.485353+0000","last_clean":"2024-09-17T05:34:43.485353+0000","last_became_active":"2024-09-17T05:34:43.483453+0000","last_became_peered":"2024-09-17T05:34:43.483453+0000","last_unstale":"2024-09-17T05:34:43.485353+0000","last_undegraded":"2024-09-17T05:34:43.485353+0000","last_fullsized":"2024-09-17T05:34:43.485353+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:39.691222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166308+0000","last_change":"2024-09-17T05:34:44.497676+0000","last_active":"2024-09-17T05:34:46.166308+0000","last_peered":"2024-09-17T05:34:46.166308+0000","last_clean":"2024-09-17T05:34:46.166308+0000","last_became_active":"2024-09-17T05:34:44.496021+0000","last_became_peered":"2024-09-17T05:34:44.496021+0000","last_unstale":"2024-09-17T05:34:46.166308+0000","last_undegraded":"2024-09-17T05:34:46.166308+0000","last_fullsized":"2024-09-17T05:34:46.166308+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.816080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491701+0000","last_change":"2024-09-17T05:34:43.486891+0000","last_active":"2024-09-17T05:34:43.491701+0000","last_peered":"2024-09-17T05:34:43.491701+0000","last_clean":"2024-09-17T05:34:43.491701+0000","last_became_active":"2024-09-17T05:34:43.485568+0000","last_became_peered":"2024-09-17T05:34:43.485568+0000","last_unstale":"2024-09-17T05:34:43.491701+0000","last_undegraded":"2024-09-17T05:34:43.491701+0000","last_fullsized":"2024-09-17T05:34:43.491701+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:48.239606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467555+0000","last_change":"2024-09-17T05:34:43.467555+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467555+0000","last_undegraded":"2024-09-17T05:34:43.467555+0000","last_fullsized":"2024-09-17T05:34:43.467555+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166350+0000","last_change":"2024-09-17T05:34:43.490087+0000","last_active":"2024-09-17T05:34:46.166350+0000","last_peered":"2024-09-17T05:34:46.166350+0000","last_clean":"2024-09-17T05:34:46.166350+0000","last_became_active":"2024-09-17T05:34:43.488206+0000","last_became_peered":"2024-09-17T05:34:43.488206+0000","last_unstale":"2024-09-17T05:34:46.166350+0000","last_undegraded":"2024-09-17T05:34:46.166350+0000","last_fullsized":"2024-09-17T05:34:46.166350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:09.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467792+0000","last_change":"2024-09-17T05:34:43.467792+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467792+0000","last_undegraded":"2024-09-17T05:34:43.467792+0000","last_fullsized":"2024-09-17T05:34:43.467792+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482792+0000","last_change":"2024-09-17T05:34:43.480480+0000","last_active":"2024-09-17T05:34:43.482792+0000","last_peered":"2024-09-17T05:34:43.482792+0000","last_clean":"2024-09-17T05:34:43.482792+0000","last_became_active":"2024-09-17T05:34:43.480158+0000","last_became_peered":"2024-09-17T05:34:43.480158+0000","last_unstale":"2024-09-17T05:34:43.482792+0000","last_undegraded":"2024-09-17T05:34:43.482792+0000","last_fullsized":"2024-09-17T05:34:43.482792+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:51.685830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466318+0000","last_change":"2024-09-17T05:34:43.466318+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466318+0000","last_undegraded":"2024-09-17T05:34:43.466318+0000","last_fullsized":"2024-09-17T05:34:43.466318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"138'8","reported_seq":23,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166302+0000","last_change":"2024-09-17T05:34:43.480821+0000","last_active":"2024-09-17T05:34:46.166302+0000","last_peered":"2024-09-17T05:34:46.166302+0000","last_clean":"2024-09-17T05:34:46.166302+0000","last_became_active":"2024-09-17T05:34:43.480465+0000","last_became_peered":"2024-09-17T05:34:43.480465+0000","last_unstale":"2024-09-17T05:34:46.166302+0000","last_undegraded":"2024-09-17T05:34:46.166302+0000","last_fullsized":"2024-09-17T05:34:46.166302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:55.811669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477704+0000","last_change":"2024-09-17T05:34:43.477704+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477704+0000","last_undegraded":"2024-09-17T05:34:43.477704+0000","last_fullsized":"2024-09-17T05:34:43.477704+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.490985+0000","last_change":"2024-09-17T05:34:43.489754+0000","last_active":"2024-09-17T05:34:43.490985+0000","last_peered":"2024-09-17T05:34:43.490985+0000","last_clean":"2024-09-17T05:34:43.490985+0000","last_became_active":"2024-09-17T05:34:43.489338+0000","last_became_peered":"2024-09-17T05:34:43.489338+0000","last_unstale":"2024-09-17T05:34:43.490985+0000","last_undegraded":"2024-09-17T05:34:43.490985+0000","last_fullsized":"2024-09-17T05:34:43.490985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:00.720082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472867+0000","last_change":"2024-09-17T05:34:43.472867+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472867+0000","last_undegraded":"2024-09-17T05:34:43.472867+0000","last_fullsized":"2024-09-17T05:34:43.472867+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.485935+0000","last_change":"2024-09-17T05:34:43.484097+0000","last_active":"2024-09-17T05:34:43.485935+0000","last_peered":"2024-09-17T05:34:43.485935+0000","last_clean":"2024-09-17T05:34:43.485935+0000","last_became_active":"2024-09-17T05:34:43.483720+0000","last_became_peered":"2024-09-17T05:34:43.483720+0000","last_unstale":"2024-09-17T05:34:43.485935+0000","last_undegraded":"2024-09-17T05:34:43.485935+0000","last_fullsized":"2024-09-17T05:34:43.485935+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:54.550132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464690+0000","last_change":"2024-09-17T05:34:43.464690+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464690+0000","last_undegraded":"2024-09-17T05:34:43.464690+0000","last_fullsized":"2024-09-17T05:34:43.464690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491259+0000","last_change":"2024-09-17T05:34:43.484124+0000","last_active":"2024-09-17T05:34:43.491259+0000","last_peered":"2024-09-17T05:34:43.491259+0000","last_clean":"2024-09-17T05:34:43.491259+0000","last_became_active":"2024-09-17T05:34:43.483814+0000","last_became_peered":"2024-09-17T05:34:43.483814+0000","last_unstale":"2024-09-17T05:34:43.491259+0000","last_undegraded":"2024-09-17T05:34:43.491259+0000","last_fullsized":"2024-09-17T05:34:43.491259+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:24.723614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472433+0000","last_change":"2024-09-17T05:34:43.472433+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472433+0000","last_undegraded":"2024-09-17T05:34:43.472433+0000","last_fullsized":"2024-09-17T05:34:43.472433+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166410+0000","last_change":"2024-09-17T05:34:43.488724+0000","last_active":"2024-09-17T05:34:46.166410+0000","last_peered":"2024-09-17T05:34:46.166410+0000","last_clean":"2024-09-17T05:34:46.166410+0000","last_became_active":"2024-09-17T05:34:43.488321+0000","last_became_peered":"2024-09-17T05:34:43.488321+0000","last_unstale":"2024-09-17T05:34:46.166410+0000","last_undegraded":"2024-09-17T05:34:46.166410+0000","last_fullsized":"2024-09-17T05:34:46.166410+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:31.408633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.473697+0000","last_change":"2024-09-17T05:34:43.473697+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.473697+0000","last_undegraded":"2024-09-17T05:34:43.473697+0000","last_fullsized":"2024-09-17T05:34:43.473697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488270+0000","last_change":"2024-09-17T05:34:43.486138+0000","last_active":"2024-09-17T05:34:43.488270+0000","last_peered":"2024-09-17T05:34:43.488270+0000","last_clean":"2024-09-17T05:34:43.488270+0000","last_became_active":"2024-09-17T05:34:43.485752+0000","last_became_peered":"2024-09-17T05:34:43.485752+0000","last_unstale":"2024-09-17T05:34:43.488270+0000","last_undegraded":"2024-09-17T05:34:43.488270+0000","last_fullsized":"2024-09-17T05:34:43.488270+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:25.830977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166207+0000","last_change":"2024-09-17T05:34:44.498312+0000","last_active":"2024-09-17T05:34:46.166207+0000","last_peered":"2024-09-17T05:34:46.166207+0000","last_clean":"2024-09-17T05:34:46.166207+0000","last_became_active":"2024-09-17T05:34:44.495324+0000","last_became_peered":"2024-09-17T05:34:44.495324+0000","last_unstale":"2024-09-17T05:34:46.166207+0000","last_undegraded":"2024-09-17T05:34:46.166207+0000","last_fullsized":"2024-09-17T05:34:46.166207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.787871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.482748+0000","last_change":"2024-09-17T05:34:43.477604+0000","last_active":"2024-09-17T05:34:43.482748+0000","last_peered":"2024-09-17T05:34:43.482748+0000","last_clean":"2024-09-17T05:34:43.482748+0000","last_became_active":"2024-09-17T05:34:43.477206+0000","last_became_peered":"2024-09-17T05:34:43.477206+0000","last_unstale":"2024-09-17T05:34:43.482748+0000","last_undegraded":"2024-09-17T05:34:43.482748+0000","last_fullsized":"2024-09-17T05:34:43.482748+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:55.300288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.472859+0000","last_change":"2024-09-17T05:34:43.472859+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.472859+0000","last_undegraded":"2024-09-17T05:34:43.472859+0000","last_fullsized":"2024-09-17T05:34:43.472859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494923+0000","last_change":"2024-09-17T05:34:43.487120+0000","last_active":"2024-09-17T05:34:43.494923+0000","last_peered":"2024-09-17T05:34:43.494923+0000","last_clean":"2024-09-17T05:34:43.494923+0000","last_became_active":"2024-09-17T05:34:43.486767+0000","last_became_peered":"2024-09-17T05:34:43.486767+0000","last_unstale":"2024-09-17T05:34:43.494923+0000","last_undegraded":"2024-09-17T05:34:43.494923+0000","last_fullsized":"2024-09-17T05:34:43.494923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:43.570302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.477665+0000","last_change":"2024-09-17T05:34:43.477665+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.477665+0000","last_undegraded":"2024-09-17T05:34:43.477665+0000","last_fullsized":"2024-09-17T05:34:43.477665+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.484272+0000","last_change":"2024-09-17T05:34:43.482392+0000","last_active":"2024-09-17T05:34:43.484272+0000","last_peered":"2024-09-17T05:34:43.484272+0000","last_clean":"2024-09-17T05:34:43.484272+0000","last_became_active":"2024-09-17T05:34:43.482075+0000","last_became_peered":"2024-09-17T05:34:43.482075+0000","last_unstale":"2024-09-17T05:34:43.484272+0000","last_undegraded":"2024-09-17T05:34:43.484272+0000","last_fullsized":"2024-09-17T05:34:43.484272+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.474970+0000","last_change":"2024-09-17T05:34:43.474970+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.474970+0000","last_undegraded":"2024-09-17T05:34:43.474970+0000","last_fullsized":"2024-09-17T05:34:43.474970+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498847+0000","last_change":"2024-09-17T05:34:43.493846+0000","last_active":"2024-09-17T05:34:43.498847+0000","last_peered":"2024-09-17T05:34:43.498847+0000","last_clean":"2024-09-17T05:34:43.498847+0000","last_became_active":"2024-09-17T05:34:43.493493+0000","last_became_peered":"2024-09-17T05:34:43.493493+0000","last_unstale":"2024-09-17T05:34:43.498847+0000","last_undegraded":"2024-09-17T05:34:43.498847+0000","last_fullsized":"2024-09-17T05:34:43.498847+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:46.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.466430+0000","last_change":"2024-09-17T05:34:43.466430+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.466430+0000","last_undegraded":"2024-09-17T05:34:43.466430+0000","last_fullsized":"2024-09-17T05:34:43.466430+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.491753+0000","last_change":"2024-09-17T05:34:43.487845+0000","last_active":"2024-09-17T05:34:43.491753+0000","last_peered":"2024-09-17T05:34:43.491753+0000","last_clean":"2024-09-17T05:34:43.491753+0000","last_became_active":"2024-09-17T05:34:43.487537+0000","last_became_peered":"2024-09-17T05:34:43.487537+0000","last_unstale":"2024-09-17T05:34:43.491753+0000","last_undegraded":"2024-09-17T05:34:43.491753+0000","last_fullsized":"2024-09-17T05:34:43.491753+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:10.323580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471452+0000","last_change":"2024-09-17T05:34:43.471452+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471452+0000","last_undegraded":"2024-09-17T05:34:43.471452+0000","last_fullsized":"2024-09-17T05:34:43.471452+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.488918+0000","last_change":"2024-09-17T05:34:43.487549+0000","last_active":"2024-09-17T05:34:43.488918+0000","last_peered":"2024-09-17T05:34:43.488918+0000","last_clean":"2024-09-17T05:34:43.488918+0000","last_became_active":"2024-09-17T05:34:43.487089+0000","last_became_peered":"2024-09-17T05:34:43.487089+0000","last_unstale":"2024-09-17T05:34:43.488918+0000","last_undegraded":"2024-09-17T05:34:43.488918+0000","last_fullsized":"2024-09-17T05:34:43.488918+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:33.360373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.471776+0000","last_change":"2024-09-17T05:34:43.471776+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.471776+0000","last_undegraded":"2024-09-17T05:34:43.471776+0000","last_fullsized":"2024-09-17T05:34:43.471776+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.498988+0000","last_change":"2024-09-17T05:34:43.495216+0000","last_active":"2024-09-17T05:34:43.498988+0000","last_peered":"2024-09-17T05:34:43.498988+0000","last_clean":"2024-09-17T05:34:43.498988+0000","last_became_active":"2024-09-17T05:34:43.494800+0000","last_became_peered":"2024-09-17T05:34:43.494800+0000","last_unstale":"2024-09-17T05:34:43.498988+0000","last_undegraded":"2024-09-17T05:34:43.498988+0000","last_fullsized":"2024-09-17T05:34:43.498988+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:03.650153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.468214+0000","last_change":"2024-09-17T05:34:43.468214+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.468214+0000","last_undegraded":"2024-09-17T05:34:43.468214+0000","last_fullsized":"2024-09-17T05:34:43.468214+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.492048+0000","last_change":"2024-09-17T05:34:43.491113+0000","last_active":"2024-09-17T05:34:43.492048+0000","last_peered":"2024-09-17T05:34:43.492048+0000","last_clean":"2024-09-17T05:34:43.492048+0000","last_became_active":"2024-09-17T05:34:43.490776+0000","last_became_peered":"2024-09-17T05:34:43.490776+0000","last_unstale":"2024-09-17T05:34:43.492048+0000","last_undegraded":"2024-09-17T05:34:43.492048+0000","last_fullsized":"2024-09-17T05:34:43.492048+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:22.282026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.476081+0000","last_change":"2024-09-17T05:34:43.476081+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.476081+0000","last_undegraded":"2024-09-17T05:34:43.476081+0000","last_fullsized":"2024-09-17T05:34:43.476081+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.496135+0000","last_change":"2024-09-17T05:34:43.490904+0000","last_active":"2024-09-17T05:34:43.496135+0000","last_peered":"2024-09-17T05:34:43.496135+0000","last_clean":"2024-09-17T05:34:43.496135+0000","last_became_active":"2024-09-17T05:34:43.490596+0000","last_became_peered":"2024-09-17T05:34:43.490596+0000","last_unstale":"2024-09-17T05:34:43.496135+0000","last_undegraded":"2024-09-17T05:34:43.496135+0000","last_fullsized":"2024-09-17T05:34:43.496135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:45.332471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.464313+0000","last_change":"2024-09-17T05:34:43.464313+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.464313+0000","last_undegraded":"2024-09-17T05:34:43.464313+0000","last_fullsized":"2024-09-17T05:34:43.464313+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.494892+0000","last_change":"2024-09-17T05:34:43.485943+0000","last_active":"2024-09-17T05:34:43.494892+0000","last_peered":"2024-09-17T05:34:43.494892+0000","last_clean":"2024-09-17T05:34:43.494892+0000","last_became_active":"2024-09-17T05:34:43.485578+0000","last_became_peered":"2024-09-17T05:34:43.485578+0000","last_unstale":"2024-09-17T05:34:43.494892+0000","last_undegraded":"2024-09-17T05:34:43.494892+0000","last_fullsized":"2024-09-17T05:34:43.494892+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:03.562618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-09-17T05:34:43.467211+0000","last_change":"2024-09-17T05:34:43.467211+0000","last_active":"2024-09-17T05:34:43.454042+0000","last_peered":"2024-09-17T05:34:43.454042+0000","last_clean":"2024-09-17T05:34:43.454042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:43.467211+0000","last_undegraded":"2024-09-17T05:34:43.467211+0000","last_fullsized":"2024-09-17T05:34:43.467211+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.486113+0000","last_change":"2024-09-17T05:34:43.483784+0000","last_active":"2024-09-17T05:34:43.486113+0000","last_peered":"2024-09-17T05:34:43.486113+0000","last_clean":"2024-09-17T05:34:43.486113+0000","last_became_active":"2024-09-17T05:34:43.483417+0000","last_became_peered":"2024-09-17T05:34:43.483417+0000","last_unstale":"2024-09-17T05:34:43.486113+0000","last_undegraded":"2024-09-17T05:34:43.486113+0000","last_fullsized":"2024-09-17T05:34:43.486113+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:39.573633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":337,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-17T05:34:43.470110+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:34:43.470110+0000","last_peered":"2024-09-17T05:34:43.470110+0000","last_clean":"2024-09-17T05:34:43.470110+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:34:43.470110+0000","last_undegraded":"2024-09-17T05:34:43.470110+0000","last_fullsized":"2024-09-17T05:34:43.470110+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509582,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":834542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45356,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326484,"statfs":{"total":96636764160,"available":96590319616,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":2334720,"data_stored":1424910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37036,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334804,"statfs":{"total":96636764160,"available":96598839296,"internally_reserved":0,"allocated":1728512,"data_stored":828267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41232,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330608,"statfs":{"total":96636764160,"available":96594542592,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:49.791 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3d is not active+clean 2024-09-17T05:34:49.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.466484+0000', 'last_change': '2024-09-17T05:34:43.466484+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.466484+0000', 'last_undegraded': '2024-09-17T05:34:43.466484+0000', 'last_fullsized': '2024-09-17T05:34:43.466484+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3b is not active+clean 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.469446+0000', 'last_change': '2024-09-17T05:34:43.469446+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.469446+0000', 'last_undegraded': '2024-09-17T05:34:43.469446+0000', 'last_fullsized': '2024-09-17T05:34:43.469446+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3a is not active+clean 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467679+0000', 'last_change': '2024-09-17T05:34:43.467679+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467679+0000', 'last_undegraded': '2024-09-17T05:34:43.467679+0000', 'last_fullsized': '2024-09-17T05:34:43.467679+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:PG 15.39 is not active+clean 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.468846+0000', 'last_change': '2024-09-17T05:34:43.468846+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.468846+0000', 'last_undegraded': '2024-09-17T05:34:43.468846+0000', 'last_fullsized': '2024-09-17T05:34:43.468846+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:PG 15.38 is not active+clean 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.466911+0000', 'last_change': '2024-09-17T05:34:43.466911+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.466911+0000', 'last_undegraded': '2024-09-17T05:34:43.466911+0000', 'last_fullsized': '2024-09-17T05:34:43.466911+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:PG 15.37 is not active+clean 2024-09-17T05:34:49.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.471899+0000', 'last_change': '2024-09-17T05:34:43.471899+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.471899+0000', 'last_undegraded': '2024-09-17T05:34:43.471899+0000', 'last_fullsized': '2024-09-17T05:34:43.471899+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:PG 15.36 is not active+clean 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.465962+0000', 'last_change': '2024-09-17T05:34:43.465962+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.465962+0000', 'last_undegraded': '2024-09-17T05:34:43.465962+0000', 'last_fullsized': '2024-09-17T05:34:43.465962+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:PG 15.35 is not active+clean 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.471985+0000', 'last_change': '2024-09-17T05:34:43.471985+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.471985+0000', 'last_undegraded': '2024-09-17T05:34:43.471985+0000', 'last_fullsized': '2024-09-17T05:34:43.471985+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:PG 15.34 is not active+clean 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467723+0000', 'last_change': '2024-09-17T05:34:43.467723+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467723+0000', 'last_undegraded': '2024-09-17T05:34:43.467723+0000', 'last_fullsized': '2024-09-17T05:34:43.467723+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:PG 15.33 is not active+clean 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.474063+0000', 'last_change': '2024-09-17T05:34:43.474063+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.474063+0000', 'last_undegraded': '2024-09-17T05:34:43.474063+0000', 'last_fullsized': '2024-09-17T05:34:43.474063+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:PG 15.30 is not active+clean 2024-09-17T05:34:49.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.468782+0000', 'last_change': '2024-09-17T05:34:43.468782+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.468782+0000', 'last_undegraded': '2024-09-17T05:34:43.468782+0000', 'last_fullsized': '2024-09-17T05:34:43.468782+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2f is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.475494+0000', 'last_change': '2024-09-17T05:34:43.475494+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.475494+0000', 'last_undegraded': '2024-09-17T05:34:43.475494+0000', 'last_fullsized': '2024-09-17T05:34:43.475494+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.473976+0000', 'last_change': '2024-09-17T05:34:43.473976+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.473976+0000', 'last_undegraded': '2024-09-17T05:34:43.473976+0000', 'last_fullsized': '2024-09-17T05:34:43.473976+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2c is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.465620+0000', 'last_change': '2024-09-17T05:34:43.465620+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.465620+0000', 'last_undegraded': '2024-09-17T05:34:43.465620+0000', 'last_fullsized': '2024-09-17T05:34:43.465620+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.468129+0000', 'last_change': '2024-09-17T05:34:43.468129+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.468129+0000', 'last_undegraded': '2024-09-17T05:34:43.468129+0000', 'last_fullsized': '2024-09-17T05:34:43.468129+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2a is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467304+0000', 'last_change': '2024-09-17T05:34:43.467304+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467304+0000', 'last_undegraded': '2024-09-17T05:34:43.467304+0000', 'last_fullsized': '2024-09-17T05:34:43.467304+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:PG 15.28 is not active+clean 2024-09-17T05:34:49.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.472479+0000', 'last_change': '2024-09-17T05:34:43.472479+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.472479+0000', 'last_undegraded': '2024-09-17T05:34:43.472479+0000', 'last_fullsized': '2024-09-17T05:34:43.472479+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:PG 15.27 is not active+clean 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.465223+0000', 'last_change': '2024-09-17T05:34:43.465223+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.465223+0000', 'last_undegraded': '2024-09-17T05:34:43.465223+0000', 'last_fullsized': '2024-09-17T05:34:43.465223+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:PG 15.26 is not active+clean 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.475574+0000', 'last_change': '2024-09-17T05:34:43.475574+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.475574+0000', 'last_undegraded': '2024-09-17T05:34:43.475574+0000', 'last_fullsized': '2024-09-17T05:34:43.475574+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:PG 15.25 is not active+clean 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.471952+0000', 'last_change': '2024-09-17T05:34:43.471952+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.471952+0000', 'last_undegraded': '2024-09-17T05:34:43.471952+0000', 'last_fullsized': '2024-09-17T05:34:43.471952+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:PG 15.24 is not active+clean 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.476158+0000', 'last_change': '2024-09-17T05:34:43.476158+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.476158+0000', 'last_undegraded': '2024-09-17T05:34:43.476158+0000', 'last_fullsized': '2024-09-17T05:34:43.476158+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:PG 15.23 is not active+clean 2024-09-17T05:34:49.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.473479+0000', 'last_change': '2024-09-17T05:34:43.473479+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.473479+0000', 'last_undegraded': '2024-09-17T05:34:43.473479+0000', 'last_fullsized': '2024-09-17T05:34:43.473479+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.22 is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.473815+0000', 'last_change': '2024-09-17T05:34:43.473815+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.473815+0000', 'last_undegraded': '2024-09-17T05:34:43.473815+0000', 'last_fullsized': '2024-09-17T05:34:43.473815+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.21 is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467581+0000', 'last_change': '2024-09-17T05:34:43.467581+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467581+0000', 'last_undegraded': '2024-09-17T05:34:43.467581+0000', 'last_fullsized': '2024-09-17T05:34:43.467581+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.478231+0000', 'last_change': '2024-09-17T05:34:43.478231+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.478231+0000', 'last_undegraded': '2024-09-17T05:34:43.478231+0000', 'last_fullsized': '2024-09-17T05:34:43.478231+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1f is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.476119+0000', 'last_change': '2024-09-17T05:34:43.476119+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.476119+0000', 'last_undegraded': '2024-09-17T05:34:43.476119+0000', 'last_fullsized': '2024-09-17T05:34:43.476119+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1d is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.465255+0000', 'last_change': '2024-09-17T05:34:43.465255+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.465255+0000', 'last_undegraded': '2024-09-17T05:34:43.465255+0000', 'last_fullsized': '2024-09-17T05:34:43.465255+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1a is not active+clean 2024-09-17T05:34:49.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.464326+0000', 'last_change': '2024-09-17T05:34:43.464326+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.464326+0000', 'last_undegraded': '2024-09-17T05:34:43.464326+0000', 'last_fullsized': '2024-09-17T05:34:43.464326+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1b is not active+clean 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.468869+0000', 'last_change': '2024-09-17T05:34:43.468869+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.468869+0000', 'last_undegraded': '2024-09-17T05:34:43.468869+0000', 'last_fullsized': '2024-09-17T05:34:43.468869+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:PG 15.18 is not active+clean 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.474995+0000', 'last_change': '2024-09-17T05:34:43.474995+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.474995+0000', 'last_undegraded': '2024-09-17T05:34:43.474995+0000', 'last_fullsized': '2024-09-17T05:34:43.474995+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:PG 15.19 is not active+clean 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.465390+0000', 'last_change': '2024-09-17T05:34:43.465390+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.465390+0000', 'last_undegraded': '2024-09-17T05:34:43.465390+0000', 'last_fullsized': '2024-09-17T05:34:43.465390+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:PG 15.17 is not active+clean 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.479109+0000', 'last_change': '2024-09-17T05:34:43.479109+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.479109+0000', 'last_undegraded': '2024-09-17T05:34:43.479109+0000', 'last_fullsized': '2024-09-17T05:34:43.479109+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:PG 15.14 is not active+clean 2024-09-17T05:34:49.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.470033+0000', 'last_change': '2024-09-17T05:34:43.470033+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.470033+0000', 'last_undegraded': '2024-09-17T05:34:43.470033+0000', 'last_fullsized': '2024-09-17T05:34:43.470033+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.15 is not active+clean 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.473951+0000', 'last_change': '2024-09-17T05:34:43.473951+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.473951+0000', 'last_undegraded': '2024-09-17T05:34:43.473951+0000', 'last_fullsized': '2024-09-17T05:34:43.473951+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467555+0000', 'last_change': '2024-09-17T05:34:43.467555+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467555+0000', 'last_undegraded': '2024-09-17T05:34:43.467555+0000', 'last_fullsized': '2024-09-17T05:34:43.467555+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.10 is not active+clean 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467792+0000', 'last_change': '2024-09-17T05:34:43.467792+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467792+0000', 'last_undegraded': '2024-09-17T05:34:43.467792+0000', 'last_fullsized': '2024-09-17T05:34:43.467792+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.466318+0000', 'last_change': '2024-09-17T05:34:43.466318+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.466318+0000', 'last_undegraded': '2024-09-17T05:34:43.466318+0000', 'last_fullsized': '2024-09-17T05:34:43.466318+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.f is not active+clean 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.477704+0000', 'last_change': '2024-09-17T05:34:43.477704+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.477704+0000', 'last_undegraded': '2024-09-17T05:34:43.477704+0000', 'last_fullsized': '2024-09-17T05:34:43.477704+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.798 INFO:tasks.ceph.ceph_manager.ceph:PG 15.c is not active+clean 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.472867+0000', 'last_change': '2024-09-17T05:34:43.472867+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.472867+0000', 'last_undegraded': '2024-09-17T05:34:43.472867+0000', 'last_fullsized': '2024-09-17T05:34:43.472867+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:PG 15.d is not active+clean 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.464690+0000', 'last_change': '2024-09-17T05:34:43.464690+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.464690+0000', 'last_undegraded': '2024-09-17T05:34:43.464690+0000', 'last_fullsized': '2024-09-17T05:34:43.464690+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.472433+0000', 'last_change': '2024-09-17T05:34:43.472433+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.472433+0000', 'last_undegraded': '2024-09-17T05:34:43.472433+0000', 'last_fullsized': '2024-09-17T05:34:43.472433+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.473697+0000', 'last_change': '2024-09-17T05:34:43.473697+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.473697+0000', 'last_undegraded': '2024-09-17T05:34:43.473697+0000', 'last_fullsized': '2024-09-17T05:34:43.473697+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:PG 15.9 is not active+clean 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.472859+0000', 'last_change': '2024-09-17T05:34:43.472859+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.472859+0000', 'last_undegraded': '2024-09-17T05:34:43.472859+0000', 'last_fullsized': '2024-09-17T05:34:43.472859+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:34:49.799 INFO:tasks.ceph.ceph_manager.ceph:PG 15.6 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.477665+0000', 'last_change': '2024-09-17T05:34:43.477665+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.477665+0000', 'last_undegraded': '2024-09-17T05:34:43.477665+0000', 'last_fullsized': '2024-09-17T05:34:43.477665+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.474970+0000', 'last_change': '2024-09-17T05:34:43.474970+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.474970+0000', 'last_undegraded': '2024-09-17T05:34:43.474970+0000', 'last_fullsized': '2024-09-17T05:34:43.474970+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.466430+0000', 'last_change': '2024-09-17T05:34:43.466430+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.466430+0000', 'last_undegraded': '2024-09-17T05:34:43.466430+0000', 'last_fullsized': '2024-09-17T05:34:43.466430+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:PG 15.5 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.471452+0000', 'last_change': '2024-09-17T05:34:43.471452+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.471452+0000', 'last_undegraded': '2024-09-17T05:34:43.471452+0000', 'last_fullsized': '2024-09-17T05:34:43.471452+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.471776+0000', 'last_change': '2024-09-17T05:34:43.471776+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.471776+0000', 'last_undegraded': '2024-09-17T05:34:43.471776+0000', 'last_fullsized': '2024-09-17T05:34:43.471776+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3 is not active+clean 2024-09-17T05:34:49.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.468214+0000', 'last_change': '2024-09-17T05:34:43.468214+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.468214+0000', 'last_undegraded': '2024-09-17T05:34:43.468214+0000', 'last_fullsized': '2024-09-17T05:34:43.468214+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.476081+0000', 'last_change': '2024-09-17T05:34:43.476081+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.476081+0000', 'last_undegraded': '2024-09-17T05:34:43.476081+0000', 'last_fullsized': '2024-09-17T05:34:43.476081+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1 is not active+clean 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.464313+0000', 'last_change': '2024-09-17T05:34:43.464313+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.464313+0000', 'last_undegraded': '2024-09-17T05:34:43.464313+0000', 'last_fullsized': '2024-09-17T05:34:43.464313+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-09-17T05:34:49.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:34:43.467211+0000', 'last_change': '2024-09-17T05:34:43.467211+0000', 'last_active': '2024-09-17T05:34:43.454042+0000', 'last_peered': '2024-09-17T05:34:43.454042+0000', 'last_clean': '2024-09-17T05:34:43.454042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:34:43.467211+0000', 'last_undegraded': '2024-09-17T05:34:43.467211+0000', 'last_fullsized': '2024-09-17T05:34:43.467211+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'last_clean_scrub_stamp': '2024-09-17T05:34:43.454042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:34:49.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:34:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- 172.21.15.43:0/3494670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b80b3af0 msgr2=0x7f57b80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 --2- 172.21.15.43:0/3494670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b80b3af0 0x7f57b80ad0b0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009c80 tx=0x7f57b001cce0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- 172.21.15.43:0/3494670841 shutdown_connections 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 --2- 172.21.15.43:0/3494670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b80b3af0 0x7f57b80ad0b0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 --2- 172.21.15.43:0/3494670841 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b8155d70 0x7f57b815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- 172.21.15.43:0/3494670841 >> 172.21.15.43:0/3494670841 conn(0x7f57b80b23d0 msgr2=0x7f57b80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- 172.21.15.43:0/3494670841 shutdown_connections 2024-09-17T05:34:49.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- 172.21.15.43:0/3494670841 wait complete. 2024-09-17T05:34:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 Processor -- start 2024-09-17T05:34:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.985+0000 7f57bfc00640 1 -- start start 2024-09-17T05:34:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b806f110 0x7f57b81fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b80cd530 0x7f57b80cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8168c80 con 0x7f57b8155d70 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8168b00 con 0x7f57b806f110 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f57b8168e00 con 0x7f57b80cd530 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54226/0 (socket says 172.21.15.43:54226) 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 -- 172.21.15.43:0/1052600878 learned_addr learned my addr 172.21.15.43:0/1052600878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bd200640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b806f110 0x7f57b81fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bdc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b80cd530 0x7f57b80cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 -- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b80cd530 msgr2=0x7f57b80cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b80cd530 0x7f57b80cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 -- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b806f110 msgr2=0x7f57b81fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b806f110 0x7f57b81fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 -- 172.21.15.43:0/1052600878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8165460 con 0x7f57b8155d70 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57afe00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c3c0 tx=0x7f57a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0017070 con 0x7f57b8155d70 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57a00053c0 con 0x7f57b8155d70 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b8165740 con 0x7f57b8155d70 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0003860 con 0x7f57b8155d70 2024-09-17T05:34:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81f9180 con 0x7f57b8155d70 2024-09-17T05:34:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.989+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b80ad0b0 con 0x7f57b8155d70 2024-09-17T05:34:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.993+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57a001a070 con 0x7f57b8155d70 2024-09-17T05:34:49.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.993+0000 7f57b5600640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 0x7f5794075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:49.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.993+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f57a0093950 con 0x7f57b8155d70 2024-09-17T05:34:49.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.993+0000 7f57bd200640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 0x7f5794075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:49.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.993+0000 7f57bd200640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 0x7f5794075710 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f57b80715c0 tx=0x7f57a800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:49.997+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f57a0021020 con 0x7f57b8155d70 2024-09-17T05:34:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f57b806dd30 con 0x7f57b8155d70 2024-09-17T05:34:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57b5600640 1 -- 172.21.15.43:0/1052600878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7f57a005e390 con 0x7f57b8155d70 2024-09-17T05:34:50.236 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":139,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550593,"num_in_osds":8,"osd_in_since":1726550582,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":52},{"state_name":"active+clean","count":45}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":590368,"bytes_used":308928512,"bytes_avail":772785184768,"bytes_total":773094113280,"inactive_pgs_ratio":0.53608244657516479,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":345,"btime":"2024-09-17T05:34:47:511187+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":10464}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":50,"modified":"2024-09-17T05:34:20.023180+0000","services":{}},"progress_events":{}} 2024-09-17T05:34:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 msgr2=0x7f5794075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 0x7f5794075710 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f57b80715c0 tx=0x7f57a800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 msgr2=0x7f57b809d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c3c0 tx=0x7f57a000c880 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 shutdown_connections 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5794073200 0x7f5794075710 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f57b80cd530 0x7f57b80cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57b8155d70 0x7f57b809d8d0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 --2- 172.21.15.43:0/1052600878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57b806f110 0x7f57b81fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 >> 172.21.15.43:0/1052600878 conn(0x7f57b80b23d0 msgr2=0x7f57b815b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 shutdown_connections 2024-09-17T05:34:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.233+0000 7f57bfc00640 1 -- 172.21.15.43:0/1052600878 wait complete. 2024-09-17T05:34:50.259 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:34:50.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- 172.21.15.43:0/2638429251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7904003af0 msgr2=0x7f7904003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 --2- 172.21.15.43:0/2638429251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7904003af0 0x7f7904003f60 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f791000c260 tx=0x7f791001cf30 comp rx=0 tx=0).stop 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7925c00640 1 -- 172.21.15.43:0/2638429251 reap_dead start 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- 172.21.15.43:0/2638429251 shutdown_connections 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- 172.21.15.43:0/2638429251 >> 172.21.15.43:0/2638429251 conn(0x7f79200b23d0 msgr2=0x7f79200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- 172.21.15.43:0/2638429251 shutdown_connections 2024-09-17T05:34:50.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- 172.21.15.43:0/2638429251 wait complete. 2024-09-17T05:34:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 Processor -- start 2024-09-17T05:34:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.417+0000 7f7927c00640 1 -- start start 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7904003af0 0x7f79200ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79200c90d0 0x7f79200c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f792015cb50 con 0x7f79200cc100 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f792015c9d0 con 0x7f79200c90d0 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f79201663b0 con 0x7f7904003af0 2024-09-17T05:34:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54252/0 (socket says 172.21.15.43:54252) 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 -- 172.21.15.43:0/2723511586 learned_addr learned my addr 172.21.15.43:0/2723511586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925200640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7904003af0 0x7f79200ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7917e00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79200c90d0 0x7f79200c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 -- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7904003af0 msgr2=0x7f79200ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7904003af0 0x7f79200ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 -- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79200c90d0 msgr2=0x7f79200c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79200c90d0 0x7f79200c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 -- 172.21.15.43:0/2723511586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79200afa20 con 0x7f79200cc100 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7925c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79200b5290 tx=0x7f7918010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791800ee10 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7918015070 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79201fcc10 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918003d50 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79201fc370 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.421+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79201fbe00 con 0x7f79200cc100 2024-09-17T05:34:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.425+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7918004030 con 0x7f79200cc100 2024-09-17T05:34:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.425+0000 7f791d600640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 0x7f78fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.425+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f7918094b70 con 0x7f79200cc100 2024-09-17T05:34:50.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.425+0000 7f7925200640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 0x7f78fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:50.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.425+0000 7f7925200640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 0x7f78fc075710 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f79100ac000 tx=0x7f7910019d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.429+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f791805f530 con 0x7f79200cc100 2024-09-17T05:34:50.449 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:34:50.445+0000 7ff2efa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.601+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79200ad0b0 con 0x7f78fc073200 2024-09-17T05:34:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.609+0000 7f791d600640 1 -- 172.21.15.43:0/2723511586 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f79200ad0b0 con 0x7f78fc073200 2024-09-17T05:34:50.615 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:50.616 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 msgr2=0x7f78fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 0x7f78fc075710 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f79100ac000 tx=0x7f7910019d70 comp rx=0 tx=0).stop 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 msgr2=0x7f79200cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f79200b5290 tx=0x7f7918010e80 comp rx=0 tx=0).stop 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 shutdown_connections 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f78fc073200 0x7f78fc075710 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79200c90d0 0x7f79200c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79200cc100 0x7f79200cc510 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 --2- 172.21.15.43:0/2723511586 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7904003af0 0x7f79200ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 >> 172.21.15.43:0/2723511586 conn(0x7f79200b23d0 msgr2=0x7f79200d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 shutdown_connections 2024-09-17T05:34:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:50.613+0000 7f7927c00640 1 -- 172.21.15.43:0/2723511586 wait complete. 2024-09-17T05:34:50.638 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-17T05:34:50.031017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285356,"kb_used_data":15016,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754689364,"statfs":{"total":773094113280,"available":772801908736,"internally_reserved":0,"allocated":15376384,"data_stored":7966492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.567324"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167073+0000","last_change":"2024-09-17T05:34:44.497120+0000","last_active":"2024-09-17T05:34:46.167073+0000","last_peered":"2024-09-17T05:34:46.167073+0000","last_clean":"2024-09-17T05:34:46.167073+0000","last_became_active":"2024-09-17T05:34:44.496734+0000","last_became_peered":"2024-09-17T05:34:44.496734+0000","last_unstale":"2024-09-17T05:34:46.167073+0000","last_undegraded":"2024-09-17T05:34:46.167073+0000","last_fullsized":"2024-09-17T05:34:46.167073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:55.766746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166945+0000","last_change":"2024-09-17T05:34:44.504496+0000","last_active":"2024-09-17T05:34:46.166945+0000","last_peered":"2024-09-17T05:34:46.166945+0000","last_clean":"2024-09-17T05:34:46.166945+0000","last_became_active":"2024-09-17T05:34:44.504097+0000","last_became_peered":"2024-09-17T05:34:44.504097+0000","last_unstale":"2024-09-17T05:34:46.166945+0000","last_undegraded":"2024-09-17T05:34:46.166945+0000","last_fullsized":"2024-09-17T05:34:46.166945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:31.928876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480275+0000","last_change":"2024-09-17T05:34:44.512793+0000","last_active":"2024-09-17T05:34:45.480275+0000","last_peered":"2024-09-17T05:34:45.480275+0000","last_clean":"2024-09-17T05:34:45.480275+0000","last_became_active":"2024-09-17T05:34:44.512396+0000","last_became_peered":"2024-09-17T05:34:44.512396+0000","last_unstale":"2024-09-17T05:34:45.480275+0000","last_undegraded":"2024-09-17T05:34:45.480275+0000","last_fullsized":"2024-09-17T05:34:45.480275+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:47.200644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166881+0000","last_change":"2024-09-17T05:34:44.511242+0000","last_active":"2024-09-17T05:34:46.166881+0000","last_peered":"2024-09-17T05:34:46.166881+0000","last_clean":"2024-09-17T05:34:46.166881+0000","last_became_active":"2024-09-17T05:34:44.510544+0000","last_became_peered":"2024-09-17T05:34:44.510544+0000","last_unstale":"2024-09-17T05:34:46.166881+0000","last_undegraded":"2024-09-17T05:34:46.166881+0000","last_fullsized":"2024-09-17T05:34:46.166881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:16.675435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163757+0000","last_change":"2024-09-17T05:34:44.487817+0000","last_active":"2024-09-17T05:34:46.163757+0000","last_peered":"2024-09-17T05:34:46.163757+0000","last_clean":"2024-09-17T05:34:46.163757+0000","last_became_active":"2024-09-17T05:34:44.487468+0000","last_became_peered":"2024-09-17T05:34:44.487468+0000","last_unstale":"2024-09-17T05:34:46.163757+0000","last_undegraded":"2024-09-17T05:34:46.163757+0000","last_fullsized":"2024-09-17T05:34:46.163757+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:16.910527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163739+0000","last_change":"2024-09-17T05:34:44.506024+0000","last_active":"2024-09-17T05:34:46.163739+0000","last_peered":"2024-09-17T05:34:46.163739+0000","last_clean":"2024-09-17T05:34:46.163739+0000","last_became_active":"2024-09-17T05:34:44.505665+0000","last_became_peered":"2024-09-17T05:34:44.505665+0000","last_unstale":"2024-09-17T05:34:46.163739+0000","last_undegraded":"2024-09-17T05:34:46.163739+0000","last_fullsized":"2024-09-17T05:34:46.163739+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:05.236790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163546+0000","last_change":"2024-09-17T05:34:44.482676+0000","last_active":"2024-09-17T05:34:46.163546+0000","last_peered":"2024-09-17T05:34:46.163546+0000","last_clean":"2024-09-17T05:34:46.163546+0000","last_became_active":"2024-09-17T05:34:44.482313+0000","last_became_peered":"2024-09-17T05:34:44.482313+0000","last_unstale":"2024-09-17T05:34:46.163546+0000","last_undegraded":"2024-09-17T05:34:46.163546+0000","last_fullsized":"2024-09-17T05:34:46.163546+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.045565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479447+0000","last_change":"2024-09-17T05:34:44.496923+0000","last_active":"2024-09-17T05:34:45.479447+0000","last_peered":"2024-09-17T05:34:45.479447+0000","last_clean":"2024-09-17T05:34:45.479447+0000","last_became_active":"2024-09-17T05:34:44.496522+0000","last_became_peered":"2024-09-17T05:34:44.496522+0000","last_unstale":"2024-09-17T05:34:45.479447+0000","last_undegraded":"2024-09-17T05:34:45.479447+0000","last_fullsized":"2024-09-17T05:34:45.479447+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:51.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164361+0000","last_change":"2024-09-17T05:34:44.493741+0000","last_active":"2024-09-17T05:34:46.164361+0000","last_peered":"2024-09-17T05:34:46.164361+0000","last_clean":"2024-09-17T05:34:46.164361+0000","last_became_active":"2024-09-17T05:34:44.493381+0000","last_became_peered":"2024-09-17T05:34:44.493381+0000","last_unstale":"2024-09-17T05:34:46.164361+0000","last_undegraded":"2024-09-17T05:34:46.164361+0000","last_fullsized":"2024-09-17T05:34:46.164361+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:35.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475879+0000","last_change":"2024-09-17T05:34:44.494112+0000","last_active":"2024-09-17T05:34:45.475879+0000","last_peered":"2024-09-17T05:34:45.475879+0000","last_clean":"2024-09-17T05:34:45.475879+0000","last_became_active":"2024-09-17T05:34:44.493782+0000","last_became_peered":"2024-09-17T05:34:44.493782+0000","last_unstale":"2024-09-17T05:34:45.475879+0000","last_undegraded":"2024-09-17T05:34:45.475879+0000","last_fullsized":"2024-09-17T05:34:45.475879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:41.344789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163847+0000","last_change":"2024-09-17T05:34:44.506473+0000","last_active":"2024-09-17T05:34:46.163847+0000","last_peered":"2024-09-17T05:34:46.163847+0000","last_clean":"2024-09-17T05:34:46.163847+0000","last_became_active":"2024-09-17T05:34:44.505979+0000","last_became_peered":"2024-09-17T05:34:44.505979+0000","last_unstale":"2024-09-17T05:34:46.163847+0000","last_undegraded":"2024-09-17T05:34:46.163847+0000","last_fullsized":"2024-09-17T05:34:46.163847+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:41.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477173+0000","last_change":"2024-09-17T05:34:44.499822+0000","last_active":"2024-09-17T05:34:45.477173+0000","last_peered":"2024-09-17T05:34:45.477173+0000","last_clean":"2024-09-17T05:34:45.477173+0000","last_became_active":"2024-09-17T05:34:44.499335+0000","last_became_peered":"2024-09-17T05:34:44.499335+0000","last_unstale":"2024-09-17T05:34:45.477173+0000","last_undegraded":"2024-09-17T05:34:45.477173+0000","last_fullsized":"2024-09-17T05:34:45.477173+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:26.838840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664124+0000","last_change":"2024-09-17T05:34:44.501617+0000","last_active":"2024-09-17T05:34:45.664124+0000","last_peered":"2024-09-17T05:34:45.664124+0000","last_clean":"2024-09-17T05:34:45.664124+0000","last_became_active":"2024-09-17T05:34:44.501185+0000","last_became_peered":"2024-09-17T05:34:44.501185+0000","last_unstale":"2024-09-17T05:34:45.664124+0000","last_undegraded":"2024-09-17T05:34:45.664124+0000","last_fullsized":"2024-09-17T05:34:45.664124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:10.459297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166241+0000","last_change":"2024-09-17T05:34:44.498246+0000","last_active":"2024-09-17T05:34:46.166241+0000","last_peered":"2024-09-17T05:34:46.166241+0000","last_clean":"2024-09-17T05:34:46.166241+0000","last_became_active":"2024-09-17T05:34:44.495932+0000","last_became_peered":"2024-09-17T05:34:44.495932+0000","last_unstale":"2024-09-17T05:34:46.166241+0000","last_undegraded":"2024-09-17T05:34:46.166241+0000","last_fullsized":"2024-09-17T05:34:46.166241+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:11.396023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166253+0000","last_change":"2024-09-17T05:34:44.503954+0000","last_active":"2024-09-17T05:34:46.166253+0000","last_peered":"2024-09-17T05:34:46.166253+0000","last_clean":"2024-09-17T05:34:46.166253+0000","last_became_active":"2024-09-17T05:34:44.502592+0000","last_became_peered":"2024-09-17T05:34:44.502592+0000","last_unstale":"2024-09-17T05:34:46.166253+0000","last_undegraded":"2024-09-17T05:34:46.166253+0000","last_fullsized":"2024-09-17T05:34:46.166253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:48.068029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163497+0000","last_change":"2024-09-17T05:34:44.512531+0000","last_active":"2024-09-17T05:34:46.163497+0000","last_peered":"2024-09-17T05:34:46.163497+0000","last_clean":"2024-09-17T05:34:46.163497+0000","last_became_active":"2024-09-17T05:34:44.511975+0000","last_became_peered":"2024-09-17T05:34:44.511975+0000","last_unstale":"2024-09-17T05:34:46.163497+0000","last_undegraded":"2024-09-17T05:34:46.163497+0000","last_fullsized":"2024-09-17T05:34:46.163497+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:42.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508302+0000","last_change":"2024-09-17T05:34:44.504891+0000","last_active":"2024-09-17T05:34:45.508302+0000","last_peered":"2024-09-17T05:34:45.508302+0000","last_clean":"2024-09-17T05:34:45.508302+0000","last_became_active":"2024-09-17T05:34:44.504514+0000","last_became_peered":"2024-09-17T05:34:44.504514+0000","last_unstale":"2024-09-17T05:34:45.508302+0000","last_undegraded":"2024-09-17T05:34:45.508302+0000","last_fullsized":"2024-09-17T05:34:45.508302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:39.318985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167005+0000","last_change":"2024-09-17T05:34:44.501872+0000","last_active":"2024-09-17T05:34:46.167005+0000","last_peered":"2024-09-17T05:34:46.167005+0000","last_clean":"2024-09-17T05:34:46.167005+0000","last_became_active":"2024-09-17T05:34:44.498268+0000","last_became_peered":"2024-09-17T05:34:44.498268+0000","last_unstale":"2024-09-17T05:34:46.167005+0000","last_undegraded":"2024-09-17T05:34:46.167005+0000","last_fullsized":"2024-09-17T05:34:46.167005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:27.861661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508072+0000","last_change":"2024-09-17T05:34:44.503701+0000","last_active":"2024-09-17T05:34:45.508072+0000","last_peered":"2024-09-17T05:34:45.508072+0000","last_clean":"2024-09-17T05:34:45.508072+0000","last_became_active":"2024-09-17T05:34:44.503358+0000","last_became_peered":"2024-09-17T05:34:44.503358+0000","last_unstale":"2024-09-17T05:34:45.508072+0000","last_undegraded":"2024-09-17T05:34:45.508072+0000","last_fullsized":"2024-09-17T05:34:45.508072+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:58.779306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477267+0000","last_change":"2024-09-17T05:34:44.502211+0000","last_active":"2024-09-17T05:34:45.477267+0000","last_peered":"2024-09-17T05:34:45.477267+0000","last_clean":"2024-09-17T05:34:45.477267+0000","last_became_active":"2024-09-17T05:34:44.501903+0000","last_became_peered":"2024-09-17T05:34:44.501903+0000","last_unstale":"2024-09-17T05:34:45.477267+0000","last_undegraded":"2024-09-17T05:34:45.477267+0000","last_fullsized":"2024-09-17T05:34:45.477267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:34.368235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479917+0000","last_change":"2024-09-17T05:34:44.484938+0000","last_active":"2024-09-17T05:34:45.479917+0000","last_peered":"2024-09-17T05:34:45.479917+0000","last_clean":"2024-09-17T05:34:45.479917+0000","last_became_active":"2024-09-17T05:34:44.484582+0000","last_became_peered":"2024-09-17T05:34:44.484582+0000","last_unstale":"2024-09-17T05:34:45.479917+0000","last_undegraded":"2024-09-17T05:34:45.479917+0000","last_fullsized":"2024-09-17T05:34:45.479917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:14.329439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479631+0000","last_change":"2024-09-17T05:34:44.486932+0000","last_active":"2024-09-17T05:34:45.479631+0000","last_peered":"2024-09-17T05:34:45.479631+0000","last_clean":"2024-09-17T05:34:45.479631+0000","last_became_active":"2024-09-17T05:34:44.486573+0000","last_became_peered":"2024-09-17T05:34:44.486573+0000","last_unstale":"2024-09-17T05:34:45.479631+0000","last_undegraded":"2024-09-17T05:34:45.479631+0000","last_fullsized":"2024-09-17T05:34:45.479631+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:51.872331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166332+0000","last_change":"2024-09-17T05:34:44.495822+0000","last_active":"2024-09-17T05:34:46.166332+0000","last_peered":"2024-09-17T05:34:46.166332+0000","last_clean":"2024-09-17T05:34:46.166332+0000","last_became_active":"2024-09-17T05:34:44.495403+0000","last_became_peered":"2024-09-17T05:34:44.495403+0000","last_unstale":"2024-09-17T05:34:46.166332+0000","last_undegraded":"2024-09-17T05:34:46.166332+0000","last_fullsized":"2024-09-17T05:34:46.166332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:15.566322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662156+0000","last_change":"2024-09-17T05:34:44.493682+0000","last_active":"2024-09-17T05:34:45.662156+0000","last_peered":"2024-09-17T05:34:45.662156+0000","last_clean":"2024-09-17T05:34:45.662156+0000","last_became_active":"2024-09-17T05:34:44.493265+0000","last_became_peered":"2024-09-17T05:34:44.493265+0000","last_unstale":"2024-09-17T05:34:45.662156+0000","last_undegraded":"2024-09-17T05:34:45.662156+0000","last_fullsized":"2024-09-17T05:34:45.662156+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.817263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476199+0000","last_change":"2024-09-17T05:34:44.503721+0000","last_active":"2024-09-17T05:34:45.476199+0000","last_peered":"2024-09-17T05:34:45.476199+0000","last_clean":"2024-09-17T05:34:45.476199+0000","last_became_active":"2024-09-17T05:34:44.503327+0000","last_became_peered":"2024-09-17T05:34:44.503327+0000","last_unstale":"2024-09-17T05:34:45.476199+0000","last_undegraded":"2024-09-17T05:34:45.476199+0000","last_fullsized":"2024-09-17T05:34:45.476199+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:54.328028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664287+0000","last_change":"2024-09-17T05:34:44.501660+0000","last_active":"2024-09-17T05:34:45.664287+0000","last_peered":"2024-09-17T05:34:45.664287+0000","last_clean":"2024-09-17T05:34:45.664287+0000","last_became_active":"2024-09-17T05:34:44.500487+0000","last_became_peered":"2024-09-17T05:34:44.500487+0000","last_unstale":"2024-09-17T05:34:45.664287+0000","last_undegraded":"2024-09-17T05:34:45.664287+0000","last_fullsized":"2024-09-17T05:34:45.664287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:14.041266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662663+0000","last_change":"2024-09-17T05:34:44.498883+0000","last_active":"2024-09-17T05:34:45.662663+0000","last_peered":"2024-09-17T05:34:45.662663+0000","last_clean":"2024-09-17T05:34:45.662663+0000","last_became_active":"2024-09-17T05:34:44.498564+0000","last_became_peered":"2024-09-17T05:34:44.498564+0000","last_unstale":"2024-09-17T05:34:45.662663+0000","last_undegraded":"2024-09-17T05:34:45.662663+0000","last_fullsized":"2024-09-17T05:34:45.662663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:34.487351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662439+0000","last_change":"2024-09-17T05:34:44.498716+0000","last_active":"2024-09-17T05:34:45.662439+0000","last_peered":"2024-09-17T05:34:45.662439+0000","last_clean":"2024-09-17T05:34:45.662439+0000","last_became_active":"2024-09-17T05:34:44.498423+0000","last_became_peered":"2024-09-17T05:34:44.498423+0000","last_unstale":"2024-09-17T05:34:45.662439+0000","last_undegraded":"2024-09-17T05:34:45.662439+0000","last_fullsized":"2024-09-17T05:34:45.662439+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:03.599412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664416+0000","last_change":"2024-09-17T05:34:44.496130+0000","last_active":"2024-09-17T05:34:45.664416+0000","last_peered":"2024-09-17T05:34:45.664416+0000","last_clean":"2024-09-17T05:34:45.664416+0000","last_became_active":"2024-09-17T05:34:44.495826+0000","last_became_peered":"2024-09-17T05:34:44.495826+0000","last_unstale":"2024-09-17T05:34:45.664416+0000","last_undegraded":"2024-09-17T05:34:45.664416+0000","last_fullsized":"2024-09-17T05:34:45.664416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:04.942761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664166+0000","last_change":"2024-09-17T05:34:44.505596+0000","last_active":"2024-09-17T05:34:45.664166+0000","last_peered":"2024-09-17T05:34:45.664166+0000","last_clean":"2024-09-17T05:34:45.664166+0000","last_became_active":"2024-09-17T05:34:44.505284+0000","last_became_peered":"2024-09-17T05:34:44.505284+0000","last_unstale":"2024-09-17T05:34:45.664166+0000","last_undegraded":"2024-09-17T05:34:45.664166+0000","last_fullsized":"2024-09-17T05:34:45.664166+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:46.903568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479588+0000","last_change":"2024-09-17T05:34:44.482924+0000","last_active":"2024-09-17T05:34:45.479588+0000","last_peered":"2024-09-17T05:34:45.479588+0000","last_clean":"2024-09-17T05:34:45.479588+0000","last_became_active":"2024-09-17T05:34:44.482374+0000","last_became_peered":"2024-09-17T05:34:44.482374+0000","last_unstale":"2024-09-17T05:34:45.479588+0000","last_undegraded":"2024-09-17T05:34:45.479588+0000","last_fullsized":"2024-09-17T05:34:45.479588+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.289888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166227+0000","last_change":"2024-09-17T05:34:44.515160+0000","last_active":"2024-09-17T05:34:46.166227+0000","last_peered":"2024-09-17T05:34:46.166227+0000","last_clean":"2024-09-17T05:34:46.166227+0000","last_became_active":"2024-09-17T05:34:44.514775+0000","last_became_peered":"2024-09-17T05:34:44.514775+0000","last_unstale":"2024-09-17T05:34:46.166227+0000","last_undegraded":"2024-09-17T05:34:46.166227+0000","last_fullsized":"2024-09-17T05:34:46.166227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:18.490230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508439+0000","last_change":"2024-09-17T05:34:44.515141+0000","last_active":"2024-09-17T05:34:45.508439+0000","last_peered":"2024-09-17T05:34:45.508439+0000","last_clean":"2024-09-17T05:34:45.508439+0000","last_became_active":"2024-09-17T05:34:44.514837+0000","last_became_peered":"2024-09-17T05:34:44.514837+0000","last_unstale":"2024-09-17T05:34:45.508439+0000","last_undegraded":"2024-09-17T05:34:45.508439+0000","last_fullsized":"2024-09-17T05:34:45.508439+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:32.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"138'4","reported_seq":19,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664457+0000","last_change":"2024-09-17T05:34:43.492447+0000","last_active":"2024-09-17T05:34:45.664457+0000","last_peered":"2024-09-17T05:34:45.664457+0000","last_clean":"2024-09-17T05:34:45.664457+0000","last_became_active":"2024-09-17T05:34:43.492128+0000","last_became_peered":"2024-09-17T05:34:43.492128+0000","last_unstale":"2024-09-17T05:34:45.664457+0000","last_undegraded":"2024-09-17T05:34:45.664457+0000","last_fullsized":"2024-09-17T05:34:45.664457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:38.840126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664393+0000","last_change":"2024-09-17T05:34:44.492812+0000","last_active":"2024-09-17T05:34:45.664393+0000","last_peered":"2024-09-17T05:34:45.664393+0000","last_clean":"2024-09-17T05:34:45.664393+0000","last_became_active":"2024-09-17T05:34:44.492300+0000","last_became_peered":"2024-09-17T05:34:44.492300+0000","last_unstale":"2024-09-17T05:34:45.664393+0000","last_undegraded":"2024-09-17T05:34:45.664393+0000","last_fullsized":"2024-09-17T05:34:45.664393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:06.195523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164199+0000","last_change":"2024-09-17T05:34:43.488048+0000","last_active":"2024-09-17T05:34:46.164199+0000","last_peered":"2024-09-17T05:34:46.164199+0000","last_clean":"2024-09-17T05:34:46.164199+0000","last_became_active":"2024-09-17T05:34:43.487637+0000","last_became_peered":"2024-09-17T05:34:43.487637+0000","last_unstale":"2024-09-17T05:34:46.164199+0000","last_undegraded":"2024-09-17T05:34:46.164199+0000","last_fullsized":"2024-09-17T05:34:46.164199+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:21.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166891+0000","last_change":"2024-09-17T05:34:44.497033+0000","last_active":"2024-09-17T05:34:46.166891+0000","last_peered":"2024-09-17T05:34:46.166891+0000","last_clean":"2024-09-17T05:34:46.166891+0000","last_became_active":"2024-09-17T05:34:44.496645+0000","last_became_peered":"2024-09-17T05:34:44.496645+0000","last_unstale":"2024-09-17T05:34:46.166891+0000","last_undegraded":"2024-09-17T05:34:46.166891+0000","last_fullsized":"2024-09-17T05:34:46.166891+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:37.177571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480349+0000","last_change":"2024-09-17T05:34:43.490759+0000","last_active":"2024-09-17T05:34:45.480349+0000","last_peered":"2024-09-17T05:34:45.480349+0000","last_clean":"2024-09-17T05:34:45.480349+0000","last_became_active":"2024-09-17T05:34:43.489512+0000","last_became_peered":"2024-09-17T05:34:43.489512+0000","last_unstale":"2024-09-17T05:34:45.480349+0000","last_undegraded":"2024-09-17T05:34:45.480349+0000","last_fullsized":"2024-09-17T05:34:45.480349+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:41.753643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475854+0000","last_change":"2024-09-17T05:34:44.481916+0000","last_active":"2024-09-17T05:34:45.475854+0000","last_peered":"2024-09-17T05:34:45.475854+0000","last_clean":"2024-09-17T05:34:45.475854+0000","last_became_active":"2024-09-17T05:34:44.481344+0000","last_became_peered":"2024-09-17T05:34:44.481344+0000","last_unstale":"2024-09-17T05:34:45.475854+0000","last_undegraded":"2024-09-17T05:34:45.475854+0000","last_fullsized":"2024-09-17T05:34:45.475854+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:34.184609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166985+0000","last_change":"2024-09-17T05:34:43.497073+0000","last_active":"2024-09-17T05:34:46.166985+0000","last_peered":"2024-09-17T05:34:46.166985+0000","last_clean":"2024-09-17T05:34:46.166985+0000","last_became_active":"2024-09-17T05:34:43.496723+0000","last_became_peered":"2024-09-17T05:34:43.496723+0000","last_unstale":"2024-09-17T05:34:46.166985+0000","last_undegraded":"2024-09-17T05:34:46.166985+0000","last_fullsized":"2024-09-17T05:34:46.166985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:36.943277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476796+0000","last_change":"2024-09-17T05:34:44.495188+0000","last_active":"2024-09-17T05:34:45.476796+0000","last_peered":"2024-09-17T05:34:45.476796+0000","last_clean":"2024-09-17T05:34:45.476796+0000","last_became_active":"2024-09-17T05:34:44.494832+0000","last_became_peered":"2024-09-17T05:34:44.494832+0000","last_unstale":"2024-09-17T05:34:45.476796+0000","last_undegraded":"2024-09-17T05:34:45.476796+0000","last_fullsized":"2024-09-17T05:34:45.476796+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:23.590308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166073+0000","last_change":"2024-09-17T05:34:43.484224+0000","last_active":"2024-09-17T05:34:46.166073+0000","last_peered":"2024-09-17T05:34:46.166073+0000","last_clean":"2024-09-17T05:34:46.166073+0000","last_became_active":"2024-09-17T05:34:43.483865+0000","last_became_peered":"2024-09-17T05:34:43.483865+0000","last_unstale":"2024-09-17T05:34:46.166073+0000","last_undegraded":"2024-09-17T05:34:46.166073+0000","last_fullsized":"2024-09-17T05:34:46.166073+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.065970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163982+0000","last_change":"2024-09-17T05:34:44.506162+0000","last_active":"2024-09-17T05:34:46.163982+0000","last_peered":"2024-09-17T05:34:46.163982+0000","last_clean":"2024-09-17T05:34:46.163982+0000","last_became_active":"2024-09-17T05:34:44.505833+0000","last_became_peered":"2024-09-17T05:34:44.505833+0000","last_unstale":"2024-09-17T05:34:46.163982+0000","last_undegraded":"2024-09-17T05:34:46.163982+0000","last_fullsized":"2024-09-17T05:34:46.163982+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:19.108188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479631+0000","last_change":"2024-09-17T05:34:43.479766+0000","last_active":"2024-09-17T05:34:45.479631+0000","last_peered":"2024-09-17T05:34:45.479631+0000","last_clean":"2024-09-17T05:34:45.479631+0000","last_became_active":"2024-09-17T05:34:43.479411+0000","last_became_peered":"2024-09-17T05:34:43.479411+0000","last_unstale":"2024-09-17T05:34:45.479631+0000","last_undegraded":"2024-09-17T05:34:45.479631+0000","last_fullsized":"2024-09-17T05:34:45.479631+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:09.451434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.663981+0000","last_change":"2024-09-17T05:34:44.495941+0000","last_active":"2024-09-17T05:34:45.663981+0000","last_peered":"2024-09-17T05:34:45.663981+0000","last_clean":"2024-09-17T05:34:45.663981+0000","last_became_active":"2024-09-17T05:34:44.495571+0000","last_became_peered":"2024-09-17T05:34:44.495571+0000","last_unstale":"2024-09-17T05:34:45.663981+0000","last_undegraded":"2024-09-17T05:34:45.663981+0000","last_fullsized":"2024-09-17T05:34:45.663981+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:38.331985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163733+0000","last_change":"2024-09-17T05:34:43.485518+0000","last_active":"2024-09-17T05:34:46.163733+0000","last_peered":"2024-09-17T05:34:46.163733+0000","last_clean":"2024-09-17T05:34:46.163733+0000","last_became_active":"2024-09-17T05:34:43.485155+0000","last_became_peered":"2024-09-17T05:34:43.485155+0000","last_unstale":"2024-09-17T05:34:46.163733+0000","last_undegraded":"2024-09-17T05:34:46.163733+0000","last_fullsized":"2024-09-17T05:34:46.163733+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:48.488113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476604+0000","last_change":"2024-09-17T05:34:44.496613+0000","last_active":"2024-09-17T05:34:45.476604+0000","last_peered":"2024-09-17T05:34:45.476604+0000","last_clean":"2024-09-17T05:34:45.476604+0000","last_became_active":"2024-09-17T05:34:44.496299+0000","last_became_peered":"2024-09-17T05:34:44.496299+0000","last_unstale":"2024-09-17T05:34:45.476604+0000","last_undegraded":"2024-09-17T05:34:45.476604+0000","last_fullsized":"2024-09-17T05:34:45.476604+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.437484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507899+0000","last_change":"2024-09-17T05:34:43.492218+0000","last_active":"2024-09-17T05:34:45.507899+0000","last_peered":"2024-09-17T05:34:45.507899+0000","last_clean":"2024-09-17T05:34:45.507899+0000","last_became_active":"2024-09-17T05:34:43.491883+0000","last_became_peered":"2024-09-17T05:34:43.491883+0000","last_unstale":"2024-09-17T05:34:45.507899+0000","last_undegraded":"2024-09-17T05:34:45.507899+0000","last_fullsized":"2024-09-17T05:34:45.507899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:45.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167150+0000","last_change":"2024-09-17T05:34:44.510566+0000","last_active":"2024-09-17T05:34:46.167150+0000","last_peered":"2024-09-17T05:34:46.167150+0000","last_clean":"2024-09-17T05:34:46.167150+0000","last_became_active":"2024-09-17T05:34:44.510105+0000","last_became_peered":"2024-09-17T05:34:44.510105+0000","last_unstale":"2024-09-17T05:34:46.167150+0000","last_undegraded":"2024-09-17T05:34:46.167150+0000","last_fullsized":"2024-09-17T05:34:46.167150+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.973701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480522+0000","last_change":"2024-09-17T05:34:43.486372+0000","last_active":"2024-09-17T05:34:45.480522+0000","last_peered":"2024-09-17T05:34:45.480522+0000","last_clean":"2024-09-17T05:34:45.480522+0000","last_became_active":"2024-09-17T05:34:43.485972+0000","last_became_peered":"2024-09-17T05:34:43.485972+0000","last_unstale":"2024-09-17T05:34:45.480522+0000","last_undegraded":"2024-09-17T05:34:45.480522+0000","last_fullsized":"2024-09-17T05:34:45.480522+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:45.895706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662839+0000","last_change":"2024-09-17T05:34:44.495845+0000","last_active":"2024-09-17T05:34:45.662839+0000","last_peered":"2024-09-17T05:34:45.662839+0000","last_clean":"2024-09-17T05:34:45.662839+0000","last_became_active":"2024-09-17T05:34:44.495506+0000","last_became_peered":"2024-09-17T05:34:44.495506+0000","last_unstale":"2024-09-17T05:34:45.662839+0000","last_undegraded":"2024-09-17T05:34:45.662839+0000","last_fullsized":"2024-09-17T05:34:45.662839+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:38.279303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167064+0000","last_change":"2024-09-17T05:34:43.488432+0000","last_active":"2024-09-17T05:34:46.167064+0000","last_peered":"2024-09-17T05:34:46.167064+0000","last_clean":"2024-09-17T05:34:46.167064+0000","last_became_active":"2024-09-17T05:34:43.487723+0000","last_became_peered":"2024-09-17T05:34:43.487723+0000","last_unstale":"2024-09-17T05:34:46.167064+0000","last_undegraded":"2024-09-17T05:34:46.167064+0000","last_fullsized":"2024-09-17T05:34:46.167064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:03.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163800+0000","last_change":"2024-09-17T05:34:44.494489+0000","last_active":"2024-09-17T05:34:46.163800+0000","last_peered":"2024-09-17T05:34:46.163800+0000","last_clean":"2024-09-17T05:34:46.163800+0000","last_became_active":"2024-09-17T05:34:44.493891+0000","last_became_peered":"2024-09-17T05:34:44.493891+0000","last_unstale":"2024-09-17T05:34:46.163800+0000","last_undegraded":"2024-09-17T05:34:46.163800+0000","last_fullsized":"2024-09-17T05:34:46.163800+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:15.830029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"138'3","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164215+0000","last_change":"2024-09-17T05:34:43.482270+0000","last_active":"2024-09-17T05:34:46.164215+0000","last_peered":"2024-09-17T05:34:46.164215+0000","last_clean":"2024-09-17T05:34:46.164215+0000","last_became_active":"2024-09-17T05:34:43.481870+0000","last_became_peered":"2024-09-17T05:34:43.481870+0000","last_unstale":"2024-09-17T05:34:46.164215+0000","last_undegraded":"2024-09-17T05:34:46.164215+0000","last_fullsized":"2024-09-17T05:34:46.164215+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:37.726005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662550+0000","last_change":"2024-09-17T05:34:44.509391+0000","last_active":"2024-09-17T05:34:45.662550+0000","last_peered":"2024-09-17T05:34:45.662550+0000","last_clean":"2024-09-17T05:34:45.662550+0000","last_became_active":"2024-09-17T05:34:44.508982+0000","last_became_peered":"2024-09-17T05:34:44.508982+0000","last_unstale":"2024-09-17T05:34:45.662550+0000","last_undegraded":"2024-09-17T05:34:45.662550+0000","last_fullsized":"2024-09-17T05:34:45.662550+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:07.144511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"138'6","reported_seq":21,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477361+0000","last_change":"2024-09-17T05:34:43.483794+0000","last_active":"2024-09-17T05:34:45.477361+0000","last_peered":"2024-09-17T05:34:45.477361+0000","last_clean":"2024-09-17T05:34:45.477361+0000","last_became_active":"2024-09-17T05:34:43.483453+0000","last_became_peered":"2024-09-17T05:34:43.483453+0000","last_unstale":"2024-09-17T05:34:45.477361+0000","last_undegraded":"2024-09-17T05:34:45.477361+0000","last_fullsized":"2024-09-17T05:34:45.477361+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:09.451434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166308+0000","last_change":"2024-09-17T05:34:44.497676+0000","last_active":"2024-09-17T05:34:46.166308+0000","last_peered":"2024-09-17T05:34:46.166308+0000","last_clean":"2024-09-17T05:34:46.166308+0000","last_became_active":"2024-09-17T05:34:44.496021+0000","last_became_peered":"2024-09-17T05:34:44.496021+0000","last_unstale":"2024-09-17T05:34:46.166308+0000","last_undegraded":"2024-09-17T05:34:46.166308+0000","last_fullsized":"2024-09-17T05:34:46.166308+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.816080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508002+0000","last_change":"2024-09-17T05:34:43.486891+0000","last_active":"2024-09-17T05:34:45.508002+0000","last_peered":"2024-09-17T05:34:45.508002+0000","last_clean":"2024-09-17T05:34:45.508002+0000","last_became_active":"2024-09-17T05:34:43.485568+0000","last_became_peered":"2024-09-17T05:34:43.485568+0000","last_unstale":"2024-09-17T05:34:45.508002+0000","last_undegraded":"2024-09-17T05:34:45.508002+0000","last_fullsized":"2024-09-17T05:34:45.508002+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:23.523550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479963+0000","last_change":"2024-09-17T05:34:44.508551+0000","last_active":"2024-09-17T05:34:45.479963+0000","last_peered":"2024-09-17T05:34:45.479963+0000","last_clean":"2024-09-17T05:34:45.479963+0000","last_became_active":"2024-09-17T05:34:44.507267+0000","last_became_peered":"2024-09-17T05:34:44.507267+0000","last_unstale":"2024-09-17T05:34:45.479963+0000","last_undegraded":"2024-09-17T05:34:45.479963+0000","last_fullsized":"2024-09-17T05:34:45.479963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:37.197155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166350+0000","last_change":"2024-09-17T05:34:43.490087+0000","last_active":"2024-09-17T05:34:46.166350+0000","last_peered":"2024-09-17T05:34:46.166350+0000","last_clean":"2024-09-17T05:34:46.166350+0000","last_became_active":"2024-09-17T05:34:43.488206+0000","last_became_peered":"2024-09-17T05:34:43.488206+0000","last_unstale":"2024-09-17T05:34:46.166350+0000","last_undegraded":"2024-09-17T05:34:46.166350+0000","last_fullsized":"2024-09-17T05:34:46.166350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:09.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163851+0000","last_change":"2024-09-17T05:34:44.499193+0000","last_active":"2024-09-17T05:34:46.163851+0000","last_peered":"2024-09-17T05:34:46.163851+0000","last_clean":"2024-09-17T05:34:46.163851+0000","last_became_active":"2024-09-17T05:34:44.496499+0000","last_became_peered":"2024-09-17T05:34:44.496499+0000","last_unstale":"2024-09-17T05:34:46.163851+0000","last_undegraded":"2024-09-17T05:34:46.163851+0000","last_fullsized":"2024-09-17T05:34:46.163851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:43.110453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476812+0000","last_change":"2024-09-17T05:34:43.480480+0000","last_active":"2024-09-17T05:34:45.476812+0000","last_peered":"2024-09-17T05:34:45.476812+0000","last_clean":"2024-09-17T05:34:45.476812+0000","last_became_active":"2024-09-17T05:34:43.480158+0000","last_became_peered":"2024-09-17T05:34:43.480158+0000","last_unstale":"2024-09-17T05:34:45.476812+0000","last_undegraded":"2024-09-17T05:34:45.476812+0000","last_fullsized":"2024-09-17T05:34:45.476812+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:03.650153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480104+0000","last_change":"2024-09-17T05:34:44.500353+0000","last_active":"2024-09-17T05:34:45.480104+0000","last_peered":"2024-09-17T05:34:45.480104+0000","last_clean":"2024-09-17T05:34:45.480104+0000","last_became_active":"2024-09-17T05:34:44.499995+0000","last_became_peered":"2024-09-17T05:34:44.499995+0000","last_unstale":"2024-09-17T05:34:45.480104+0000","last_undegraded":"2024-09-17T05:34:45.480104+0000","last_fullsized":"2024-09-17T05:34:45.480104+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:28.682718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"138'8","reported_seq":23,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166302+0000","last_change":"2024-09-17T05:34:43.480821+0000","last_active":"2024-09-17T05:34:46.166302+0000","last_peered":"2024-09-17T05:34:46.166302+0000","last_clean":"2024-09-17T05:34:46.166302+0000","last_became_active":"2024-09-17T05:34:43.480465+0000","last_became_peered":"2024-09-17T05:34:43.480465+0000","last_unstale":"2024-09-17T05:34:46.166302+0000","last_undegraded":"2024-09-17T05:34:46.166302+0000","last_fullsized":"2024-09-17T05:34:46.166302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:55.811669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662825+0000","last_change":"2024-09-17T05:34:44.507622+0000","last_active":"2024-09-17T05:34:45.662825+0000","last_peered":"2024-09-17T05:34:45.662825+0000","last_clean":"2024-09-17T05:34:45.662825+0000","last_became_active":"2024-09-17T05:34:44.507284+0000","last_became_peered":"2024-09-17T05:34:44.507284+0000","last_unstale":"2024-09-17T05:34:45.662825+0000","last_undegraded":"2024-09-17T05:34:45.662825+0000","last_fullsized":"2024-09-17T05:34:45.662825+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:39.302757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477412+0000","last_change":"2024-09-17T05:34:43.489754+0000","last_active":"2024-09-17T05:34:45.477412+0000","last_peered":"2024-09-17T05:34:45.477412+0000","last_clean":"2024-09-17T05:34:45.477412+0000","last_became_active":"2024-09-17T05:34:43.489338+0000","last_became_peered":"2024-09-17T05:34:43.489338+0000","last_unstale":"2024-09-17T05:34:45.477412+0000","last_undegraded":"2024-09-17T05:34:45.477412+0000","last_fullsized":"2024-09-17T05:34:45.477412+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:45.895706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662568+0000","last_change":"2024-09-17T05:34:44.512234+0000","last_active":"2024-09-17T05:34:45.662568+0000","last_peered":"2024-09-17T05:34:45.662568+0000","last_clean":"2024-09-17T05:34:45.662568+0000","last_became_active":"2024-09-17T05:34:44.510561+0000","last_became_peered":"2024-09-17T05:34:44.510561+0000","last_unstale":"2024-09-17T05:34:45.662568+0000","last_undegraded":"2024-09-17T05:34:45.662568+0000","last_fullsized":"2024-09-17T05:34:45.662568+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164273+0000","last_change":"2024-09-17T05:34:43.484097+0000","last_active":"2024-09-17T05:34:46.164273+0000","last_peered":"2024-09-17T05:34:46.164273+0000","last_clean":"2024-09-17T05:34:46.164273+0000","last_became_active":"2024-09-17T05:34:43.483720+0000","last_became_peered":"2024-09-17T05:34:43.483720+0000","last_unstale":"2024-09-17T05:34:46.164273+0000","last_undegraded":"2024-09-17T05:34:46.164273+0000","last_fullsized":"2024-09-17T05:34:46.164273+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:38.311122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477391+0000","last_change":"2024-09-17T05:34:44.501177+0000","last_active":"2024-09-17T05:34:45.477391+0000","last_peered":"2024-09-17T05:34:45.477391+0000","last_clean":"2024-09-17T05:34:45.477391+0000","last_became_active":"2024-09-17T05:34:44.500864+0000","last_became_peered":"2024-09-17T05:34:44.500864+0000","last_unstale":"2024-09-17T05:34:45.477391+0000","last_undegraded":"2024-09-17T05:34:45.477391+0000","last_fullsized":"2024-09-17T05:34:45.477391+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.289888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508067+0000","last_change":"2024-09-17T05:34:43.484124+0000","last_active":"2024-09-17T05:34:45.508067+0000","last_peered":"2024-09-17T05:34:45.508067+0000","last_clean":"2024-09-17T05:34:45.508067+0000","last_became_active":"2024-09-17T05:34:43.483814+0000","last_became_peered":"2024-09-17T05:34:43.483814+0000","last_unstale":"2024-09-17T05:34:45.508067+0000","last_undegraded":"2024-09-17T05:34:45.508067+0000","last_fullsized":"2024-09-17T05:34:45.508067+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:50.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508149+0000","last_change":"2024-09-17T05:34:44.505921+0000","last_active":"2024-09-17T05:34:45.508149+0000","last_peered":"2024-09-17T05:34:45.508149+0000","last_clean":"2024-09-17T05:34:45.508149+0000","last_became_active":"2024-09-17T05:34:44.504624+0000","last_became_peered":"2024-09-17T05:34:44.504624+0000","last_unstale":"2024-09-17T05:34:45.508149+0000","last_undegraded":"2024-09-17T05:34:45.508149+0000","last_fullsized":"2024-09-17T05:34:45.508149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:23.001564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166410+0000","last_change":"2024-09-17T05:34:43.488724+0000","last_active":"2024-09-17T05:34:46.166410+0000","last_peered":"2024-09-17T05:34:46.166410+0000","last_clean":"2024-09-17T05:34:46.166410+0000","last_became_active":"2024-09-17T05:34:43.488321+0000","last_became_peered":"2024-09-17T05:34:43.488321+0000","last_unstale":"2024-09-17T05:34:46.166410+0000","last_undegraded":"2024-09-17T05:34:46.166410+0000","last_fullsized":"2024-09-17T05:34:46.166410+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:31.408633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507912+0000","last_change":"2024-09-17T05:34:44.501243+0000","last_active":"2024-09-17T05:34:45.507912+0000","last_peered":"2024-09-17T05:34:45.507912+0000","last_clean":"2024-09-17T05:34:45.507912+0000","last_became_active":"2024-09-17T05:34:44.500905+0000","last_became_peered":"2024-09-17T05:34:44.500905+0000","last_unstale":"2024-09-17T05:34:45.507912+0000","last_undegraded":"2024-09-17T05:34:45.507912+0000","last_fullsized":"2024-09-17T05:34:45.507912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:16.467250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662240+0000","last_change":"2024-09-17T05:34:43.486138+0000","last_active":"2024-09-17T05:34:45.662240+0000","last_peered":"2024-09-17T05:34:45.662240+0000","last_clean":"2024-09-17T05:34:45.662240+0000","last_became_active":"2024-09-17T05:34:43.485752+0000","last_became_peered":"2024-09-17T05:34:43.485752+0000","last_unstale":"2024-09-17T05:34:45.662240+0000","last_undegraded":"2024-09-17T05:34:45.662240+0000","last_fullsized":"2024-09-17T05:34:45.662240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.178231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166207+0000","last_change":"2024-09-17T05:34:44.498312+0000","last_active":"2024-09-17T05:34:46.166207+0000","last_peered":"2024-09-17T05:34:46.166207+0000","last_clean":"2024-09-17T05:34:46.166207+0000","last_became_active":"2024-09-17T05:34:44.495324+0000","last_became_peered":"2024-09-17T05:34:44.495324+0000","last_unstale":"2024-09-17T05:34:46.166207+0000","last_undegraded":"2024-09-17T05:34:46.166207+0000","last_fullsized":"2024-09-17T05:34:46.166207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.787871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475529+0000","last_change":"2024-09-17T05:34:43.477604+0000","last_active":"2024-09-17T05:34:45.475529+0000","last_peered":"2024-09-17T05:34:45.475529+0000","last_clean":"2024-09-17T05:34:45.475529+0000","last_became_active":"2024-09-17T05:34:43.477206+0000","last_became_peered":"2024-09-17T05:34:43.477206+0000","last_unstale":"2024-09-17T05:34:45.475529+0000","last_undegraded":"2024-09-17T05:34:45.475529+0000","last_fullsized":"2024-09-17T05:34:45.475529+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:23.859235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662251+0000","last_change":"2024-09-17T05:34:44.499504+0000","last_active":"2024-09-17T05:34:45.662251+0000","last_peered":"2024-09-17T05:34:45.662251+0000","last_clean":"2024-09-17T05:34:45.662251+0000","last_became_active":"2024-09-17T05:34:44.499169+0000","last_became_peered":"2024-09-17T05:34:44.499169+0000","last_unstale":"2024-09-17T05:34:45.662251+0000","last_undegraded":"2024-09-17T05:34:45.662251+0000","last_fullsized":"2024-09-17T05:34:45.662251+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:09.411623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479442+0000","last_change":"2024-09-17T05:34:43.487120+0000","last_active":"2024-09-17T05:34:45.479442+0000","last_peered":"2024-09-17T05:34:45.479442+0000","last_clean":"2024-09-17T05:34:45.479442+0000","last_became_active":"2024-09-17T05:34:43.486767+0000","last_became_peered":"2024-09-17T05:34:43.486767+0000","last_unstale":"2024-09-17T05:34:45.479442+0000","last_undegraded":"2024-09-17T05:34:45.479442+0000","last_fullsized":"2024-09-17T05:34:45.479442+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:21.331700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508424+0000","last_change":"2024-09-17T05:34:44.511186+0000","last_active":"2024-09-17T05:34:45.508424+0000","last_peered":"2024-09-17T05:34:45.508424+0000","last_clean":"2024-09-17T05:34:45.508424+0000","last_became_active":"2024-09-17T05:34:44.510799+0000","last_became_peered":"2024-09-17T05:34:44.510799+0000","last_unstale":"2024-09-17T05:34:45.508424+0000","last_undegraded":"2024-09-17T05:34:45.508424+0000","last_fullsized":"2024-09-17T05:34:45.508424+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:46.750796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477470+0000","last_change":"2024-09-17T05:34:43.482392+0000","last_active":"2024-09-17T05:34:45.477470+0000","last_peered":"2024-09-17T05:34:45.477470+0000","last_clean":"2024-09-17T05:34:45.477470+0000","last_became_active":"2024-09-17T05:34:43.482075+0000","last_became_peered":"2024-09-17T05:34:43.482075+0000","last_unstale":"2024-09-17T05:34:45.477470+0000","last_undegraded":"2024-09-17T05:34:45.477470+0000","last_fullsized":"2024-09-17T05:34:45.477470+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:37.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508320+0000","last_change":"2024-09-17T05:34:44.513231+0000","last_active":"2024-09-17T05:34:45.508320+0000","last_peered":"2024-09-17T05:34:45.508320+0000","last_clean":"2024-09-17T05:34:45.508320+0000","last_became_active":"2024-09-17T05:34:44.508966+0000","last_became_peered":"2024-09-17T05:34:44.508966+0000","last_unstale":"2024-09-17T05:34:45.508320+0000","last_undegraded":"2024-09-17T05:34:45.508320+0000","last_fullsized":"2024-09-17T05:34:45.508320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:45.193953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"139'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508650+0000","last_change":"2024-09-17T05:34:43.493846+0000","last_active":"2024-09-17T05:34:45.508650+0000","last_peered":"2024-09-17T05:34:45.508650+0000","last_clean":"2024-09-17T05:34:45.508650+0000","last_became_active":"2024-09-17T05:34:43.493493+0000","last_became_peered":"2024-09-17T05:34:43.493493+0000","last_unstale":"2024-09-17T05:34:45.508650+0000","last_undegraded":"2024-09-17T05:34:45.508650+0000","last_fullsized":"2024-09-17T05:34:45.508650+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:38.144157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480097+0000","last_change":"2024-09-17T05:34:44.499337+0000","last_active":"2024-09-17T05:34:45.480097+0000","last_peered":"2024-09-17T05:34:45.480097+0000","last_clean":"2024-09-17T05:34:45.480097+0000","last_became_active":"2024-09-17T05:34:44.498950+0000","last_became_peered":"2024-09-17T05:34:44.498950+0000","last_unstale":"2024-09-17T05:34:45.480097+0000","last_undegraded":"2024-09-17T05:34:45.480097+0000","last_fullsized":"2024-09-17T05:34:45.480097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:56.926090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508277+0000","last_change":"2024-09-17T05:34:43.487845+0000","last_active":"2024-09-17T05:34:45.508277+0000","last_peered":"2024-09-17T05:34:45.508277+0000","last_clean":"2024-09-17T05:34:45.508277+0000","last_became_active":"2024-09-17T05:34:43.487537+0000","last_became_peered":"2024-09-17T05:34:43.487537+0000","last_unstale":"2024-09-17T05:34:45.508277+0000","last_undegraded":"2024-09-17T05:34:45.508277+0000","last_fullsized":"2024-09-17T05:34:45.508277+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:34.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164080+0000","last_change":"2024-09-17T05:34:44.484583+0000","last_active":"2024-09-17T05:34:46.164080+0000","last_peered":"2024-09-17T05:34:46.164080+0000","last_clean":"2024-09-17T05:34:46.164080+0000","last_became_active":"2024-09-17T05:34:44.483761+0000","last_became_peered":"2024-09-17T05:34:44.483761+0000","last_unstale":"2024-09-17T05:34:46.164080+0000","last_undegraded":"2024-09-17T05:34:46.164080+0000","last_fullsized":"2024-09-17T05:34:46.164080+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:22.822388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662588+0000","last_change":"2024-09-17T05:34:43.487549+0000","last_active":"2024-09-17T05:34:45.662588+0000","last_peered":"2024-09-17T05:34:45.662588+0000","last_clean":"2024-09-17T05:34:45.662588+0000","last_became_active":"2024-09-17T05:34:43.487089+0000","last_became_peered":"2024-09-17T05:34:43.487089+0000","last_unstale":"2024-09-17T05:34:45.662588+0000","last_undegraded":"2024-09-17T05:34:45.662588+0000","last_fullsized":"2024-09-17T05:34:45.662588+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:13.321577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507984+0000","last_change":"2024-09-17T05:34:44.509810+0000","last_active":"2024-09-17T05:34:45.507984+0000","last_peered":"2024-09-17T05:34:45.507984+0000","last_clean":"2024-09-17T05:34:45.507984+0000","last_became_active":"2024-09-17T05:34:44.509326+0000","last_became_peered":"2024-09-17T05:34:44.509326+0000","last_unstale":"2024-09-17T05:34:45.507984+0000","last_undegraded":"2024-09-17T05:34:45.507984+0000","last_fullsized":"2024-09-17T05:34:45.507984+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.495976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480335+0000","last_change":"2024-09-17T05:34:43.495216+0000","last_active":"2024-09-17T05:34:45.480335+0000","last_peered":"2024-09-17T05:34:45.480335+0000","last_clean":"2024-09-17T05:34:45.480335+0000","last_became_active":"2024-09-17T05:34:43.494800+0000","last_became_peered":"2024-09-17T05:34:43.494800+0000","last_unstale":"2024-09-17T05:34:45.480335+0000","last_undegraded":"2024-09-17T05:34:45.480335+0000","last_fullsized":"2024-09-17T05:34:45.480335+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.033404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163994+0000","last_change":"2024-09-17T05:34:44.488673+0000","last_active":"2024-09-17T05:34:46.163994+0000","last_peered":"2024-09-17T05:34:46.163994+0000","last_clean":"2024-09-17T05:34:46.163994+0000","last_became_active":"2024-09-17T05:34:44.488336+0000","last_became_peered":"2024-09-17T05:34:44.488336+0000","last_unstale":"2024-09-17T05:34:46.163994+0000","last_undegraded":"2024-09-17T05:34:46.163994+0000","last_fullsized":"2024-09-17T05:34:46.163994+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:43.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662314+0000","last_change":"2024-09-17T05:34:43.491113+0000","last_active":"2024-09-17T05:34:45.662314+0000","last_peered":"2024-09-17T05:34:45.662314+0000","last_clean":"2024-09-17T05:34:45.662314+0000","last_became_active":"2024-09-17T05:34:43.490776+0000","last_became_peered":"2024-09-17T05:34:43.490776+0000","last_unstale":"2024-09-17T05:34:45.662314+0000","last_undegraded":"2024-09-17T05:34:45.662314+0000","last_fullsized":"2024-09-17T05:34:45.662314+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:50.864469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508185+0000","last_change":"2024-09-17T05:34:44.510255+0000","last_active":"2024-09-17T05:34:45.508185+0000","last_peered":"2024-09-17T05:34:45.508185+0000","last_clean":"2024-09-17T05:34:45.508185+0000","last_became_active":"2024-09-17T05:34:44.509308+0000","last_became_peered":"2024-09-17T05:34:44.509308+0000","last_unstale":"2024-09-17T05:34:45.508185+0000","last_undegraded":"2024-09-17T05:34:45.508185+0000","last_fullsized":"2024-09-17T05:34:45.508185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:44.223673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664039+0000","last_change":"2024-09-17T05:34:43.490904+0000","last_active":"2024-09-17T05:34:45.664039+0000","last_peered":"2024-09-17T05:34:45.664039+0000","last_clean":"2024-09-17T05:34:45.664039+0000","last_became_active":"2024-09-17T05:34:43.490596+0000","last_became_peered":"2024-09-17T05:34:43.490596+0000","last_unstale":"2024-09-17T05:34:45.664039+0000","last_undegraded":"2024-09-17T05:34:45.664039+0000","last_fullsized":"2024-09-17T05:34:45.664039+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.673465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476626+0000","last_change":"2024-09-17T05:34:44.494342+0000","last_active":"2024-09-17T05:34:45.476626+0000","last_peered":"2024-09-17T05:34:45.476626+0000","last_clean":"2024-09-17T05:34:45.476626+0000","last_became_active":"2024-09-17T05:34:44.494018+0000","last_became_peered":"2024-09-17T05:34:44.494018+0000","last_unstale":"2024-09-17T05:34:45.476626+0000","last_undegraded":"2024-09-17T05:34:45.476626+0000","last_fullsized":"2024-09-17T05:34:45.476626+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:54.902278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479602+0000","last_change":"2024-09-17T05:34:43.485943+0000","last_active":"2024-09-17T05:34:45.479602+0000","last_peered":"2024-09-17T05:34:45.479602+0000","last_clean":"2024-09-17T05:34:45.479602+0000","last_became_active":"2024-09-17T05:34:43.485578+0000","last_became_peered":"2024-09-17T05:34:43.485578+0000","last_unstale":"2024-09-17T05:34:45.479602+0000","last_undegraded":"2024-09-17T05:34:45.479602+0000","last_fullsized":"2024-09-17T05:34:45.479602+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:37.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480507+0000","last_change":"2024-09-17T05:34:44.496944+0000","last_active":"2024-09-17T05:34:45.480507+0000","last_peered":"2024-09-17T05:34:45.480507+0000","last_clean":"2024-09-17T05:34:45.480507+0000","last_became_active":"2024-09-17T05:34:44.496229+0000","last_became_peered":"2024-09-17T05:34:44.496229+0000","last_unstale":"2024-09-17T05:34:45.480507+0000","last_undegraded":"2024-09-17T05:34:45.480507+0000","last_fullsized":"2024-09-17T05:34:45.480507+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:40.652251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477365+0000","last_change":"2024-09-17T05:34:43.483784+0000","last_active":"2024-09-17T05:34:45.477365+0000","last_peered":"2024-09-17T05:34:45.477365+0000","last_clean":"2024-09-17T05:34:45.477365+0000","last_became_active":"2024-09-17T05:34:43.483417+0000","last_became_peered":"2024-09-17T05:34:43.483417+0000","last_unstale":"2024-09-17T05:34:45.477365+0000","last_undegraded":"2024-09-17T05:34:45.477365+0000","last_fullsized":"2024-09-17T05:34:45.477365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:21.331700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":341,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476599+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:34:45.476599+0000","last_peered":"2024-09-17T05:34:45.476599+0000","last_clean":"2024-09-17T05:34:45.476599+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:34:45.476599+0000","last_undegraded":"2024-09-17T05:34:45.476599+0000","last_fullsized":"2024-09-17T05:34:45.476599+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":848199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2379776,"data_stored":1439653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":848199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28768,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343072,"statfs":{"total":96636764160,"available":96607305728,"internally_reserved":0,"allocated":1777664,"data_stored":847811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":847687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2379776,"data_stored":1439709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41232,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330608,"statfs":{"total":96636764160,"available":96594542592,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:52.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:52.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:53.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:53.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.809+0000 7fb877000640 1 -- 172.21.15.43:0/1809364008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8700b3af0 msgr2=0x7fb87015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.809+0000 7fb877000640 1 --2- 172.21.15.43:0/1809364008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8700b3af0 0x7fb87015c7e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fb8700d0c40 tx=0x7fb85801c800 comp rx=0 tx=0).stop 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 -- 172.21.15.43:0/1809364008 shutdown_connections 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 --2- 172.21.15.43:0/1809364008 >> v2:172.21.15.62:3300/0 conn(0x7fb87006f110 0x7fb8700ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 --2- 172.21.15.43:0/1809364008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8700b3af0 0x7fb87015c7e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 -- 172.21.15.43:0/1809364008 >> 172.21.15.43:0/1809364008 conn(0x7fb8700b23d0 msgr2=0x7fb8700d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 -- 172.21.15.43:0/1809364008 shutdown_connections 2024-09-17T05:34:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 -- 172.21.15.43:0/1809364008 wait complete. 2024-09-17T05:34:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 Processor -- start 2024-09-17T05:34:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 -- start start 2024-09-17T05:34:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:53.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.813+0000 7fb877000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb870155d70 0x7fb87009d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb87009d990 0x7fb870164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb870168c30 con 0x7fb87006f110 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb870168ab0 con 0x7fb870155d70 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb870168db0 con 0x7fb87009d990 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41644/0 (socket says 172.21.15.43:41644) 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 -- 172.21.15.43:0/2039326881 learned_addr learned my addr 172.21.15.43:0/2039326881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb867e00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb870155d70 0x7fb87009d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb875000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb87009d990 0x7fb870164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 -- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb87009d990 msgr2=0x7fb870164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb87009d990 0x7fb870164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 -- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb870155d70 msgr2=0x7fb87009d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb870155d70 0x7fb87009d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 -- 172.21.15.43:0/2039326881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8701f87b0 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86fe00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b410 tx=0x7fb86000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860015030 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb860004020 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb870156b50 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860004470 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701f8d50 con 0x7fb87006f110 2024-09-17T05:34:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8700ad0b0 con 0x7fb87006f110 2024-09-17T05:34:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.817+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb860018070 con 0x7fb87006f110 2024-09-17T05:34:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.821+0000 7fb86cc00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 0x7fb84c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.821+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fb860093c60 con 0x7fb87006f110 2024-09-17T05:34:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.821+0000 7fb867e00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 0x7fb84c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.821+0000 7fb867e00640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 0x7fb84c075710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb8580af000 tx=0x7fb8580a7040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.825+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb86005e620 con 0x7fb87006f110 2024-09-17T05:34:54.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:53.997+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8700af030 con 0x7fb84c073200 2024-09-17T05:34:54.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.001+0000 7fb86cc00640 1 -- 172.21.15.43:0/2039326881 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7fb8700af030 con 0x7fb84c073200 2024-09-17T05:34:54.008 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:34:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 msgr2=0x7fb84c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 0x7fb84c075710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb8580af000 tx=0x7fb8580a7040 comp rx=0 tx=0).stop 2024-09-17T05:34:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 msgr2=0x7fb870156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b410 tx=0x7fb86000b8d0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 shutdown_connections 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb84c073200 0x7fb84c075710 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb870155d70 0x7fb87009d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb87009d990 0x7fb870164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 --2- 172.21.15.43:0/2039326881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb87006f110 0x7fb870156300 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 >> 172.21.15.43:0/2039326881 conn(0x7fb8700b23d0 msgr2=0x7fb870072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 shutdown_connections 2024-09-17T05:34:54.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.005+0000 7fb877000640 1 -- 172.21.15.43:0/2039326881 wait complete. 2024-09-17T05:34:54.029 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-17T05:34:52.031851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285356,"kb_used_data":15016,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754689364,"statfs":{"total":773094113280,"available":772801908736,"internally_reserved":0,"allocated":15376384,"data_stored":7966492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":276614706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002511"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167073+0000","last_change":"2024-09-17T05:34:44.497120+0000","last_active":"2024-09-17T05:34:46.167073+0000","last_peered":"2024-09-17T05:34:46.167073+0000","last_clean":"2024-09-17T05:34:46.167073+0000","last_became_active":"2024-09-17T05:34:44.496734+0000","last_became_peered":"2024-09-17T05:34:44.496734+0000","last_unstale":"2024-09-17T05:34:46.167073+0000","last_undegraded":"2024-09-17T05:34:46.167073+0000","last_fullsized":"2024-09-17T05:34:46.167073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:55.766746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166945+0000","last_change":"2024-09-17T05:34:44.504496+0000","last_active":"2024-09-17T05:34:46.166945+0000","last_peered":"2024-09-17T05:34:46.166945+0000","last_clean":"2024-09-17T05:34:46.166945+0000","last_became_active":"2024-09-17T05:34:44.504097+0000","last_became_peered":"2024-09-17T05:34:44.504097+0000","last_unstale":"2024-09-17T05:34:46.166945+0000","last_undegraded":"2024-09-17T05:34:46.166945+0000","last_fullsized":"2024-09-17T05:34:46.166945+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:31.928876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480275+0000","last_change":"2024-09-17T05:34:44.512793+0000","last_active":"2024-09-17T05:34:45.480275+0000","last_peered":"2024-09-17T05:34:45.480275+0000","last_clean":"2024-09-17T05:34:45.480275+0000","last_became_active":"2024-09-17T05:34:44.512396+0000","last_became_peered":"2024-09-17T05:34:44.512396+0000","last_unstale":"2024-09-17T05:34:45.480275+0000","last_undegraded":"2024-09-17T05:34:45.480275+0000","last_fullsized":"2024-09-17T05:34:45.480275+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:47.200644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166881+0000","last_change":"2024-09-17T05:34:44.511242+0000","last_active":"2024-09-17T05:34:46.166881+0000","last_peered":"2024-09-17T05:34:46.166881+0000","last_clean":"2024-09-17T05:34:46.166881+0000","last_became_active":"2024-09-17T05:34:44.510544+0000","last_became_peered":"2024-09-17T05:34:44.510544+0000","last_unstale":"2024-09-17T05:34:46.166881+0000","last_undegraded":"2024-09-17T05:34:46.166881+0000","last_fullsized":"2024-09-17T05:34:46.166881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:16.675435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163757+0000","last_change":"2024-09-17T05:34:44.487817+0000","last_active":"2024-09-17T05:34:46.163757+0000","last_peered":"2024-09-17T05:34:46.163757+0000","last_clean":"2024-09-17T05:34:46.163757+0000","last_became_active":"2024-09-17T05:34:44.487468+0000","last_became_peered":"2024-09-17T05:34:44.487468+0000","last_unstale":"2024-09-17T05:34:46.163757+0000","last_undegraded":"2024-09-17T05:34:46.163757+0000","last_fullsized":"2024-09-17T05:34:46.163757+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:16.910527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163739+0000","last_change":"2024-09-17T05:34:44.506024+0000","last_active":"2024-09-17T05:34:46.163739+0000","last_peered":"2024-09-17T05:34:46.163739+0000","last_clean":"2024-09-17T05:34:46.163739+0000","last_became_active":"2024-09-17T05:34:44.505665+0000","last_became_peered":"2024-09-17T05:34:44.505665+0000","last_unstale":"2024-09-17T05:34:46.163739+0000","last_undegraded":"2024-09-17T05:34:46.163739+0000","last_fullsized":"2024-09-17T05:34:46.163739+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:05.236790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163546+0000","last_change":"2024-09-17T05:34:44.482676+0000","last_active":"2024-09-17T05:34:46.163546+0000","last_peered":"2024-09-17T05:34:46.163546+0000","last_clean":"2024-09-17T05:34:46.163546+0000","last_became_active":"2024-09-17T05:34:44.482313+0000","last_became_peered":"2024-09-17T05:34:44.482313+0000","last_unstale":"2024-09-17T05:34:46.163546+0000","last_undegraded":"2024-09-17T05:34:46.163546+0000","last_fullsized":"2024-09-17T05:34:46.163546+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.045565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479447+0000","last_change":"2024-09-17T05:34:44.496923+0000","last_active":"2024-09-17T05:34:45.479447+0000","last_peered":"2024-09-17T05:34:45.479447+0000","last_clean":"2024-09-17T05:34:45.479447+0000","last_became_active":"2024-09-17T05:34:44.496522+0000","last_became_peered":"2024-09-17T05:34:44.496522+0000","last_unstale":"2024-09-17T05:34:45.479447+0000","last_undegraded":"2024-09-17T05:34:45.479447+0000","last_fullsized":"2024-09-17T05:34:45.479447+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:51.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164361+0000","last_change":"2024-09-17T05:34:44.493741+0000","last_active":"2024-09-17T05:34:46.164361+0000","last_peered":"2024-09-17T05:34:46.164361+0000","last_clean":"2024-09-17T05:34:46.164361+0000","last_became_active":"2024-09-17T05:34:44.493381+0000","last_became_peered":"2024-09-17T05:34:44.493381+0000","last_unstale":"2024-09-17T05:34:46.164361+0000","last_undegraded":"2024-09-17T05:34:46.164361+0000","last_fullsized":"2024-09-17T05:34:46.164361+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:35.467399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475879+0000","last_change":"2024-09-17T05:34:44.494112+0000","last_active":"2024-09-17T05:34:45.475879+0000","last_peered":"2024-09-17T05:34:45.475879+0000","last_clean":"2024-09-17T05:34:45.475879+0000","last_became_active":"2024-09-17T05:34:44.493782+0000","last_became_peered":"2024-09-17T05:34:44.493782+0000","last_unstale":"2024-09-17T05:34:45.475879+0000","last_undegraded":"2024-09-17T05:34:45.475879+0000","last_fullsized":"2024-09-17T05:34:45.475879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:41.344789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163847+0000","last_change":"2024-09-17T05:34:44.506473+0000","last_active":"2024-09-17T05:34:46.163847+0000","last_peered":"2024-09-17T05:34:46.163847+0000","last_clean":"2024-09-17T05:34:46.163847+0000","last_became_active":"2024-09-17T05:34:44.505979+0000","last_became_peered":"2024-09-17T05:34:44.505979+0000","last_unstale":"2024-09-17T05:34:46.163847+0000","last_undegraded":"2024-09-17T05:34:46.163847+0000","last_fullsized":"2024-09-17T05:34:46.163847+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:41.868093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477173+0000","last_change":"2024-09-17T05:34:44.499822+0000","last_active":"2024-09-17T05:34:45.477173+0000","last_peered":"2024-09-17T05:34:45.477173+0000","last_clean":"2024-09-17T05:34:45.477173+0000","last_became_active":"2024-09-17T05:34:44.499335+0000","last_became_peered":"2024-09-17T05:34:44.499335+0000","last_unstale":"2024-09-17T05:34:45.477173+0000","last_undegraded":"2024-09-17T05:34:45.477173+0000","last_fullsized":"2024-09-17T05:34:45.477173+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:26.838840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664124+0000","last_change":"2024-09-17T05:34:44.501617+0000","last_active":"2024-09-17T05:34:45.664124+0000","last_peered":"2024-09-17T05:34:45.664124+0000","last_clean":"2024-09-17T05:34:45.664124+0000","last_became_active":"2024-09-17T05:34:44.501185+0000","last_became_peered":"2024-09-17T05:34:44.501185+0000","last_unstale":"2024-09-17T05:34:45.664124+0000","last_undegraded":"2024-09-17T05:34:45.664124+0000","last_fullsized":"2024-09-17T05:34:45.664124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:10.459297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166241+0000","last_change":"2024-09-17T05:34:44.498246+0000","last_active":"2024-09-17T05:34:46.166241+0000","last_peered":"2024-09-17T05:34:46.166241+0000","last_clean":"2024-09-17T05:34:46.166241+0000","last_became_active":"2024-09-17T05:34:44.495932+0000","last_became_peered":"2024-09-17T05:34:44.495932+0000","last_unstale":"2024-09-17T05:34:46.166241+0000","last_undegraded":"2024-09-17T05:34:46.166241+0000","last_fullsized":"2024-09-17T05:34:46.166241+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:11.396023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166253+0000","last_change":"2024-09-17T05:34:44.503954+0000","last_active":"2024-09-17T05:34:46.166253+0000","last_peered":"2024-09-17T05:34:46.166253+0000","last_clean":"2024-09-17T05:34:46.166253+0000","last_became_active":"2024-09-17T05:34:44.502592+0000","last_became_peered":"2024-09-17T05:34:44.502592+0000","last_unstale":"2024-09-17T05:34:46.166253+0000","last_undegraded":"2024-09-17T05:34:46.166253+0000","last_fullsized":"2024-09-17T05:34:46.166253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:48.068029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163497+0000","last_change":"2024-09-17T05:34:44.512531+0000","last_active":"2024-09-17T05:34:46.163497+0000","last_peered":"2024-09-17T05:34:46.163497+0000","last_clean":"2024-09-17T05:34:46.163497+0000","last_became_active":"2024-09-17T05:34:44.511975+0000","last_became_peered":"2024-09-17T05:34:44.511975+0000","last_unstale":"2024-09-17T05:34:46.163497+0000","last_undegraded":"2024-09-17T05:34:46.163497+0000","last_fullsized":"2024-09-17T05:34:46.163497+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:42.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508302+0000","last_change":"2024-09-17T05:34:44.504891+0000","last_active":"2024-09-17T05:34:45.508302+0000","last_peered":"2024-09-17T05:34:45.508302+0000","last_clean":"2024-09-17T05:34:45.508302+0000","last_became_active":"2024-09-17T05:34:44.504514+0000","last_became_peered":"2024-09-17T05:34:44.504514+0000","last_unstale":"2024-09-17T05:34:45.508302+0000","last_undegraded":"2024-09-17T05:34:45.508302+0000","last_fullsized":"2024-09-17T05:34:45.508302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:39.318985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167005+0000","last_change":"2024-09-17T05:34:44.501872+0000","last_active":"2024-09-17T05:34:46.167005+0000","last_peered":"2024-09-17T05:34:46.167005+0000","last_clean":"2024-09-17T05:34:46.167005+0000","last_became_active":"2024-09-17T05:34:44.498268+0000","last_became_peered":"2024-09-17T05:34:44.498268+0000","last_unstale":"2024-09-17T05:34:46.167005+0000","last_undegraded":"2024-09-17T05:34:46.167005+0000","last_fullsized":"2024-09-17T05:34:46.167005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:27.861661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508072+0000","last_change":"2024-09-17T05:34:44.503701+0000","last_active":"2024-09-17T05:34:45.508072+0000","last_peered":"2024-09-17T05:34:45.508072+0000","last_clean":"2024-09-17T05:34:45.508072+0000","last_became_active":"2024-09-17T05:34:44.503358+0000","last_became_peered":"2024-09-17T05:34:44.503358+0000","last_unstale":"2024-09-17T05:34:45.508072+0000","last_undegraded":"2024-09-17T05:34:45.508072+0000","last_fullsized":"2024-09-17T05:34:45.508072+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:58.779306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477267+0000","last_change":"2024-09-17T05:34:44.502211+0000","last_active":"2024-09-17T05:34:45.477267+0000","last_peered":"2024-09-17T05:34:45.477267+0000","last_clean":"2024-09-17T05:34:45.477267+0000","last_became_active":"2024-09-17T05:34:44.501903+0000","last_became_peered":"2024-09-17T05:34:44.501903+0000","last_unstale":"2024-09-17T05:34:45.477267+0000","last_undegraded":"2024-09-17T05:34:45.477267+0000","last_fullsized":"2024-09-17T05:34:45.477267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:34.368235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479917+0000","last_change":"2024-09-17T05:34:44.484938+0000","last_active":"2024-09-17T05:34:45.479917+0000","last_peered":"2024-09-17T05:34:45.479917+0000","last_clean":"2024-09-17T05:34:45.479917+0000","last_became_active":"2024-09-17T05:34:44.484582+0000","last_became_peered":"2024-09-17T05:34:44.484582+0000","last_unstale":"2024-09-17T05:34:45.479917+0000","last_undegraded":"2024-09-17T05:34:45.479917+0000","last_fullsized":"2024-09-17T05:34:45.479917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:14.329439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479631+0000","last_change":"2024-09-17T05:34:44.486932+0000","last_active":"2024-09-17T05:34:45.479631+0000","last_peered":"2024-09-17T05:34:45.479631+0000","last_clean":"2024-09-17T05:34:45.479631+0000","last_became_active":"2024-09-17T05:34:44.486573+0000","last_became_peered":"2024-09-17T05:34:44.486573+0000","last_unstale":"2024-09-17T05:34:45.479631+0000","last_undegraded":"2024-09-17T05:34:45.479631+0000","last_fullsized":"2024-09-17T05:34:45.479631+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:51.872331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166332+0000","last_change":"2024-09-17T05:34:44.495822+0000","last_active":"2024-09-17T05:34:46.166332+0000","last_peered":"2024-09-17T05:34:46.166332+0000","last_clean":"2024-09-17T05:34:46.166332+0000","last_became_active":"2024-09-17T05:34:44.495403+0000","last_became_peered":"2024-09-17T05:34:44.495403+0000","last_unstale":"2024-09-17T05:34:46.166332+0000","last_undegraded":"2024-09-17T05:34:46.166332+0000","last_fullsized":"2024-09-17T05:34:46.166332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:15.566322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662156+0000","last_change":"2024-09-17T05:34:44.493682+0000","last_active":"2024-09-17T05:34:45.662156+0000","last_peered":"2024-09-17T05:34:45.662156+0000","last_clean":"2024-09-17T05:34:45.662156+0000","last_became_active":"2024-09-17T05:34:44.493265+0000","last_became_peered":"2024-09-17T05:34:44.493265+0000","last_unstale":"2024-09-17T05:34:45.662156+0000","last_undegraded":"2024-09-17T05:34:45.662156+0000","last_fullsized":"2024-09-17T05:34:45.662156+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.817263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476199+0000","last_change":"2024-09-17T05:34:44.503721+0000","last_active":"2024-09-17T05:34:45.476199+0000","last_peered":"2024-09-17T05:34:45.476199+0000","last_clean":"2024-09-17T05:34:45.476199+0000","last_became_active":"2024-09-17T05:34:44.503327+0000","last_became_peered":"2024-09-17T05:34:44.503327+0000","last_unstale":"2024-09-17T05:34:45.476199+0000","last_undegraded":"2024-09-17T05:34:45.476199+0000","last_fullsized":"2024-09-17T05:34:45.476199+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:54.328028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664287+0000","last_change":"2024-09-17T05:34:44.501660+0000","last_active":"2024-09-17T05:34:45.664287+0000","last_peered":"2024-09-17T05:34:45.664287+0000","last_clean":"2024-09-17T05:34:45.664287+0000","last_became_active":"2024-09-17T05:34:44.500487+0000","last_became_peered":"2024-09-17T05:34:44.500487+0000","last_unstale":"2024-09-17T05:34:45.664287+0000","last_undegraded":"2024-09-17T05:34:45.664287+0000","last_fullsized":"2024-09-17T05:34:45.664287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:14.041266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662663+0000","last_change":"2024-09-17T05:34:44.498883+0000","last_active":"2024-09-17T05:34:45.662663+0000","last_peered":"2024-09-17T05:34:45.662663+0000","last_clean":"2024-09-17T05:34:45.662663+0000","last_became_active":"2024-09-17T05:34:44.498564+0000","last_became_peered":"2024-09-17T05:34:44.498564+0000","last_unstale":"2024-09-17T05:34:45.662663+0000","last_undegraded":"2024-09-17T05:34:45.662663+0000","last_fullsized":"2024-09-17T05:34:45.662663+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:34.487351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662439+0000","last_change":"2024-09-17T05:34:44.498716+0000","last_active":"2024-09-17T05:34:45.662439+0000","last_peered":"2024-09-17T05:34:45.662439+0000","last_clean":"2024-09-17T05:34:45.662439+0000","last_became_active":"2024-09-17T05:34:44.498423+0000","last_became_peered":"2024-09-17T05:34:44.498423+0000","last_unstale":"2024-09-17T05:34:45.662439+0000","last_undegraded":"2024-09-17T05:34:45.662439+0000","last_fullsized":"2024-09-17T05:34:45.662439+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:03.599412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664416+0000","last_change":"2024-09-17T05:34:44.496130+0000","last_active":"2024-09-17T05:34:45.664416+0000","last_peered":"2024-09-17T05:34:45.664416+0000","last_clean":"2024-09-17T05:34:45.664416+0000","last_became_active":"2024-09-17T05:34:44.495826+0000","last_became_peered":"2024-09-17T05:34:44.495826+0000","last_unstale":"2024-09-17T05:34:45.664416+0000","last_undegraded":"2024-09-17T05:34:45.664416+0000","last_fullsized":"2024-09-17T05:34:45.664416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:04.942761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664166+0000","last_change":"2024-09-17T05:34:44.505596+0000","last_active":"2024-09-17T05:34:45.664166+0000","last_peered":"2024-09-17T05:34:45.664166+0000","last_clean":"2024-09-17T05:34:45.664166+0000","last_became_active":"2024-09-17T05:34:44.505284+0000","last_became_peered":"2024-09-17T05:34:44.505284+0000","last_unstale":"2024-09-17T05:34:45.664166+0000","last_undegraded":"2024-09-17T05:34:45.664166+0000","last_fullsized":"2024-09-17T05:34:45.664166+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:46.903568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479588+0000","last_change":"2024-09-17T05:34:44.482924+0000","last_active":"2024-09-17T05:34:45.479588+0000","last_peered":"2024-09-17T05:34:45.479588+0000","last_clean":"2024-09-17T05:34:45.479588+0000","last_became_active":"2024-09-17T05:34:44.482374+0000","last_became_peered":"2024-09-17T05:34:44.482374+0000","last_unstale":"2024-09-17T05:34:45.479588+0000","last_undegraded":"2024-09-17T05:34:45.479588+0000","last_fullsized":"2024-09-17T05:34:45.479588+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.289888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166227+0000","last_change":"2024-09-17T05:34:44.515160+0000","last_active":"2024-09-17T05:34:46.166227+0000","last_peered":"2024-09-17T05:34:46.166227+0000","last_clean":"2024-09-17T05:34:46.166227+0000","last_became_active":"2024-09-17T05:34:44.514775+0000","last_became_peered":"2024-09-17T05:34:44.514775+0000","last_unstale":"2024-09-17T05:34:46.166227+0000","last_undegraded":"2024-09-17T05:34:46.166227+0000","last_fullsized":"2024-09-17T05:34:46.166227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:18.490230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508439+0000","last_change":"2024-09-17T05:34:44.515141+0000","last_active":"2024-09-17T05:34:45.508439+0000","last_peered":"2024-09-17T05:34:45.508439+0000","last_clean":"2024-09-17T05:34:45.508439+0000","last_became_active":"2024-09-17T05:34:44.514837+0000","last_became_peered":"2024-09-17T05:34:44.514837+0000","last_unstale":"2024-09-17T05:34:45.508439+0000","last_undegraded":"2024-09-17T05:34:45.508439+0000","last_fullsized":"2024-09-17T05:34:45.508439+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:32.530173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"138'4","reported_seq":19,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664457+0000","last_change":"2024-09-17T05:34:43.492447+0000","last_active":"2024-09-17T05:34:45.664457+0000","last_peered":"2024-09-17T05:34:45.664457+0000","last_clean":"2024-09-17T05:34:45.664457+0000","last_became_active":"2024-09-17T05:34:43.492128+0000","last_became_peered":"2024-09-17T05:34:43.492128+0000","last_unstale":"2024-09-17T05:34:45.664457+0000","last_undegraded":"2024-09-17T05:34:45.664457+0000","last_fullsized":"2024-09-17T05:34:45.664457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:38.840126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664393+0000","last_change":"2024-09-17T05:34:44.492812+0000","last_active":"2024-09-17T05:34:45.664393+0000","last_peered":"2024-09-17T05:34:45.664393+0000","last_clean":"2024-09-17T05:34:45.664393+0000","last_became_active":"2024-09-17T05:34:44.492300+0000","last_became_peered":"2024-09-17T05:34:44.492300+0000","last_unstale":"2024-09-17T05:34:45.664393+0000","last_undegraded":"2024-09-17T05:34:45.664393+0000","last_fullsized":"2024-09-17T05:34:45.664393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:06.195523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164199+0000","last_change":"2024-09-17T05:34:43.488048+0000","last_active":"2024-09-17T05:34:46.164199+0000","last_peered":"2024-09-17T05:34:46.164199+0000","last_clean":"2024-09-17T05:34:46.164199+0000","last_became_active":"2024-09-17T05:34:43.487637+0000","last_became_peered":"2024-09-17T05:34:43.487637+0000","last_unstale":"2024-09-17T05:34:46.164199+0000","last_undegraded":"2024-09-17T05:34:46.164199+0000","last_fullsized":"2024-09-17T05:34:46.164199+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:21.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166891+0000","last_change":"2024-09-17T05:34:44.497033+0000","last_active":"2024-09-17T05:34:46.166891+0000","last_peered":"2024-09-17T05:34:46.166891+0000","last_clean":"2024-09-17T05:34:46.166891+0000","last_became_active":"2024-09-17T05:34:44.496645+0000","last_became_peered":"2024-09-17T05:34:44.496645+0000","last_unstale":"2024-09-17T05:34:46.166891+0000","last_undegraded":"2024-09-17T05:34:46.166891+0000","last_fullsized":"2024-09-17T05:34:46.166891+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:37.177571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480349+0000","last_change":"2024-09-17T05:34:43.490759+0000","last_active":"2024-09-17T05:34:45.480349+0000","last_peered":"2024-09-17T05:34:45.480349+0000","last_clean":"2024-09-17T05:34:45.480349+0000","last_became_active":"2024-09-17T05:34:43.489512+0000","last_became_peered":"2024-09-17T05:34:43.489512+0000","last_unstale":"2024-09-17T05:34:45.480349+0000","last_undegraded":"2024-09-17T05:34:45.480349+0000","last_fullsized":"2024-09-17T05:34:45.480349+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:41.753643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475854+0000","last_change":"2024-09-17T05:34:44.481916+0000","last_active":"2024-09-17T05:34:45.475854+0000","last_peered":"2024-09-17T05:34:45.475854+0000","last_clean":"2024-09-17T05:34:45.475854+0000","last_became_active":"2024-09-17T05:34:44.481344+0000","last_became_peered":"2024-09-17T05:34:44.481344+0000","last_unstale":"2024-09-17T05:34:45.475854+0000","last_undegraded":"2024-09-17T05:34:45.475854+0000","last_fullsized":"2024-09-17T05:34:45.475854+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:34.184609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166985+0000","last_change":"2024-09-17T05:34:43.497073+0000","last_active":"2024-09-17T05:34:46.166985+0000","last_peered":"2024-09-17T05:34:46.166985+0000","last_clean":"2024-09-17T05:34:46.166985+0000","last_became_active":"2024-09-17T05:34:43.496723+0000","last_became_peered":"2024-09-17T05:34:43.496723+0000","last_unstale":"2024-09-17T05:34:46.166985+0000","last_undegraded":"2024-09-17T05:34:46.166985+0000","last_fullsized":"2024-09-17T05:34:46.166985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:36.943277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476796+0000","last_change":"2024-09-17T05:34:44.495188+0000","last_active":"2024-09-17T05:34:45.476796+0000","last_peered":"2024-09-17T05:34:45.476796+0000","last_clean":"2024-09-17T05:34:45.476796+0000","last_became_active":"2024-09-17T05:34:44.494832+0000","last_became_peered":"2024-09-17T05:34:44.494832+0000","last_unstale":"2024-09-17T05:34:45.476796+0000","last_undegraded":"2024-09-17T05:34:45.476796+0000","last_fullsized":"2024-09-17T05:34:45.476796+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:23.590308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166073+0000","last_change":"2024-09-17T05:34:43.484224+0000","last_active":"2024-09-17T05:34:46.166073+0000","last_peered":"2024-09-17T05:34:46.166073+0000","last_clean":"2024-09-17T05:34:46.166073+0000","last_became_active":"2024-09-17T05:34:43.483865+0000","last_became_peered":"2024-09-17T05:34:43.483865+0000","last_unstale":"2024-09-17T05:34:46.166073+0000","last_undegraded":"2024-09-17T05:34:46.166073+0000","last_fullsized":"2024-09-17T05:34:46.166073+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:10.065970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163982+0000","last_change":"2024-09-17T05:34:44.506162+0000","last_active":"2024-09-17T05:34:46.163982+0000","last_peered":"2024-09-17T05:34:46.163982+0000","last_clean":"2024-09-17T05:34:46.163982+0000","last_became_active":"2024-09-17T05:34:44.505833+0000","last_became_peered":"2024-09-17T05:34:44.505833+0000","last_unstale":"2024-09-17T05:34:46.163982+0000","last_undegraded":"2024-09-17T05:34:46.163982+0000","last_fullsized":"2024-09-17T05:34:46.163982+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:19.108188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479631+0000","last_change":"2024-09-17T05:34:43.479766+0000","last_active":"2024-09-17T05:34:45.479631+0000","last_peered":"2024-09-17T05:34:45.479631+0000","last_clean":"2024-09-17T05:34:45.479631+0000","last_became_active":"2024-09-17T05:34:43.479411+0000","last_became_peered":"2024-09-17T05:34:43.479411+0000","last_unstale":"2024-09-17T05:34:45.479631+0000","last_undegraded":"2024-09-17T05:34:45.479631+0000","last_fullsized":"2024-09-17T05:34:45.479631+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:09.451434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.663981+0000","last_change":"2024-09-17T05:34:44.495941+0000","last_active":"2024-09-17T05:34:45.663981+0000","last_peered":"2024-09-17T05:34:45.663981+0000","last_clean":"2024-09-17T05:34:45.663981+0000","last_became_active":"2024-09-17T05:34:44.495571+0000","last_became_peered":"2024-09-17T05:34:44.495571+0000","last_unstale":"2024-09-17T05:34:45.663981+0000","last_undegraded":"2024-09-17T05:34:45.663981+0000","last_fullsized":"2024-09-17T05:34:45.663981+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:38.331985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163733+0000","last_change":"2024-09-17T05:34:43.485518+0000","last_active":"2024-09-17T05:34:46.163733+0000","last_peered":"2024-09-17T05:34:46.163733+0000","last_clean":"2024-09-17T05:34:46.163733+0000","last_became_active":"2024-09-17T05:34:43.485155+0000","last_became_peered":"2024-09-17T05:34:43.485155+0000","last_unstale":"2024-09-17T05:34:46.163733+0000","last_undegraded":"2024-09-17T05:34:46.163733+0000","last_fullsized":"2024-09-17T05:34:46.163733+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:48.488113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476604+0000","last_change":"2024-09-17T05:34:44.496613+0000","last_active":"2024-09-17T05:34:45.476604+0000","last_peered":"2024-09-17T05:34:45.476604+0000","last_clean":"2024-09-17T05:34:45.476604+0000","last_became_active":"2024-09-17T05:34:44.496299+0000","last_became_peered":"2024-09-17T05:34:44.496299+0000","last_unstale":"2024-09-17T05:34:45.476604+0000","last_undegraded":"2024-09-17T05:34:45.476604+0000","last_fullsized":"2024-09-17T05:34:45.476604+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.437484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507899+0000","last_change":"2024-09-17T05:34:43.492218+0000","last_active":"2024-09-17T05:34:45.507899+0000","last_peered":"2024-09-17T05:34:45.507899+0000","last_clean":"2024-09-17T05:34:45.507899+0000","last_became_active":"2024-09-17T05:34:43.491883+0000","last_became_peered":"2024-09-17T05:34:43.491883+0000","last_unstale":"2024-09-17T05:34:45.507899+0000","last_undegraded":"2024-09-17T05:34:45.507899+0000","last_fullsized":"2024-09-17T05:34:45.507899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:45.500739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167150+0000","last_change":"2024-09-17T05:34:44.510566+0000","last_active":"2024-09-17T05:34:46.167150+0000","last_peered":"2024-09-17T05:34:46.167150+0000","last_clean":"2024-09-17T05:34:46.167150+0000","last_became_active":"2024-09-17T05:34:44.510105+0000","last_became_peered":"2024-09-17T05:34:44.510105+0000","last_unstale":"2024-09-17T05:34:46.167150+0000","last_undegraded":"2024-09-17T05:34:46.167150+0000","last_fullsized":"2024-09-17T05:34:46.167150+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:44.973701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480522+0000","last_change":"2024-09-17T05:34:43.486372+0000","last_active":"2024-09-17T05:34:45.480522+0000","last_peered":"2024-09-17T05:34:45.480522+0000","last_clean":"2024-09-17T05:34:45.480522+0000","last_became_active":"2024-09-17T05:34:43.485972+0000","last_became_peered":"2024-09-17T05:34:43.485972+0000","last_unstale":"2024-09-17T05:34:45.480522+0000","last_undegraded":"2024-09-17T05:34:45.480522+0000","last_fullsized":"2024-09-17T05:34:45.480522+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:45.895706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662839+0000","last_change":"2024-09-17T05:34:44.495845+0000","last_active":"2024-09-17T05:34:45.662839+0000","last_peered":"2024-09-17T05:34:45.662839+0000","last_clean":"2024-09-17T05:34:45.662839+0000","last_became_active":"2024-09-17T05:34:44.495506+0000","last_became_peered":"2024-09-17T05:34:44.495506+0000","last_unstale":"2024-09-17T05:34:45.662839+0000","last_undegraded":"2024-09-17T05:34:45.662839+0000","last_fullsized":"2024-09-17T05:34:45.662839+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:38.279303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.167064+0000","last_change":"2024-09-17T05:34:43.488432+0000","last_active":"2024-09-17T05:34:46.167064+0000","last_peered":"2024-09-17T05:34:46.167064+0000","last_clean":"2024-09-17T05:34:46.167064+0000","last_became_active":"2024-09-17T05:34:43.487723+0000","last_became_peered":"2024-09-17T05:34:43.487723+0000","last_unstale":"2024-09-17T05:34:46.167064+0000","last_undegraded":"2024-09-17T05:34:46.167064+0000","last_fullsized":"2024-09-17T05:34:46.167064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:03.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163800+0000","last_change":"2024-09-17T05:34:44.494489+0000","last_active":"2024-09-17T05:34:46.163800+0000","last_peered":"2024-09-17T05:34:46.163800+0000","last_clean":"2024-09-17T05:34:46.163800+0000","last_became_active":"2024-09-17T05:34:44.493891+0000","last_became_peered":"2024-09-17T05:34:44.493891+0000","last_unstale":"2024-09-17T05:34:46.163800+0000","last_undegraded":"2024-09-17T05:34:46.163800+0000","last_fullsized":"2024-09-17T05:34:46.163800+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:15.830029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"138'3","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164215+0000","last_change":"2024-09-17T05:34:43.482270+0000","last_active":"2024-09-17T05:34:46.164215+0000","last_peered":"2024-09-17T05:34:46.164215+0000","last_clean":"2024-09-17T05:34:46.164215+0000","last_became_active":"2024-09-17T05:34:43.481870+0000","last_became_peered":"2024-09-17T05:34:43.481870+0000","last_unstale":"2024-09-17T05:34:46.164215+0000","last_undegraded":"2024-09-17T05:34:46.164215+0000","last_fullsized":"2024-09-17T05:34:46.164215+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:37.726005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662550+0000","last_change":"2024-09-17T05:34:44.509391+0000","last_active":"2024-09-17T05:34:45.662550+0000","last_peered":"2024-09-17T05:34:45.662550+0000","last_clean":"2024-09-17T05:34:45.662550+0000","last_became_active":"2024-09-17T05:34:44.508982+0000","last_became_peered":"2024-09-17T05:34:44.508982+0000","last_unstale":"2024-09-17T05:34:45.662550+0000","last_undegraded":"2024-09-17T05:34:45.662550+0000","last_fullsized":"2024-09-17T05:34:45.662550+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:07.144511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"138'6","reported_seq":21,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477361+0000","last_change":"2024-09-17T05:34:43.483794+0000","last_active":"2024-09-17T05:34:45.477361+0000","last_peered":"2024-09-17T05:34:45.477361+0000","last_clean":"2024-09-17T05:34:45.477361+0000","last_became_active":"2024-09-17T05:34:43.483453+0000","last_became_peered":"2024-09-17T05:34:43.483453+0000","last_unstale":"2024-09-17T05:34:45.477361+0000","last_undegraded":"2024-09-17T05:34:45.477361+0000","last_fullsized":"2024-09-17T05:34:45.477361+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:13:09.451434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166308+0000","last_change":"2024-09-17T05:34:44.497676+0000","last_active":"2024-09-17T05:34:46.166308+0000","last_peered":"2024-09-17T05:34:46.166308+0000","last_clean":"2024-09-17T05:34:46.166308+0000","last_became_active":"2024-09-17T05:34:44.496021+0000","last_became_peered":"2024-09-17T05:34:44.496021+0000","last_unstale":"2024-09-17T05:34:46.166308+0000","last_undegraded":"2024-09-17T05:34:46.166308+0000","last_fullsized":"2024-09-17T05:34:46.166308+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.816080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508002+0000","last_change":"2024-09-17T05:34:43.486891+0000","last_active":"2024-09-17T05:34:45.508002+0000","last_peered":"2024-09-17T05:34:45.508002+0000","last_clean":"2024-09-17T05:34:45.508002+0000","last_became_active":"2024-09-17T05:34:43.485568+0000","last_became_peered":"2024-09-17T05:34:43.485568+0000","last_unstale":"2024-09-17T05:34:45.508002+0000","last_undegraded":"2024-09-17T05:34:45.508002+0000","last_fullsized":"2024-09-17T05:34:45.508002+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:23.523550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479963+0000","last_change":"2024-09-17T05:34:44.508551+0000","last_active":"2024-09-17T05:34:45.479963+0000","last_peered":"2024-09-17T05:34:45.479963+0000","last_clean":"2024-09-17T05:34:45.479963+0000","last_became_active":"2024-09-17T05:34:44.507267+0000","last_became_peered":"2024-09-17T05:34:44.507267+0000","last_unstale":"2024-09-17T05:34:45.479963+0000","last_undegraded":"2024-09-17T05:34:45.479963+0000","last_fullsized":"2024-09-17T05:34:45.479963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:37.197155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166350+0000","last_change":"2024-09-17T05:34:43.490087+0000","last_active":"2024-09-17T05:34:46.166350+0000","last_peered":"2024-09-17T05:34:46.166350+0000","last_clean":"2024-09-17T05:34:46.166350+0000","last_became_active":"2024-09-17T05:34:43.488206+0000","last_became_peered":"2024-09-17T05:34:43.488206+0000","last_unstale":"2024-09-17T05:34:46.166350+0000","last_undegraded":"2024-09-17T05:34:46.166350+0000","last_fullsized":"2024-09-17T05:34:46.166350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:09.538409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163851+0000","last_change":"2024-09-17T05:34:44.499193+0000","last_active":"2024-09-17T05:34:46.163851+0000","last_peered":"2024-09-17T05:34:46.163851+0000","last_clean":"2024-09-17T05:34:46.163851+0000","last_became_active":"2024-09-17T05:34:44.496499+0000","last_became_peered":"2024-09-17T05:34:44.496499+0000","last_unstale":"2024-09-17T05:34:46.163851+0000","last_undegraded":"2024-09-17T05:34:46.163851+0000","last_fullsized":"2024-09-17T05:34:46.163851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:43.110453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476812+0000","last_change":"2024-09-17T05:34:43.480480+0000","last_active":"2024-09-17T05:34:45.476812+0000","last_peered":"2024-09-17T05:34:45.476812+0000","last_clean":"2024-09-17T05:34:45.476812+0000","last_became_active":"2024-09-17T05:34:43.480158+0000","last_became_peered":"2024-09-17T05:34:43.480158+0000","last_unstale":"2024-09-17T05:34:45.476812+0000","last_undegraded":"2024-09-17T05:34:45.476812+0000","last_fullsized":"2024-09-17T05:34:45.476812+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:03.650153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480104+0000","last_change":"2024-09-17T05:34:44.500353+0000","last_active":"2024-09-17T05:34:45.480104+0000","last_peered":"2024-09-17T05:34:45.480104+0000","last_clean":"2024-09-17T05:34:45.480104+0000","last_became_active":"2024-09-17T05:34:44.499995+0000","last_became_peered":"2024-09-17T05:34:44.499995+0000","last_unstale":"2024-09-17T05:34:45.480104+0000","last_undegraded":"2024-09-17T05:34:45.480104+0000","last_fullsized":"2024-09-17T05:34:45.480104+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:28.682718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"138'8","reported_seq":23,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166302+0000","last_change":"2024-09-17T05:34:43.480821+0000","last_active":"2024-09-17T05:34:46.166302+0000","last_peered":"2024-09-17T05:34:46.166302+0000","last_clean":"2024-09-17T05:34:46.166302+0000","last_became_active":"2024-09-17T05:34:43.480465+0000","last_became_peered":"2024-09-17T05:34:43.480465+0000","last_unstale":"2024-09-17T05:34:46.166302+0000","last_undegraded":"2024-09-17T05:34:46.166302+0000","last_fullsized":"2024-09-17T05:34:46.166302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:55.811669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662825+0000","last_change":"2024-09-17T05:34:44.507622+0000","last_active":"2024-09-17T05:34:45.662825+0000","last_peered":"2024-09-17T05:34:45.662825+0000","last_clean":"2024-09-17T05:34:45.662825+0000","last_became_active":"2024-09-17T05:34:44.507284+0000","last_became_peered":"2024-09-17T05:34:44.507284+0000","last_unstale":"2024-09-17T05:34:45.662825+0000","last_undegraded":"2024-09-17T05:34:45.662825+0000","last_fullsized":"2024-09-17T05:34:45.662825+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:39.302757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477412+0000","last_change":"2024-09-17T05:34:43.489754+0000","last_active":"2024-09-17T05:34:45.477412+0000","last_peered":"2024-09-17T05:34:45.477412+0000","last_clean":"2024-09-17T05:34:45.477412+0000","last_became_active":"2024-09-17T05:34:43.489338+0000","last_became_peered":"2024-09-17T05:34:43.489338+0000","last_unstale":"2024-09-17T05:34:45.477412+0000","last_undegraded":"2024-09-17T05:34:45.477412+0000","last_fullsized":"2024-09-17T05:34:45.477412+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:45.895706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662568+0000","last_change":"2024-09-17T05:34:44.512234+0000","last_active":"2024-09-17T05:34:45.662568+0000","last_peered":"2024-09-17T05:34:45.662568+0000","last_clean":"2024-09-17T05:34:45.662568+0000","last_became_active":"2024-09-17T05:34:44.510561+0000","last_became_peered":"2024-09-17T05:34:44.510561+0000","last_unstale":"2024-09-17T05:34:45.662568+0000","last_undegraded":"2024-09-17T05:34:45.662568+0000","last_fullsized":"2024-09-17T05:34:45.662568+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:16.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164273+0000","last_change":"2024-09-17T05:34:43.484097+0000","last_active":"2024-09-17T05:34:46.164273+0000","last_peered":"2024-09-17T05:34:46.164273+0000","last_clean":"2024-09-17T05:34:46.164273+0000","last_became_active":"2024-09-17T05:34:43.483720+0000","last_became_peered":"2024-09-17T05:34:43.483720+0000","last_unstale":"2024-09-17T05:34:46.164273+0000","last_undegraded":"2024-09-17T05:34:46.164273+0000","last_fullsized":"2024-09-17T05:34:46.164273+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:38.311122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477391+0000","last_change":"2024-09-17T05:34:44.501177+0000","last_active":"2024-09-17T05:34:45.477391+0000","last_peered":"2024-09-17T05:34:45.477391+0000","last_clean":"2024-09-17T05:34:45.477391+0000","last_became_active":"2024-09-17T05:34:44.500864+0000","last_became_peered":"2024-09-17T05:34:44.500864+0000","last_unstale":"2024-09-17T05:34:45.477391+0000","last_undegraded":"2024-09-17T05:34:45.477391+0000","last_fullsized":"2024-09-17T05:34:45.477391+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:56:23.289888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508067+0000","last_change":"2024-09-17T05:34:43.484124+0000","last_active":"2024-09-17T05:34:45.508067+0000","last_peered":"2024-09-17T05:34:45.508067+0000","last_clean":"2024-09-17T05:34:45.508067+0000","last_became_active":"2024-09-17T05:34:43.483814+0000","last_became_peered":"2024-09-17T05:34:43.483814+0000","last_unstale":"2024-09-17T05:34:45.508067+0000","last_undegraded":"2024-09-17T05:34:45.508067+0000","last_fullsized":"2024-09-17T05:34:45.508067+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:50.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508149+0000","last_change":"2024-09-17T05:34:44.505921+0000","last_active":"2024-09-17T05:34:45.508149+0000","last_peered":"2024-09-17T05:34:45.508149+0000","last_clean":"2024-09-17T05:34:45.508149+0000","last_became_active":"2024-09-17T05:34:44.504624+0000","last_became_peered":"2024-09-17T05:34:44.504624+0000","last_unstale":"2024-09-17T05:34:45.508149+0000","last_undegraded":"2024-09-17T05:34:45.508149+0000","last_fullsized":"2024-09-17T05:34:45.508149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:23.001564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166410+0000","last_change":"2024-09-17T05:34:43.488724+0000","last_active":"2024-09-17T05:34:46.166410+0000","last_peered":"2024-09-17T05:34:46.166410+0000","last_clean":"2024-09-17T05:34:46.166410+0000","last_became_active":"2024-09-17T05:34:43.488321+0000","last_became_peered":"2024-09-17T05:34:43.488321+0000","last_unstale":"2024-09-17T05:34:46.166410+0000","last_undegraded":"2024-09-17T05:34:46.166410+0000","last_fullsized":"2024-09-17T05:34:46.166410+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:31.408633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507912+0000","last_change":"2024-09-17T05:34:44.501243+0000","last_active":"2024-09-17T05:34:45.507912+0000","last_peered":"2024-09-17T05:34:45.507912+0000","last_clean":"2024-09-17T05:34:45.507912+0000","last_became_active":"2024-09-17T05:34:44.500905+0000","last_became_peered":"2024-09-17T05:34:44.500905+0000","last_unstale":"2024-09-17T05:34:45.507912+0000","last_undegraded":"2024-09-17T05:34:45.507912+0000","last_fullsized":"2024-09-17T05:34:45.507912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:16.467250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662240+0000","last_change":"2024-09-17T05:34:43.486138+0000","last_active":"2024-09-17T05:34:45.662240+0000","last_peered":"2024-09-17T05:34:45.662240+0000","last_clean":"2024-09-17T05:34:45.662240+0000","last_became_active":"2024-09-17T05:34:43.485752+0000","last_became_peered":"2024-09-17T05:34:43.485752+0000","last_unstale":"2024-09-17T05:34:45.662240+0000","last_undegraded":"2024-09-17T05:34:45.662240+0000","last_fullsized":"2024-09-17T05:34:45.662240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.178231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.166207+0000","last_change":"2024-09-17T05:34:44.498312+0000","last_active":"2024-09-17T05:34:46.166207+0000","last_peered":"2024-09-17T05:34:46.166207+0000","last_clean":"2024-09-17T05:34:46.166207+0000","last_became_active":"2024-09-17T05:34:44.495324+0000","last_became_peered":"2024-09-17T05:34:44.495324+0000","last_unstale":"2024-09-17T05:34:46.166207+0000","last_undegraded":"2024-09-17T05:34:46.166207+0000","last_fullsized":"2024-09-17T05:34:46.166207+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.787871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.475529+0000","last_change":"2024-09-17T05:34:43.477604+0000","last_active":"2024-09-17T05:34:45.475529+0000","last_peered":"2024-09-17T05:34:45.475529+0000","last_clean":"2024-09-17T05:34:45.475529+0000","last_became_active":"2024-09-17T05:34:43.477206+0000","last_became_peered":"2024-09-17T05:34:43.477206+0000","last_unstale":"2024-09-17T05:34:45.475529+0000","last_undegraded":"2024-09-17T05:34:45.475529+0000","last_fullsized":"2024-09-17T05:34:45.475529+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:23.859235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662251+0000","last_change":"2024-09-17T05:34:44.499504+0000","last_active":"2024-09-17T05:34:45.662251+0000","last_peered":"2024-09-17T05:34:45.662251+0000","last_clean":"2024-09-17T05:34:45.662251+0000","last_became_active":"2024-09-17T05:34:44.499169+0000","last_became_peered":"2024-09-17T05:34:44.499169+0000","last_unstale":"2024-09-17T05:34:45.662251+0000","last_undegraded":"2024-09-17T05:34:45.662251+0000","last_fullsized":"2024-09-17T05:34:45.662251+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:09.411623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479442+0000","last_change":"2024-09-17T05:34:43.487120+0000","last_active":"2024-09-17T05:34:45.479442+0000","last_peered":"2024-09-17T05:34:45.479442+0000","last_clean":"2024-09-17T05:34:45.479442+0000","last_became_active":"2024-09-17T05:34:43.486767+0000","last_became_peered":"2024-09-17T05:34:43.486767+0000","last_unstale":"2024-09-17T05:34:45.479442+0000","last_undegraded":"2024-09-17T05:34:45.479442+0000","last_fullsized":"2024-09-17T05:34:45.479442+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:21.331700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508424+0000","last_change":"2024-09-17T05:34:44.511186+0000","last_active":"2024-09-17T05:34:45.508424+0000","last_peered":"2024-09-17T05:34:45.508424+0000","last_clean":"2024-09-17T05:34:45.508424+0000","last_became_active":"2024-09-17T05:34:44.510799+0000","last_became_peered":"2024-09-17T05:34:44.510799+0000","last_unstale":"2024-09-17T05:34:45.508424+0000","last_undegraded":"2024-09-17T05:34:45.508424+0000","last_fullsized":"2024-09-17T05:34:45.508424+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:46.750796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"138'2","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477470+0000","last_change":"2024-09-17T05:34:43.482392+0000","last_active":"2024-09-17T05:34:45.477470+0000","last_peered":"2024-09-17T05:34:45.477470+0000","last_clean":"2024-09-17T05:34:45.477470+0000","last_became_active":"2024-09-17T05:34:43.482075+0000","last_became_peered":"2024-09-17T05:34:43.482075+0000","last_unstale":"2024-09-17T05:34:45.477470+0000","last_undegraded":"2024-09-17T05:34:45.477470+0000","last_fullsized":"2024-09-17T05:34:45.477470+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:37.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508320+0000","last_change":"2024-09-17T05:34:44.513231+0000","last_active":"2024-09-17T05:34:45.508320+0000","last_peered":"2024-09-17T05:34:45.508320+0000","last_clean":"2024-09-17T05:34:45.508320+0000","last_became_active":"2024-09-17T05:34:44.508966+0000","last_became_peered":"2024-09-17T05:34:44.508966+0000","last_unstale":"2024-09-17T05:34:45.508320+0000","last_undegraded":"2024-09-17T05:34:45.508320+0000","last_fullsized":"2024-09-17T05:34:45.508320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:45.193953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"139'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508650+0000","last_change":"2024-09-17T05:34:43.493846+0000","last_active":"2024-09-17T05:34:45.508650+0000","last_peered":"2024-09-17T05:34:45.508650+0000","last_clean":"2024-09-17T05:34:45.508650+0000","last_became_active":"2024-09-17T05:34:43.493493+0000","last_became_peered":"2024-09-17T05:34:43.493493+0000","last_unstale":"2024-09-17T05:34:45.508650+0000","last_undegraded":"2024-09-17T05:34:45.508650+0000","last_fullsized":"2024-09-17T05:34:45.508650+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:38.144157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480097+0000","last_change":"2024-09-17T05:34:44.499337+0000","last_active":"2024-09-17T05:34:45.480097+0000","last_peered":"2024-09-17T05:34:45.480097+0000","last_clean":"2024-09-17T05:34:45.480097+0000","last_became_active":"2024-09-17T05:34:44.498950+0000","last_became_peered":"2024-09-17T05:34:44.498950+0000","last_unstale":"2024-09-17T05:34:45.480097+0000","last_undegraded":"2024-09-17T05:34:45.480097+0000","last_fullsized":"2024-09-17T05:34:45.480097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:56.926090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508277+0000","last_change":"2024-09-17T05:34:43.487845+0000","last_active":"2024-09-17T05:34:45.508277+0000","last_peered":"2024-09-17T05:34:45.508277+0000","last_clean":"2024-09-17T05:34:45.508277+0000","last_became_active":"2024-09-17T05:34:43.487537+0000","last_became_peered":"2024-09-17T05:34:43.487537+0000","last_unstale":"2024-09-17T05:34:45.508277+0000","last_undegraded":"2024-09-17T05:34:45.508277+0000","last_fullsized":"2024-09-17T05:34:45.508277+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:34.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.164080+0000","last_change":"2024-09-17T05:34:44.484583+0000","last_active":"2024-09-17T05:34:46.164080+0000","last_peered":"2024-09-17T05:34:46.164080+0000","last_clean":"2024-09-17T05:34:46.164080+0000","last_became_active":"2024-09-17T05:34:44.483761+0000","last_became_peered":"2024-09-17T05:34:44.483761+0000","last_unstale":"2024-09-17T05:34:46.164080+0000","last_undegraded":"2024-09-17T05:34:46.164080+0000","last_fullsized":"2024-09-17T05:34:46.164080+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:22.822388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662588+0000","last_change":"2024-09-17T05:34:43.487549+0000","last_active":"2024-09-17T05:34:45.662588+0000","last_peered":"2024-09-17T05:34:45.662588+0000","last_clean":"2024-09-17T05:34:45.662588+0000","last_became_active":"2024-09-17T05:34:43.487089+0000","last_became_peered":"2024-09-17T05:34:43.487089+0000","last_unstale":"2024-09-17T05:34:45.662588+0000","last_undegraded":"2024-09-17T05:34:45.662588+0000","last_fullsized":"2024-09-17T05:34:45.662588+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:13.321577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.507984+0000","last_change":"2024-09-17T05:34:44.509810+0000","last_active":"2024-09-17T05:34:45.507984+0000","last_peered":"2024-09-17T05:34:45.507984+0000","last_clean":"2024-09-17T05:34:45.507984+0000","last_became_active":"2024-09-17T05:34:44.509326+0000","last_became_peered":"2024-09-17T05:34:44.509326+0000","last_unstale":"2024-09-17T05:34:45.507984+0000","last_undegraded":"2024-09-17T05:34:45.507984+0000","last_fullsized":"2024-09-17T05:34:45.507984+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.495976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480335+0000","last_change":"2024-09-17T05:34:43.495216+0000","last_active":"2024-09-17T05:34:45.480335+0000","last_peered":"2024-09-17T05:34:45.480335+0000","last_clean":"2024-09-17T05:34:45.480335+0000","last_became_active":"2024-09-17T05:34:43.494800+0000","last_became_peered":"2024-09-17T05:34:43.494800+0000","last_unstale":"2024-09-17T05:34:45.480335+0000","last_undegraded":"2024-09-17T05:34:45.480335+0000","last_fullsized":"2024-09-17T05:34:45.480335+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.033404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:46.163994+0000","last_change":"2024-09-17T05:34:44.488673+0000","last_active":"2024-09-17T05:34:46.163994+0000","last_peered":"2024-09-17T05:34:46.163994+0000","last_clean":"2024-09-17T05:34:46.163994+0000","last_became_active":"2024-09-17T05:34:44.488336+0000","last_became_peered":"2024-09-17T05:34:44.488336+0000","last_unstale":"2024-09-17T05:34:46.163994+0000","last_undegraded":"2024-09-17T05:34:46.163994+0000","last_fullsized":"2024-09-17T05:34:46.163994+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:43.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.662314+0000","last_change":"2024-09-17T05:34:43.491113+0000","last_active":"2024-09-17T05:34:45.662314+0000","last_peered":"2024-09-17T05:34:45.662314+0000","last_clean":"2024-09-17T05:34:45.662314+0000","last_became_active":"2024-09-17T05:34:43.490776+0000","last_became_peered":"2024-09-17T05:34:43.490776+0000","last_unstale":"2024-09-17T05:34:45.662314+0000","last_undegraded":"2024-09-17T05:34:45.662314+0000","last_fullsized":"2024-09-17T05:34:45.662314+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:50.864469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.508185+0000","last_change":"2024-09-17T05:34:44.510255+0000","last_active":"2024-09-17T05:34:45.508185+0000","last_peered":"2024-09-17T05:34:45.508185+0000","last_clean":"2024-09-17T05:34:45.508185+0000","last_became_active":"2024-09-17T05:34:44.509308+0000","last_became_peered":"2024-09-17T05:34:44.509308+0000","last_unstale":"2024-09-17T05:34:45.508185+0000","last_undegraded":"2024-09-17T05:34:45.508185+0000","last_fullsized":"2024-09-17T05:34:45.508185+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:44.223673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.664039+0000","last_change":"2024-09-17T05:34:43.490904+0000","last_active":"2024-09-17T05:34:45.664039+0000","last_peered":"2024-09-17T05:34:45.664039+0000","last_clean":"2024-09-17T05:34:45.664039+0000","last_became_active":"2024-09-17T05:34:43.490596+0000","last_became_peered":"2024-09-17T05:34:43.490596+0000","last_unstale":"2024-09-17T05:34:45.664039+0000","last_undegraded":"2024-09-17T05:34:45.664039+0000","last_fullsized":"2024-09-17T05:34:45.664039+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:09.673465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476626+0000","last_change":"2024-09-17T05:34:44.494342+0000","last_active":"2024-09-17T05:34:45.476626+0000","last_peered":"2024-09-17T05:34:45.476626+0000","last_clean":"2024-09-17T05:34:45.476626+0000","last_became_active":"2024-09-17T05:34:44.494018+0000","last_became_peered":"2024-09-17T05:34:44.494018+0000","last_unstale":"2024-09-17T05:34:45.476626+0000","last_undegraded":"2024-09-17T05:34:45.476626+0000","last_fullsized":"2024-09-17T05:34:45.476626+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:54.902278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.479602+0000","last_change":"2024-09-17T05:34:43.485943+0000","last_active":"2024-09-17T05:34:45.479602+0000","last_peered":"2024-09-17T05:34:45.479602+0000","last_clean":"2024-09-17T05:34:45.479602+0000","last_became_active":"2024-09-17T05:34:43.485578+0000","last_became_peered":"2024-09-17T05:34:43.485578+0000","last_unstale":"2024-09-17T05:34:45.479602+0000","last_undegraded":"2024-09-17T05:34:45.479602+0000","last_fullsized":"2024-09-17T05:34:45.479602+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:37.324122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.480507+0000","last_change":"2024-09-17T05:34:44.496944+0000","last_active":"2024-09-17T05:34:45.480507+0000","last_peered":"2024-09-17T05:34:45.480507+0000","last_clean":"2024-09-17T05:34:45.480507+0000","last_became_active":"2024-09-17T05:34:44.496229+0000","last_became_peered":"2024-09-17T05:34:44.496229+0000","last_unstale":"2024-09-17T05:34:45.480507+0000","last_undegraded":"2024-09-17T05:34:45.480507+0000","last_fullsized":"2024-09-17T05:34:45.480507+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:43.454042+0000","last_clean_scrub_stamp":"2024-09-17T05:34:43.454042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:40.652251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"138'1","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.477365+0000","last_change":"2024-09-17T05:34:43.483784+0000","last_active":"2024-09-17T05:34:45.477365+0000","last_peered":"2024-09-17T05:34:45.477365+0000","last_clean":"2024-09-17T05:34:45.477365+0000","last_became_active":"2024-09-17T05:34:43.483417+0000","last_became_peered":"2024-09-17T05:34:43.483417+0000","last_unstale":"2024-09-17T05:34:45.477365+0000","last_undegraded":"2024-09-17T05:34:45.477365+0000","last_fullsized":"2024-09-17T05:34:45.477365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:42.446180+0000","last_clean_scrub_stamp":"2024-09-17T05:34:42.446180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:21.331700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":341,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T05:34:45.476599+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:34:45.476599+0000","last_peered":"2024-09-17T05:34:45.476599+0000","last_clean":"2024-09-17T05:34:45.476599+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:34:45.476599+0000","last_undegraded":"2024-09-17T05:34:45.476599+0000","last_fullsized":"2024-09-17T05:34:45.476599+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":848199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2379776,"data_stored":1439653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509582,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":848199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28768,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343072,"statfs":{"total":96636764160,"available":96607305728,"internally_reserved":0,"allocated":1777664,"data_stored":847811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1769472,"data_stored":847687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2379776,"data_stored":1439709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542286,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41232,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330608,"statfs":{"total":96636764160,"available":96594542592,"internally_reserved":0,"allocated":1765376,"data_stored":847617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:54.033 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:34:54.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:34:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- 172.21.15.43:0/3857617755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb806f110 msgr2=0x7f5bb80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 --2- 172.21.15.43:0/3857617755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb806f110 0x7f5bb80ad0b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009bc0 tx=0x7f5bb001cae0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- 172.21.15.43:0/3857617755 shutdown_connections 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 --2- 172.21.15.43:0/3857617755 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb806f110 0x7f5bb80ad0b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 --2- 172.21.15.43:0/3857617755 >> v2:172.21.15.62:3300/0 conn(0x7f5bb80b3af0 0x7f5bb815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- 172.21.15.43:0/3857617755 >> 172.21.15.43:0/3857617755 conn(0x7f5bb80b23d0 msgr2=0x7f5bb80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- 172.21.15.43:0/3857617755 shutdown_connections 2024-09-17T05:34:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- 172.21.15.43:0/3857617755 wait complete. 2024-09-17T05:34:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 Processor -- start 2024-09-17T05:34:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.233+0000 7f5bbca00640 1 -- start start 2024-09-17T05:34:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb806f110 0x7f5bb81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb815e3f0 0x7f5bb815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8168c10 con 0x7f5bb8155d70 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8168a90 con 0x7f5bb806f110 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb8168d90 con 0x7f5bb815e3f0 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41654/0 (socket says 172.21.15.43:41654) 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 -- 172.21.15.43:0/1298780695 learned_addr learned my addr 172.21.15.43:0/1298780695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5e00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb806f110 0x7f5bb81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb6800640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb815e3f0 0x7f5bb815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 -- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb815e3f0 msgr2=0x7f5bb815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb815e3f0 0x7f5bb815e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 -- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb806f110 msgr2=0x7f5bb81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb806f110 0x7f5bb81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 -- 172.21.15.43:0/1298780695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb815ed40 con 0x7f5bb8155d70 2024-09-17T05:34:54.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bb5400640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400e7a0 tx=0x7f5ba400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400ccc0 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ba400ce60 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4005260 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81f98c0 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb80735c0 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.237+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb80ad0b0 con 0x7f5bb8155d70 2024-09-17T05:34:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.241+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ba4005400 con 0x7f5bb8155d70 2024-09-17T05:34:54.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.241+0000 7f5bae000640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 0x7f5b98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.241+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f5ba4094a60 con 0x7f5bb8155d70 2024-09-17T05:34:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.241+0000 7f5bb5e00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 0x7f5b98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.241+0000 7f5bb5e00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 0x7f5b98075740 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80715c0 tx=0x7f5ba8001380 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.245+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5bb815ed40 con 0x7f5bb8155d70 2024-09-17T05:34:54.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.481+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5bb815bee0 con 0x7f5bb8155d70 2024-09-17T05:34:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bae000640 1 -- 172.21.15.43:0/1298780695 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f5ba405f420 con 0x7f5bb8155d70 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPEulmt9xEERAAgYyxj0+leq0/lC25lh6Vpw== 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:34:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 msgr2=0x7f5b98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 0x7f5b98075740 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80715c0 tx=0x7f5ba8001380 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 msgr2=0x7f5bb80cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400e7a0 tx=0x7f5ba400ec60 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 shutdown_connections 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5b98073230 0x7f5b98075740 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5bb815e3f0 0x7f5bb815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bb8155d70 0x7f5bb80cc930 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 --2- 172.21.15.43:0/1298780695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bb806f110 0x7f5bb81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 >> 172.21.15.43:0/1298780695 conn(0x7f5bb80b23d0 msgr2=0x7f5bb80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 shutdown_connections 2024-09-17T05:34:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.485+0000 7f5bbca00640 1 -- 172.21.15.43:0/1298780695 wait complete. 2024-09-17T05:34:54.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- 172.21.15.43:0/1947770192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e0155d70 msgr2=0x7f52e015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 --2- 172.21.15.43:0/1947770192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e0155d70 0x7f52e015c7e0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f52e00d0c40 tx=0x7f52d401c740 comp rx=0 tx=0).stop 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- 172.21.15.43:0/1947770192 shutdown_connections 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 --2- 172.21.15.43:0/1947770192 >> v2:172.21.15.62:3300/0 conn(0x7f52e00b3af0 0x7f52e00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 --2- 172.21.15.43:0/1947770192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e0155d70 0x7f52e015c7e0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- 172.21.15.43:0/1947770192 >> 172.21.15.43:0/1947770192 conn(0x7f52e00b23d0 msgr2=0x7f52e00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- 172.21.15.43:0/1947770192 shutdown_connections 2024-09-17T05:34:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- 172.21.15.43:0/1947770192 wait complete. 2024-09-17T05:34:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 Processor -- start 2024-09-17T05:34:54.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.669+0000 7f52e8200640 1 -- start start 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e0155d70 0x7f52e009d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 0x7f52e015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0168b60 con 0x7f52e006f110 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01689e0 con 0x7f52e00cc350 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f52e0168ce0 con 0x7f52e0155d70 2024-09-17T05:34:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41666/0 (socket says 172.21.15.43:41666) 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 -- 172.21.15.43:0/2001430084 learned_addr learned my addr 172.21.15.43:0/2001430084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e4e00640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e0155d70 0x7f52e009d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e6200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 0x7f52e015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 -- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e0155d70 msgr2=0x7f52e009d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e0155d70 0x7f52e009d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 -- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 msgr2=0x7f52e015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 0x7f52e015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 -- 172.21.15.43:0/2001430084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e009dd50 con 0x7f52e006f110 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e6200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 0x7f52e015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e5800640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f52d4098320 tx=0x7f52d4005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40b2040 con 0x7f52e006f110 2024-09-17T05:34:54.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d40bb070 con 0x7f52e006f110 2024-09-17T05:34:54.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4003600 con 0x7f52e006f110 2024-09-17T05:34:54.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e01fc440 con 0x7f52e006f110 2024-09-17T05:34:54.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01fca00 con 0x7f52e006f110 2024-09-17T05:34:54.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e00ad0b0 con 0x7f52e006f110 2024-09-17T05:34:54.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.673+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f52d40037a0 con 0x7f52e006f110 2024-09-17T05:34:54.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.677+0000 7f52dd600640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 0x7f52c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.677+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f52d412f100 con 0x7f52e006f110 2024-09-17T05:34:54.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.677+0000 7f52e4e00640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 0x7f52c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.677+0000 7f52e4e00640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 0x7f52c0075710 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f52e015ef20 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.677+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52d40163f0 con 0x7f52e006f110 2024-09-17T05:34:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.889+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52e015b6e0 con 0x7f52e006f110 2024-09-17T05:34:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.889+0000 7f52dd600640 1 -- 172.21.15.43:0/2001430084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) ==== 95+0+5233 (secure 0 0 0) 0x7f52d40fa320 con 0x7f52e006f110 2024-09-17T05:34:54.895 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:54.895 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":345,"btime":"2024-09-17T05:34:47:511187+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:34:46.927801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10464":{"gid":10464,"name":"c","rank":0,"incarnation":341,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/4038368577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4038368577},{"type":"v1","addr":"172.21.15.43:6834","nonce":4038368577}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10464,"qdb_cluster":[10464]},"id":7}]} 2024-09-17T05:34:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 345 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 msgr2=0x7f52c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 0x7f52c0075710 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f52e015ef20 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 msgr2=0x7f52e00ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f52d4098320 tx=0x7f52d4005ca0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 shutdown_connections 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f52c0073200 0x7f52c0075710 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52e00cc350 0x7f52e015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f52e0155d70 0x7f52e009d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 --2- 172.21.15.43:0/2001430084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52e006f110 0x7f52e00ca260 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 >> 172.21.15.43:0/2001430084 conn(0x7f52e00b23d0 msgr2=0x7f52e006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 shutdown_connections 2024-09-17T05:34:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:54.893+0000 7f52e8200640 1 -- 172.21.15.43:0/2001430084 wait complete. 2024-09-17T05:34:55.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 345, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:34:43.887777+0000', 'modified': '2024-09-17T05:34:46.927801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10464}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10464': {'gid': 10464, 'name': 'c', 'rank': 0, 'incarnation': 341, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/4038368577', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 4038368577}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 4038368577}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10464, 'qdb_cluster': [10464]} 2024-09-17T05:34:55.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:34:55.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:34:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 -- 172.21.15.43:0/3646936852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c155d70 msgr2=0x7f9e6c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 --2- 172.21.15.43:0/3646936852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c155d70 0x7f9e6c15c7e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0af0a0 tx=0x7f9e5c01c800 comp rx=0 tx=0).stop 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 -- 172.21.15.43:0/3646936852 shutdown_connections 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 --2- 172.21.15.43:0/3646936852 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c0b3920 0x7f9e6c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 --2- 172.21.15.43:0/3646936852 >> v2:172.21.15.62:3300/0 conn(0x7f9e6c06f110 0x7f9e6c0b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 --2- 172.21.15.43:0/3646936852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c155d70 0x7f9e6c15c7e0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 -- 172.21.15.43:0/3646936852 >> 172.21.15.43:0/3646936852 conn(0x7f9e6c0b23d0 msgr2=0x7f9e6c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 -- 172.21.15.43:0/3646936852 shutdown_connections 2024-09-17T05:34:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.481+0000 7f9e73200640 1 -- 172.21.15.43:0/3646936852 wait complete. 2024-09-17T05:34:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 Processor -- start 2024-09-17T05:34:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- start start 2024-09-17T05:34:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c0b3920 0x7f9e6c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c155d70 0x7f9e6c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c15f8c0 con 0x7f9e6c06f110 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c15f740 con 0x7f9e6c0b3920 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c15fa40 con 0x7f9e6c155d70 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41694/0 (socket says 172.21.15.43:41694) 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 -- 172.21.15.43:0/425055138 learned_addr learned my addr 172.21.15.43:0/425055138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e63e00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c0b3920 0x7f9e6c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e71200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c155d70 0x7f9e6c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 -- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c155d70 msgr2=0x7f9e6c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c155d70 0x7f9e6c0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 -- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c0b3920 msgr2=0x7f9e6c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c0b3920 0x7f9e6c0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 -- 172.21.15.43:0/425055138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c166bf0 con 0x7f9e6c06f110 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e6be00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0b0040 tx=0x7f9e5c0a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e68c00640 1 -- 172.21.15.43:0/425055138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0b2040 con 0x7f9e6c06f110 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e68c00640 1 -- 172.21.15.43:0/425055138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c004430 con 0x7f9e6c06f110 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e68c00640 1 -- 172.21.15.43:0/425055138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c005290 con 0x7f9e6c06f110 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c166ed0 con 0x7f9e6c06f110 2024-09-17T05:34:55.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.485+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1679b0 con 0x7f9e6c06f110 2024-09-17T05:34:55.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e68c00640 1 -- 172.21.15.43:0/425055138 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9e5c0bd060 con 0x7f9e6c06f110 2024-09-17T05:34:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e68c00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 0x7f9e48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e68c00640 1 -- 172.21.15.43:0/425055138 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c12d7d0 con 0x7f9e6c06f110 2024-09-17T05:34:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e63e00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 0x7f9e48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e7aa33000 1 Processor -- start 2024-09-17T05:34:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e7aa33000 1 -- start start 2024-09-17T05:34:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.489+0000 7f9e63e00640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 0x7f9e48075740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9e540058d0 tx=0x7f9e54005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561482366950 0x5614824b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x5614824b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56148238c410 con 0x561481f76460 2024-09-17T05:34:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5614822c9b40 con 0x561482366950 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5614822b8f70 con 0x56148234fca0 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41696/0 (socket says 172.21.15.43:41696) 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 -- 172.21.15.43:0/187162282 learned_addr learned my addr 172.21.15.43:0/187162282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e63e00640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561482366950 0x5614824b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e71200640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x5614824b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 -- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 msgr2=0x5614824b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x5614824b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 -- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561482366950 msgr2=0x5614824b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561482366950 0x5614824b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 -- 172.21.15.43:0/187162282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614822805b0 con 0x561481f76460 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e71200640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x5614824b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e6be00640 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01cd70 tx=0x7f9e5c0f40b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b5dab1157a848e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e60c00640 1 -- 172.21.15.43:0/187162282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0f7bf0 con 0x561481f76460 2024-09-17T05:34:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e60c00640 1 -- 172.21.15.43:0/187162282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c0f87e0 con 0x561481f76460 2024-09-17T05:34:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e60c00640 1 -- 172.21.15.43:0/187162282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0ff0e0 con 0x561481f76460 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- 172.21.15.43:0/187162282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 msgr2=0x5614824b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01cd70 tx=0x7f9e5c0f40b0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- 172.21.15.43:0/187162282 shutdown_connections 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561482366950 0x5614824b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x5614824b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/187162282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824b76a0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- 172.21.15.43:0/187162282 >> 172.21.15.43:0/187162282 conn(0x56148243cd90 msgr2=0x56148246c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- 172.21.15.43:0/187162282 shutdown_connections 2024-09-17T05:34:55.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.493+0000 7f9e7aa33000 1 -- 172.21.15.43:0/187162282 wait complete. 2024-09-17T05:34:55.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- start start 2024-09-17T05:34:55.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 Processor -- start 2024-09-17T05:34:55.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x5614824bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561482366950 0x5614824bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614822535f0 con 0x561481f76460 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5614822bcd80 con 0x56148234fca0 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56148225f520 con 0x561482366950 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 10 client.0 _ll_register_callbacks cb 0x561482263830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41708/0 (socket says 172.21.15.43:41708) 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 -- 172.21.15.43:0/1846268434 learned_addr learned my addr 172.21.15.43:0/1846268434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e63e00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x5614824bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e71200640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561482366950 0x5614824bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 -- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561482366950 msgr2=0x5614824bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561482366950 0x5614824bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 -- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 msgr2=0x5614824bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x5614824bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 -- 172.21.15.43:0/1846268434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561482257190 con 0x561481f76460 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e6be00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01cd70 tx=0x7f9e5c0f7c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cf89126f2424df server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e59600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:34:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c117060 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c0ff030 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561482251110 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=345) ==== 24+0+0 (secure 0 0 0) 0x7f9e5c0f80e0 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 10 client.10635 fetch_fsmap learned FSMap version 345 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=345,monmap=2+}) -- 0x56148233df10 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.497+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561482251110 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 345) ==== 3502+0+0 (secure 0 0 0) 0x7f9e5c005430 con 0x561481f76460 2024-09-17T05:34:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 10 client.10635 fetch_fsmap finished waiting for FSMap version 345 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 10 client.10635 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 20 client.10635 populate_metadata read hostname 'smithi043' 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f5b7d0 0x5614824c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 4 client.10635 mds_command: new command op to 10464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 --> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614822cba90 con 0x561481f5b7d0 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e63e00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f5b7d0 0x5614824c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e59600640 10 client.10635 ms_handle_connect on v2:172.21.15.43:6832/4038368577 2024-09-17T05:34:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e63e00640 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f5b7d0 0x5614824c8d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70639b7ef55eab3d server_cookie=ad557414d51c88d in_seq=0 out_seq=0 2024-09-17T05:34:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e59600640 1 -- 172.21.15.43:0/1846268434 <== mds.0 v2:172.21.15.43:6832/4038368577 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614822cba90 con 0x561481f5b7d0 2024-09-17T05:34:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e59600640 10 client.10635 handle_command_reply: tid=0 2024-09-17T05:34:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 client.10635 shutdown 2024-09-17T05:34:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 msgr2=0x5614824be920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01cd70 tx=0x7f9e5c0f7c70 comp rx=0 tx=0).stop 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 shutdown_connections 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f5b7d0 0x5614824c8d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x5614824bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e59600640 0 client.10635 ms_handle_reset on v2:172.21.15.43:6832/4038368577 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561482366950 0x5614824bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/1846268434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561481f76460 0x5614824be920 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 >> 172.21.15.43:0/1846268434 conn(0x56148243cd90 msgr2=0x561482473440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 shutdown_connections 2024-09-17T05:34:55.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 1 -- 172.21.15.43:0/1846268434 wait complete. 2024-09-17T05:34:55.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.501+0000 7f9e7aa33000 20 client.10635 trim_cache size 0 max 16384 2024-09-17T05:34:55.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 Processor -- start 2024-09-17T05:34:55.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 -- start start 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 0x561482529630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x561482529b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1f9a60 con 0x561482366950 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1fd0e0 con 0x56148234fca0 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c1fc440 con 0x561481f76460 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e71200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e71200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41716/0 (socket says 172.21.15.43:41716) 2024-09-17T05:34:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.521+0000 7f9e71200640 1 -- 172.21.15.43:0/3574000070 learned_addr learned my addr 172.21.15.43:0/3574000070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e63e00640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x561482529b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e6be00640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 0x561482529630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 -- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 msgr2=0x561482529630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 0x561482529630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 -- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 msgr2=0x561482529b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x561482529b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 -- 172.21.15.43:0/3574000070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561482314db0 con 0x561482366950 2024-09-17T05:34:55.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e63e00640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x561482529b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e6be00640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 0x561482529630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e71200640 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0749b0 tx=0x7f9e64013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc53a1407bd7328 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e5be00640 1 -- 172.21.15.43:0/3574000070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004d30 con 0x561482366950 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e5be00640 1 -- 172.21.15.43:0/3574000070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e64005020 con 0x561482366950 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e5be00640 1 -- 172.21.15.43:0/3574000070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640053f0 con 0x561482366950 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3574000070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 msgr2=0x56148252a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0749b0 tx=0x7f9e64013830 comp rx=0 tx=0).stop 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3574000070 shutdown_connections 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56148234fca0 0x561482529b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252a0b0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3574000070 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x561481f76460 0x561482529630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3574000070 >> 172.21.15.43:0/3574000070 conn(0x56148243cd90 msgr2=0x56148246bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3574000070 shutdown_connections 2024-09-17T05:34:55.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3574000070 wait complete. 2024-09-17T05:34:55.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 -- start start 2024-09-17T05:34:55.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.525+0000 7f9e7aa33000 1 Processor -- start 2024-09-17T05:34:55.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561481f76460 0x56148252d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x56148252d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56148225f520 con 0x561482366950 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1fca90 con 0x561481f76460 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5614822535f0 con 0x56148234fca0 2024-09-17T05:34:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 10 client.0 _ll_register_callbacks cb 0x7f9e54015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:55.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41728/0 (socket says 172.21.15.43:41728) 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 -- 172.21.15.43:0/3125701295 learned_addr learned my addr 172.21.15.43:0/3125701295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e63e00640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x56148252d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e6be00640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561481f76460 0x56148252d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 -- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 msgr2=0x56148252d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x56148252d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 -- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561481f76460 msgr2=0x56148252d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561481f76460 0x56148252d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 -- 172.21.15.43:0/3125701295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148227fab0 con 0x561482366950 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e71200640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e60 tx=0x7f9e64004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5fcfa5c6f56bc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6402b070 con 0x561482366950 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e64005020 con 0x561482366950 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f9e64005660 con 0x561482366950 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=345) ==== 24+0+0 (secure 0 0 0) 0x7f9e64036050 con 0x561482366950 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 10 client.10641 fetch_fsmap learned FSMap version 345 2024-09-17T05:34:55.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=345,monmap=2+}) -- 0x56148225e070 con 0x561482366950 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e64005020 con 0x561482366950 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 345) ==== 3502+0+0 (secure 0 0 0) 0x7f9e64039020 con 0x561482366950 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 10 client.10641 fetch_fsmap finished waiting for FSMap version 345 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 10 client.10641 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 20 client.10641 populate_metadata read hostname 'smithi043' 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f49830 0x5614824e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 4 client.10641 mds_command: new command op to 10464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:34:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 --> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] -- command(tid 0: {"prefix": "status"}) -- 0x5614822c7540 con 0x561481f49830 2024-09-17T05:34:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e6be00640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f49830 0x5614824e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 10 client.10641 ms_handle_connect on v2:172.21.15.43:6832/4038368577 2024-09-17T05:34:55.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e6be00640 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f49830 0x5614824e9e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a693248d724306f server_cookie=7d790596b2d9ab99 in_seq=0 out_seq=0 2024-09-17T05:34:55.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 1 -- 172.21.15.43:0/3125701295 <== mds.0 v2:172.21.15.43:6832/4038368577 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5614822c7540 con 0x561481f49830 2024-09-17T05:34:55.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e59600640 10 client.10641 handle_command_reply: tid=0 2024-09-17T05:34:55.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 client.10641 shutdown 2024-09-17T05:34:55.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 msgr2=0x56148252dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.529+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e60 tx=0x7f9e64004e70 comp rx=0 tx=0).stop 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 shutdown_connections 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:6832/4038368577,v1:172.21.15.43:6834/4038368577] conn(0x561481f49830 0x5614824e9e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561482366950 0x56148252dc40 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56148234fca0 0x56148252d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e59600640 0 client.10641 ms_handle_reset on v2:172.21.15.43:6832/4038368577 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 --2- 172.21.15.43:0/3125701295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561481f76460 0x56148252d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 >> 172.21.15.43:0/3125701295 conn(0x56148243cd90 msgr2=0x56148246c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 shutdown_connections 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 1 -- 172.21.15.43:0/3125701295 wait complete. 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e7aa33000 20 client.10641 trim_cache size 0 max 16384 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:34:55.541 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10464, 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 11.042523940000001, 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 345, 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 139, 2024-09-17T05:34:55.542 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 139, 2024-09-17T05:34:55.543 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 15.453295242999999 2024-09-17T05:34:55.543 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 msgr2=0x7f9e48075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 0x7f9e48075740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9e540058d0 tx=0x7f9e54005860 comp rx=0 tx=0).stop 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 msgr2=0x7f9e6c09b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0b0040 tx=0x7f9e5c0a5790 comp rx=0 tx=0).stop 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 shutdown_connections 2024-09-17T05:34:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9e48073230 0x7f9e48075740 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e6c0b3920 0x7f9e6c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e6c155d70 0x7f9e6c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 --2- 172.21.15.43:0/425055138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c06f110 0x7f9e6c09b680 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 >> 172.21.15.43:0/425055138 conn(0x7f9e6c0b23d0 msgr2=0x7f9e6c09d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 shutdown_connections 2024-09-17T05:34:55.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.533+0000 7f9e73200640 1 -- 172.21.15.43:0/425055138 wait complete. 2024-09-17T05:34:55.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:34:55.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 -- 172.21.15.43:0/183835259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b5150 msgr2=0x7fb86c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 --2- 172.21.15.43:0/183835259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b5150 0x7fb86c0b5560 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fb86c073b50 tx=0x7fb86401cd80 comp rx=0 tx=0).stop 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 -- 172.21.15.43:0/183835259 shutdown_connections 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 --2- 172.21.15.43:0/183835259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b5150 0x7fb86c0b5560 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 --2- 172.21.15.43:0/183835259 >> v2:172.21.15.62:3300/0 conn(0x7fb86c0b64d0 0x7fb86c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 -- 172.21.15.43:0/183835259 >> 172.21.15.43:0/183835259 conn(0x7fb86c0d12f0 msgr2=0x7fb86c0d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 -- 172.21.15.43:0/183835259 shutdown_connections 2024-09-17T05:34:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.709+0000 7fb872600640 1 -- 172.21.15.43:0/183835259 wait complete. 2024-09-17T05:34:55.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 Processor -- start 2024-09-17T05:34:55.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 -- start start 2024-09-17T05:34:55.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb86c0b5150 0x7fb86c09e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86c0b64d0 0x7fb86c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c168cd0 con 0x7fb86c0b0950 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c168b50 con 0x7fb86c0b64d0 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb872600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c168e50 con 0x7fb86c0b5150 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb86b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb86b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41746/0 (socket says 172.21.15.43:41746) 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.713+0000 7fb86b400640 1 -- 172.21.15.43:0/1232377414 learned_addr learned my addr 172.21.15.43:0/1232377414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb863e00640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb86c0b5150 0x7fb86c09e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86be00640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86c0b64d0 0x7fb86c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 -- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb86c0b5150 msgr2=0x7fb86c09e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb86c0b5150 0x7fb86c09e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 -- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86c0b64d0 msgr2=0x7fb86c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86c0b64d0 0x7fb86c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 -- 172.21.15.43:0/1232377414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1fc470 con 0x7fb86c0b0950 2024-09-17T05:34:55.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb86b400640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d6d0 tx=0x7fb85c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c017070 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb85c009b10 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1fc750 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c010460 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1fcd10 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c0b5560 con 0x7fb86c0b0950 2024-09-17T05:34:55.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.717+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb85c01a070 con 0x7fb86c0b0950 2024-09-17T05:34:55.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.721+0000 7fb863400640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 0x7fb848075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.721+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fb85c094590 con 0x7fb86c0b0950 2024-09-17T05:34:55.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.721+0000 7fb863e00640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 0x7fb848075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.721+0000 7fb863e00640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 0x7fb848075710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb86c074050 tx=0x7fb854009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.725+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb85c05ef50 con 0x7fb86c0b0950 2024-09-17T05:34:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.933+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb86c0b62b0 con 0x7fb86c0b0950 2024-09-17T05:34:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb863400640 1 -- 172.21.15.43:0/1232377414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v345) ==== 53+0+73 (secure 0 0 0) 0x7fb85c063980 con 0x7fb86c0b0950 2024-09-17T05:34:55.939 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 msgr2=0x7fb848075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 0x7fb848075710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb86c074050 tx=0x7fb854009060 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 msgr2=0x7fb86c14d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d6d0 tx=0x7fb85c00db90 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 shutdown_connections 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb848073200 0x7fb848075710 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb86c0b64d0 0x7fb86c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb86c0b5150 0x7fb86c09e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 --2- 172.21.15.43:0/1232377414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb86c0b0950 0x7fb86c14d8d0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 >> 172.21.15.43:0/1232377414 conn(0x7fb86c0d12f0 msgr2=0x7fb86c164fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 shutdown_connections 2024-09-17T05:34:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:55.937+0000 7fb872600640 1 -- 172.21.15.43:0/1232377414 wait complete. 2024-09-17T05:34:55.961 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:34:55.961 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:34:55.962 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:55.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:55.963 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:34:55.972 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:34:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41748sec preferred_lft 41748sec 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:34:55.974 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:34:55.975 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:34:55.976 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.976 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:55.976 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:34:55.976 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:34:55.976 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:55.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:55.977 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:34:56.037 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:34:56.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.040 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:34:56.096 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:34:56.097 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:34:56.097 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:34:56.097 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:34:56.097 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:34:56.097 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:34:56.097 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:34:56.097 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:34:56.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:34:56.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:34:56.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:34:56.098 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:34:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:34:56.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:56.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.236 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:34:56.246 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:34:56.246 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:34:56.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:56.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:34:56.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:56.311 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:56.363 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:56.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:56.363 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:56.419 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:34:56.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:56.525 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:56.525 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:56.525 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:56.543 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:56.543 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:56.543 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:56.543 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:34:56.544 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:56.554 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:34:56.555 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:34:56.559 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:34:56.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:56.580 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:34:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:34:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:56.690 INFO:teuthology.orchestra.run.smithi043.stdout:client10650 2024-09-17T05:34:56.691 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:56.691 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:56.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.701 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-09-17T05:34:56.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.758 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-09-17T05:34:56.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.809 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-09-17T05:34:56.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.861 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-09-17T05:34:56.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.920 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-17T05:34:56.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:56.978 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-09-17T05:34:57.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:57.033 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-09-17T05:34:57.085 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:34:57.085 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:34:57.085 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:34:57.086 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:34:57.086 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-09-17T05:34:57.086 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:34:57.197 INFO:teuthology.orchestra.run.smithi043.stdout:1099511627777 2024-09-17T05:34:57.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-09-17T05:34:57.206 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:34:57.313 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628279 2024-09-17T05:34:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-09-17T05:34:57.322 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-17T05:34:57.429 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628278 2024-09-17T05:34:57.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:57.437 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:57.437 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:57.487 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:34:57.487 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:57.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:34:57.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 -- 172.21.15.43:0/918357903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06d570 msgr2=0x7ffa7c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 --2- 172.21.15.43:0/918357903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06d570 0x7ffa7c06f1a0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0b2cd0 tx=0x7ffa6401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 -- 172.21.15.43:0/918357903 shutdown_connections 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 --2- 172.21.15.43:0/918357903 >> v2:172.21.15.62:3300/0 conn(0x7ffa7c0cc6d0 0x7ffa7c06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 --2- 172.21.15.43:0/918357903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06d570 0x7ffa7c06f1a0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 -- 172.21.15.43:0/918357903 >> 172.21.15.43:0/918357903 conn(0x7ffa7c0b23d0 msgr2=0x7ffa7c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 -- 172.21.15.43:0/918357903 shutdown_connections 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.737+0000 7ffa80c00640 1 -- 172.21.15.43:0/918357903 wait complete. 2024-09-17T05:34:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 Processor -- start 2024-09-17T05:34:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 -- start start 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa7c0cc6d0 0x7ffa7c1f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c072f00 0x7ffa7c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c166560 con 0x7ffa7c15adc0 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1663e0 con 0x7ffa7c072f00 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa80c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c1666e0 con 0x7ffa7c0cc6d0 2024-09-17T05:34:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41770/0 (socket says 172.21.15.43:41770) 2024-09-17T05:34:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 -- 172.21.15.43:0/1450538076 learned_addr learned my addr 172.21.15.43:0/1450538076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 -- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa7c0cc6d0 msgr2=0x7ffa7c1f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa7c0cc6d0 0x7ffa7c1f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 -- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c072f00 msgr2=0x7ffa7c074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c072f00 0x7ffa7c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.741+0000 7ffa73e00640 1 -- 172.21.15.43:0/1450538076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c0736e0 con 0x7ffa7c15adc0 2024-09-17T05:34:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa73e00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ffa640b0040 tx=0x7ffa640a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa640bb070 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa640044b0 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c0739c0 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa640b2040 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c073f80 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c166b90 con 0x7ffa7c15adc0 2024-09-17T05:34:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa640052d0 con 0x7ffa7c15adc0 2024-09-17T05:34:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 0x7ffa58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7ffa6412db10 con 0x7ffa7c15adc0 2024-09-17T05:34:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.745+0000 7ffa79e00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 0x7ffa58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.749+0000 7ffa79e00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 0x7ffa58075740 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f70 tx=0x7ffa6c0079c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.753+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa640164b0 con 0x7ffa7c15adc0 2024-09-17T05:34:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa7c074150 con 0x7ffa7c15adc0 2024-09-17T05:34:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa72000640 1 -- 172.21.15.43:0/1450538076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) ==== 80+0+0 (secure 0 0 0) 0x7ffa640f8a90 con 0x7ffa7c15adc0 2024-09-17T05:34:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 msgr2=0x7ffa58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 0x7ffa58075740 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000f70 tx=0x7ffa6c0079c0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 msgr2=0x7ffa7c09d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ffa640b0040 tx=0x7ffa640a57f0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 shutdown_connections 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ffa58073230 0x7ffa58075740 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa7c072f00 0x7ffa7c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c15adc0 0x7ffa7c09d8b0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 --2- 172.21.15.43:0/1450538076 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ffa7c0cc6d0 0x7ffa7c1f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.937+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 >> 172.21.15.43:0/1450538076 conn(0x7ffa7c0b23d0 msgr2=0x7ffa7c0ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.941+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 shutdown_connections 2024-09-17T05:34:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:57.941+0000 7ffa80c00640 1 -- 172.21.15.43:0/1450538076 wait complete. 2024-09-17T05:34:57.964 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:34:57.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:57.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:58.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:34:58.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:58.026 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:34:58.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:58.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 -- 172.21.15.43:0/765128239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 msgr2=0x7fb33c0cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 --2- 172.21.15.43:0/765128239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c0cbc70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb334009bc0 tx=0x7fb33401cb10 comp rx=0 tx=0).stop 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 -- 172.21.15.43:0/765128239 shutdown_connections 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 --2- 172.21.15.43:0/765128239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c0cbc70 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 --2- 172.21.15.43:0/765128239 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c070620 0x7fb33c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 -- 172.21.15.43:0/765128239 >> 172.21.15.43:0/765128239 conn(0x7fb33c0b23d0 msgr2=0x7fb33c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 -- 172.21.15.43:0/765128239 shutdown_connections 2024-09-17T05:34:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.245+0000 7fb343e00640 1 -- 172.21.15.43:0/765128239 wait complete. 2024-09-17T05:34:58.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 Processor -- start 2024-09-17T05:34:58.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- start start 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c06ce60 0x7fb33c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb33c070620 0x7fb33c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0ce680 con 0x7fb33c0cb860 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0ce500 con 0x7fb33c070620 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c0ce800 con 0x7fb33c06ce60 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41784/0 (socket says 172.21.15.43:41784) 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 -- 172.21.15.43:0/498417793 learned_addr learned my addr 172.21.15.43:0/498417793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341400640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c06ce60 0x7fb33c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 -- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c06ce60 msgr2=0x7fb33c0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c06ce60 0x7fb33c0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 -- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb33c070620 msgr2=0x7fb33c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb33c070620 0x7fb33c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 -- 172.21.15.43:0/498417793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c0cd290 con 0x7fb33c0cb860 2024-09-17T05:34:58.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb341e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb334098500 tx=0x7fb334003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340b2040 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3340bb070 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334004370 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c0ad250 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.249+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c0ad810 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33c0aeea0 con 0x7fb33c0cb860 2024-09-17T05:34:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb334004510 con 0x7fb33c0cb860 2024-09-17T05:34:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb339600640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 0x7fb31c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fb33412eb40 con 0x7fb33c0cb860 2024-09-17T05:34:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb341400640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 0x7fb31c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.253+0000 7fb341400640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 0x7fb31c075740 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb330009a10 tx=0x7fb3300091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:58.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.257+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb334016790 con 0x7fb33c0cb860 2024-09-17T05:34:58.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.465+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33c0ced90 con 0x7fb33c0cb860 2024-09-17T05:34:58.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.465+0000 7fb339600640 1 -- 172.21.15.43:0/498417793 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) ==== 95+0+5233 (secure 0 0 0) 0x7fb3340f9bf0 con 0x7fb33c0cb860 2024-09-17T05:34:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:34:58.470 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":345,"btime":"2024-09-17T05:34:47:511187+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:34:46.927801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10464":{"gid":10464,"name":"c","rank":0,"incarnation":341,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/4038368577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4038368577},{"type":"v1","addr":"172.21.15.43:6834","nonce":4038368577}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10464,"qdb_cluster":[10464]},"id":7}]} 2024-09-17T05:34:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 345 2024-09-17T05:34:58.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 msgr2=0x7fb31c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 0x7fb31c075740 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb330009a10 tx=0x7fb3300091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 msgr2=0x7fb33c09b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb334098500 tx=0x7fb334003f40 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 shutdown_connections 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb31c073230 0x7fb31c075740 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb33c070620 0x7fb33c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c0cb860 0x7fb33c09b990 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 --2- 172.21.15.43:0/498417793 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb33c06ce60 0x7fb33c0ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 >> 172.21.15.43:0/498417793 conn(0x7fb33c0b23d0 msgr2=0x7fb33c0ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 shutdown_connections 2024-09-17T05:34:58.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.469+0000 7fb343e00640 1 -- 172.21.15.43:0/498417793 wait complete. 2024-09-17T05:34:58.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T05:34:58.676 INFO:teuthology.orchestra.run.smithi043.stdout:{"message":"","return_code":0} 2024-09-17T05:34:58.676 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T05:34:58.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:34:58.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- 172.21.15.43:0/2731174684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c003a60 msgr2=0x7f580c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 --2- 172.21.15.43:0/2731174684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c003a60 0x7f580c003ed0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f58180016f0 tx=0x7f581801c790 comp rx=0 tx=0).stop 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- 172.21.15.43:0/2731174684 shutdown_connections 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 --2- 172.21.15.43:0/2731174684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f580c003a60 0x7f580c003ed0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- 172.21.15.43:0/2731174684 >> 172.21.15.43:0/2731174684 conn(0x7f58280d0010 msgr2=0x7f58280d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- 172.21.15.43:0/2731174684 shutdown_connections 2024-09-17T05:34:58.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- 172.21.15.43:0/2731174684 wait complete. 2024-09-17T05:34:58.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 Processor -- start 2024-09-17T05:34:58.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.833+0000 7f582dc00640 1 -- start start 2024-09-17T05:34:58.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c003a60 0x7f582809d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f582809aa50 0x7f58281489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58280706f0 con 0x7f5828157870 2024-09-17T05:34:58.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5828070570 con 0x7f582809aa50 2024-09-17T05:34:58.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f58281603c0 con 0x7f580c003a60 2024-09-17T05:34:58.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41798/0 (socket says 172.21.15.43:41798) 2024-09-17T05:34:58.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 -- 172.21.15.43:0/1065717859 learned_addr learned my addr 172.21.15.43:0/1065717859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:58.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f5827e00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c003a60 0x7f582809d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f581f400640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f582809aa50 0x7f58281489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 -- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c003a60 msgr2=0x7f582809d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c003a60 0x7f582809d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 -- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f582809aa50 msgr2=0x7f58281489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f582809aa50 0x7f58281489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 -- 172.21.15.43:0/1065717859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58281581c0 con 0x7f5828157870 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582d200640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f58280ae950 tx=0x7f582000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:58.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582000ca10 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5820018070 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58281f4c00 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820011460 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58281f51c0 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.837+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582809dc50 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.841+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5820011600 con 0x7f5828157870 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.841+0000 7f5824c00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 0x7f5804075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.841+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f5820095050 con 0x7f5828157870 2024-09-17T05:34:58.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.841+0000 7f5827e00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 0x7f5804075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.841+0000 7f5827e00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 0x7f5804075710 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f58180016c0 tx=0x7f5818019580 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:58.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:58.845+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58281581c0 con 0x7f5828157870 2024-09-17T05:34:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.053+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5828071030 con 0x7f5828157870 2024-09-17T05:34:59.058 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:34:59.053+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:34:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f5824c00640 1 -- 172.21.15.43:0/1065717859 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v346) ==== 161+0+0 (secure 0 0 0) 0x7f582005fa90 con 0x7f5828157870 2024-09-17T05:34:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 msgr2=0x7f5804075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 0x7f5804075710 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f58180016c0 tx=0x7f5818019580 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 msgr2=0x7f5828157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f58280ae950 tx=0x7f582000eb60 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 shutdown_connections 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5804073200 0x7f5804075710 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f582809aa50 0x7f58281489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828157870 0x7f5828157c80 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 --2- 172.21.15.43:0/1065717859 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f580c003a60 0x7f582809d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 >> 172.21.15.43:0/1065717859 conn(0x7f58280d0010 msgr2=0x7f5828159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 shutdown_connections 2024-09-17T05:34:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.057+0000 7f582dc00640 1 -- 172.21.15.43:0/1065717859 wait complete. 2024-09-17T05:34:59.076 INFO:tasks.ceph.mds.c.smithi043.stderr: -7040> 2024-09-17T05:34:50.445+0000 7ff2efa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:59.085 DEBUG:teuthology.orchestra.run.smithi043:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-17T05:34:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- start start 2024-09-17T05:34:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 Processor -- start 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 0x5610daf66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 0x5610daf66b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7c600 con 0x5610da26bb00 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7c480 con 0x5610da26b680 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5610daf7c300 con 0x5610da26c400 2024-09-17T05:34:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41812/0 (socket says 172.21.15.43:41812) 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 -- 172.21.15.43:0/3275153691 learned_addr learned my addr 172.21.15.43:0/3275153691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bda00640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 0x5610daf66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99be400640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 0x5610daf66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 -- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 msgr2=0x5610daf66b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 0x5610daf66b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 -- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 msgr2=0x5610daf66000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 0x5610daf66000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 -- 172.21.15.43:0/3275153691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610da2a0960 con 0x5610da26bb00 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99be400640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 0x5610daf66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bda00640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 0x5610daf66000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bd000640 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x5610daff39b0 tx=0x5610da2a26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6244d0db1ec61d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99ba800640 1 -- 172.21.15.43:0/3275153691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf520 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99ba800640 1 -- 172.21.15.43:0/3275153691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5610da1dc540 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99ba800640 1 -- 172.21.15.43:0/3275153691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf1e0 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610da2a0d20 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5610da2a0f00 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99bf8a8400 4 waiting for MDS map... 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.141+0000 7f99ba800640 1 -- 172.21.15.43:0/3275153691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x5610daf71000 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99ba800640 1 -- 172.21.15.43:0/3275153691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 346) ==== 2893+0+0 (secure 0 0 0) 0x5610da1df180 con 0x5610da26bb00 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 4 Got MDS map 346 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 10 main: JournalTool::main 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 4 main: JournalTool: connecting to RADOS... 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 Processor -- start 2024-09-17T05:34:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 -- start start 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7c180 con 0x5610da26ad80 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610da1e9380 con 0x5610da26a900 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5610da1e9800 con 0x5610da26a480 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41824/0 (socket says 172.21.15.43:41824) 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 -- 172.21.15.43:0/3515581157 learned_addr learned my addr 172.21.15.43:0/3515581157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bda00640 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99bd000640 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 -- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26a480 msgr2=0x5610daf67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 -- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a900 msgr2=0x5610daf67600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 -- 172.21.15.43:0/3515581157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610da2a0d20 con 0x5610da26ad80 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99be400640 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x5610db0acf30 tx=0x5610da4be060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5745160a1440dd01 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99b8000640 1 -- 172.21.15.43:0/3515581157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf520 con 0x5610da26ad80 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99b8000640 1 -- 172.21.15.43:0/3515581157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5610da1dce00 con 0x5610da26ad80 2024-09-17T05:34:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.145+0000 7f99b8000640 1 -- 172.21.15.43:0/3515581157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf1e0 con 0x5610da26ad80 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3515581157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 msgr2=0x5610daf67080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x5610db0acf30 tx=0x5610da4be060 comp rx=0 tx=0).stop 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3515581157 shutdown_connections 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3515581157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3515581157 >> 172.21.15.43:0/3515581157 conn(0x5610da26b200 msgr2=0x5610d6182800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3515581157 shutdown_connections 2024-09-17T05:34:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3515581157 wait complete. 2024-09-17T05:34:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 Processor -- start 2024-09-17T05:34:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- start start 2024-09-17T05:34:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7d380 con 0x5610da26a900 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7c180 con 0x5610da26a480 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5610daf7d500 con 0x5610da26ad80 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41840/0 (socket says 172.21.15.43:41840) 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bda00640 1 -- 172.21.15.43:0/3753989411 learned_addr learned my addr 172.21.15.43:0/3753989411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99bd000640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.149+0000 7f99be400640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 -- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26ad80 msgr2=0x5610daf67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 -- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a480 msgr2=0x5610daf67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 -- 172.21.15.43:0/3753989411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610da2a1a40 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bda00640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x5610db0adc20 tx=0x5610daf7a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 -- 172.21.15.43:0/3753989411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf380 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 -- 172.21.15.43:0/3753989411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5610da1dcc40 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 -- 172.21.15.43:0/3753989411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1bf520 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610da2a1c20 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610da2a01e0 con 0x5610da26a900 2024-09-17T05:34:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bf8a8400 4 main: JournalTool: resolving pool 14 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 -- 172.21.15.43:0/3753989411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x5610da4d0c00 con 0x5610da26a900 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 0x5610daf68100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99b6200640 1 -- 172.21.15.43:0/3753989411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x5610da4c8a00 con 0x5610da26a900 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bf8a8400 4 main: JournalTool: creating IoCtx.. 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99bf8a8400 4 main: Executing for rank 0 2024-09-17T05:34:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.153+0000 7f99be400640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 0x5610daf68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 0x5610daf68100 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5610db0acf30 tx=0x5610da2a2e40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- start start 2024-09-17T05:34:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 Processor -- start 2024-09-17T05:34:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 0x5610daf68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26cd00 0x5610daf68c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26d600 0x5610daf69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7d800 con 0x5610da26c880 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5610daf7d980 con 0x5610da26cd00 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5610da1e9800 con 0x5610da26d600 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 0x5610daf68680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 0x5610daf68680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41844/0 (socket says 172.21.15.43:41844) 2024-09-17T05:34:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 -- 172.21.15.43:0/1959722039 learned_addr learned my addr 172.21.15.43:0/1959722039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:59.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bda00640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26cd00 0x5610daf68c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bd000640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26d600 0x5610daf69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26d600 msgr2=0x5610daf69180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26d600 0x5610daf69180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26cd00 msgr2=0x5610daf68c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26cd00 0x5610daf68c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610da61a5a0 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99be400640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 0x5610daf68680 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x5610da622150 tx=0x5610da5d2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cd3153d78ef9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99b2600640 1 -- 172.21.15.43:0/1959722039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da1be820 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99b2600640 1 -- 172.21.15.43:0/1959722039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5610da1dc000 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99b2600640 1 -- 172.21.15.43:0/1959722039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5610da505d40 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610da61a780 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5610da61a960 con 0x5610da26c880 2024-09-17T05:34:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.157+0000 7f99bf8a8400 4 waiting for MDS map... 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99b2600640 1 -- 172.21.15.43:0/1959722039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x5610da62e000 con 0x5610da26c880 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99b2600640 1 -- 172.21.15.43:0/1959722039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 346) ==== 2893+0+0 (secure 0 0 0) 0x5610da1df500 con 0x5610da26c880 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99bf8a8400 4 Got MDS map 346 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610da26da80 0x5610daf69700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e140) -- 0x5610da4c7c00 con 0x5610da26da80 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99bda00640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610da26da80 0x5610daf69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.161+0000 7f99bda00640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610da26da80 0x5610daf69700 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=40a885f8ed48fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5610da292000 con 0x5610da26da80 2024-09-17T05:34:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e140) -- 0x5610da642000 con 0x5610da26da80 2024-09-17T05:34:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv21 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5610da292000 con 0x5610da26da80 2024-09-17T05:34:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99b0800640 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e140) -- 0x5610da642c00 con 0x5610da26da80 2024-09-17T05:34:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99b0800640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5610da646000 0x5610da64e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99b0800640 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e140) -- 0x5610da643000 con 0x5610da646000 2024-09-17T05:34:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.1 v2:172.21.15.43:6816/1939991414 3 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5610da292000 con 0x5610da26da80 2024-09-17T05:34:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bd000640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5610da646000 0x5610da64e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.165+0000 7f99bd000640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5610da646000 0x5610da64e000 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=c85cc3b462f94db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292500 con 0x5610da646000 2024-09-17T05:34:59.173 INFO:teuthology.orchestra.run.smithi043.stdout:old journal was 4225216~2030 2024-09-17T05:34:59.173 INFO:teuthology.orchestra.run.smithi043.stdout:new journal start will be 8388608 (4161362 bytes past old end) 2024-09-17T05:34:59.173 INFO:teuthology.orchestra.run.smithi043.stdout:writing journal head 2024-09-17T05:34:59.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- osd_op(unknown.0.0:5 14.14 14:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da642400 con 0x5610da26da80 2024-09-17T05:34:59.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.1 v2:172.21.15.43:6816/1939991414 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v140'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5610da292000 con 0x5610da26da80 2024-09-17T05:34:59.174 INFO:teuthology.orchestra.run.smithi043.stdout:writing EResetJournal entry 2024-09-17T05:34:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:6 14.1b 14:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da642800 con 0x5610da646000 2024-09-17T05:34:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:7 14.b 14:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da643800 con 0x5610da646000 2024-09-17T05:34:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610da646480 0x5610da64e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- osd_op(unknown.0.0:8 14.1f 14:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da643c00 con 0x5610da646480 2024-09-17T05:34:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x5610da646900 0x5610da64eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:9 14.1a 14:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da643000 con 0x5610da646900 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610da646d80 0x5610da64f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- osd_op(unknown.0.0:10 14.1e 14:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da642800 con 0x5610da646d80 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- osd_op(unknown.0.0:11 14.1d 14:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da643800 con 0x5610da646900 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bd000640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610da646d80 0x5610da64f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.169+0000 7f99bda00640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x5610da646900 0x5610da64eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99be400640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610da646480 0x5610da64e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bd000640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610da646d80 0x5610da64f080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=fb2220328e5af9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99be400640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610da646480 0x5610da64e580 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=de3f7563376d4db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bda00640 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x5610da646900 0x5610da64eb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=c03bcc25aac38903 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== osd_op_reply(7 200.00000003 [delete] v140'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292500 con 0x5610da646000 2024-09-17T05:34:59.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 <== osd.6 v2:172.21.15.62:6802/2388352512 3 ==== osd_op_reply(6 200.00000002 [delete] v140'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292500 con 0x5610da646000 2024-09-17T05:34:59.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99b0800640 1 -- 172.21.15.43:0/1959722039 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- osd_op(unknown.0.0:12 14.1b 14:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140) -- 0x5610da643400 con 0x5610da646000 2024-09-17T05:34:59.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== osd_op_reply(10 200.00000006 [delete] v140'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292500 con 0x5610da646d80 2024-09-17T05:34:59.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.4 v2:172.21.15.62:6818/1078882315 1 ==== osd_op_reply(11 200.00000007 [delete] v140'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292000 con 0x5610da646900 2024-09-17T05:34:59.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.173+0000 7f99bda00640 1 -- 172.21.15.43:0/1959722039 <== osd.4 v2:172.21.15.62:6818/1078882315 2 ==== osd_op_reply(9 200.00000005 [delete] v140'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292000 con 0x5610da646900 2024-09-17T05:34:59.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 <== osd.6 v2:172.21.15.62:6802/2388352512 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v140'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5610da292500 con 0x5610da646000 2024-09-17T05:34:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99be400640 1 -- 172.21.15.43:0/1959722039 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== osd_op_reply(8 200.00000004 [delete] v140'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5610da292780 con 0x5610da646480 2024-09-17T05:34:59.182 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-09-17T05:34:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610da26da80 msgr2=0x5610daf69700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x5610da26da80 0x5610daf69700 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610da646d80 msgr2=0x5610da64f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x5610da646d80 0x5610da64f080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x5610da646900 msgr2=0x5610da64eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x5610da646900 0x5610da64eb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5610da646000 msgr2=0x5610da64e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x5610da646000 0x5610da64e000 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610da646480 msgr2=0x5610da64e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x5610da646480 0x5610da64e580 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 msgr2=0x5610daf68680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/1959722039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26c880 0x5610daf68680 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x5610da622150 tx=0x5610da5d2e40 comp rx=0 tx=0).stop 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 reap_dead start 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 reap_dead start 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 reap_dead start 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bd000640 1 -- 172.21.15.43:0/1959722039 reap_dead start 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 shutdown_connections 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 >> 172.21.15.43:0/1959722039 conn(0x5610da26a000 msgr2=0x5610da5ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 shutdown_connections 2024-09-17T05:34:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/1959722039 wait complete. 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 msgr2=0x5610daf68100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 0x5610daf68100 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5610db0acf30 tx=0x5610da2a2e40 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 msgr2=0x5610daf67600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x5610db0adc20 tx=0x5610daf7a360 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 shutdown_connections 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x5610da26d180 0x5610daf68100 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26ad80 0x5610daf67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26a900 0x5610daf67600 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3753989411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26a480 0x5610daf67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 >> 172.21.15.43:0/3753989411 conn(0x5610da26b200 msgr2=0x5610d6182800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 shutdown_connections 2024-09-17T05:34:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.177+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3753989411 wait complete. 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 msgr2=0x5610daf66580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x5610daff39b0 tx=0x5610da2a26c0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 shutdown_connections 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5610da26c400 0x5610daf66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5610da26bb00 0x5610daf66580 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 --2- 172.21.15.43:0/3275153691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610da26b680 0x5610daf66000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 >> 172.21.15.43:0/3275153691 conn(0x5610da26bf80 msgr2=0x5610d6183800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 shutdown_connections 2024-09-17T05:34:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.181+0000 7f99bf8a8400 1 -- 172.21.15.43:0/3275153691 wait complete. 2024-09-17T05:34:59.190 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.105238, result: old journal was 4225216~2030 new journal start will be 8388608 (4161362 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-17T05:34:59.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T05:34:59.204 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:34:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 -- 172.21.15.43:0/3448560549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 msgr2=0x7f369815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 --2- 172.21.15.43:0/3448560549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f369815be00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3688009840 tx=0x7f368801c740 comp rx=0 tx=0).stop 2024-09-17T05:34:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 -- 172.21.15.43:0/3448560549 shutdown_connections 2024-09-17T05:34:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 --2- 172.21.15.43:0/3448560549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f369815be00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 -- 172.21.15.43:0/3448560549 >> 172.21.15.43:0/3448560549 conn(0x7f369806d9f0 msgr2=0x7f36980b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 -- 172.21.15.43:0/3448560549 shutdown_connections 2024-09-17T05:34:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.349+0000 7f3697e00640 1 -- 172.21.15.43:0/3448560549 wait complete. 2024-09-17T05:34:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 Processor -- start 2024-09-17T05:34:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 -- start start 2024-09-17T05:34:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36980af0c0 0x7f36980b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f36980b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3698169ba0 con 0x7f369815c340 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f369815c890 con 0x7f369815b9f0 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3698169d20 con 0x7f36980af0c0 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41888/0 (socket says 172.21.15.43:41888) 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 -- 172.21.15.43:0/386917393 learned_addr learned my addr 172.21.15.43:0/386917393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3696a00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36980af0c0 0x7f36980b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f368fe00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f36980b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 -- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36980af0c0 msgr2=0x7f36980b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36980af0c0 0x7f36980b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 -- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 msgr2=0x7f36980b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f36980b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 -- 172.21.15.43:0/386917393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36980b12f0 con 0x7f369815c340 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f3697400640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f3690001710 tx=0x7f369000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.353+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690016020 con 0x7f369815c340 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f369000ab60 con 0x7f369815c340 2024-09-17T05:34:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36980b15d0 con 0x7f369815c340 2024-09-17T05:34:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690012420 con 0x7f369815c340 2024-09-17T05:34:59.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36980af900 con 0x7f369815c340 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369815be00 con 0x7f369815c340 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3690019020 con 0x7f369815c340 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f368ea00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 0x7f3674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f3690094600 con 0x7f369815c340 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.357+0000 7f3696a00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 0x7f3674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.361+0000 7f3696a00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 0x7f3674075710 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f36880af000 tx=0x7f36880b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.361+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3690020020 con 0x7f369815c340 2024-09-17T05:34:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:34:59.581+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f36980b1010 con 0x7f369815c340 2024-09-17T05:35:00.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.057+0000 7f368ea00640 1 -- 172.21.15.43:0/386917393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v347) ==== 162+0+0 (secure 0 0 0) 0x7f369005eed0 con 0x7f369815c340 2024-09-17T05:35:00.062 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T05:35:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 msgr2=0x7f3674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 0x7f3674075710 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f36880af000 tx=0x7f36880b2040 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 msgr2=0x7f3698156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f3690001710 tx=0x7f369000cc80 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 shutdown_connections 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3674073200 0x7f3674075710 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f369815b9f0 0x7f36980b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369815c340 0x7f3698156ef0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 --2- 172.21.15.43:0/386917393 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f36980af0c0 0x7f36980b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 >> 172.21.15.43:0/386917393 conn(0x7f369806d9f0 msgr2=0x7f36980b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 shutdown_connections 2024-09-17T05:35:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.061+0000 7f3697e00640 1 -- 172.21.15.43:0/386917393 wait complete. 2024-09-17T05:35:00.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:00.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 -- 172.21.15.43:0/2040980377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0910 msgr2=0x7f43ec06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 --2- 172.21.15.43:0/2040980377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0910 0x7f43ec06ce60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009840 tx=0x7f43d801c790 comp rx=0 tx=0).stop 2024-09-17T05:35:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 -- 172.21.15.43:0/2040980377 shutdown_connections 2024-09-17T05:35:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 --2- 172.21.15.43:0/2040980377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0910 0x7f43ec06ce60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 -- 172.21.15.43:0/2040980377 >> 172.21.15.43:0/2040980377 conn(0x7f43ec0b23d0 msgr2=0x7f43ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 -- 172.21.15.43:0/2040980377 shutdown_connections 2024-09-17T05:35:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.293+0000 7f43f4000640 1 -- 172.21.15.43:0/2040980377 wait complete. 2024-09-17T05:35:00.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 Processor -- start 2024-09-17T05:35:00.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 -- start start 2024-09-17T05:35:00.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 0x7f43ec158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0d20 0x7f43ec1fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec0ce830 con 0x7f43ec0d0910 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec0ce6b0 con 0x7f43ec0d0d20 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f4000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f43ec0ce9b0 con 0x7f43ec070620 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41904/0 (socket says 172.21.15.43:41904) 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 -- 172.21.15.43:0/1719645124 learned_addr learned my addr 172.21.15.43:0/1719645124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f2000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0d20 0x7f43ec1fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f1600640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 0x7f43ec158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 -- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 msgr2=0x7f43ec158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 0x7f43ec158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 -- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0d20 msgr2=0x7f43ec1fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0d20 0x7f43ec1fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 -- 172.21.15.43:0/1719645124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec0cb4e0 con 0x7f43ec0d0910 2024-09-17T05:35:00.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f1600640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 0x7f43ec158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:00.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.297+0000 7f43f0c00640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c10 tx=0x7f43d80a8650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80a8aa0 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43d80a8c40 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80050c0 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec0c97f0 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec0c9db0 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec0c9560 con 0x7f43ec0d0910 2024-09-17T05:35:00.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43d8005260 con 0x7f43ec0d0910 2024-09-17T05:35:00.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.301+0000 7f43e9600640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 0x7f43cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.305+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f43d812e510 con 0x7f43ec0d0910 2024-09-17T05:35:00.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.305+0000 7f43f1600640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 0x7f43cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.305+0000 7f43f1600640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 0x7f43cc075710 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000f70 tx=0x7f43e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.309+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43d80c1030 con 0x7f43ec0d0910 2024-09-17T05:35:00.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43ec06ce60 con 0x7f43ec0d0910 2024-09-17T05:35:00.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43e9600640 1 -- 172.21.15.43:0/1719645124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 348 v348) ==== 95+0+5226 (secure 0 0 0) 0x7f43d80ac030 con 0x7f43ec0d0910 2024-09-17T05:35:00.525 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:00.525 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":348,"btime":"2024-09-17T05:35:00:065325+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":348,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:00.065297+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T05:35:00.525 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 348 2024-09-17T05:35:00.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 msgr2=0x7f43cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 0x7f43cc075710 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000f70 tx=0x7f43e0009040 comp rx=0 tx=0).stop 2024-09-17T05:35:00.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 msgr2=0x7f43ec0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000c10 tx=0x7f43d80a8650 comp rx=0 tx=0).stop 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 shutdown_connections 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f43cc073200 0x7f43cc075710 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ec0d0d20 0x7f43ec1fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.521+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec0d0910 0x7f43ec0cafa0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.525+0000 7f43f4000640 1 --2- 172.21.15.43:0/1719645124 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f43ec070620 0x7f43ec158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.525+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 >> 172.21.15.43:0/1719645124 conn(0x7f43ec0b23d0 msgr2=0x7f43ec0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.525+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 shutdown_connections 2024-09-17T05:35:00.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:00.525+0000 7f43f4000640 1 -- 172.21.15.43:0/1719645124 wait complete. 2024-09-17T05:35:00.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 348, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:34:43.887777+0000', 'modified': '2024-09-17T05:35:00.065297+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 140, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10036': {'gid': 10036, 'name': 'c', 'rank': 0, 'incarnation': 348, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6834/1735972562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1735972562}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 1735972562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:35:00.950 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10036:up:replay 2024-09-17T05:35:01.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- 172.21.15.43:0/4076061990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49200b3af0 msgr2=0x7f492015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 --2- 172.21.15.43:0/4076061990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49200b3af0 0x7f492015c7e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4908009900 tx=0x7f490801c820 comp rx=0 tx=0).stop 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- 172.21.15.43:0/4076061990 shutdown_connections 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 --2- 172.21.15.43:0/4076061990 >> v2:172.21.15.62:3300/0 conn(0x7f492006f110 0x7f49200ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 --2- 172.21.15.43:0/4076061990 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f49200b3af0 0x7f492015c7e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- 172.21.15.43:0/4076061990 >> 172.21.15.43:0/4076061990 conn(0x7f49200b23d0 msgr2=0x7f49200d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:02.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- 172.21.15.43:0/4076061990 shutdown_connections 2024-09-17T05:35:02.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- 172.21.15.43:0/4076061990 wait complete. 2024-09-17T05:35:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 Processor -- start 2024-09-17T05:35:02.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.121+0000 7f4926800640 1 -- start start 2024-09-17T05:35:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492006f110 0x7f49200c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920155d70 0x7f492009d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4920168cb0 con 0x7f492009da00 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4920168b30 con 0x7f4920155d70 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f4920168e30 con 0x7f492006f110 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41908/0 (socket says 172.21.15.43:41908) 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 -- 172.21.15.43:0/3306785686 learned_addr learned my addr 172.21.15.43:0/3306785686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491f400640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492006f110 0x7f49200c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4916a00640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920155d70 0x7f492009d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 -- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492006f110 msgr2=0x7f49200c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492006f110 0x7f49200c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 -- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920155d70 msgr2=0x7f492009d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920155d70 0x7f492009d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 -- 172.21.15.43:0/3306785686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200af9a0 con 0x7f492009da00 2024-09-17T05:35:02.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f491fe00640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f491800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491800cdc0 con 0x7f492009da00 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4918004b30 con 0x7f492009da00 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49200c8650 con 0x7f492009da00 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918005020 con 0x7f492009da00 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.125+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49200c7350 con 0x7f492009da00 2024-09-17T05:35:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49200ad0b0 con 0x7f492009da00 2024-09-17T05:35:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f49180051c0 con 0x7f492009da00 2024-09-17T05:35:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f4917e00640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 0x7f48fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f4918094e60 con 0x7f492009da00 2024-09-17T05:35:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f491f400640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 0x7f48fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.129+0000 7f491f400640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 0x7f48fc075710 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4910007fd0 tx=0x7f4910009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.133+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f491805f730 con 0x7f492009da00 2024-09-17T05:35:02.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f492006e070 con 0x7f492009da00 2024-09-17T05:35:02.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4917e00640 1 -- 172.21.15.43:0/3306785686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5226 (secure 0 0 0) 0x7f4918064160 con 0x7f492009da00 2024-09-17T05:35:02.349 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:02.349 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":350,"btime":"2024-09-17T05:35:02:070481+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:01.073150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T05:35:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 350 2024-09-17T05:35:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 msgr2=0x7f48fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 0x7f48fc075710 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4910007fd0 tx=0x7f4910009040 comp rx=0 tx=0).stop 2024-09-17T05:35:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 msgr2=0x7f49200c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f491800ef10 comp rx=0 tx=0).stop 2024-09-17T05:35:02.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 shutdown_connections 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f48fc073200 0x7f48fc075710 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4920155d70 0x7f492009d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f492009da00 0x7f49200c88b0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 --2- 172.21.15.43:0/3306785686 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f492006f110 0x7f49200c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 >> 172.21.15.43:0/3306785686 conn(0x7f49200b23d0 msgr2=0x7f49200af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 shutdown_connections 2024-09-17T05:35:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:02.345+0000 7f4926800640 1 -- 172.21.15.43:0/3306785686 wait complete. 2024-09-17T05:35:02.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 350, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:34:43.887777+0000', 'modified': '2024-09-17T05:35:01.073150+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 140, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10036': {'gid': 10036, 'name': 'c', 'rank': 0, 'incarnation': 348, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.43:6834/1735972562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1735972562}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 1735972562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T05:35:02.774 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10036:up:rejoin 2024-09-17T05:35:03.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:03.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 -- 172.21.15.43:0/1225310040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a940737c0 msgr2=0x7f1a940700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 --2- 172.21.15.43:0/1225310040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a940737c0 0x7f1a940700a0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009900 tx=0x7f1a8c01c780 comp rx=0 tx=0).stop 2024-09-17T05:35:03.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 -- 172.21.15.43:0/1225310040 shutdown_connections 2024-09-17T05:35:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 --2- 172.21.15.43:0/1225310040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a940737c0 0x7f1a940700a0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 --2- 172.21.15.43:0/1225310040 >> v2:172.21.15.62:3300/0 conn(0x7f1a9415fe10 0x7f1a940b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 -- 172.21.15.43:0/1225310040 >> 172.21.15.43:0/1225310040 conn(0x7f1a940b23d0 msgr2=0x7f1a940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 -- 172.21.15.43:0/1225310040 shutdown_connections 2024-09-17T05:35:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.941+0000 7f1a98a00640 1 -- 172.21.15.43:0/1225310040 wait complete. 2024-09-17T05:35:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 Processor -- start 2024-09-17T05:35:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 -- start start 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a9406fc90 0x7f1a941f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a9415fe10 0x7f1a9415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94169fc0 con 0x7f1a94159550 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94169e40 con 0x7f1a9415fe10 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a98a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9416a140 con 0x7f1a9406fc90 2024-09-17T05:35:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a92800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a92800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37470/0 (socket says 172.21.15.43:37470) 2024-09-17T05:35:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.945+0000 7f1a92800640 1 -- 172.21.15.43:0/2835448026 learned_addr learned my addr 172.21.15.43:0/2835448026 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a89400640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a9415fe10 0x7f1a9415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a91e00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a9406fc90 0x7f1a941f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 -- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a9406fc90 msgr2=0x7f1a941f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a9406fc90 0x7f1a941f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 -- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a9415fe10 msgr2=0x7f1a9415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a9415fe10 0x7f1a9415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 -- 172.21.15.43:0/2835448026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a941f7620 con 0x7f1a94159550 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a92800640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c01a3b0 tx=0x7f1a8c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0bb070 con 0x7f1a94159550 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a8c004320 con 0x7f1a94159550 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0b2040 con 0x7f1a94159550 2024-09-17T05:35:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a941f7b20 con 0x7f1a94159550 2024-09-17T05:35:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a94153840 con 0x7f1a94159550 2024-09-17T05:35:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a940700a0 con 0x7f1a94159550 2024-09-17T05:35:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.949+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a8c0052d0 con 0x7f1a94159550 2024-09-17T05:35:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.953+0000 7f1a8aa00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 0x7f1a70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.953+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f1a8c12dac0 con 0x7f1a94159550 2024-09-17T05:35:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.953+0000 7f1a91e00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 0x7f1a70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.953+0000 7f1a91e00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 0x7f1a70075710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009b40 tx=0x7f1a840091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:03.957+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1a8c016490 con 0x7f1a94159550 2024-09-17T05:35:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a940aee40 con 0x7f1a94159550 2024-09-17T05:35:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a8aa00640 1 -- 172.21.15.43:0/2835448026 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7f1a8c0f8310 con 0x7f1a94159550 2024-09-17T05:35:04.169 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:04.170 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 msgr2=0x7f1a70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 0x7f1a70075710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009b40 tx=0x7f1a840091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 msgr2=0x7f1a9415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.165+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c01a3b0 tx=0x7f1a8c005d60 comp rx=0 tx=0).stop 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 shutdown_connections 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f1a70073200 0x7f1a70075710 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a9415fe10 0x7f1a9415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94159550 0x7f1a9415b620 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 --2- 172.21.15.43:0/2835448026 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f1a9406fc90 0x7f1a941f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 >> 172.21.15.43:0/2835448026 conn(0x7f1a940b23d0 msgr2=0x7f1a94073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 shutdown_connections 2024-09-17T05:35:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.169+0000 7f1a98a00640 1 -- 172.21.15.43:0/2835448026 wait complete. 2024-09-17T05:35:04.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 351, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:34:43.887777+0000', 'modified': '2024-09-17T05:35:03.076405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 140, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10036': {'gid': 10036, 'name': 'c', 'rank': 0, 'incarnation': 348, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6834/1735972562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1735972562}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 1735972562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10036, 'qdb_cluster': [10036]} 2024-09-17T05:35:04.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:35:04.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:35:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- 172.21.15.43:0/3753969228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c155d70 msgr2=0x7f9f2c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 --2- 172.21.15.43:0/3753969228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c155d70 0x7f9f2c15c7e0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0d0c40 tx=0x7f9f1c01c800 comp rx=0 tx=0).stop 2024-09-17T05:35:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- 172.21.15.43:0/3753969228 shutdown_connections 2024-09-17T05:35:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 --2- 172.21.15.43:0/3753969228 >> v2:172.21.15.62:3300/0 conn(0x7f9f2c0b3af0 0x7f9f2c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 --2- 172.21.15.43:0/3753969228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c155d70 0x7f9f2c15c7e0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- 172.21.15.43:0/3753969228 >> 172.21.15.43:0/3753969228 conn(0x7f9f2c0b23d0 msgr2=0x7f9f2c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- 172.21.15.43:0/3753969228 shutdown_connections 2024-09-17T05:35:04.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- 172.21.15.43:0/3753969228 wait complete. 2024-09-17T05:35:04.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 Processor -- start 2024-09-17T05:35:04.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.757+0000 7f9f33600640 1 -- start start 2024-09-17T05:35:04.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 0x7f9f2c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 0x7f9f2c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c168b60 con 0x7f9f2c06f110 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1689e0 con 0x7f9f2c160530 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c168ce0 con 0x7f9f2c155d70 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37488/0 (socket says 172.21.15.43:37488) 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 -- 172.21.15.43:0/2300844049 learned_addr learned my addr 172.21.15.43:0/2300844049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 0x7f9f2c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f31600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 0x7f9f2c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 -- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 msgr2=0x7f9f2c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 0x7f9f2c0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 -- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 msgr2=0x7f9f2c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 0x7f9f2c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 -- 172.21.15.43:0/2300844049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c09d3b0 con 0x7f9f2c06f110 2024-09-17T05:35:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 0x7f9f2c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f31600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 0x7f9f2c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f30c00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0b0040 tx=0x7f9f1c0a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f29600640 1 -- 172.21.15.43:0/2300844049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0bd070 con 0x7f9f2c06f110 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f29600640 1 -- 172.21.15.43:0/2300844049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f1c0043b0 con 0x7f9f2c06f110 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f29600640 1 -- 172.21.15.43:0/2300844049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0b2040 con 0x7f9f2c06f110 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c09d650 con 0x7f9f2c06f110 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.761+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c09dc10 con 0x7f9f2c06f110 2024-09-17T05:35:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f29600640 1 -- 172.21.15.43:0/2300844049 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f1c005290 con 0x7f9f2c06f110 2024-09-17T05:35:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f29600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 0x7f9f08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f29600640 1 -- 172.21.15.43:0/2300844049 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f9f1c12e170 con 0x7f9f2c06f110 2024-09-17T05:35:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 0x7f9f08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 Processor -- start 2024-09-17T05:35:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 -- start start 2024-09-17T05:35:04.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 0x7f9f08075710 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0cdca0 tx=0x7f9f14009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 0x56023b72b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5c3ca0 0x56023b72c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56023b600410 con 0x56023b5da950 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56023b53db40 con 0x56023b1ea460 2024-09-17T05:35:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.765+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56023b52cf70 con 0x56023b5c3ca0 2024-09-17T05:35:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37490/0 (socket says 172.21.15.43:37490) 2024-09-17T05:35:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 0x56023b72b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 -- 172.21.15.43:0/2718752524 learned_addr learned my addr 172.21.15.43:0/2718752524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f31600640 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5c3ca0 0x56023b72c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 -- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5c3ca0 msgr2=0x56023b72c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5c3ca0 0x56023b72c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 -- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 msgr2=0x56023b72b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 0x56023b72b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 -- 172.21.15.43:0/2718752524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56023b4f45b0 con 0x56023b5da950 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f30c00640 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 0x56023b72b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f23e00640 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9f14001e30 tx=0x7f9f14011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f6582686550023 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f21200640 1 -- 172.21.15.43:0/2718752524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003600 con 0x56023b5da950 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f21200640 1 -- 172.21.15.43:0/2718752524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f14003890 con 0x56023b5da950 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f21200640 1 -- 172.21.15.43:0/2718752524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003cc0 con 0x56023b5da950 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- 172.21.15.43:0/2718752524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 msgr2=0x56023b72bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f9f14001e30 tx=0x7f9f14011000 comp rx=0 tx=0).stop 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- 172.21.15.43:0/2718752524 shutdown_connections 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5c3ca0 0x56023b72c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b72bbe0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/2718752524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b1ea460 0x56023b72b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- 172.21.15.43:0/2718752524 >> 172.21.15.43:0/2718752524 conn(0x56023b6b0d90 msgr2=0x56023b6e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- 172.21.15.43:0/2718752524 shutdown_connections 2024-09-17T05:35:04.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- 172.21.15.43:0/2718752524 wait complete. 2024-09-17T05:35:04.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 -- start start 2024-09-17T05:35:04.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.769+0000 7f9f3acbc000 1 Processor -- start 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b732e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56023b4c75f0 con 0x56023b1ea460 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56023b530d80 con 0x56023b5c3ca0 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56023b4d3520 con 0x56023b5da950 2024-09-17T05:35:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 10 client.0 _ll_register_callbacks cb 0x56023b4d7830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37494/0 (socket says 172.21.15.43:37494) 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 -- 172.21.15.43:0/1237686210 learned_addr learned my addr 172.21.15.43:0/1237686210 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f31600640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f23e00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b732e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 -- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 msgr2=0x56023b7333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 -- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 msgr2=0x56023b732e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b732e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 -- 172.21.15.43:0/1237686210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56023b4cb190 con 0x56023b1ea460 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f31600640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f30c00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c019410 tx=0x7f9f1c0f88c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37084dc2a831ff1e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0fd310 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f9f1c102130 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56023b4c5110 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c0f8540 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 10 client.10734 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x56023b5b1f10 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56023b4c5110 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f9f1c11a030 con 0x56023b1ea460 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 10 client.10734 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 10 client.10734 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 20 client.10734 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1cf7d0 0x56023b73cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 4 client.10734 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56023b53fa90 con 0x56023b1cf7d0 2024-09-17T05:35:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f23e00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1cf7d0 0x56023b73cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f23e00640 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1cf7d0 0x56023b73cd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa04fd10d2f5718 server_cookie=dd7e1a3d8de4c39c in_seq=0 out_seq=0 2024-09-17T05:35:04.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.773+0000 7f9f19600640 10 client.10734 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f19600640 1 -- 172.21.15.43:0/1237686210 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56023b53fa90 con 0x56023b1cf7d0 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f19600640 10 client.10734 handle_command_reply: tid=0 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 client.10734 shutdown 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 msgr2=0x56023b732920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c019410 tx=0x7f9f1c0f88c0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 shutdown_connections 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1cf7d0 0x56023b73cd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b732e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1237686210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b732920 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f19600640 0 client.10734 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 >> 172.21.15.43:0/1237686210 conn(0x56023b6b0d90 msgr2=0x56023b6e7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 shutdown_connections 2024-09-17T05:35:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1237686210 wait complete. 2024-09-17T05:35:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.777+0000 7f9f3acbc000 20 client.10734 trim_cache size 0 max 16384 2024-09-17T05:35:04.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 Processor -- start 2024-09-17T05:35:04.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- start start 2024-09-17T05:35:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b1ea460 0x56023b79d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b79dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c2053d0 con 0x56023b5da950 2024-09-17T05:35:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c204470 con 0x56023b5c3ca0 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c1636c0 con 0x56023b1ea460 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37508/0 (socket says 172.21.15.43:37508) 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 -- 172.21.15.43:0/1477972358 learned_addr learned my addr 172.21.15.43:0/1477972358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f30c00640 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b1ea460 0x56023b79d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f23e00640 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b79dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 -- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b1ea460 msgr2=0x56023b79d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b1ea460 0x56023b79d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 -- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 msgr2=0x56023b79dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b79dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 -- 172.21.15.43:0/1477972358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56023b588db0 con 0x56023b5da950 2024-09-17T05:35:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f31600640 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011600 tx=0x7f9f24011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a568c81ca5d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f1be00640 1 -- 172.21.15.43:0/1477972358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24020070 con 0x56023b5da950 2024-09-17T05:35:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f1be00640 1 -- 172.21.15.43:0/1477972358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f240108e0 con 0x56023b5da950 2024-09-17T05:35:04.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f1be00640 1 -- 172.21.15.43:0/1477972358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010cb0 con 0x56023b5da950 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1477972358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 msgr2=0x56023b79e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011600 tx=0x7f9f24011ac0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1477972358 shutdown_connections 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b79dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b5da950 0x56023b79e2c0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/1477972358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b1ea460 0x56023b79d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.797+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1477972358 >> 172.21.15.43:0/1477972358 conn(0x56023b6b0d90 msgr2=0x56023b6dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1477972358 shutdown_connections 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- 172.21.15.43:0/1477972358 wait complete. 2024-09-17T05:35:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- start start 2024-09-17T05:35:04.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 Processor -- start 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b7a1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56023b4c75f0 con 0x56023b1ea460 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c163d10 con 0x56023b5c3ca0 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56023b530d80 con 0x56023b5da950 2024-09-17T05:35:04.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f3acbc000 10 client.0 _ll_register_callbacks cb 0x7f9f1c0f48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37512/0 (socket says 172.21.15.43:37512) 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 -- 172.21.15.43:0/966978496 learned_addr learned my addr 172.21.15.43:0/966978496 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f31600640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f23e00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b7a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 -- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 msgr2=0x56023b7a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 -- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 msgr2=0x56023b7a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b7a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 -- 172.21.15.43:0/966978496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56023b4f3ab0 con 0x56023b1ea460 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f30c00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c01cd70 tx=0x7f9f1c11a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a919d7ad1e89d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f19600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.801+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0f81a0 con 0x56023b1ea460 2024-09-17T05:35:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f9f1c10eb50 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f240108e0 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c0f9130 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 10 client.10740 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x56023b4d2070 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f240108e0 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f9f1c0f4380 con 0x56023b1ea460 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 10 client.10740 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 10 client.10740 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 20 client.10740 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1bd830 0x56023b75d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 4 client.10740 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:35:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "status"}) -- 0x56023b53b540 con 0x56023b1bd830 2024-09-17T05:35:04.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f23e00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1bd830 0x56023b75d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 10 client.10740 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f23e00640 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1bd830 0x56023b75d1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e0bb0192a99417 server_cookie=46caafdeae970107 in_seq=0 out_seq=0 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 1 -- 172.21.15.43:0/966978496 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x56023b53b540 con 0x56023b1bd830 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 10 client.10740 handle_command_reply: tid=0 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 client.10740 shutdown 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 msgr2=0x56023b7a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c01cd70 tx=0x7f9f1c11a280 comp rx=0 tx=0).stop 2024-09-17T05:35:04.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 shutdown_connections 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56023b1bd830 0x56023b75d1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56023b5c3ca0 0x56023b7a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56023b5da950 0x56023b7a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 --2- 172.21.15.43:0/966978496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56023b1ea460 0x56023b7a13d0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 >> 172.21.15.43:0/966978496 conn(0x56023b6b0d90 msgr2=0x56023b6e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f19600640 0 client.10740 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 shutdown_connections 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 1 -- 172.21.15.43:0/966978496 wait complete. 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f3acbc000 20 client.10740 trim_cache size 0 max 16384 2024-09-17T05:35:04.813 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "1cac0027-7a1b-472a-9fc8-e0027668aa6d", 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10036, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.7375673110000003, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 351, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 140, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 140, 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 5.601814708 2024-09-17T05:35:04.814 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 msgr2=0x7f9f08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 0x7f9f08075710 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0cdca0 tx=0x7f9f14009040 comp rx=0 tx=0).stop 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 msgr2=0x7f9f2c09c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0b0040 tx=0x7f9f1c0a5740 comp rx=0 tx=0).stop 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 shutdown_connections 2024-09-17T05:35:04.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9f08073200 0x7f9f08075710 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f2c160530 0x7f9f2c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9f2c155d70 0x7f9f2c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 --2- 172.21.15.43:0/2300844049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f2c06f110 0x7f9f2c09c090 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 >> 172.21.15.43:0/2300844049 conn(0x7f9f2c0b23d0 msgr2=0x7f9f2c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 shutdown_connections 2024-09-17T05:35:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.805+0000 7f9f33600640 1 -- 172.21.15.43:0/2300844049 wait complete. 2024-09-17T05:35:04.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:35:04.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- 172.21.15.43:0/706100914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c06f110 msgr2=0x7f744c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 --2- 172.21.15.43:0/706100914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c06f110 0x7f744c0b33e0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f744c071050 tx=0x7f743401c800 comp rx=0 tx=0).stop 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- 172.21.15.43:0/706100914 shutdown_connections 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 --2- 172.21.15.43:0/706100914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c06f110 0x7f744c0b33e0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 --2- 172.21.15.43:0/706100914 >> v2:172.21.15.62:3300/0 conn(0x7f744c0b3920 0x7f744c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 --2- 172.21.15.43:0/706100914 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c155d70 0x7f744c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- 172.21.15.43:0/706100914 >> 172.21.15.43:0/706100914 conn(0x7f744c0b23d0 msgr2=0x7f744c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- 172.21.15.43:0/706100914 shutdown_connections 2024-09-17T05:35:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- 172.21.15.43:0/706100914 wait complete. 2024-09-17T05:35:04.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 Processor -- start 2024-09-17T05:35:04.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.985+0000 7f7453200640 1 -- start start 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c06f110 0x7f744c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c0b3920 0x7f744c1f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f744c15f870 con 0x7f744c155d70 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f744c15f6f0 con 0x7f744c06f110 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f744c15f9f0 con 0x7f744c0b3920 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37538/0 (socket says 172.21.15.43:37538) 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 -- 172.21.15.43:0/2295926802 learned_addr learned my addr 172.21.15.43:0/2295926802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f744be00640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c06f110 0x7f744c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7443e00640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c0b3920 0x7f744c1f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 -- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c0b3920 msgr2=0x7f744c1f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c0b3920 0x7f744c1f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 -- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c06f110 msgr2=0x7f744c153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c06f110 0x7f744c153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 -- 172.21.15.43:0/2295926802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c1ed170 con 0x7f744c155d70 2024-09-17T05:35:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7451200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f744400d9a0 tx=0x7f744400de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444017070 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7444004e20 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744400bd50 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c0c72b0 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c1ed410 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.989+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744c15fea0 con 0x7f744c155d70 2024-09-17T05:35:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.993+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7444003980 con 0x7f744c155d70 2024-09-17T05:35:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.993+0000 7f7448c00640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 0x7f7428075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.993+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5795+0+0 (secure 0 0 0) 0x7f74440949c0 con 0x7f744c155d70 2024-09-17T05:35:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.993+0000 7f744be00640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 0x7f7428075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.993+0000 7f744be00640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 0x7f7428075710 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f744c0715c0 tx=0x7f743c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:04.997+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7444020020 con 0x7f744c155d70 2024-09-17T05:35:05.073 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:35:05.069+0000 7ff593e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:35:05.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f744c0b00c0 con 0x7f744c155d70 2024-09-17T05:35:05.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7448c00640 1 -- 172.21.15.43:0/2295926802 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v351) ==== 53+0+73 (secure 0 0 0) 0x7f7444003c70 con 0x7f744c155d70 2024-09-17T05:35:05.213 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 msgr2=0x7f7428075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 0x7f7428075710 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f744c0715c0 tx=0x7f743c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 msgr2=0x7f744c1ed930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f744400d9a0 tx=0x7f744400de60 comp rx=0 tx=0).stop 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 shutdown_connections 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7428073200 0x7f7428075710 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c155d70 0x7f744c1ed930 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f744c0b3920 0x7f744c1f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 --2- 172.21.15.43:0/2295926802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f744c06f110 0x7f744c153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 >> 172.21.15.43:0/2295926802 conn(0x7f744c0b23d0 msgr2=0x7f744c15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:05.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 shutdown_connections 2024-09-17T05:35:05.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05.209+0000 7f7453200640 1 -- 172.21.15.43:0/2295926802 wait complete. 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:35:05.235 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:35:05.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:35:05.236 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:35:05.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.236 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:35:05.246 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41739sec preferred_lft 41739sec 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:35:05.247 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:a2:99:98:70 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:35:05.248 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.249 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:35:05.249 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:35:05.249 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:35:05.249 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:35:05.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.249 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:35:05.309 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-17T05:35:05.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.311 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-17T05:35:05.368 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-17T05:35:05.369 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:35:05.369 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:35:05.369 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:35:05.369 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:35:05.369 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:35:05.369 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:35:05.369 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-17T05:35:05.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:35:05.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:35:05.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:35:05.370 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-17T05:35:05.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:35:05.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:35:05.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.507 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:35:05.517 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:35:05.517 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:35:05.517 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:35:05.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-17T05:35:05.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:35:05.579 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:35:05.635 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:35:05.635 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:35:05.635 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:35:05.690 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:35:05.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:35:05.793 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:35:05.793 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:35:05.793 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:35:05.811 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:35:05.811 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:35:05.811 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:35:05.811 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:35:05.811 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:35:05.820 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:35:05.820 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:35:05.820 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2024-09-17T05:35:05.821 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.62:3300,172.21.15.62:3301" 2024-09-17T05:35:05.821 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:35:05.821 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790" 2024-09-17T05:35:05.821 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.62:6789,172.21.15.62:6790:/ 2024-09-17T05:35:05.821 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1cac0027-7a1b-472a-9fc8-e0027668aa6d" will pass to kernel 2024-09-17T05:35:05.824 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:35:05.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:05.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:35:05.844 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:35:05.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:35:05.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:35:05.948 INFO:teuthology.orchestra.run.smithi043.stdout:client10749 2024-09-17T05:35:05.948 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:05.948 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:05.999 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-17T05:35:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.101+0000 7f7b89536080 1 Processor -- start 2024-09-17T05:35:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.101+0000 7f7b89536080 1 -- start start 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 0x560932b32100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932b9fe40 0x560932bbca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560932a37a00 con 0x560932b18cb0 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560932a95f40 con 0x560932ae9d70 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560932ba0890 con 0x560932b9fe40 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37554/0 (socket says 172.21.15.43:37554) 2024-09-17T05:35:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 -- 172.21.15.43:0/3266633233 learned_addr learned my addr 172.21.15.43:0/3266633233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b82600640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 0x560932b32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b83000640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932b9fe40 0x560932bbca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 -- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932b9fe40 msgr2=0x560932bbca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932b9fe40 0x560932bbca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 -- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 msgr2=0x560932b32100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 0x560932b32100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 -- 172.21.15.43:0/3266633233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560932a10c10 con 0x560932b18cb0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b82600640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 0x560932b32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b81c00640 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000b250 tx=0x7f7b7000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c6174875302dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b7be00640 1 -- 172.21.15.43:0/3266633233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000a040 con 0x560932b18cb0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b7be00640 1 -- 172.21.15.43:0/3266633233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b7000f620 con 0x560932b18cb0 2024-09-17T05:35:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b7be00640 1 -- 172.21.15.43:0/3266633233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000fa70 con 0x560932b18cb0 2024-09-17T05:35:06.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- 172.21.15.43:0/3266633233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 msgr2=0x560932bbe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000b250 tx=0x7f7b7000b710 comp rx=0 tx=0).stop 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- 172.21.15.43:0/3266633233 shutdown_connections 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932b9fe40 0x560932bbca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b18cb0 0x560932bbe140 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 --2- 172.21.15.43:0/3266633233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932ae9d70 0x560932b32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- 172.21.15.43:0/3266633233 >> 172.21.15.43:0/3266633233 conn(0x560932ad5d80 msgr2=0x560932b183c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- 172.21.15.43:0/3266633233 shutdown_connections 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.105+0000 7f7b89536080 1 -- 172.21.15.43:0/3266633233 wait complete. 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 Processor -- start 2024-09-17T05:35:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- start start 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932ae9d70 0x560932bbc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932b18cb0 0x560932b14030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 0x560932b14570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560932bbf820 con 0x560932b9fe40 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560932bbfac0 con 0x560932b18cb0 2024-09-17T05:35:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560932b9f9d0 con 0x560932ae9d70 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 0x560932b14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 0x560932b14570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37566/0 (socket says 172.21.15.43:37566) 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 -- 172.21.15.43:0/1055588628 learned_addr learned my addr 172.21.15.43:0/1055588628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b82600640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932ae9d70 0x560932bbc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b81c00640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932b18cb0 0x560932b14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 -- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932ae9d70 msgr2=0x560932bbc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560932ae9d70 0x560932bbc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 -- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932b18cb0 msgr2=0x560932b14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560932b18cb0 0x560932b14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 -- 172.21.15.43:0/1055588628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560932b14ab0 con 0x560932b9fe40 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b83000640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 0x560932b14570 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c010820 tx=0x7f7b7c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b7aa00640 1 -- 172.21.15.43:0/1055588628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00ecd0 con 0x560932b9fe40 2024-09-17T05:35:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b7aa00640 1 -- 172.21.15.43:0/1055588628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b7c015070 con 0x560932b9fe40 2024-09-17T05:35:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b7aa00640 1 -- 172.21.15.43:0/1055588628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0116b0 con 0x560932b9fe40 2024-09-17T05:35:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560932b14d40 con 0x560932b9fe40 2024-09-17T05:35:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.109+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560932b15280 con 0x560932b9fe40 2024-09-17T05:35:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b7aa00640 1 -- 172.21.15.43:0/1055588628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b7c017030 con 0x560932b9fe40 2024-09-17T05:35:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b7aa00640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7b64071be0 0x7f7b640740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b7aa00640 1 -- 172.21.15.43:0/1055588628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5853+0+0 (secure 0 0 0) 0x7f7b7c093c20 con 0x560932b9fe40 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b89536080 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560932b10620 0x560932b2e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b82600640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7b64071be0 0x7f7b640740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 15.7 15:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e141) -- 0x560932b2e890 con 0x560932b10620 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b81c00640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560932b10620 0x560932b2e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b82600640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7b64071be0 0x7f7b640740f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7b740af000 tx=0x7f7b7401c250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.113+0000 7f7b81c00640 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560932b10620 0x560932b2e350 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b81c00640 1 -- 172.21.15.43:0/1055588628 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7f7b7000d040 con 0x560932b10620 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560932b10620 msgr2=0x560932b2e350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x560932b10620 0x560932b2e350 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7b64071be0 msgr2=0x7f7b640740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7b64071be0 0x7f7b640740f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7b740af000 tx=0x7f7b7401c250 comp rx=0 tx=0).stop 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 msgr2=0x560932b14570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 --2- 172.21.15.43:0/1055588628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560932b9fe40 0x560932b14570 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c010820 tx=0x7f7b7c010ce0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b83000640 1 -- 172.21.15.43:0/1055588628 reap_dead start 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 shutdown_connections 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 >> 172.21.15.43:0/1055588628 conn(0x560932ad5d80 msgr2=0x560932bbca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 shutdown_connections 2024-09-17T05:35:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.117+0000 7f7b89536080 1 -- 172.21.15.43:0/1055588628 wait complete. 2024-09-17T05:35:06.125 INFO:teuthology.orchestra.run.smithi043.stdout:Ufile1dir1 2024-09-17T05:35:06.125 DEBUG:teuthology.orchestra.run.smithi043:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-17T05:35:06.192 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511627777, 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "ancestors": [ 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "dirino": 1099511627776, 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "dname": "file1", 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 8 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "dirino": 1, 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "dname": "dir1", 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 29 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:35:06.193 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-17T05:35:06.194 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 15, 2024-09-17T05:35:06.194 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [] 2024-09-17T05:35:06.194 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:06.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:06.198 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2024-09-17T05:35:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.301+0000 7f583304e080 1 Processor -- start 2024-09-17T05:35:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.301+0000 7f583304e080 1 -- start start 2024-09-17T05:35:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898907d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559898881330 0x559898904fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559898877500 con 0x55989887f910 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559898786a00 con 0x559898881330 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559898836000 con 0x559898877960 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37592/0 (socket says 172.21.15.43:37592) 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 -- 172.21.15.43:0/2284047373 learned_addr learned my addr 172.21.15.43:0/2284047373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582cc00640 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559898881330 0x559898904fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582c200640 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898907d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 -- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 msgr2=0x559898907d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898907d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 -- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559898881330 msgr2=0x559898904fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559898881330 0x559898904fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 -- 172.21.15.43:0/2284047373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559898785a20 con 0x55989887f910 2024-09-17T05:35:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582b800640 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f581800c590 tx=0x7f581800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5f0f4f6e5275fc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582a400640 1 -- 172.21.15.43:0/2284047373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581800a9f0 con 0x55989887f910 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582a400640 1 -- 172.21.15.43:0/2284047373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f581800ab90 con 0x55989887f910 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f582a400640 1 -- 172.21.15.43:0/2284047373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818015440 con 0x55989887f910 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- 172.21.15.43:0/2284047373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 msgr2=0x55989890b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f581800c590 tx=0x7f581800c990 comp rx=0 tx=0).stop 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- 172.21.15.43:0/2284047373 shutdown_connections 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559898881330 0x559898904fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55989887f910 0x55989890b5a0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 --2- 172.21.15.43:0/2284047373 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898907d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- 172.21.15.43:0/2284047373 >> 172.21.15.43:0/2284047373 conn(0x559898824d80 msgr2=0x559898909350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- 172.21.15.43:0/2284047373 shutdown_connections 2024-09-17T05:35:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- 172.21.15.43:0/2284047373 wait complete. 2024-09-17T05:35:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 Processor -- start 2024-09-17T05:35:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.305+0000 7f583304e080 1 -- start start 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898900a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55989887f910 0x559898905e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 0x5598989a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55989887d500 con 0x559898881330 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559898867290 con 0x55989887f910 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559898867530 con 0x559898877960 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 0x5598989a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582c200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898900a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 0x5598989a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37594/0 (socket says 172.21.15.43:37594) 2024-09-17T05:35:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 -- 172.21.15.43:0/4021647810 learned_addr learned my addr 172.21.15.43:0/4021647810 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582b800640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55989887f910 0x559898905e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 -- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 msgr2=0x559898900a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559898877960 0x559898900a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 -- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55989887f910 msgr2=0x559898905e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55989887f910 0x559898905e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 -- 172.21.15.43:0/4021647810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989875fc10 con 0x559898881330 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f582cc00640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 0x5598989a9aa0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f582400f6b0 tx=0x7f582400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f5829000640 1 -- 172.21.15.43:0/4021647810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824004220 con 0x559898881330 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f5829000640 1 -- 172.21.15.43:0/4021647810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5824014070 con 0x559898881330 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598989aa500 con 0x559898881330 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f5829000640 1 -- 172.21.15.43:0/4021647810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400cb20 con 0x559898881330 2024-09-17T05:35:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.309+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559898901370 con 0x559898881330 2024-09-17T05:35:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f5829000640 1 -- 172.21.15.43:0/4021647810 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5824016030 con 0x559898881330 2024-09-17T05:35:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f5829000640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f580c071be0 0x7f580c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f5829000640 1 -- 172.21.15.43:0/4021647810 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f5824093990 con 0x559898881330 2024-09-17T05:35:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f582c200640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f580c071be0 0x7f580c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f583304e080 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5598989a9fe0 0x559898901ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- osd_op(unknown.0.0:1 15.7 15:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142) -- 0x559898901ff0 con 0x5598989a9fe0 2024-09-17T05:35:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f582b800640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5598989a9fe0 0x559898901ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f582c200640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f580c071be0 0x7f580c0740f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ab000 tx=0x7f581c002040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.313+0000 7f582b800640 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5598989a9fe0 0x559898901ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f582b800640 1 -- 172.21.15.43:0/4021647810 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v142'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f581800a7f0 con 0x5598989a9fe0 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5598989a9fe0 msgr2=0x559898901ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x5598989a9fe0 0x559898901ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f580c071be0 msgr2=0x7f580c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f580c071be0 0x7f580c0740f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ab000 tx=0x7f581c002040 comp rx=0 tx=0).stop 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 msgr2=0x5598989a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 --2- 172.21.15.43:0/4021647810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559898881330 0x5598989a9aa0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f582400f6b0 tx=0x7f582400fb70 comp rx=0 tx=0).stop 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f582cc00640 1 -- 172.21.15.43:0/4021647810 reap_dead start 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 shutdown_connections 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 >> 172.21.15.43:0/4021647810 conn(0x559898824d80 msgr2=0x559898907ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 shutdown_connections 2024-09-17T05:35:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.317+0000 7f583304e080 1 -- 172.21.15.43:0/4021647810 wait complete. 2024-09-17T05:35:06.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:06.327 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2024-09-17T05:35:06.338 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:35:06.333+0000 7ff58ee00640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-17T05:35:06.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:35:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2024-09-17T05:35:06.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:06.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 -- 172.21.15.43:0/1455174712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b43c0 msgr2=0x7feda80b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 --2- 172.21.15.43:0/1455174712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b43c0 0x7feda80b47d0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7feda80738f0 tx=0x7feda001cd50 comp rx=0 tx=0).stop 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 -- 172.21.15.43:0/1455174712 shutdown_connections 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 --2- 172.21.15.43:0/1455174712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b43c0 0x7feda80b47d0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 --2- 172.21.15.43:0/1455174712 >> v2:172.21.15.62:3300/0 conn(0x7feda80b5fe0 0x7feda815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 -- 172.21.15.43:0/1455174712 >> 172.21.15.43:0/1455174712 conn(0x7feda80b2e80 msgr2=0x7feda80b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 -- 172.21.15.43:0/1455174712 shutdown_connections 2024-09-17T05:35:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.537+0000 7fedae600640 1 -- 172.21.15.43:0/1455174712 wait complete. 2024-09-17T05:35:06.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 Processor -- start 2024-09-17T05:35:06.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- start start 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda80b43c0 0x7feda814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 0x7feda80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feda8169110 con 0x7feda80b5fe0 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feda8168f90 con 0x7feda815fa70 2024-09-17T05:35:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7feda8169290 con 0x7feda80b43c0 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37616/0 (socket says 172.21.15.43:37616) 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 -- 172.21.15.43:0/3435144999 learned_addr learned my addr 172.21.15.43:0/3435144999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7feda7400640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda80b43c0 0x7feda814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7feda7e00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 0x7feda80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 -- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda80b43c0 msgr2=0x7feda814e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda80b43c0 0x7feda814e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 -- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 msgr2=0x7feda80c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 0x7feda80c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 -- 172.21.15.43:0/3435144999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda80cb520 con 0x7feda80b5fe0 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7feda7e00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 0x7feda80c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9ea00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b440 tx=0x7fed9000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90015020 con 0x7feda80b5fe0 2024-09-17T05:35:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed90011980 con 0x7feda80b5fe0 2024-09-17T05:35:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90011dd0 con 0x7feda80b5fe0 2024-09-17T05:35:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda80ca220 con 0x7feda80b5fe0 2024-09-17T05:35:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.541+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda80ca7e0 con 0x7feda80b5fe0 2024-09-17T05:35:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.545+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda80c88b0 con 0x7feda80b5fe0 2024-09-17T05:35:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.545+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fed90018020 con 0x7feda80b5fe0 2024-09-17T05:35:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.545+0000 7fed9fe00640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 0x7fed84075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.545+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fed90094770 con 0x7feda80b5fe0 2024-09-17T05:35:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.545+0000 7feda7400640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 0x7fed84075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.549+0000 7feda7400640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 0x7fed84075740 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7feda8073990 tx=0x7fed9800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.549+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fed9001f020 con 0x7feda80b5fe0 2024-09-17T05:35:06.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.765+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feda80cf380 con 0x7feda80b5fe0 2024-09-17T05:35:06.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.765+0000 7fed9fe00640 1 -- 172.21.15.43:0/3435144999 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7feda80cb520 con 0x7feda80b5fe0 2024-09-17T05:35:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:06.771 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:06.771 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 msgr2=0x7fed84075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 0x7fed84075740 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7feda8073990 tx=0x7fed9800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 msgr2=0x7feda80cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed9000b440 tx=0x7fed9000b900 comp rx=0 tx=0).stop 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 shutdown_connections 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fed84073230 0x7fed84075740 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feda815fa70 0x7feda80c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda80b5fe0 0x7feda80cafe0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 --2- 172.21.15.43:0/3435144999 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7feda80b43c0 0x7feda814e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 >> 172.21.15.43:0/3435144999 conn(0x7feda80b2e80 msgr2=0x7feda8070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 shutdown_connections 2024-09-17T05:35:06.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.769+0000 7fedae600640 1 -- 172.21.15.43:0/3435144999 wait complete. 2024-09-17T05:35:06.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T05:35:06.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 -- 172.21.15.43:0/2840638200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 msgr2=0x7fa4e40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2840638200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e40ad0b0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009c80 tx=0x7fa4e001cce0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 -- 172.21.15.43:0/2840638200 shutdown_connections 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2840638200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e40ad0b0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2840638200 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e40b3af0 0x7fa4e415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 -- 172.21.15.43:0/2840638200 >> 172.21.15.43:0/2840638200 conn(0x7fa4e40b23d0 msgr2=0x7fa4e40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 -- 172.21.15.43:0/2840638200 shutdown_connections 2024-09-17T05:35:06.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.949+0000 7fa4ec800640 1 -- 172.21.15.43:0/2840638200 wait complete. 2024-09-17T05:35:06.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 Processor -- start 2024-09-17T05:35:06.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- start start 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e4155d70 0x7fa4e408f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e408f550 0x7fa4e408f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4168c80 con 0x7fa4e406f110 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4168b00 con 0x7fa4e4155d70 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e4168e00 con 0x7fa4e408f550 2024-09-17T05:35:06.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37638/0 (socket says 172.21.15.43:37638) 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 -- 172.21.15.43:0/2707620572 learned_addr learned my addr 172.21.15.43:0/2707620572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ea800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e408f550 0x7fa4e408f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9400640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e4155d70 0x7fa4e408f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 -- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e408f550 msgr2=0x7fa4e408f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e408f550 0x7fa4e408f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 -- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e4155d70 msgr2=0x7fa4e408f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e4155d70 0x7fa4e408f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 -- 172.21.15.43:0/2707620572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e4157810 con 0x7fa4e406f110 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800b410 tx=0x7fa4d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4de000640 1 -- 172.21.15.43:0/2707620572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8015030 con 0x7fa4e406f110 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4de000640 1 -- 172.21.15.43:0/2707620572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d8004020 con 0x7fa4e406f110 2024-09-17T05:35:06.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4de000640 1 -- 172.21.15.43:0/2707620572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8004470 con 0x7fa4e406f110 2024-09-17T05:35:06.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e4157af0 con 0x7fa4e406f110 2024-09-17T05:35:06.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.953+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e41599c0 con 0x7fa4e406f110 2024-09-17T05:35:06.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4de000640 1 -- 172.21.15.43:0/2707620572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4d8018070 con 0x7fa4e406f110 2024-09-17T05:35:06.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4de000640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 0x7fa4c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4de000640 1 -- 172.21.15.43:0/2707620572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fa4d8093cc0 con 0x7fa4e406f110 2024-09-17T05:35:06.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4f3f16000 1 Processor -- start 2024-09-17T05:35:06.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4f3f16000 1 -- start start 2024-09-17T05:35:06.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4e9400640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 0x7fa4c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.957+0000 7fa4e9400640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 0x7fa4c8075740 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40058d0 tx=0x7fa4d4005860 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646dd489d00 con 0x5646dd4a5b60 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646dd3fdc90 con 0x5646dd49b1c0 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5646dd3ed0c0 con 0x5646dd484d90 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37652/0 (socket says 172.21.15.43:37652) 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 -- 172.21.15.43:0/3125130784 learned_addr learned my addr 172.21.15.43:0/3125130784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4ea800640 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9400640 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 -- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 msgr2=0x5646dd5ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 -- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 msgr2=0x5646dd5eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 -- 172.21.15.43:0/3125130784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dd3c8290 con 0x5646dd4a5b60 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80920c0 tx=0x7fa4d805ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e0d46a77b2a9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4d2600640 1 -- 172.21.15.43:0/3125130784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8067f60 con 0x5646dd4a5b60 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4d2600640 1 -- 172.21.15.43:0/3125130784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d805d830 con 0x5646dd4a5b60 2024-09-17T05:35:06.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4d2600640 1 -- 172.21.15.43:0/3125130784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80652c0 con 0x5646dd4a5b60 2024-09-17T05:35:06.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3125130784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 msgr2=0x5646dd5eb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80920c0 tx=0x7fa4d805ecd0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3125130784 shutdown_connections 2024-09-17T05:35:06.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3125130784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5eb2e0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3125130784 >> 172.21.15.43:0/3125130784 conn(0x5646dd0aa760 msgr2=0x5646dd5ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3125130784 shutdown_connections 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3125130784 wait complete. 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 -- start start 2024-09-17T05:35:06.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.961+0000 7fa4f3f16000 1 Processor -- start 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 0x5646dd5cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646dd387940 con 0x5646dd484d90 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646dd3f0ed0 con 0x5646dd49b1c0 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5646dd0b2820 con 0x5646dd4a5b60 2024-09-17T05:35:06.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 10 client.0 _ll_register_callbacks cb 0x5646dd3cb7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37658/0 (socket says 172.21.15.43:37658) 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 -- 172.21.15.43:0/885417947 learned_addr learned my addr 172.21.15.43:0/885417947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4ea800640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 0x5646dd5cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9400640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 -- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 msgr2=0x5646dd5cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 0x5646dd5cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 -- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 msgr2=0x5646dd5a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 -- 172.21.15.43:0/885417947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dd3a0550 con 0x5646dd484d90 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4ea800640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 0x5646dd5cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9400640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:06.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8067400 tx=0x7fa4d805aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432480fa968ec361 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d807d060 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fa4d80652c0 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646dd385400 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7fa4d80654e0 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 10 client.10782 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x5646dd38b480 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5646dd385400 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7fa4d8022060 con 0x5646dd484d90 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 10 client.10782 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 10 client.10782 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 20 client.10782 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:06.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd08fad0 0x5646dd5fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 4 client.10782 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:06.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646dd3ffbe0 con 0x5646dd08fad0 2024-09-17T05:35:06.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9400640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd08fad0 0x5646dd5fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4e9400640 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd08fad0 0x5646dd5fc8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22735ea87eca3cd8 server_cookie=56effb066ed44327 in_seq=0 out_seq=0 2024-09-17T05:35:06.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.965+0000 7fa4c6a00640 10 client.10782 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:06.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4c6a00640 1 -- 172.21.15.43:0/885417947 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646dd3ffbe0 con 0x5646dd08fad0 2024-09-17T05:35:06.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4c6a00640 10 client.10782 handle_command_reply: tid=0 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 client.10782 shutdown 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 msgr2=0x5646dd5a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8067400 tx=0x7fa4d805aaf0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 shutdown_connections 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd08fad0 0x5646dd5fc8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd4a5b60 0x5646dd5cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/885417947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd484d90 0x5646dd5a7070 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4c6a00640 0 client.10782 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:06.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 >> 172.21.15.43:0/885417947 conn(0x5646dd0aa760 msgr2=0x5646dd59f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 shutdown_connections 2024-09-17T05:35:06.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 1 -- 172.21.15.43:0/885417947 wait complete. 2024-09-17T05:35:06.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.969+0000 7fa4f3f16000 20 client.10782 trim_cache size 0 max 16384 2024-09-17T05:35:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.981+0000 7fa4f3f16000 1 Processor -- start 2024-09-17T05:35:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.981+0000 7fa4f3f16000 1 -- start start 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5a1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646dd5ab510 con 0x5646dd4a5b60 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646dd398300 con 0x5646dd49b1c0 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5646dd3fca60 con 0x5646dd484d90 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37660/0 (socket says 172.21.15.43:37660) 2024-09-17T05:35:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 -- 172.21.15.43:0/1682423150 learned_addr learned my addr 172.21.15.43:0/1682423150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 -- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 msgr2=0x5646dd5a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4e9400640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 -- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 msgr2=0x5646dd5a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 -- 172.21.15.43:0/1682423150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dd472d10 con 0x5646dd4a5b60 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4e9400640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4ea800640 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009c80 tx=0x7fa4e0019a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0ebe4ba7a44467 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4d1c00640 1 -- 172.21.15.43:0/1682423150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b2040 con 0x5646dd4a5b60 2024-09-17T05:35:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4d1c00640 1 -- 172.21.15.43:0/1682423150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e0003a70 con 0x5646dd4a5b60 2024-09-17T05:35:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4d1c00640 1 -- 172.21.15.43:0/1682423150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003e40 con 0x5646dd4a5b60 2024-09-17T05:35:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- 172.21.15.43:0/1682423150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 msgr2=0x5646dd5a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009c80 tx=0x7fa4e0019a80 comp rx=0 tx=0).stop 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- 172.21.15.43:0/1682423150 shutdown_connections 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd5a2110 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/1682423150 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- 172.21.15.43:0/1682423150 >> 172.21.15.43:0/1682423150 conn(0x5646dd0aa760 msgr2=0x5646dd5cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- 172.21.15.43:0/1682423150 shutdown_connections 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.985+0000 7fa4f3f16000 1 -- 172.21.15.43:0/1682423150 wait complete. 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- start start 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 Processor -- start 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5feed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5646dd3b9300 con 0x5646dd4a5b60 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646dd3aa670 con 0x5646dd49b1c0 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5646dd3f01e0 con 0x5646dd484d90 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 10 client.0 _ll_register_callbacks cb 0x7fa4d8077070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37670/0 (socket says 172.21.15.43:37670) 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 -- 172.21.15.43:0/3845071155 learned_addr learned my addr 172.21.15.43:0/3845071155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4e9400640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 -- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 msgr2=0x5646dd5feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 -- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 msgr2=0x5646dd5ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 -- 172.21.15.43:0/3845071155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dd3a3e50 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4ea800640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001c750 tx=0x7fa4e00a5c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53818b07ed98a491 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00c3020 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fa4e0004200 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e0003a70 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7fa4e00cd050 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 10 client.10788 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x5646dd3922d0 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e0003a70 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.989+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7fa4e00d0020 con 0x5646dd4a5b60 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 10 client.10788 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 10 client.10788 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 20 client.10788 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd07db30 0x5646dd61d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 4 client.10788 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T05:35:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5646dd3fb690 con 0x5646dd07db30 2024-09-17T05:35:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd07db30 0x5646dd61d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4c6a00640 10 client.10788 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4e9e00640 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd07db30 0x5646dd61d780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfa5ef0bef68d1b server_cookie=92d9bcf55ea4a1a4 in_seq=0 out_seq=0 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4c6a00640 1 -- 172.21.15.43:0/3845071155 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+145 (crc 0 0 0) 0x5646dd3fb690 con 0x5646dd07db30 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4c6a00640 10 client.10788 handle_command_reply: tid=0 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 client.10788 shutdown 2024-09-17T05:35:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 msgr2=0x5646dd61cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001c750 tx=0x7fa4e00a5c70 comp rx=0 tx=0).stop 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 shutdown_connections 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5646dd07db30 0x5646dd61d780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646dd49b1c0 0x5646dd5ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5646dd4a5b60 0x5646dd61cdc0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 --2- 172.21.15.43:0/3845071155 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5646dd484d90 0x5646dd5feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4c6a00640 0 client.10788 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 >> 172.21.15.43:0/3845071155 conn(0x5646dd0aa760 msgr2=0x5646dd5f7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 shutdown_connections 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 1 -- 172.21.15.43:0/3845071155 wait complete. 2024-09-17T05:35:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4f3f16000 20 client.10788 trim_cache size 0 max 16384 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "backtrace", 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 1253238885, 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511627777, 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/testdir/hardlink1" 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:35:07.002 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 msgr2=0x7fa4c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 0x7fa4c8075740 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40058d0 tx=0x7fa4d4005860 comp rx=0 tx=0).stop 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 msgr2=0x7fa4e4159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800b410 tx=0x7fa4d800b8d0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 shutdown_connections 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa4c8073230 0x7fa4c8075740 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e4155d70 0x7fa4e408f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa4e408f550 0x7fa4e408f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 --2- 172.21.15.43:0/2707620572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4e406f110 0x7fa4e4159fe0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 >> 172.21.15.43:0/2707620572 conn(0x7fa4e40b23d0 msgr2=0x7fa4e4074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 shutdown_connections 2024-09-17T05:35:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:06.993+0000 7fa4ec800640 1 -- 172.21.15.43:0/2707620572 wait complete. 2024-09-17T05:35:07.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- 172.21.15.43:0/3508270071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815fe10 msgr2=0x7fcb180b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 --2- 172.21.15.43:0/3508270071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815fe10 0x7fcb180b4b80 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009c80 tx=0x7fcb0801cc40 comp rx=0 tx=0).stop 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- 172.21.15.43:0/3508270071 shutdown_connections 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 --2- 172.21.15.43:0/3508270071 >> v2:172.21.15.62:3300/0 conn(0x7fcb180737c0 0x7fcb180700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 --2- 172.21.15.43:0/3508270071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815fe10 0x7fcb180b4b80 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- 172.21.15.43:0/3508270071 >> 172.21.15.43:0/3508270071 conn(0x7fcb180b23d0 msgr2=0x7fcb180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- 172.21.15.43:0/3508270071 shutdown_connections 2024-09-17T05:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- 172.21.15.43:0/3508270071 wait complete. 2024-09-17T05:35:07.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 Processor -- start 2024-09-17T05:35:07.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.173+0000 7fcb1e200640 1 -- start start 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 0x7fcb18098790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1815fe10 0x7fcb180c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1816a020 con 0x7fcb1815c780 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18169ea0 con 0x7fcb1806fc90 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1816a1a0 con 0x7fcb1815fe10 2024-09-17T05:35:07.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37682/0 (socket says 172.21.15.43:37682) 2024-09-17T05:35:07.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 -- 172.21.15.43:0/2171250419 learned_addr learned my addr 172.21.15.43:0/2171250419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17400640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 0x7fcb18098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb0ea00640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1815fe10 0x7fcb180c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 -- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1815fe10 msgr2=0x7fcb180c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1815fe10 0x7fcb180c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 -- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 msgr2=0x7fcb18098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 0x7fcb18098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 -- 172.21.15.43:0/2171250419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb18099170 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17400640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 0x7fcb18098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb17e00640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e770 tx=0x7fcb1000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000cc60 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb1000ce00 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1815d450 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005280 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb180c3520 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.177+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb18098ee0 con 0x7fcb1815c780 2024-09-17T05:35:07.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.181+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb10005420 con 0x7fcb1815c780 2024-09-17T05:35:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.181+0000 7fcb0fe00640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 0x7fcaf4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.181+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fcb10094a60 con 0x7fcb1815c780 2024-09-17T05:35:07.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.181+0000 7fcb17400640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 0x7fcaf4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.181+0000 7fcb17400640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 0x7fcaf4075740 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcb080af000 tx=0x7fcb0801b480 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.185+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb1005f390 con 0x7fcb1815c780 2024-09-17T05:35:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.397+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb180700a0 con 0x7fcb1815c780 2024-09-17T05:35:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb0fe00640 1 -- 172.21.15.43:0/2171250419 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7fcb10063dc0 con 0x7fcb1815c780 2024-09-17T05:35:07.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:07.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 msgr2=0x7fcaf4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 0x7fcaf4075740 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcb080af000 tx=0x7fcb0801b480 comp rx=0 tx=0).stop 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 msgr2=0x7fcb1815cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e770 tx=0x7fcb1000ec30 comp rx=0 tx=0).stop 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 shutdown_connections 2024-09-17T05:35:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcaf4073230 0x7fcaf4075740 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb1815c780 0x7fcb1815cb90 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb1815fe10 0x7fcb180c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 --2- 172.21.15.43:0/2171250419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb1806fc90 0x7fcb18098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 >> 172.21.15.43:0/2171250419 conn(0x7fcb180b23d0 msgr2=0x7fcb1815ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 shutdown_connections 2024-09-17T05:35:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.401+0000 7fcb1e200640 1 -- 172.21.15.43:0/2171250419 wait complete. 2024-09-17T05:35:07.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1253238885 2024-09-17T05:35:07.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 -- 172.21.15.43:0/784845118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 msgr2=0x7f28480d2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 --2- 172.21.15.43:0/784845118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28480d2190 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f28480d1390 tx=0x7f283c01c770 comp rx=0 tx=0).stop 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 -- 172.21.15.43:0/784845118 shutdown_connections 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 --2- 172.21.15.43:0/784845118 >> v2:172.21.15.62:3300/0 conn(0x7f28480cf320 0x7f2848167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 --2- 172.21.15.43:0/784845118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28480d2190 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 -- 172.21.15.43:0/784845118 >> 172.21.15.43:0/784845118 conn(0x7f28480691b0 msgr2=0x7f28480695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 -- 172.21.15.43:0/784845118 shutdown_connections 2024-09-17T05:35:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.585+0000 7f284ee00640 1 -- 172.21.15.43:0/784845118 wait complete. 2024-09-17T05:35:07.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 Processor -- start 2024-09-17T05:35:07.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- start start 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 0x7f284806e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28480d26d0 0x7f28481596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2848169f90 con 0x7f284815c610 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b80d0 con 0x7f28480cf320 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f284816a110 con 0x7f28480d26d0 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37712/0 (socket says 172.21.15.43:37712) 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 -- 172.21.15.43:0/1253952869 learned_addr learned my addr 172.21.15.43:0/1253952869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284d000640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28480d26d0 0x7f28481596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284da00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 0x7f284806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 -- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28480d26d0 msgr2=0x7f28481596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28480d26d0 0x7f28481596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 -- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 msgr2=0x7f284806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 0x7f284806e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 -- 172.21.15.43:0/1253952869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284806fd60 con 0x7f284815c610 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284da00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 0x7f284806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284e400640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f284000e940 tx=0x7f284000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f2845600640 1 -- 172.21.15.43:0/1253952869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840004d30 con 0x7f284815c610 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f2845600640 1 -- 172.21.15.43:0/1253952869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f284000cb20 con 0x7f284815c610 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28481fc270 con 0x7f284815c610 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f2845600640 1 -- 172.21.15.43:0/1253952869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840005020 con 0x7f284815c610 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.589+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2848159ef0 con 0x7f284815c610 2024-09-17T05:35:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f2845600640 1 -- 172.21.15.43:0/1253952869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f28400051c0 con 0x7f284815c610 2024-09-17T05:35:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f2845600640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 0x7f2828075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f2845600640 1 -- 172.21.15.43:0/1253952869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f2840094bc0 con 0x7f284815c610 2024-09-17T05:35:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f2857c77000 1 Processor -- start 2024-09-17T05:35:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f2857c77000 1 -- start start 2024-09-17T05:35:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f284da00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 0x7f2828075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.593+0000 7f284da00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 0x7f2828075710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f283c0af000 tx=0x7f283c01afb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a3866eaa0 con 0x559a38669b30 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a385e3cb0 con 0x559a38680400 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559a385d30e0 con 0x559a3868ada0 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37714/0 (socket says 172.21.15.43:37714) 2024-09-17T05:35:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 -- 172.21.15.43:0/3108446829 learned_addr learned my addr 172.21.15.43:0/3108446829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284d000640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284da00640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 -- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 msgr2=0x559a387d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 -- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 msgr2=0x559a387d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 -- 172.21.15.43:0/3108446829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a38577370 con 0x559a38669b30 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284d000640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f284e400640 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f284005f480 tx=0x7f284005f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed99c0acdc0dfedd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f283a000640 1 -- 172.21.15.43:0/3108446829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840084020 con 0x559a38669b30 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f283a000640 1 -- 172.21.15.43:0/3108446829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2840068dd0 con 0x559a38669b30 2024-09-17T05:35:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f283a000640 1 -- 172.21.15.43:0/3108446829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284001f040 con 0x559a38669b30 2024-09-17T05:35:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- 172.21.15.43:0/3108446829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 msgr2=0x559a387d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f284005f480 tx=0x7f284005f3f0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- 172.21.15.43:0/3108446829 shutdown_connections 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d1b70 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 --2- 172.21.15.43:0/3108446829 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- 172.21.15.43:0/3108446829 >> 172.21.15.43:0/3108446829 conn(0x559a38290700 msgr2=0x559a387b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- 172.21.15.43:0/3108446829 shutdown_connections 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- 172.21.15.43:0/3108446829 wait complete. 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 -- start start 2024-09-17T05:35:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.597+0000 7f2857c77000 1 Processor -- start 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a38657620 con 0x559a38669b30 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a38692200 con 0x559a38680400 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559a3857cb30 con 0x559a3868ada0 2024-09-17T05:35:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 10 client.0 _ll_register_callbacks cb 0x559a38772cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37720/0 (socket says 172.21.15.43:37720) 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 -- 172.21.15.43:0/1043372945 learned_addr learned my addr 172.21.15.43:0/1043372945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284d000640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284e400640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 -- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 msgr2=0x559a387d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 -- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 msgr2=0x559a387d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 -- 172.21.15.43:0/1043372945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a38571280 con 0x559a38669b30 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284d000640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284da00640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c10 tx=0x7f283c0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981c2885e0e2b04e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0ca060 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f283c0a5ad0 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a3856b200 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f283c0d0050 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 10 client.10809 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x559a3859cae0 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559a3856b200 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f283c0d3020 con 0x559a38669b30 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 10 client.10809 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 10 client.10809 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 20 client.10809 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38275a70 0x559a387e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 4 client.10809 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a385e5c00 con 0x559a38275a70 2024-09-17T05:35:07.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.601+0000 7f284d000640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38275a70 0x559a387e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f284d000640 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38275a70 0x559a387e2960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8589ee94aef748f2 server_cookie=ea7d825f02020f20 in_seq=0 out_seq=0 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f282ea00640 10 client.10809 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f282ea00640 1 -- 172.21.15.43:0/1043372945 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a385e5c00 con 0x559a38275a70 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f282ea00640 10 client.10809 handle_command_reply: tid=0 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 client.10809 shutdown 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 msgr2=0x559a387d8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c10 tx=0x7f283c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 shutdown_connections 2024-09-17T05:35:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38275a70 0x559a387e2960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a387d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f282ea00640 0 client.10809 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 --2- 172.21.15.43:0/1043372945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a387d8580 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 >> 172.21.15.43:0/1043372945 conn(0x559a38290700 msgr2=0x559a38785b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 shutdown_connections 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 1 -- 172.21.15.43:0/1043372945 wait complete. 2024-09-17T05:35:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.605+0000 7f2857c77000 20 client.10809 trim_cache size 0 max 16384 2024-09-17T05:35:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 Processor -- start 2024-09-17T05:35:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- start start 2024-09-17T05:35:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38669b30 0x559a387de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28482046c0 con 0x559a38680400 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f28480c6a30 con 0x559a38669b30 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f28481f9750 con 0x559a3868ada0 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37730/0 (socket says 172.21.15.43:37730) 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 -- 172.21.15.43:0/1643029089 learned_addr learned my addr 172.21.15.43:0/1643029089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284da00640 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38669b30 0x559a387de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284e400640 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 -- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 msgr2=0x559a387e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 -- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38669b30 msgr2=0x559a387de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38669b30 0x559a387de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 -- 172.21.15.43:0/1643029089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a38657ab0 con 0x559a38680400 2024-09-17T05:35:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f284d000640 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f28480b7d40 tx=0x7f2834009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0687cba360aa207 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2839600640 1 -- 172.21.15.43:0/1643029089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834017bb0 con 0x559a38680400 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2839600640 1 -- 172.21.15.43:0/1643029089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2834021020 con 0x559a38680400 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2839600640 1 -- 172.21.15.43:0/1643029089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834017dd0 con 0x559a38680400 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- 172.21.15.43:0/1643029089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 msgr2=0x559a387de6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f28480b7d40 tx=0x7f2834009040 comp rx=0 tx=0).stop 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- 172.21.15.43:0/1643029089 shutdown_connections 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38680400 0x559a387de6a0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 --2- 172.21.15.43:0/1643029089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38669b30 0x559a387de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- 172.21.15.43:0/1643029089 >> 172.21.15.43:0/1643029089 conn(0x559a38290700 msgr2=0x559a387e84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- 172.21.15.43:0/1643029089 shutdown_connections 2024-09-17T05:35:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.621+0000 7f2857c77000 1 -- 172.21.15.43:0/1643029089 wait complete. 2024-09-17T05:35:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 -- start start 2024-09-17T05:35:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 Processor -- start 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a3878cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a385e2a80 con 0x559a38669b30 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a38577b30 con 0x559a38680400 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x559a385d6200 con 0x559a3868ada0 2024-09-17T05:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 10 client.0 _ll_register_callbacks cb 0x7f283c0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37746/0 (socket says 172.21.15.43:37746) 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 -- 172.21.15.43:0/1365853281 learned_addr learned my addr 172.21.15.43:0/1365853281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284e400640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284d000640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a3878cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 -- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 msgr2=0x559a387e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 -- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 msgr2=0x559a3878cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a3878cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 -- 172.21.15.43:0/1365853281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a38578350 con 0x559a38669b30 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f284da00640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b0040 tx=0x7f283c0a5d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa76a08d1e909 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f282ea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b2040 con 0x559a38669b30 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f283c0d5b30 con 0x559a38669b30 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.625+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2834021020 con 0x559a38669b30 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f283c0d5d30 con 0x559a38669b30 2024-09-17T05:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 10 client.10815 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x559a3859bcd0 con 0x559a38669b30 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2834021020 con 0x559a38669b30 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f283c0d3040 con 0x559a38669b30 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 10 client.10815 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 10 client.10815 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 20 client.10815 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38263ad0 0x559a388464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 4 client.10815 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1253238885}] 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1253238885}) -- 0x559a385e16b0 con 0x559a38263ad0 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284d000640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38263ad0 0x559a388464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284d000640 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38263ad0 0x559a388464b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f81f9cd9e7ccdb server_cookie=6f052b10bc899e64 in_seq=0 out_seq=0 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 10 client.10815 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 1 -- 172.21.15.43:0/1365853281 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x559a385e16b0 con 0x559a38263ad0 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 10 client.10815 handle_command_reply: tid=0 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 client.10815 shutdown 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 msgr2=0x559a3878c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b0040 tx=0x7f283c0a5d70 comp rx=0 tx=0).stop 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 shutdown_connections 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x559a38263ad0 0x559a388464b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a38680400 0x559a3878cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x559a3868ada0 0x559a387e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 --2- 172.21.15.43:0/1365853281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a38669b30 0x559a3878c9d0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f282ea00640 0 client.10815 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 >> 172.21.15.43:0/1365853281 conn(0x559a38290700 msgr2=0x559a3878c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 shutdown_connections 2024-09-17T05:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 1 -- 172.21.15.43:0/1365853281 wait complete. 2024-09-17T05:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f2857c77000 20 client.10815 trim_cache size 0 max 16384 2024-09-17T05:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 msgr2=0x7f2828075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 0x7f2828075710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f283c0af000 tx=0x7f283c01afb0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 msgr2=0x7f28481665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f284000e940 tx=0x7f284000ee00 comp rx=0 tx=0).stop 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 shutdown_connections 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2828073200 0x7f2828075710 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f284815c610 0x7f28481665d0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f28480d26d0 0x7f28481596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 --2- 172.21.15.43:0/1253952869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28480cf320 0x7f284806e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 >> 172.21.15.43:0/1253952869 conn(0x7f28480691b0 msgr2=0x7f284815eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 shutdown_connections 2024-09-17T05:35:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.629+0000 7f284ee00640 1 -- 172.21.15.43:0/1253952869 wait complete. 2024-09-17T05:35:07.653 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2024-09-17T05:35:07.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 Processor -- start 2024-09-17T05:35:07.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- start start 2024-09-17T05:35:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfb0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f7ce4aa00 con 0x555f7cfb35b0 2024-09-17T05:35:07.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f7cea8f40 con 0x555f7cfb39c0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555f7cfdd2d0 con 0x555f7cf019c0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37762/0 (socket says 172.21.15.43:37762) 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 -- 172.21.15.43:0/2442902509 learned_addr learned my addr 172.21.15.43:0/2442902509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa07200640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa06800640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfb0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 -- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 msgr2=0x555f7cfb0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfb0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 -- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 msgr2=0x555f7cf465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 -- 172.21.15.43:0/2442902509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7ce23c10 con 0x555f7cfb35b0 2024-09-17T05:35:07.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa07200640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:07.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa06800640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfb0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:07.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa05e00640 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4018d80 tx=0x7ff9f4017ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cadb9c6704a0f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa04a00640 1 -- 172.21.15.43:0/2442902509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40ab980 con 0x555f7cfb35b0 2024-09-17T05:35:07.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa04a00640 1 -- 172.21.15.43:0/2442902509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9f40b9070 con 0x555f7cfb35b0 2024-09-17T05:35:07.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa04a00640 1 -- 172.21.15.43:0/2442902509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40b4400 con 0x555f7cfb35b0 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- 172.21.15.43:0/2442902509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 msgr2=0x555f7cf46080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4018d80 tx=0x7ff9f4017ef0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.709+0000 7ffa0d633080 1 -- 172.21.15.43:0/2442902509 shutdown_connections 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf46080 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- 172.21.15.43:0/2442902509 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfb0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- 172.21.15.43:0/2442902509 >> 172.21.15.43:0/2442902509 conn(0x555f7cee8d80 msgr2=0x555f7cfb2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- 172.21.15.43:0/2442902509 shutdown_connections 2024-09-17T05:35:07.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- 172.21.15.43:0/2442902509 wait complete. 2024-09-17T05:35:07.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 Processor -- start 2024-09-17T05:35:07.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- start start 2024-09-17T05:35:07.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfd2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf44930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf44e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555f7cefce80 con 0x555f7cfb35b0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f7cfdaee0 con 0x555f7cfb39c0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa0d633080 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x555f7cefd120 con 0x555f7cf019c0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf44930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf44930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37768/0 (socket says 172.21.15.43:37768) 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 -- 172.21.15.43:0/150152545 learned_addr learned my addr 172.21.15.43:0/150152545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa07200640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf44e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa06800640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfd2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 -- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 msgr2=0x555f7cfd2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x555f7cf019c0 0x555f7cfd2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 -- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 msgr2=0x555f7cf44e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f7cfb39c0 0x555f7cf44e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 -- 172.21.15.43:0/150152545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7cfd3790 con 0x555f7cfb35b0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.713+0000 7ffa05e00640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf44930 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4096b30 tx=0x7ff9f40abd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 -- 172.21.15.43:0/150152545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40be400 con 0x555f7cfb35b0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 -- 172.21.15.43:0/150152545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9f40c4070 con 0x555f7cfb35b0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555f7cf441f0 con 0x555f7cfb35b0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 -- 172.21.15.43:0/150152545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40be8f0 con 0x555f7cfb35b0 2024-09-17T05:35:07.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7cf46570 con 0x555f7cfb35b0 2024-09-17T05:35:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 -- 172.21.15.43:0/150152545 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9f40b3070 con 0x555f7cfb35b0 2024-09-17T05:35:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff9e8071be0 0x7ff9e80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ff9ff400640 1 -- 172.21.15.43:0/150152545 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7ff9f412ee20 con 0x555f7cfb35b0 2024-09-17T05:35:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ffa06800640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff9e8071be0 0x7ff9e80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ffa0d633080 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x555f7cf96ec0 0x555f7cf972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.717+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- osd_op(unknown.0.0:1 14.a 14:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e142) -- 0x555f7cf97810 con 0x555f7cf96ec0 2024-09-17T05:35:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa07200640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x555f7cf96ec0 0x555f7cf972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa06800640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff9e8071be0 0x7ff9e80740f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c570 tx=0x7ff9f80049c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa07200640 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x555f7cf96ec0 0x555f7cf972d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa07200640 1 -- 172.21.15.43:0/150152545 <== osd.2 v2:172.21.15.43:6808/3128122281 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v142'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa00010650 con 0x555f7cf96ec0 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x555f7cf96ec0 msgr2=0x555f7cf972d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x555f7cf96ec0 0x555f7cf972d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff9e8071be0 msgr2=0x7ff9e80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff9e8071be0 0x7ff9e80740f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c570 tx=0x7ff9f80049c0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 msgr2=0x555f7cf44930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 --2- 172.21.15.43:0/150152545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555f7cfb35b0 0x555f7cf44930 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4096b30 tx=0x7ff9f40abd20 comp rx=0 tx=0).stop 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa07200640 1 -- 172.21.15.43:0/150152545 reap_dead start 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 shutdown_connections 2024-09-17T05:35:07.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 >> 172.21.15.43:0/150152545 conn(0x555f7cee8d80 msgr2=0x555f7cf3f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 shutdown_connections 2024-09-17T05:35:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.721+0000 7ffa0d633080 1 -- 172.21.15.43:0/150152545 wait complete. 2024-09-17T05:35:07.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:07.731 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2024-09-17T05:35:07.743 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-09-17T05:35:07.741+0000 7ff58ee00640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-17T05:35:07.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:35:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2024-09-17T05:35:07.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:07.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 -- 172.21.15.43:0/3365745084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0155d70 msgr2=0x7f26e015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 --2- 172.21.15.43:0/3365745084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0155d70 0x7f26e015c7e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009840 tx=0x7f26d001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 -- 172.21.15.43:0/3365745084 shutdown_connections 2024-09-17T05:35:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 --2- 172.21.15.43:0/3365745084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e0155d70 0x7f26e015c7e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 -- 172.21.15.43:0/3365745084 >> 172.21.15.43:0/3365745084 conn(0x7f26e00b23d0 msgr2=0x7f26e00d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 -- 172.21.15.43:0/3365745084 shutdown_connections 2024-09-17T05:35:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.941+0000 7f26e6400640 1 -- 172.21.15.43:0/3365745084 wait complete. 2024-09-17T05:35:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 Processor -- start 2024-09-17T05:35:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- start start 2024-09-17T05:35:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 0x7f26e0165b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26e00cf150 0x7f26e00cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0169dc0 con 0x7f26e0155d70 2024-09-17T05:35:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0169c40 con 0x7f26e006f110 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f26e0169f40 con 0x7f26e00cf150 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37782/0 (socket says 172.21.15.43:37782) 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 -- 172.21.15.43:0/763115203 learned_addr learned my addr 172.21.15.43:0/763115203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26df400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 0x7f26e0165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26dfe00640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26e00cf150 0x7f26e00cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 -- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26e00cf150 msgr2=0x7f26e00cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26e00cf150 0x7f26e00cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 -- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 msgr2=0x7f26e0165b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 0x7f26e0165b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 -- 172.21.15.43:0/763115203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00b3d70 con 0x7f26e0155d70 2024-09-17T05:35:07.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26df400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 0x7f26e0165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7e00640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c430 tx=0x7f26c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8017070 con 0x7f26e0155d70 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c80053e0 con 0x7f26e0155d70 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8003860 con 0x7f26e0155d70 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e00cfaa0 con 0x7f26e0155d70 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.945+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e009dcc0 con 0x7f26e0155d70 2024-09-17T05:35:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e015c7e0 con 0x7f26e0155d70 2024-09-17T05:35:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26c801a070 con 0x7f26e0155d70 2024-09-17T05:35:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26d7400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 0x7f26bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f26c8093630 con 0x7f26e0155d70 2024-09-17T05:35:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26df400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 0x7f26bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.949+0000 7f26df400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 0x7f26bc075710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f26d00ae000 tx=0x7f26d001aef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:07.953+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26e00cfaa0 con 0x7f26e0155d70 2024-09-17T05:35:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26e00ad4c0 con 0x7f26e0155d70 2024-09-17T05:35:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26d7400640 1 -- 172.21.15.43:0/763115203 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7f26c805df60 con 0x7f26e0155d70 2024-09-17T05:35:08.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:08.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 msgr2=0x7f26bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 0x7f26bc075710 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f26d00ae000 tx=0x7f26d001aef0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 msgr2=0x7f26e0159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c430 tx=0x7f26c800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 shutdown_connections 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f26bc073200 0x7f26bc075710 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f26e00cf150 0x7f26e00cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e0155d70 0x7f26e0159fa0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 --2- 172.21.15.43:0/763115203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26e006f110 0x7f26e0165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 >> 172.21.15.43:0/763115203 conn(0x7f26e00b23d0 msgr2=0x7f26e00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 shutdown_connections 2024-09-17T05:35:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.161+0000 7f26e6400640 1 -- 172.21.15.43:0/763115203 wait complete. 2024-09-17T05:35:08.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 -- 172.21.15.43:0/4000788567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a40053a0 msgr2=0x7f70a400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 --2- 172.21.15.43:0/4000788567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a40053a0 0x7f70a400ba30 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009e90 tx=0x7f70b801d5f0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 -- 172.21.15.43:0/4000788567 shutdown_connections 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 --2- 172.21.15.43:0/4000788567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a40053a0 0x7f70a400ba30 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 -- 172.21.15.43:0/4000788567 >> 172.21.15.43:0/4000788567 conn(0x7f70c00b2e80 msgr2=0x7f70c00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 -- 172.21.15.43:0/4000788567 shutdown_connections 2024-09-17T05:35:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.345+0000 7f70c7200640 1 -- 172.21.15.43:0/4000788567 wait complete. 2024-09-17T05:35:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 Processor -- start 2024-09-17T05:35:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- start start 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a40053a0 0x7f70c015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70c00ce4f0 0x7f70c0157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70c015f8c0 con 0x7f70c015d810 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70c015f740 con 0x7f70a40053a0 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f70c015fa40 con 0x7f70c00ce4f0 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37804/0 (socket says 172.21.15.43:37804) 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 -- 172.21.15.43:0/4156292069 learned_addr learned my addr 172.21.15.43:0/4156292069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70b7400640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70c00ce4f0 0x7f70c0157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70bfe00640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a40053a0 0x7f70c015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 -- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70c00ce4f0 msgr2=0x7f70c0157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70c00ce4f0 0x7f70c0157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 -- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a40053a0 msgr2=0x7f70c015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a40053a0 0x7f70c015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 -- 172.21.15.43:0/4156292069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c015e160 con 0x7f70c015d810 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c5200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f70b8099010 tx=0x7f70b8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70bcc00640 1 -- 172.21.15.43:0/4156292069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80bb070 con 0x7f70c015d810 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70bcc00640 1 -- 172.21.15.43:0/4156292069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70b80aa960 con 0x7f70c015d810 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70bcc00640 1 -- 172.21.15.43:0/4156292069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80af040 con 0x7f70c015d810 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c00cc8d0 con 0x7f70c015d810 2024-09-17T05:35:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.349+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c015cab0 con 0x7f70c015d810 2024-09-17T05:35:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70bcc00640 1 -- 172.21.15.43:0/4156292069 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70b80be070 con 0x7f70c015d810 2024-09-17T05:35:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70bcc00640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 0x7f709c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70bcc00640 1 -- 172.21.15.43:0/4156292069 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f70b812e8d0 con 0x7f70c015d810 2024-09-17T05:35:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70ce8fc000 1 Processor -- start 2024-09-17T05:35:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70bfe00640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 0x7f709c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.353+0000 7f70ce8fc000 1 -- start start 2024-09-17T05:35:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70bfe00640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 0x7f709c075740 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006b20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ffeb60 0x5573a71442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6fddd90 0x5573a7144d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573a6fe2d00 con 0x5573a6ff41c0 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573a6f56c90 con 0x5573a6ffeb60 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573a6f460c0 con 0x5573a6fddd90 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 -- 172.21.15.43:0/2459409942 learned_addr learned my addr 172.21.15.43:0/2459409942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70c5200640 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6fddd90 0x5573a7144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70bfe00640 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ffeb60 0x5573a71442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 -- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6fddd90 msgr2=0x5573a7144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6fddd90 0x5573a7144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 -- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ffeb60 msgr2=0x5573a71442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ffeb60 0x5573a71442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 -- 172.21.15.43:0/2459409942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a6f21290 con 0x5573a6ff41c0 2024-09-17T05:35:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b7400640 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f70a8013820 tx=0x7f70a8013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c062c2a22649d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b4c00640 1 -- 172.21.15.43:0/2459409942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8013230 con 0x5573a6ff41c0 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b4c00640 1 -- 172.21.15.43:0/2459409942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a80119b0 con 0x5573a6ff41c0 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70b4c00640 1 -- 172.21.15.43:0/2459409942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011dc0 con 0x5573a6ff41c0 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2459409942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 msgr2=0x5573a7144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f70a8013820 tx=0x7f70a8013ce0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2459409942 shutdown_connections 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6fddd90 0x5573a7144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7144820 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2459409942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ffeb60 0x5573a71442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2459409942 >> 172.21.15.43:0/2459409942 conn(0x5573a6c03760 msgr2=0x5573a7127790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2459409942 shutdown_connections 2024-09-17T05:35:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.357+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2459409942 wait complete. 2024-09-17T05:35:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- start start 2024-09-17T05:35:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 Processor -- start 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a7100060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ff41c0 0x5573a71005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573a6ee0940 con 0x5573a6ffeb60 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573a6f49ed0 con 0x5573a6fddd90 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573a6c0b820 con 0x5573a6ff41c0 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 10 client.0 _ll_register_callbacks cb 0x5573a6f247f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a7100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37820/0 (socket says 172.21.15.43:37820) 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 -- 172.21.15.43:0/2364263639 learned_addr learned my addr 172.21.15.43:0/2364263639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70b7400640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ff41c0 0x5573a71005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 -- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ff41c0 msgr2=0x5573a71005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ff41c0 0x5573a71005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 -- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 msgr2=0x5573a7100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a7100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 -- 172.21.15.43:0/2364263639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a6ef9550 con 0x5573a6ffeb60 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70c5200640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f70b812ccd0 tx=0x7f70b80f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf656382d57ab72 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b811e030 con 0x5573a6ffeb60 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f70b8102850 con 0x5573a6ffeb60 2024-09-17T05:35:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573a6ede400 con 0x5573a6ffeb60 2024-09-17T05:35:08.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f70b80a7040 con 0x5573a6ffeb60 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 10 client.10845 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x5573a6ee4480 con 0x5573a6ffeb60 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5573a6ede400 con 0x5573a6ffeb60 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f70b80b2070 con 0x5573a6ffeb60 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 10 client.10845 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 10 client.10845 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 20 client.10845 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6be8ad0 0x5573a71272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 4 client.10845 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.361+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a6f58be0 con 0x5573a6be8ad0 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70bfe00640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6be8ad0 0x5573a71272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70bfe00640 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6be8ad0 0x5573a71272e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de2f5a1674e5e25 server_cookie=cef3048ead97a175 in_seq=0 out_seq=0 2024-09-17T05:35:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ad600640 10 client.10845 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ad600640 1 -- 172.21.15.43:0/2364263639 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573a6f58be0 con 0x5573a6be8ad0 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ad600640 10 client.10845 handle_command_reply: tid=0 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 client.10845 shutdown 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 msgr2=0x5573a71268d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f70b812ccd0 tx=0x7f70b80f8b80 comp rx=0 tx=0).stop 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 shutdown_connections 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ffeb60 0x5573a71268d0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ff41c0 0x5573a71005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6be8ad0 0x5573a71272e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/2364263639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a7100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ad600640 0 client.10845 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 >> 172.21.15.43:0/2364263639 conn(0x5573a6c03760 msgr2=0x5573a70f8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 shutdown_connections 2024-09-17T05:35:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 1 -- 172.21.15.43:0/2364263639 wait complete. 2024-09-17T05:35:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.365+0000 7f70ce8fc000 20 client.10845 trim_cache size 0 max 16384 2024-09-17T05:35:08.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 Processor -- start 2024-09-17T05:35:08.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- start start 2024-09-17T05:35:08.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a70ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a70fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573a7005fc0 con 0x5573a6ff41c0 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573a6ef1300 con 0x5573a6fddd90 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573a6f55a60 con 0x5573a6ffeb60 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37834/0 (socket says 172.21.15.43:37834) 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 -- 172.21.15.43:0/3079020749 learned_addr learned my addr 172.21.15.43:0/3079020749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 -- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 msgr2=0x5573a70fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70bfe00640 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a70ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a70fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 -- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 msgr2=0x5573a70ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a70ffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 -- 172.21.15.43:0/3079020749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a6fcbd10 con 0x5573a6ff41c0 2024-09-17T05:35:08.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70bfe00640 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a70ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:08.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70b7400640 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x5573a6ff84b0 tx=0x7f70c00b21e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0897805e03f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70afe00640 1 -- 172.21.15.43:0/3079020749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011ce0 con 0x5573a6ff41c0 2024-09-17T05:35:08.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70afe00640 1 -- 172.21.15.43:0/3079020749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a8011e80 con 0x5573a6ff41c0 2024-09-17T05:35:08.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70afe00640 1 -- 172.21.15.43:0/3079020749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a801f670 con 0x5573a6ff41c0 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- 172.21.15.43:0/3079020749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 msgr2=0x5573a7100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x5573a6ff84b0 tx=0x7f70c00b21e0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- 172.21.15.43:0/3079020749 shutdown_connections 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a70fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6ff41c0 0x5573a7100120 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/3079020749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6fddd90 0x5573a70ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.381+0000 7f70ce8fc000 1 -- 172.21.15.43:0/3079020749 >> 172.21.15.43:0/3079020749 conn(0x5573a6c03760 msgr2=0x5573a71252b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- 172.21.15.43:0/3079020749 shutdown_connections 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- 172.21.15.43:0/3079020749 wait complete. 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- start start 2024-09-17T05:35:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 Processor -- start 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ff41c0 0x5573a7100080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a71568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573a6f12300 con 0x5573a6fddd90 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573a6f03670 con 0x5573a6ff41c0 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x5573a6f491e0 con 0x5573a6ffeb60 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 10 client.0 _ll_register_callbacks cb 0x7f70b8009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37838/0 (socket says 172.21.15.43:37838) 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 -- 172.21.15.43:0/7843431 learned_addr learned my addr 172.21.15.43:0/7843431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70c5200640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a71568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70b7400640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ff41c0 0x5573a7100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 -- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 msgr2=0x5573a71568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a71568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 -- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ff41c0 msgr2=0x5573a7100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ff41c0 0x5573a7100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 -- 172.21.15.43:0/7843431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a6efce50 con 0x5573a6fddd90 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70bfe00640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f70b00063d0 tx=0x7f70b0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865184c68aae589 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ad600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0013820 con 0x5573a6fddd90 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f70b0029040 con 0x5573a6fddd90 2024-09-17T05:35:08.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.385+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573a6f54690 con 0x5573a6fddd90 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f70b0039050 con 0x5573a6fddd90 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 10 client.10851 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x5573a6eeb2d0 con 0x5573a6fddd90 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5573a6f54690 con 0x5573a6fddd90 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f70b0030030 con 0x5573a6fddd90 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 10 client.10851 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 10 client.10851 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 20 client.10851 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6bd6b30 0x5573a71b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 4 client.10851 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5573a6f51ac0 con 0x5573a6bd6b30 2024-09-17T05:35:08.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70b7400640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6bd6b30 0x5573a71b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70b7400640 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6bd6b30 0x5573a71b90b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379976ed9a38a57b server_cookie=24e1b55c290d29b9 in_seq=0 out_seq=0 2024-09-17T05:35:08.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 10 client.10851 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 1 -- 172.21.15.43:0/7843431 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+292 (crc 0 0 0) 0x5573a6f51ac0 con 0x5573a6bd6b30 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 10 client.10851 handle_command_reply: tid=0 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 client.10851 shutdown 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 msgr2=0x5573a70ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f70b00063d0 tx=0x7f70b0014040 comp rx=0 tx=0).stop 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 shutdown_connections 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x5573a6bd6b30 0x5573a71b90b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573a6ff41c0 0x5573a7100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x5573a6ffeb60 0x5573a71568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 --2- 172.21.15.43:0/7843431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573a6fddd90 0x5573a70ffb40 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ad600640 0 client.10851 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 >> 172.21.15.43:0/7843431 conn(0x5573a6c03760 msgr2=0x5573a7150ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 shutdown_connections 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 1 -- 172.21.15.43:0/7843431 wait complete. 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70ce8fc000 20 client.10851 trim_cache size 0 max 16384 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "dir_frag", 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 170608276, 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628278, 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: "frag": "*", 2024-09-17T05:35:08.397 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/dir2" 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "backtrace", 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 849322527, 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628279, 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/testdir/hardlink2" 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-17T05:35:08.398 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 msgr2=0x7f709c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 0x7f709c075740 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006b20 comp rx=0 tx=0).stop 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 msgr2=0x7f70c015dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f70b8099010 tx=0x7f70b8004f10 comp rx=0 tx=0).stop 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 shutdown_connections 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f709c073230 0x7f709c075740 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c015d810 0x7f70c015dc20 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f70c00ce4f0 0x7f70c0157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 --2- 172.21.15.43:0/4156292069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70a40053a0 0x7f70c015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 >> 172.21.15.43:0/4156292069 conn(0x7f70c00b2e80 msgr2=0x7f70c006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 shutdown_connections 2024-09-17T05:35:08.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.389+0000 7f70c7200640 1 -- 172.21.15.43:0/4156292069 wait complete. 2024-09-17T05:35:08.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 -- 172.21.15.43:0/1774333437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c06f110 msgr2=0x7fcf2c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 --2- 172.21.15.43:0/1774333437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c06f110 0x7fcf2c0b33e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c072b60 tx=0x7fcf1801c740 comp rx=0 tx=0).stop 2024-09-17T05:35:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 -- 172.21.15.43:0/1774333437 shutdown_connections 2024-09-17T05:35:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 --2- 172.21.15.43:0/1774333437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c06f110 0x7fcf2c0b33e0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 --2- 172.21.15.43:0/1774333437 >> v2:172.21.15.62:3300/0 conn(0x7fcf2c155d70 0x7fcf2c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 -- 172.21.15.43:0/1774333437 >> 172.21.15.43:0/1774333437 conn(0x7fcf2c0b23d0 msgr2=0x7fcf2c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 -- 172.21.15.43:0/1774333437 shutdown_connections 2024-09-17T05:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.569+0000 7fcf30e00640 1 -- 172.21.15.43:0/1774333437 wait complete. 2024-09-17T05:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 Processor -- start 2024-09-17T05:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 -- start start 2024-09-17T05:35:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 0x7fcf2c09b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 0x7fcf2c1f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c168a60 con 0x7fcf2c0b3920 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1688e0 con 0x7fcf2c155d70 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf30e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c168be0 con 0x7fcf2c06f110 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37862/0 (socket says 172.21.15.43:37862) 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 -- 172.21.15.43:0/2220814100 learned_addr learned my addr 172.21.15.43:0/2220814100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 0x7fcf2c09b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 -- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 msgr2=0x7fcf2c09b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf2a800640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 0x7fcf2c1f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 0x7fcf2c09b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 -- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 msgr2=0x7fcf2c1f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 0x7fcf2c1f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 -- 172.21.15.43:0/2220814100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c15fb40 con 0x7fcf2c0b3920 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 0x7fcf2c09b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf2a800640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 0x7fcf2c1f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.573+0000 7fcf29400640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c10 tx=0x7fcf18005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf180b2040 con 0x7fcf2c0b3920 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf180043e0 con 0x7fcf2c0b3920 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c0b0080 con 0x7fcf2c0b3920 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf180052b0 con 0x7fcf2c0b3920 2024-09-17T05:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c074160 con 0x7fcf2c0b3920 2024-09-17T05:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c09bb40 con 0x7fcf2c0b3920 2024-09-17T05:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf18005450 con 0x7fcf2c0b3920 2024-09-17T05:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 0x7fcf0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fcf1812e1e0 con 0x7fcf2c0b3920 2024-09-17T05:35:08.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.577+0000 7fcf29e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 0x7fcf0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.581+0000 7fcf29e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 0x7fcf0c075740 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0715c0 tx=0x7fcf1c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.585+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcf180bd060 con 0x7fcf2c0b3920 2024-09-17T05:35:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf2c0b33e0 con 0x7fcf2c0b3920 2024-09-17T05:35:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf22000640 1 -- 172.21.15.43:0/2220814100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7fcf180163f0 con 0x7fcf2c0b3920 2024-09-17T05:35:08.797 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:08.797 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 msgr2=0x7fcf0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 0x7fcf0c075740 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0715c0 tx=0x7fcf1c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 msgr2=0x7fcf2c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fcf18000c10 tx=0x7fcf18005dd0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 shutdown_connections 2024-09-17T05:35:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcf0c073230 0x7fcf0c075740 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2c155d70 0x7fcf2c1f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf2c0b3920 0x7fcf2c0cf280 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 --2- 172.21.15.43:0/2220814100 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcf2c06f110 0x7fcf2c09b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 >> 172.21.15.43:0/2220814100 conn(0x7fcf2c0b23d0 msgr2=0x7fcf2c09dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 shutdown_connections 2024-09-17T05:35:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.793+0000 7fcf30e00640 1 -- 172.21.15.43:0/2220814100 wait complete. 2024-09-17T05:35:08.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 170608276 2024-09-17T05:35:08.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 -- 172.21.15.43:0/4135813141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c155d70 msgr2=0x7f140c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 --2- 172.21.15.43:0/4135813141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c155d70 0x7f140c15c7e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009900 tx=0x7f13fc01c820 comp rx=0 tx=0).stop 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 -- 172.21.15.43:0/4135813141 shutdown_connections 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 --2- 172.21.15.43:0/4135813141 >> v2:172.21.15.62:3300/0 conn(0x7f140c0b3af0 0x7f140c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 --2- 172.21.15.43:0/4135813141 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c155d70 0x7f140c15c7e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 -- 172.21.15.43:0/4135813141 >> 172.21.15.43:0/4135813141 conn(0x7f140c0b23d0 msgr2=0x7f140c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 -- 172.21.15.43:0/4135813141 shutdown_connections 2024-09-17T05:35:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.977+0000 7f1412e00640 1 -- 172.21.15.43:0/4135813141 wait complete. 2024-09-17T05:35:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 Processor -- start 2024-09-17T05:35:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 -- start start 2024-09-17T05:35:08.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c06f110 0x7f140c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f140c0ccab0 0x7f140c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f140c168c80 con 0x7f140c155d70 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f140c168b00 con 0x7f140c0ccab0 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1412e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f140c168e00 con 0x7f140c06f110 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37864/0 (socket says 172.21.15.43:37864) 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 -- 172.21.15.43:0/3456756229 learned_addr learned my addr 172.21.15.43:0/3456756229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140be00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c06f110 0x7f140c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f1410e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f140c0ccab0 0x7f140c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 -- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c06f110 msgr2=0x7f140c0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c06f110 0x7f140c0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 -- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f140c0ccab0 msgr2=0x7f140c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f140c0ccab0 0x7f140c0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.981+0000 7f140b400640 1 -- 172.21.15.43:0/3456756229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c0cd400 con 0x7f140c155d70 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f140b400640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c3f0 tx=0x7f13f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 -- 172.21.15.43:0/3456756229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8017070 con 0x7f140c155d70 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c0ca1c0 con 0x7f140c155d70 2024-09-17T05:35:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 -- 172.21.15.43:0/3456756229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13f80053c0 con 0x7f140c155d70 2024-09-17T05:35:08.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 -- 172.21.15.43:0/3456756229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8003860 con 0x7f140c155d70 2024-09-17T05:35:08.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c15ad00 con 0x7f140c155d70 2024-09-17T05:35:08.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 -- 172.21.15.43:0/3456756229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13f801a070 con 0x7f140c155d70 2024-09-17T05:35:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 0x7f13ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.985+0000 7f1403e00640 1 -- 172.21.15.43:0/3456756229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f13f8093630 con 0x7f140c155d70 2024-09-17T05:35:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 0x7f13ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 Processor -- start 2024-09-17T05:35:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 -- start start 2024-09-17T05:35:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 0x7f13ec075740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0af000 tx=0x7f13fc01b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d143120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d143660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f9cfe1810 con 0x560f9cfd2430 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f9cf54980 con 0x560f9cfc9b80 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f9cf43db0 con 0x560f9cfc9370 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37872/0 (socket says 172.21.15.43:37872) 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 -- 172.21.15.43:0/2934849015 learned_addr learned my addr 172.21.15.43:0/2934849015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140b400640 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f1410e00640 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 -- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 msgr2=0x560f9d143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 -- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 msgr2=0x560f9d143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 -- 172.21.15.43:0/2934849015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9cf11af0 con 0x560f9cfd2430 2024-09-17T05:35:08.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f140be00640 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c10 tx=0x7f13fc0b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1867b3dc20dd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:08.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f1400c00640 1 -- 172.21.15.43:0/2934849015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0c8070 con 0x560f9cfd2430 2024-09-17T05:35:08.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f1400c00640 1 -- 172.21.15.43:0/2934849015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13fc0041f0 con 0x560f9cfd2430 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.989+0000 7f1400c00640 1 -- 172.21.15.43:0/2934849015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0b2040 con 0x560f9cfd2430 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- 172.21.15.43:0/2934849015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 msgr2=0x560f9d142be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f13fc000c10 tx=0x7f13fc0b4000 comp rx=0 tx=0).stop 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- 172.21.15.43:0/2934849015 shutdown_connections 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- 172.21.15.43:0/2934849015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d142be0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- 172.21.15.43:0/2934849015 >> 172.21.15.43:0/2934849015 conn(0x560f9d0c83c0 msgr2=0x560f9d0f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- 172.21.15.43:0/2934849015 shutdown_connections 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- 172.21.15.43:0/2934849015 wait complete. 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- start start 2024-09-17T05:35:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 Processor -- start 2024-09-17T05:35:09.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfd2430 0x560f9d105880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f9cee9140 con 0x560f9cfc9370 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f9cf47bc0 con 0x560f9cfd2430 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f9ceea320 con 0x560f9cfc9b80 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f141a554000 10 client.0 _ll_register_callbacks cb 0x560f9d0e4b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37878/0 (socket says 172.21.15.43:37878) 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 -- 172.21.15.43:0/2804688688 learned_addr learned my addr 172.21.15.43:0/2804688688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140b400640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f1410e00640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfd2430 0x560f9d105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 -- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 msgr2=0x560f9d105340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d105340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 -- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfd2430 msgr2=0x560f9d105880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfd2430 0x560f9d105880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.993+0000 7f140be00640 1 -- 172.21.15.43:0/2804688688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9ceecba0 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f140be00640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f13fc01cd00 tx=0x7f13fc0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad56b82b7b5dc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0b2070 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f13fc0cdb60 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f9cedbe20 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f13fc0c8040 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 10 client.10869 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x560f9cee1f90 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560f9cedbe20 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f13fc0b5030 con 0x560f9cfc9370 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 10 client.10869 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:09.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 10 client.10869 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:09.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 20 client.10869 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:09.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbe6a10 0x560f9d1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 4 client.10869 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:09.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9cf568d0 con 0x560f9cbe6a10 2024-09-17T05:35:09.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f140b400640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbe6a10 0x560f9d1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f140b400640 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbe6a10 0x560f9d1541a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc79a12ff8e18f1 server_cookie=a866c59d44d6ad1 in_seq=0 out_seq=0 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 10 client.10869 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 1 -- 172.21.15.43:0/2804688688 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f9cf568d0 con 0x560f9cbe6a10 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 10 client.10869 handle_command_reply: tid=0 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 client.10869 shutdown 2024-09-17T05:35:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 msgr2=0x560f9d104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f13fc01cd00 tx=0x7f13fc0044e0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 shutdown_connections 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbe6a10 0x560f9d1541a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfd2430 0x560f9d105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 --2- 172.21.15.43:0/2804688688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfc9370 0x560f9d104e00 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f13f5600640 0 client.10869 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 >> 172.21.15.43:0/2804688688 conn(0x560f9d0c83c0 msgr2=0x560f9d0f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 shutdown_connections 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 1 -- 172.21.15.43:0/2804688688 wait complete. 2024-09-17T05:35:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:08.997+0000 7f141a554000 20 client.10869 trim_cache size 0 max 16384 2024-09-17T05:35:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 Processor -- start 2024-09-17T05:35:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 -- start start 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d1b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d1b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1fcbd0 con 0x560f9cfd2430 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f9cf4c1f0 con 0x560f9cfc9b80 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f9cee9140 con 0x560f9cfc9370 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37886/0 (socket says 172.21.15.43:37886) 2024-09-17T05:35:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 -- 172.21.15.43:0/3571952225 learned_addr learned my addr 172.21.15.43:0/3571952225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f140be00640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d1b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f140b400640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d1b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 -- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 msgr2=0x560f9d1b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d1b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 -- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 msgr2=0x560f9d1b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d1b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f1410e00640 1 -- 172.21.15.43:0/3571952225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9cf0d970 con 0x560f9cfd2430 2024-09-17T05:35:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.013+0000 7f140be00640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d1b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f1410e00640 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f1404003a40 tx=0x7f1404003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78067d940424cc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f13f7e00640 1 -- 172.21.15.43:0/3571952225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404020070 con 0x560f9cfd2430 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f13f7e00640 1 -- 172.21.15.43:0/3571952225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f140401b430 con 0x560f9cfd2430 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f13f7e00640 1 -- 172.21.15.43:0/3571952225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140401b840 con 0x560f9cfd2430 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- 172.21.15.43:0/3571952225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 msgr2=0x560f9d1b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f1404003a40 tx=0x7f1404003f00 comp rx=0 tx=0).stop 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- 172.21.15.43:0/3571952225 shutdown_connections 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9b80 0x560f9d1b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d1b4960 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 --2- 172.21.15.43:0/3571952225 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9370 0x560f9d1b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- 172.21.15.43:0/3571952225 >> 172.21.15.43:0/3571952225 conn(0x560f9d0c83c0 msgr2=0x560f9d152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- 172.21.15.43:0/3571952225 shutdown_connections 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- 172.21.15.43:0/3571952225 wait complete. 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 -- start start 2024-09-17T05:35:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.017+0000 7f141a554000 1 Processor -- start 2024-09-17T05:35:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9370 0x560f9d0fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d0fb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f9cf53750 con 0x560f9cfd2430 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560f9d004710 con 0x560f9cfc9370 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x560f9cec85a0 con 0x560f9cfc9b80 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 10 client.0 _ll_register_callbacks cb 0x7f13fc0c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37896/0 (socket says 172.21.15.43:37896) 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 -- 172.21.15.43:0/2622209558 learned_addr learned my addr 172.21.15.43:0/2622209558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f140be00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9370 0x560f9d0fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f140b400640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d0fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 -- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 msgr2=0x560f9d0fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d0fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 -- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9370 msgr2=0x560f9d0fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9370 0x560f9d0fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 -- 172.21.15.43:0/2622209558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9cec8f50 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f1410e00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x560f9cf77940 tx=0x7f140401b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd31f7d08266e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404020040 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f140401bcc0 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140401b430 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f1404010430 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 10 client.10875 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x560f9cf06a60 con 0x560f9cfd2430 2024-09-17T05:35:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f140401b430 con 0x560f9cfd2430 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f140401a070 con 0x560f9cfd2430 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 10 client.10875 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 10 client.10875 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 20 client.10875 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbd4a70 0x560f9d1b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 4 client.10875 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 170608276}] 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "damage rm", "damage_id": 170608276}) -- 0x560f9cf52380 con 0x560f9cbd4a70 2024-09-17T05:35:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f140be00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbd4a70 0x560f9d1b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f140be00640 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbd4a70 0x560f9d1b7fd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161d98cf89c0b2d1 server_cookie=79083bef330ea620 in_seq=0 out_seq=0 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 10 client.10875 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 1 -- 172.21.15.43:0/2622209558 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x560f9cf52380 con 0x560f9cbd4a70 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 10 client.10875 handle_command_reply: tid=0 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 client.10875 shutdown 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 msgr2=0x560f9d19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x560f9cf77940 tx=0x7f140401b810 comp rx=0 tx=0).stop 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 shutdown_connections 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f9cfd2430 0x560f9d19cba0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x560f9cfc9b80 0x560f9d0fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x560f9cbd4a70 0x560f9d1b7fd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 --2- 172.21.15.43:0/2622209558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f9cfc9370 0x560f9d0fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 >> 172.21.15.43:0/2622209558 conn(0x560f9d0c83c0 msgr2=0x560f9d0f7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f13f5600640 0 client.10875 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 shutdown_connections 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 1 -- 172.21.15.43:0/2622209558 wait complete. 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.021+0000 7f141a554000 20 client.10875 trim_cache size 0 max 16384 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 msgr2=0x7f13ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 0x7f13ec075740 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0af000 tx=0x7f13fc01b060 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 msgr2=0x7f140c0c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c3f0 tx=0x7f13f800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 shutdown_connections 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f13ec073230 0x7f13ec075740 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f140c0ccab0 0x7f140c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f140c155d70 0x7f140c0c62c0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 --2- 172.21.15.43:0/3456756229 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f140c06f110 0x7f140c0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 >> 172.21.15.43:0/3456756229 conn(0x7f140c0b23d0 msgr2=0x7f140c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 shutdown_connections 2024-09-17T05:35:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.025+0000 7f1412e00640 1 -- 172.21.15.43:0/3456756229 wait complete. 2024-09-17T05:35:09.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 -- 172.21.15.43:0/2365037351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b60155d70 msgr2=0x7f0b6015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 --2- 172.21.15.43:0/2365037351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b60155d70 0x7f0b6015c7e0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f0b600d0c40 tx=0x7f0b5001c740 comp rx=0 tx=0).stop 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 -- 172.21.15.43:0/2365037351 shutdown_connections 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 --2- 172.21.15.43:0/2365037351 >> v2:172.21.15.62:3300/0 conn(0x7f0b600b3af0 0x7f0b600ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 --2- 172.21.15.43:0/2365037351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b60155d70 0x7f0b6015c7e0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 -- 172.21.15.43:0/2365037351 >> 172.21.15.43:0/2365037351 conn(0x7f0b600b23d0 msgr2=0x7f0b600d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 -- 172.21.15.43:0/2365037351 shutdown_connections 2024-09-17T05:35:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.201+0000 7f0b65c00640 1 -- 172.21.15.43:0/2365037351 wait complete. 2024-09-17T05:35:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 Processor -- start 2024-09-17T05:35:09.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 -- start start 2024-09-17T05:35:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b6006f110 0x7f0b601fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b60155d70 0x7f0b601f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60168b60 con 0x7f0b6009c750 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601689e0 con 0x7f0b60155d70 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b65c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0b60168ce0 con 0x7f0b6006f110 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b5f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b5f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37912/0 (socket says 172.21.15.43:37912) 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b5f400640 1 -- 172.21.15.43:0/106493009 learned_addr learned my addr 172.21.15.43:0/106493009 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b5ea00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b6006f110 0x7f0b601fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.205+0000 7f0b56000640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b60155d70 0x7f0b601f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 -- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b6006f110 msgr2=0x7f0b601fa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b6006f110 0x7f0b601fa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 -- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b60155d70 msgr2=0x7f0b601f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b60155d70 0x7f0b601f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 -- 172.21.15.43:0/106493009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b600cbee0 con 0x7f0b6009c750 2024-09-17T05:35:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b5f400640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800ea80 tx=0x7f0b5800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800ce40 con 0x7f0b6009c750 2024-09-17T05:35:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b58004b30 con 0x7f0b6009c750 2024-09-17T05:35:09.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b601fac30 con 0x7f0b6009c750 2024-09-17T05:35:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58005020 con 0x7f0b6009c750 2024-09-17T05:35:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b600cc150 con 0x7f0b6009c750 2024-09-17T05:35:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6015aee0 con 0x7f0b6009c750 2024-09-17T05:35:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.209+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0b580051c0 con 0x7f0b6009c750 2024-09-17T05:35:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.213+0000 7f0b57400640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 0x7f0b3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.213+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f0b58094e80 con 0x7f0b6009c750 2024-09-17T05:35:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.213+0000 7f0b5ea00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 0x7f0b3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.213+0000 7f0b5ea00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 0x7f0b3c075710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0b500af000 tx=0x7f0b5001af80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.217+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b6015aee0 con 0x7f0b6009c750 2024-09-17T05:35:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b600ad0b0 con 0x7f0b6009c750 2024-09-17T05:35:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b57400640 1 -- 172.21.15.43:0/106493009 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 351 v351) ==== 95+0+5235 (secure 0 0 0) 0x7f0b601fac30 con 0x7f0b6009c750 2024-09-17T05:35:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:09.429 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":351,"btime":"2024-09-17T05:35:03:076415+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":351,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:03.076405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:09.429 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 351 2024-09-17T05:35:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 msgr2=0x7f0b3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 0x7f0b3c075710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0b500af000 tx=0x7f0b5001af80 comp rx=0 tx=0).stop 2024-09-17T05:35:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 msgr2=0x7f0b601f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800ea80 tx=0x7f0b5800ef40 comp rx=0 tx=0).stop 2024-09-17T05:35:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 shutdown_connections 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0b3c073200 0x7f0b3c075710 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b60155d70 0x7f0b601f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b6009c750 0x7f0b601f6c70 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 --2- 172.21.15.43:0/106493009 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0b6006f110 0x7f0b601fa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 >> 172.21.15.43:0/106493009 conn(0x7f0b600b23d0 msgr2=0x7f0b6015da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 shutdown_connections 2024-09-17T05:35:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.425+0000 7f0b65c00640 1 -- 172.21.15.43:0/106493009 wait complete. 2024-09-17T05:35:09.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 849322527 2024-09-17T05:35:09.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- 172.21.15.43:0/2382937915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980b7ce0 msgr2=0x7f39980d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 --2- 172.21.15.43:0/2382937915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980b7ce0 0x7f39980d0840 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f399809bbe0 tx=0x7f398801c600 comp rx=0 tx=0).stop 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- 172.21.15.43:0/2382937915 shutdown_connections 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 --2- 172.21.15.43:0/2382937915 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980cd580 0x7f39981609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 --2- 172.21.15.43:0/2382937915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980b7ce0 0x7f39980d0840 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- 172.21.15.43:0/2382937915 >> 172.21.15.43:0/2382937915 conn(0x7f39980cee80 msgr2=0x7f39980cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- 172.21.15.43:0/2382937915 shutdown_connections 2024-09-17T05:35:09.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- 172.21.15.43:0/2382937915 wait complete. 2024-09-17T05:35:09.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 Processor -- start 2024-09-17T05:35:09.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.609+0000 7f399ce00640 1 -- start start 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980b7ce0 0x7f39981fc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39980d0d80 0x7f3998071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f399816a020 con 0x7f39980cd580 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3998169ea0 con 0x7f39980d0d80 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f399816a1a0 con 0x7f39980b7ce0 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37928/0 (socket says 172.21.15.43:37928) 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 -- 172.21.15.43:0/1600086732 learned_addr learned my addr 172.21.15.43:0/1600086732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f3995e00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980b7ce0 0x7f39981fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f3996800640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39980d0d80 0x7f3998071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 -- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980b7ce0 msgr2=0x7f39981fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980b7ce0 0x7f39981fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 -- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39980d0d80 msgr2=0x7f3998071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39980d0d80 0x7f3998071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 -- 172.21.15.43:0/1600086732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39981fd410 con 0x7f39980cd580 2024-09-17T05:35:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398d400640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f398000b3c0 tx=0x7f398000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398ea00640 1 -- 172.21.15.43:0/1600086732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980011830 con 0x7f39980cd580 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398ea00640 1 -- 172.21.15.43:0/1600086732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3980016070 con 0x7f39980cd580 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f398ea00640 1 -- 172.21.15.43:0/1600086732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980011d80 con 0x7f39980cd580 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39981fce50 con 0x7f39980cd580 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.613+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399806e250 con 0x7f39980cd580 2024-09-17T05:35:09.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f398ea00640 1 -- 172.21.15.43:0/1600086732 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f398001b460 con 0x7f39980cd580 2024-09-17T05:35:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f398ea00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 0x7f3974075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f398ea00640 1 -- 172.21.15.43:0/1600086732 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f3980093d70 con 0x7f39980cd580 2024-09-17T05:35:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f3995e00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 0x7f3974075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 Processor -- start 2024-09-17T05:35:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 -- start start 2024-09-17T05:35:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f3995e00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 0x7f3974075710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f39880ae000 tx=0x7f398801ae40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303370 0x56257f47d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f47d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56257f31b810 con 0x56257f30c430 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56257f28e980 con 0x56257f303370 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56257f27ddb0 con 0x56257f303b80 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37938/0 (socket says 172.21.15.43:37938) 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.617+0000 7f3995e00640 1 -- 172.21.15.43:0/384819647 learned_addr learned my addr 172.21.15.43:0/384819647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f398d400640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303370 0x56257f47d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3996800640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f47d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 -- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 msgr2=0x56257f47d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f47d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 -- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303370 msgr2=0x56257f47d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303370 0x56257f47d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 -- 172.21.15.43:0/384819647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257f24baf0 con 0x56257f30c430 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3996800640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f47d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3995e00640 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f3988009840 tx=0x7f39880b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1ce5a31e1149c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3986a00640 1 -- 172.21.15.43:0/384819647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880aae60 con 0x56257f30c430 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3986a00640 1 -- 172.21.15.43:0/384819647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39880c5070 con 0x56257f30c430 2024-09-17T05:35:09.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f3986a00640 1 -- 172.21.15.43:0/384819647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988005350 con 0x56257f30c430 2024-09-17T05:35:09.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- 172.21.15.43:0/384819647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 msgr2=0x56257f47cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f3988009840 tx=0x7f39880b8040 comp rx=0 tx=0).stop 2024-09-17T05:35:09.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- 172.21.15.43:0/384819647 shutdown_connections 2024-09-17T05:35:09.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303370 0x56257f47d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f47d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 --2- 172.21.15.43:0/384819647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f30c430 0x56257f47cbe0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- 172.21.15.43:0/384819647 >> 172.21.15.43:0/384819647 conn(0x56257f4023c0 msgr2=0x56257f431be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- 172.21.15.43:0/384819647 shutdown_connections 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- 172.21.15.43:0/384819647 wait complete. 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 -- start start 2024-09-17T05:35:09.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.621+0000 7f39a452d000 1 Processor -- start 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303370 0x56257f43ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f43f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56257f223140 con 0x56257f303b80 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56257f281bc0 con 0x56257f30c430 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56257f224320 con 0x56257f303370 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 10 client.0 _ll_register_callbacks cb 0x56257f41eb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37954/0 (socket says 172.21.15.43:37954) 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 -- 172.21.15.43:0/1224666908 learned_addr learned my addr 172.21.15.43:0/1224666908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f3996800640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f43f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f3995e00640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303370 0x56257f43ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 -- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303370 msgr2=0x56257f43ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303370 0x56257f43ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 -- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 msgr2=0x56257f43f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f43f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 -- 172.21.15.43:0/1224666908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257f226ba0 con 0x56257f303b80 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f398d400640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f398000b390 tx=0x7f398005ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe1ed5a13c6bdad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980081070 con 0x56257f303b80 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f3980078b40 con 0x56257f303b80 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56257f215e20 con 0x56257f303b80 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f398007f050 con 0x56257f303b80 2024-09-17T05:35:09.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 10 client.10896 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x56257f21bf90 con 0x56257f303b80 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56257f215e20 con 0x56257f303b80 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f3980065050 con 0x56257f303b80 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 10 client.10896 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 10 client.10896 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 20 client.10896 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef20a10 0x56257f48e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 4 client.10896 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257f2908d0 con 0x56257ef20a10 2024-09-17T05:35:09.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f3995e00640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef20a10 0x56257f48e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 10 client.10896 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f3995e00640 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef20a10 0x56257f48e1a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a2748ce09c91ed server_cookie=95b3424980bf2ad8 in_seq=0 out_seq=0 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 1 -- 172.21.15.43:0/1224666908 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56257f2908d0 con 0x56257ef20a10 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.625+0000 7f397b400640 10 client.10896 handle_command_reply: tid=0 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 client.10896 shutdown 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 msgr2=0x56257f43f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f398000b390 tx=0x7f398005ac20 comp rx=0 tx=0).stop 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 shutdown_connections 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef20a10 0x56257f48e1a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f43f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303b80 0x56257f43f340 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 --2- 172.21.15.43:0/1224666908 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303370 0x56257f43ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f397b400640 0 client.10896 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 >> 172.21.15.43:0/1224666908 conn(0x56257f4023c0 msgr2=0x56257f432370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 shutdown_connections 2024-09-17T05:35:09.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 1 -- 172.21.15.43:0/1224666908 wait complete. 2024-09-17T05:35:09.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.629+0000 7f39a452d000 20 client.10896 trim_cache size 0 max 16384 2024-09-17T05:35:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.641+0000 7f39a452d000 1 Processor -- start 2024-09-17T05:35:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.641+0000 7f39a452d000 1 -- start start 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f4ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f4ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39981f82c0 con 0x56257f303370 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39981f8910 con 0x56257f30c430 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56257f280ed0 con 0x56257f303b80 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37964/0 (socket says 172.21.15.43:37964) 2024-09-17T05:35:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 -- 172.21.15.43:0/1279669489 learned_addr learned my addr 172.21.15.43:0/1279669489 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3996800640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f4ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f398d400640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f4ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 -- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 msgr2=0x56257f4ee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f4ee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 -- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 msgr2=0x56257f4ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f4ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 -- 172.21.15.43:0/1279669489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257f247970 con 0x56257f303370 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f398d400640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f4ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3996800640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f4ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3995e00640 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f398801cae0 tx=0x7f39880091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2811c9aeaf8fe38 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3986000640 1 -- 172.21.15.43:0/1279669489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880da070 con 0x56257f303370 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3986000640 1 -- 172.21.15.43:0/1279669489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39880c5070 con 0x56257f303370 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f3986000640 1 -- 172.21.15.43:0/1279669489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880aae60 con 0x56257f303370 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- 172.21.15.43:0/1279669489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 msgr2=0x56257f4edac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f398801cae0 tx=0x7f39880091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- 172.21.15.43:0/1279669489 shutdown_connections 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f30c430 0x56257f4ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f303b80 0x56257f4ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 --2- 172.21.15.43:0/1279669489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f4edac0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- 172.21.15.43:0/1279669489 >> 172.21.15.43:0/1279669489 conn(0x56257f4023c0 msgr2=0x56257f48cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- 172.21.15.43:0/1279669489 shutdown_connections 2024-09-17T05:35:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- 172.21.15.43:0/1279669489 wait complete. 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 -- start start 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.645+0000 7f39a452d000 1 Processor -- start 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303b80 0x56257f435540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f30c430 0x56257f4ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56257f33e710 con 0x56257f303370 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56257f2025a0 con 0x56257f303b80 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x56257f2861f0 con 0x56257f30c430 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 10 client.0 _ll_register_callbacks cb 0x7f398005a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37970/0 (socket says 172.21.15.43:37970) 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 -- 172.21.15.43:0/2159008337 learned_addr learned my addr 172.21.15.43:0/2159008337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f398d400640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303b80 0x56257f435540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3996800640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f30c430 0x56257f4ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 -- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f30c430 msgr2=0x56257f4ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f30c430 0x56257f4ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 -- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303b80 msgr2=0x56257f435540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303b80 0x56257f435540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 -- 172.21.15.43:0/2159008337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257f202f50 con 0x56257f303370 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f3995e00640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f398801cae0 tx=0x7f39880092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1089e13cd61da540 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880da070 con 0x56257f303370 2024-09-17T05:35:09.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f39880053b0 con 0x56257f303370 2024-09-17T05:35:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39880c5070 con 0x56257f303370 2024-09-17T05:35:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=351) ==== 24+0+0 (secure 0 0 0) 0x7f39880055b0 con 0x56257f303370 2024-09-17T05:35:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 10 client.10902 fetch_fsmap learned FSMap version 351 2024-09-17T05:35:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=351,monmap=2+}) -- 0x56257f240a60 con 0x56257f303370 2024-09-17T05:35:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39880c5070 con 0x56257f303370 2024-09-17T05:35:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 351) ==== 3502+0+0 (secure 0 0 0) 0x7f39880bd030 con 0x56257f303370 2024-09-17T05:35:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 10 client.10902 fetch_fsmap finished waiting for FSMap version 351 2024-09-17T05:35:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 10 client.10902 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 20 client.10902 populate_metadata read hostname 'smithi043' 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef0ea70 0x56257f4ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 4 client.10902 mds_command: new command op to 10036 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 849322527}] 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 --> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] -- command(tid 0: {"prefix": "damage rm", "damage_id": 849322527}) -- 0x56257f28c380 con 0x56257ef0ea70 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.649+0000 7f398d400640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef0ea70 0x56257f4ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f398d400640 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef0ea70 0x56257f4ae0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cde9d4bc2d9c86c server_cookie=8ba7316a6ca02314 in_seq=0 out_seq=0 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f397b400640 10 client.10902 ms_handle_connect on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f397b400640 1 -- 172.21.15.43:0/2159008337 <== mds.0 v2:172.21.15.43:6832/1735972562 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x56257f28c380 con 0x56257ef0ea70 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f397b400640 10 client.10902 handle_command_reply: tid=0 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 client.10902 shutdown 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 msgr2=0x56257f435000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f398801cae0 tx=0x7f39880092f0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 shutdown_connections 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:6832/1735972562,v1:172.21.15.43:6834/1735972562] conn(0x56257ef0ea70 0x56257f4ae0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56257f303b80 0x56257f435540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x56257f30c430 0x56257f4ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 --2- 172.21.15.43:0/2159008337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56257f303370 0x56257f435000 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f397b400640 0 client.10902 ms_handle_reset on v2:172.21.15.43:6832/1735972562 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 >> 172.21.15.43:0/2159008337 conn(0x56257f4023c0 msgr2=0x56257f431220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 shutdown_connections 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 1 -- 172.21.15.43:0/2159008337 wait complete. 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f39a452d000 20 client.10902 trim_cache size 0 max 16384 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 msgr2=0x7f3974075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 0x7f3974075710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f39880ae000 tx=0x7f398801ae40 comp rx=0 tx=0).stop 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 msgr2=0x7f3998153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f398000b3c0 tx=0x7f398000b880 comp rx=0 tx=0).stop 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 shutdown_connections 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3974073200 0x7f3974075710 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39980d0d80 0x7f3998071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39980cd580 0x7f3998153660 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 --2- 172.21.15.43:0/1600086732 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f39980b7ce0 0x7f39981fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 >> 172.21.15.43:0/1600086732 conn(0x7f39980cee80 msgr2=0x7f39981570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 shutdown_connections 2024-09-17T05:35:09.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:09.653+0000 7f399ce00640 1 -- 172.21.15.43:0/1600086732 wait complete. 2024-09-17T05:35:09.676 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:35:09.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:35:09.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:35:09.861 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:35:09.862 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:35:09.862 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:09.862 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:09.871 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:09.871 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:09.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:35:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- 172.21.15.43:0/1337412257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f50004d50 msgr2=0x7f7f50005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 --2- 172.21.15.43:0/1337412257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f50004d50 0x7f7f50005160 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006fd0 tx=0x7f7f6401fa20 comp rx=0 tx=0).stop 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f71800640 1 -- 172.21.15.43:0/1337412257 reap_dead start 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- 172.21.15.43:0/1337412257 shutdown_connections 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- 172.21.15.43:0/1337412257 >> 172.21.15.43:0/1337412257 conn(0x7f7f6c0d1160 msgr2=0x7f7f6c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- 172.21.15.43:0/1337412257 shutdown_connections 2024-09-17T05:35:10.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- 172.21.15.43:0/1337412257 wait complete. 2024-09-17T05:35:10.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 Processor -- start 2024-09-17T05:35:10.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.125+0000 7f7f72200640 1 -- start start 2024-09-17T05:35:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f50004d50 0x7f7f6c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c150950 0x7f7f6c150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c15c750 con 0x7f7f6c15f090 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c15c5d0 con 0x7f7f50004d50 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c15c8d0 con 0x7f7f6c150950 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37994/0 (socket says 172.21.15.43:37994) 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 -- 172.21.15.43:0/551957959 learned_addr learned my addr 172.21.15.43:0/551957959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f70e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f50004d50 0x7f7f6c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f71800640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c150950 0x7f7f6c150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 -- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c150950 msgr2=0x7f7f6c150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c150950 0x7f7f6c150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 -- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f50004d50 msgr2=0x7f7f6c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f50004d50 0x7f7f6c154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 -- 172.21.15.43:0/551957959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1512a0 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f63e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f7f54011b10 tx=0x7f7f54011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401e070 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f54004460 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400fba0 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1f8c20 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1f91e0 con 0x7f7f6c15f090 2024-09-17T05:35:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.129+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c154a30 con 0x7f7f6c15f090 2024-09-17T05:35:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.133+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7f54021070 con 0x7f7f6c15f090 2024-09-17T05:35:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.133+0000 7f7f69600640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 0x7f7f48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.133+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f7f54094c80 con 0x7f7f6c15f090 2024-09-17T05:35:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.133+0000 7f7f70e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 0x7f7f48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.133+0000 7f7f70e00640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 0x7f7f48075740 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009c60 tx=0x7f7f5c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.137+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f6c1512a0 con 0x7f7f6c15f090 2024-09-17T05:35:10.297 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:35:10.293+0000 7fcc80800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-17T05:35:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f6c073ef0 con 0x7f7f6c15f090 2024-09-17T05:35:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f69600640 1 -- 172.21.15.43:0/551957959 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v142) ==== 80+0+122 (secure 0 0 0) 0x7f7f5405f5b0 con 0x7f7f6c15f090 2024-09-17T05:35:10.324 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/4038368577 2024-09-18T05:34:59.058452+0000 2024-09-17T05:35:10.324 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6832/4038368577 2024-09-18T05:34:59.058452+0000 2024-09-17T05:35:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-09-17T05:35:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 msgr2=0x7f7f48075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 0x7f7f48075740 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009c60 tx=0x7f7f5c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 msgr2=0x7f7f6c14ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f7f54011b10 tx=0x7f7f54011f10 comp rx=0 tx=0).stop 2024-09-17T05:35:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 shutdown_connections 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7f48073230 0x7f7f48075740 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7f6c150950 0x7f7f6c150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c15f090 0x7f7f6c14ce50 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 --2- 172.21.15.43:0/551957959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f50004d50 0x7f7f6c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 >> 172.21.15.43:0/551957959 conn(0x7f7f6c0d1160 msgr2=0x7f7f6c0b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 shutdown_connections 2024-09-17T05:35:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.321+0000 7f7f72200640 1 -- 172.21.15.43:0/551957959 wait complete. 2024-09-17T05:35:10.346 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:35:10.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:10.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:35:10.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:35:10.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:10.373 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:35:10.428 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:35:10.428 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:35:10.428 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:10.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:35:10.439 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:35:10.439 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:10.439 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:10.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:35:10.479 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:35:10.479 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:35:10.479 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:10.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:35:10.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:10.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 -- 172.21.15.43:0/3065941891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240d0c20 msgr2=0x7fcb2406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 --2- 172.21.15.43:0/3065941891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240d0c20 0x7fcb2406ce60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009c80 tx=0x7fcb1401cd10 comp rx=0 tx=0).stop 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 -- 172.21.15.43:0/3065941891 shutdown_connections 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 --2- 172.21.15.43:0/3065941891 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb240ce810 0x7fcb24169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 --2- 172.21.15.43:0/3065941891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240d0c20 0x7fcb2406ce60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 -- 172.21.15.43:0/3065941891 >> 172.21.15.43:0/3065941891 conn(0x7fcb240b23d0 msgr2=0x7fcb240d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 -- 172.21.15.43:0/3065941891 shutdown_connections 2024-09-17T05:35:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.677+0000 7fcb29200640 1 -- 172.21.15.43:0/3065941891 wait complete. 2024-09-17T05:35:10.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 Processor -- start 2024-09-17T05:35:10.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- start start 2024-09-17T05:35:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb2406d3a0 0x7fcb24099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240ce810 0x7fcb240cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2416c570 con 0x7fcb240d0c20 2024-09-17T05:35:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2416c3f0 con 0x7fcb240ce810 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2416c6f0 con 0x7fcb2406d3a0 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37996/0 (socket says 172.21.15.43:37996) 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 -- 172.21.15.43:0/4013848383 learned_addr learned my addr 172.21.15.43:0/4013848383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb21e00640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb2406d3a0 0x7fcb24099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb19400640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240ce810 0x7fcb240cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 -- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb2406d3a0 msgr2=0x7fcb24099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb2406d3a0 0x7fcb24099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 -- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240ce810 msgr2=0x7fcb240cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240ce810 0x7fcb240cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 -- 172.21.15.43:0/4013848383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb240c9c10 con 0x7fcb240d0c20 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb22800640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e970 tx=0x7fcb1c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00ccc0 con 0x7fcb240d0c20 2024-09-17T05:35:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb1c010430 con 0x7fcb240d0c20 2024-09-17T05:35:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb240c9ef0 con 0x7fcb240d0c20 2024-09-17T05:35:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c010880 con 0x7fcb240d0c20 2024-09-17T05:35:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.681+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb240c8120 con 0x7fcb240d0c20 2024-09-17T05:35:10.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2406ce60 con 0x7fcb240d0c20 2024-09-17T05:35:10.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb1c01f020 con 0x7fcb240d0c20 2024-09-17T05:35:10.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb1aa00640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 0x7fcb00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7fcb1c094a80 con 0x7fcb240d0c20 2024-09-17T05:35:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb21e00640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 0x7fcb00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.685+0000 7fcb21e00640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 0x7fcb00075710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcb140af000 tx=0x7fcb140a5f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.689+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb1c096e20 con 0x7fcb240d0c20 2024-09-17T05:35:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.901+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb240ca0c0 con 0x7fcb240d0c20 2024-09-17T05:35:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb1aa00640 1 -- 172.21.15.43:0/4013848383 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+5235 (secure 0 0 0) 0x7fcb1c05fa00 con 0x7fcb240d0c20 2024-09-17T05:35:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:10.908 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":352,"btime":"2024-09-17T05:35:10:297455+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":352,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:10.080533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":6,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:10.908 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 352 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 msgr2=0x7fcb00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 0x7fcb00075710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcb140af000 tx=0x7fcb140a5f90 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 msgr2=0x7fcb240cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e970 tx=0x7fcb1c00ee30 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 shutdown_connections 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fcb00073200 0x7fcb00075710 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb240ce810 0x7fcb240cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb240d0c20 0x7fcb240cbdb0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 --2- 172.21.15.43:0/4013848383 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fcb2406d3a0 0x7fcb24099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 >> 172.21.15.43:0/4013848383 conn(0x7fcb240b23d0 msgr2=0x7fcb241694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 shutdown_connections 2024-09-17T05:35:10.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:10.905+0000 7fcb29200640 1 -- 172.21.15.43:0/4013848383 wait complete. 2024-09-17T05:35:10.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 -- 172.21.15.43:0/3348146179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 msgr2=0x7f68a815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 --2- 172.21.15.43:0/3348146179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815c7e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6898009840 tx=0x7f689801c790 comp rx=0 tx=0).stop 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 -- 172.21.15.43:0/3348146179 shutdown_connections 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 --2- 172.21.15.43:0/3348146179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815c7e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 -- 172.21.15.43:0/3348146179 >> 172.21.15.43:0/3348146179 conn(0x7f68a80b23d0 msgr2=0x7f68a80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 -- 172.21.15.43:0/3348146179 shutdown_connections 2024-09-17T05:35:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.085+0000 7f68aec00640 1 -- 172.21.15.43:0/3348146179 wait complete. 2024-09-17T05:35:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 Processor -- start 2024-09-17T05:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- start start 2024-09-17T05:35:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68a806f110 0x7f68a81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8169d70 con 0x7f68a80cd530 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8169bf0 con 0x7f68a8155d70 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f68a8169ef0 con 0x7f68a806f110 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38006/0 (socket says 172.21.15.43:38006) 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 -- 172.21.15.43:0/3521096901 learned_addr learned my addr 172.21.15.43:0/3521096901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68a7e00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68a806f110 0x7f68a81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f689fe00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 -- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68a806f110 msgr2=0x7f68a81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68a806f110 0x7f68a81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 -- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 msgr2=0x7f68a815e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 -- 172.21.15.43:0/3521096901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a80cc4a0 con 0x7f68a80cd530 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f689fe00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68acc00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f68a000c730 tx=0x7f68a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0003b80 con 0x7f68a80cd530 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a0003d20 con 0x7f68a80cd530 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a000aab0 con 0x7f68a80cd530 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a80cc780 con 0x7f68a80cd530 2024-09-17T05:35:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.089+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a809a410 con 0x7f68a80cd530 2024-09-17T05:35:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.093+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a815c7e0 con 0x7f68a80cd530 2024-09-17T05:35:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.093+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68a000ac50 con 0x7f68a80cd530 2024-09-17T05:35:11.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.093+0000 7f68a4c00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 0x7f6884075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.093+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f68a00940f0 con 0x7f68a80cd530 2024-09-17T05:35:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.093+0000 7f68a7e00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 0x7f6884075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.097+0000 7f68a7e00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 0x7f6884075710 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f68980af000 tx=0x7f689801afd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.097+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f68a005efe0 con 0x7f68a80cd530 2024-09-17T05:35:11.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68a80ad1e0 con 0x7f68a80cd530 2024-09-17T05:35:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68a4c00640 1 -- 172.21.15.43:0/3521096901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+5235 (secure 0 0 0) 0x7f68a005e8e0 con 0x7f68a80cd530 2024-09-17T05:35:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:11.313 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":352,"btime":"2024-09-17T05:35:10:297455+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":352,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:10.080533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":6,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:11.313 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 352 2024-09-17T05:35:11.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 msgr2=0x7f6884075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 0x7f6884075710 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f68980af000 tx=0x7f689801afd0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 msgr2=0x7f68a815ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f68a000c730 tx=0x7f68a000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 shutdown_connections 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f6884073200 0x7f6884075710 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68a8155d70 0x7f68a815e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a80cd530 0x7f68a815ea60 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 --2- 172.21.15.43:0/3521096901 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f68a806f110 0x7f68a81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.309+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 >> 172.21.15.43:0/3521096901 conn(0x7f68a80b23d0 msgr2=0x7f68a815f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.313+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 shutdown_connections 2024-09-17T05:35:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.313+0000 7f68aec00640 1 -- 172.21.15.43:0/3521096901 wait complete. 2024-09-17T05:35:11.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 -- 172.21.15.43:0/1718777608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0406fce0 msgr2=0x7f0d040700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 --2- 172.21.15.43:0/1718777608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0406fce0 0x7f0d040700f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009bc0 tx=0x7f0cfc01c9a0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 -- 172.21.15.43:0/1718777608 shutdown_connections 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 --2- 172.21.15.43:0/1718777608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0406fce0 0x7f0d040700f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 --2- 172.21.15.43:0/1718777608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d0415f470 0x7f0d040d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 -- 172.21.15.43:0/1718777608 >> 172.21.15.43:0/1718777608 conn(0x7f0d040b2810 msgr2=0x7f0d040b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 -- 172.21.15.43:0/1718777608 shutdown_connections 2024-09-17T05:35:11.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.489+0000 7f0d0a800640 1 -- 172.21.15.43:0/1718777608 wait complete. 2024-09-17T05:35:11.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 Processor -- start 2024-09-17T05:35:11.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 -- start start 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d040b5610 0x7f0d0414dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0415f470 0x7f0d0415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041695c0 con 0x7f0d0406fce0 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04169440 con 0x7f0d0415f470 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d0a800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f0d04169740 con 0x7f0d040b5610 2024-09-17T05:35:11.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38016/0 (socket says 172.21.15.43:38016) 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 -- 172.21.15.43:0/2318459243 learned_addr learned my addr 172.21.15.43:0/2318459243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0cfbe00640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d040b5610 0x7f0d0414dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03e00640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0415f470 0x7f0d0415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 -- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d040b5610 msgr2=0x7f0d0414dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d040b5610 0x7f0d0414dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 -- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0415f470 msgr2=0x7f0d0415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0415f470 0x7f0d0415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.493+0000 7f0d03400640 1 -- 172.21.15.43:0/2318459243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0414e270 con 0x7f0d0406fce0 2024-09-17T05:35:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0d03400640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e770 tx=0x7f0cf400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400cc60 con 0x7f0d0406fce0 2024-09-17T05:35:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cf400ce00 con 0x7f0d0406fce0 2024-09-17T05:35:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004280 con 0x7f0d0406fce0 2024-09-17T05:35:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d041f7520 con 0x7f0d0406fce0 2024-09-17T05:35:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d041f7ae0 con 0x7f0d0406fce0 2024-09-17T05:35:11.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d040700f0 con 0x7f0d0406fce0 2024-09-17T05:35:11.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0cf4003600 con 0x7f0d0406fce0 2024-09-17T05:35:11.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.497+0000 7f0cfb400640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 0x7f0ce0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.501+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f0cf4094cc0 con 0x7f0d0406fce0 2024-09-17T05:35:11.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.501+0000 7f0cfbe00640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 0x7f0ce0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.501+0000 7f0cfbe00640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 0x7f0ce0075740 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0d040cd3b0 tx=0x7f0cec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.505+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0cf409b080 con 0x7f0d0406fce0 2024-09-17T05:35:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.677+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d040749e0 con 0x7f0d0406fce0 2024-09-17T05:35:11.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.677+0000 7f0cfb400640 1 -- 172.21.15.43:0/2318459243 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16845 (secure 0 0 0) 0x7f0cf405fbb0 con 0x7f0d0406fce0 2024-09-17T05:35:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:11.683 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":142,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:35:06.281368+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:41.643835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:42.859239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.13","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/4038368577":"2024-09-18T05:34:59.058452+0000","172.21.15.43:6832/4038368577":"2024-09-18T05:34:59.058452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:35:11.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 msgr2=0x7f0ce0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 0x7f0ce0075740 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0d040cd3b0 tx=0x7f0cec009040 comp rx=0 tx=0).stop 2024-09-17T05:35:11.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 msgr2=0x7f0d04155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e770 tx=0x7f0cf400ec30 comp rx=0 tx=0).stop 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 shutdown_connections 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f0ce0073230 0x7f0ce0075740 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d0415f470 0x7f0d0415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f0d040b5610 0x7f0d0414dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 --2- 172.21.15.43:0/2318459243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d0406fce0 0x7f0d04155e50 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 >> 172.21.15.43:0/2318459243 conn(0x7f0d040b2810 msgr2=0x7f0d0415fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 shutdown_connections 2024-09-17T05:35:11.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.681+0000 7f0d0a800640 1 -- 172.21.15.43:0/2318459243 wait complete. 2024-09-17T05:35:11.706 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:35:11.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 -- 172.21.15.43:0/4173229220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec8155d70 msgr2=0x7faec815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 --2- 172.21.15.43:0/4173229220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec8155d70 0x7faec815c7e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009840 tx=0x7faeb801c760 comp rx=0 tx=0).stop 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 -- 172.21.15.43:0/4173229220 shutdown_connections 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 --2- 172.21.15.43:0/4173229220 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec80b3af0 0x7faec80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 --2- 172.21.15.43:0/4173229220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec8155d70 0x7faec815c7e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 -- 172.21.15.43:0/4173229220 >> 172.21.15.43:0/4173229220 conn(0x7faec80b23d0 msgr2=0x7faec80d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 -- 172.21.15.43:0/4173229220 shutdown_connections 2024-09-17T05:35:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.861+0000 7faece000640 1 -- 172.21.15.43:0/4173229220 wait complete. 2024-09-17T05:35:11.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 Processor -- start 2024-09-17T05:35:11.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- start start 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8155d70 0x7faec80cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 0x7faec81fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faec8168c60 con 0x7faec81f74a0 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faec8168ae0 con 0x7faec806f110 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7faec8168de0 con 0x7faec8155d70 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 0x7faec81fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.43:37956/0 (socket says 172.21.15.43:37956) 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 -- 172.21.15.43:0/2372993561 learned_addr learned my addr 172.21.15.43:0/2372993561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faebfe00640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8155d70 0x7faec80cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 -- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8155d70 msgr2=0x7faec80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8155d70 0x7faec80cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 -- 172.21.15.43:0/2372993561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 msgr2=0x7faec81fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 0x7faec81fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 -- 172.21.15.43:0/2372993561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec80c95b0 con 0x7faec806f110 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7e00640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 0x7faec81fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faec7400640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009790 tx=0x7faeb80a56a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb80bb070 con 0x7faec806f110 2024-09-17T05:35:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeb8004470 con 0x7faec806f110 2024-09-17T05:35:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb80b2040 con 0x7faec806f110 2024-09-17T05:35:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec80c9890 con 0x7faec806f110 2024-09-17T05:35:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.865+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec81f7f10 con 0x7faec806f110 2024-09-17T05:35:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.869+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec815c7d0 con 0x7faec806f110 2024-09-17T05:35:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.869+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faeb8005290 con 0x7faec806f110 2024-09-17T05:35:11.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.869+0000 7faebf400640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 0x7faea4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.869+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7faeb812e040 con 0x7faec806f110 2024-09-17T05:35:11.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.869+0000 7faebfe00640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 0x7faea4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.873+0000 7faebfe00640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 0x7faea4075710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7faeb0007fd0 tx=0x7faeb0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:11.873+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faeb8016410 con 0x7faec806f110 2024-09-17T05:35:12.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.085+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec80cc880 con 0x7faec806f110 2024-09-17T05:35:12.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.085+0000 7faebf400640 1 -- 172.21.15.43:0/2372993561 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+5235 (secure 0 0 0) 0x7faeb80f8980 con 0x7faec806f110 2024-09-17T05:35:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:12.090 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":352,"btime":"2024-09-17T05:35:10:297455+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":352,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:10.080533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":6,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:12.090 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 352 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 msgr2=0x7faea4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 0x7faea4075710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7faeb0007fd0 tx=0x7faeb0009040 comp rx=0 tx=0).stop 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 msgr2=0x7faec80ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009790 tx=0x7faeb80a56a0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 shutdown_connections 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7faea4073200 0x7faea4075710 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec81f74a0 0x7faec81fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7faec8155d70 0x7faec80cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 --2- 172.21.15.43:0/2372993561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faec806f110 0x7faec80ca8e0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 >> 172.21.15.43:0/2372993561 conn(0x7faec80b23d0 msgr2=0x7faec815d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 shutdown_connections 2024-09-17T05:35:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.089+0000 7faece000640 1 -- 172.21.15.43:0/2372993561 wait complete. 2024-09-17T05:35:12.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:12.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.265+0000 7f7233400640 1 -- 172.21.15.43:0/3610798735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0b7ce0 msgr2=0x7f722c0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.265+0000 7f7233400640 1 --2- 172.21.15.43:0/3610798735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0b7ce0 0x7f722c0d0840 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f72200098f0 tx=0x7f722001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- 172.21.15.43:0/3610798735 shutdown_connections 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 --2- 172.21.15.43:0/3610798735 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 --2- 172.21.15.43:0/3610798735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0b7ce0 0x7f722c0d0840 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- 172.21.15.43:0/3610798735 >> 172.21.15.43:0/3610798735 conn(0x7f722c0cee80 msgr2=0x7f722c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- 172.21.15.43:0/3610798735 shutdown_connections 2024-09-17T05:35:12.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- 172.21.15.43:0/3610798735 wait complete. 2024-09-17T05:35:12.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 Processor -- start 2024-09-17T05:35:12.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- start start 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f722c0b7ce0 0x7f722c1f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f722c169e10 con 0x7f722c0cd3b0 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f722c169c90 con 0x7f722c0b7ce0 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.269+0000 7f7233400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f722c169f90 con 0x7f722c0d0d80 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38054/0 (socket says 172.21.15.43:38054) 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 -- 172.21.15.43:0/589006308 learned_addr learned my addr 172.21.15.43:0/589006308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7230a00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f722c0b7ce0 0x7f722c1f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7231400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 -- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 msgr2=0x7f722c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 -- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f722c0b7ce0 msgr2=0x7f722c1f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f722c0b7ce0 0x7f722c1f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 -- 172.21.15.43:0/589006308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c0afa40 con 0x7f722c0cd3b0 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7231400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f722be00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f721800d720 tx=0x7f721800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218017070 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7218005310 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218009820 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c1f9af0 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c15db20 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722c0d0840 con 0x7f722c0cd3b0 2024-09-17T05:35:12.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.273+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f72180099c0 con 0x7f722c0cd3b0 2024-09-17T05:35:12.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.277+0000 7f7228c00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 0x7f720c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.277+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f7218094390 con 0x7f722c0cd3b0 2024-09-17T05:35:12.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.277+0000 7f7230a00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 0x7f720c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.277+0000 7f7230a00640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 0x7f720c075710 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f72200af000 tx=0x7f722001b120 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:12.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.281+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f721805ecc0 con 0x7f722c0cd3b0 2024-09-17T05:35:12.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.493+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f722c0b0b70 con 0x7f722c0cd3b0 2024-09-17T05:35:12.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.493+0000 7f7228c00640 1 -- 172.21.15.43:0/589006308 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+5235 (secure 0 0 0) 0x7f72180636f0 con 0x7f722c0cd3b0 2024-09-17T05:35:12.498 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:12.498 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":352,"btime":"2024-09-17T05:35:10:297455+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340}],"filesystems":[{"mdsmap":{"epoch":352,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:43.887777+0000","modified":"2024-09-17T05:35:10.080533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10036":{"gid":10036,"name":"c","rank":0,"incarnation":348,"state":"up:active","state_seq":6,"addr":"172.21.15.43:6834/1735972562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1735972562},{"type":"v1","addr":"172.21.15.43:6834","nonce":1735972562}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10036,"qdb_cluster":[10036]},"id":7}]} 2024-09-17T05:35:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 352 2024-09-17T05:35:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 msgr2=0x7f720c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 0x7f720c075710 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f72200af000 tx=0x7f722001b120 comp rx=0 tx=0).stop 2024-09-17T05:35:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 msgr2=0x7f722c1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f721800d720 tx=0x7f721800dbe0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 shutdown_connections 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f720c073200 0x7f720c075710 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f722c0d0d80 0x7f722c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f722c0cd3b0 0x7f722c1533c0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 --2- 172.21.15.43:0/589006308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f722c0b7ce0 0x7f722c1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 >> 172.21.15.43:0/589006308 conn(0x7f722c0cee80 msgr2=0x7f722c0af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 shutdown_connections 2024-09-17T05:35:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.497+0000 7f7233400640 1 -- 172.21.15.43:0/589006308 wait complete. 2024-09-17T05:35:12.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- 172.21.15.43:0/1741748391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 msgr2=0x7ff8e4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 --2- 172.21.15.43:0/1741748391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff8e4003f90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c310 tx=0x7ff8f001cf40 comp rx=0 tx=0).stop 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff8ff400640 1 -- 172.21.15.43:0/1741748391 reap_dead start 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- 172.21.15.43:0/1741748391 shutdown_connections 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- 172.21.15.43:0/1741748391 >> 172.21.15.43:0/1741748391 conn(0x7ff9000b23d0 msgr2=0x7ff9000d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- 172.21.15.43:0/1741748391 shutdown_connections 2024-09-17T05:35:12.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- 172.21.15.43:0/1741748391 wait complete. 2024-09-17T05:35:12.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 Processor -- start 2024-09-17T05:35:12.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.677+0000 7ff905800640 1 -- start start 2024-09-17T05:35:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9000b11d0 0x7ff900159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff900159750 0x7ff9000ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff900168ed0 con 0x7ff8e4003b20 2024-09-17T05:35:12.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff900168d50 con 0x7ff9000b11d0 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff900169050 con 0x7ff900159750 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38068/0 (socket says 172.21.15.43:38068) 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 -- 172.21.15.43:0/1257912602 learned_addr learned my addr 172.21.15.43:0/1257912602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8ff400640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff900159750 0x7ff9000ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8f7e00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9000b11d0 0x7ff900159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 -- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff900159750 msgr2=0x7ff9000ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff900159750 0x7ff9000ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 -- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9000b11d0 msgr2=0x7ff900159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9000b11d0 0x7ff900159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 -- 172.21.15.43:0/1257912602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff900159b60 con 0x7ff8e4003b20 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8fea00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7ff8f001b640 tx=0x7ff8f0001db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:12.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0001030 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f00014c0 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001fa5e0 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0006330 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001faba0 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.681+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9000b4b80 con 0x7ff8e4003b20 2024-09-17T05:35:12.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.685+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff8f00b9420 con 0x7ff8e4003b20 2024-09-17T05:35:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.685+0000 7ff8f6a00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 0x7ff8dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.685+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7ff8f012dba0 con 0x7ff8e4003b20 2024-09-17T05:35:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.685+0000 7ff8f7e00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 0x7ff8dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.685+0000 7ff8f7e00640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 0x7ff8dc075740 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff90009e610 tx=0x7ff8e80091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:12.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.689+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff900159b60 con 0x7ff8e4003b20 2024-09-17T05:35:12.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.869+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9000cd900 con 0x7ff8e4003b20 2024-09-17T05:35:12.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.869+0000 7ff8f6a00640 1 -- 172.21.15.43:0/1257912602 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16845 (secure 0 0 0) 0x7ff8f0016cf0 con 0x7ff8e4003b20 2024-09-17T05:35:12.873 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:12.874 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":142,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:35:06.281368+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:41.643835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:42.859239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.13","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/4038368577":"2024-09-18T05:34:59.058452+0000","172.21.15.43:6832/4038368577":"2024-09-18T05:34:59.058452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 msgr2=0x7ff8dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 0x7ff8dc075740 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff90009e610 tx=0x7ff8e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 msgr2=0x7ff9001f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7ff8f001b640 tx=0x7ff8f0001db0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 shutdown_connections 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7ff8dc073230 0x7ff8dc075740 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9000b11d0 0x7ff900159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff900159750 0x7ff9000ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 --2- 172.21.15.43:0/1257912602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8e4003b20 0x7ff9001f67b0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 >> 172.21.15.43:0/1257912602 conn(0x7ff9000b23d0 msgr2=0x7ff90015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 shutdown_connections 2024-09-17T05:35:12.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:12.873+0000 7ff905800640 1 -- 172.21.15.43:0/1257912602 wait complete. 2024-09-17T05:35:12.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 -- 172.21.15.43:0/315764931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f406f110 msgr2=0x7f87f40b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 --2- 172.21.15.43:0/315764931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f406f110 0x7f87f40b33e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009840 tx=0x7f87dc01c760 comp rx=0 tx=0).stop 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 -- 172.21.15.43:0/315764931 shutdown_connections 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 --2- 172.21.15.43:0/315764931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f406f110 0x7f87f40b33e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 --2- 172.21.15.43:0/315764931 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f4155d70 0x7f87f415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 -- 172.21.15.43:0/315764931 >> 172.21.15.43:0/315764931 conn(0x7f87f40b23d0 msgr2=0x7f87f40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 -- 172.21.15.43:0/315764931 shutdown_connections 2024-09-17T05:35:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.053+0000 7f87f9a00640 1 -- 172.21.15.43:0/315764931 wait complete. 2024-09-17T05:35:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 Processor -- start 2024-09-17T05:35:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- start start 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f40b3920 0x7f87f4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f4155d70 0x7f87f40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4168a60 con 0x7f87f406f110 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41688e0 con 0x7f87f4155d70 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f87f4168be0 con 0x7f87f40b3920 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38076/0 (socket says 172.21.15.43:38076) 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 -- 172.21.15.43:0/2345197329 learned_addr learned my addr 172.21.15.43:0/2345197329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f40b3920 0x7f87f4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f3400640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f4155d70 0x7f87f40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 -- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f40b3920 msgr2=0x7f87f4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f40b3920 0x7f87f4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 -- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f4155d70 msgr2=0x7f87f40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f4155d70 0x7f87f40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 -- 172.21.15.43:0/2345197329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f4074f80 con 0x7f87f406f110 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f2a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e790 tx=0x7f87e400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e400eea0 con 0x7f87f406f110 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e400cc80 con 0x7f87f406f110 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f4074ca0 con 0x7f87f406f110 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004290 con 0x7f87f406f110 2024-09-17T05:35:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.057+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f40c7b90 con 0x7f87f406f110 2024-09-17T05:35:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.061+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f40b33e0 con 0x7f87f406f110 2024-09-17T05:35:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.061+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87e4003600 con 0x7f87f406f110 2024-09-17T05:35:13.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.061+0000 7f87eaa00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 0x7f87d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.061+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5853+0+0 (secure 0 0 0) 0x7f87e4094b80 con 0x7f87f406f110 2024-09-17T05:35:13.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.061+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 0x7f87d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.065+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 0x7f87d0075710 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0af000 tx=0x7f87dc01afa0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:13.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.065+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87e405fb90 con 0x7f87f406f110 2024-09-17T05:35:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.273+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87f40b3680 con 0x7f87f406f110 2024-09-17T05:35:13.279 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-09-17T05:35:13.273+0000 7fcc7d600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:35:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.277+0000 7f87eaa00640 1 -- 172.21.15.43:0/2345197329 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v353) ==== 161+0+0 (secure 0 0 0) 0x7f87e405f490 con 0x7f87f406f110 2024-09-17T05:35:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:35:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 msgr2=0x7f87d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 0x7f87d0075710 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0af000 tx=0x7f87dc01afa0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 msgr2=0x7f87f40c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e790 tx=0x7f87e400ec50 comp rx=0 tx=0).stop 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 shutdown_connections 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f87d0073200 0x7f87d0075710 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87f4155d70 0x7f87f40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f87f40b3920 0x7f87f4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 --2- 172.21.15.43:0/2345197329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f406f110 0x7f87f40c8390 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 >> 172.21.15.43:0/2345197329 conn(0x7f87f40b23d0 msgr2=0x7f87f40d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 shutdown_connections 2024-09-17T05:35:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.281+0000 7f87f9a00640 1 -- 172.21.15.43:0/2345197329 wait complete. 2024-09-17T05:35:13.291 INFO:tasks.ceph.mds.c.smithi043.stderr: -1188> 2024-09-17T05:35:05.069+0000 7ff593e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:35:13.292 INFO:tasks.ceph.mds.c.smithi043.stderr: -735> 2024-09-17T05:35:06.333+0000 7ff58ee00640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-17T05:35:13.292 INFO:tasks.ceph.mds.c.smithi043.stderr: -493> 2024-09-17T05:35:07.741+0000 7ff58ee00640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-17T05:35:13.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:35:13.415 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- 172.21.15.43:0/440380746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c406ce60 msgr2=0x7f12c406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 --2- 172.21.15.43:0/440380746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c406ce60 0x7f12c406d270 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f12c40d12a0 tx=0x7f12ac01c830 comp rx=0 tx=0).stop 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- 172.21.15.43:0/440380746 shutdown_connections 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 --2- 172.21.15.43:0/440380746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c406ce60 0x7f12c406d270 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 --2- 172.21.15.43:0/440380746 >> v2:172.21.15.62:3300/0 conn(0x7f12c4070620 0x7f12c40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- 172.21.15.43:0/440380746 >> 172.21.15.43:0/440380746 conn(0x7f12c40b23d0 msgr2=0x7f12c40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- 172.21.15.43:0/440380746 shutdown_connections 2024-09-17T05:35:13.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- 172.21.15.43:0/440380746 wait complete. 2024-09-17T05:35:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 Processor -- start 2024-09-17T05:35:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.481+0000 7f12cb600640 1 -- start start 2024-09-17T05:35:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 0x7f12c409ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 0x7f12c4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12c40ce490 con 0x7f12c40cb860 2024-09-17T05:35:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12c40ce310 con 0x7f12c4070620 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f12c40ce610 con 0x7f12c406ce60 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c8c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 0x7f12c409ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38104/0 (socket says 172.21.15.43:38104) 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 -- 172.21.15.43:0/4109780921 learned_addr learned my addr 172.21.15.43:0/4109780921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12bbe00640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 0x7f12c4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 -- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 msgr2=0x7f12c409ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 0x7f12c409ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 -- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 msgr2=0x7f12c4154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 0x7f12c4154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 -- 172.21.15.43:0/4109780921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c40ca0f0 con 0x7f12c40cb860 2024-09-17T05:35:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c8c00640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 0x7f12c409ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:13.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12bbe00640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 0x7f12c4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:13.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c9600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c710 tx=0x7f12bc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc003b80 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12bc003d20 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c40c9e10 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00aab0 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c41588b0 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c40c9b80 con 0x7f12c40cb860 2024-09-17T05:35:13.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.485+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f12bc00ac50 con 0x7f12c40cb860 2024-09-17T05:35:13.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.489+0000 7f12c1600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 0x7f12a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:13.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.489+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5939+0+0 (secure 0 0 0) 0x7f12bc0940a0 con 0x7f12c40cb860 2024-09-17T05:35:13.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.489+0000 7f12c8c00640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 0x7f12a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:13.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.489+0000 7f12c8c00640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 0x7f12a0075710 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009f30 tx=0x7f12b40091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.493+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12bc00ce20 con 0x7f12c40cb860 2024-09-17T05:35:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:13.705+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f12c40cebd0 con 0x7f12c40cb860 2024-09-17T05:35:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.317+0000 7f12c1600640 1 -- 172.21.15.43:0/4109780921 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v354) ==== 104+0+0 (secure 0 0 0) 0x7f12bc05e8e0 con 0x7f12c40cb860 2024-09-17T05:35:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 msgr2=0x7f12a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 0x7f12a0075710 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009f30 tx=0x7f12b40091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 msgr2=0x7f12c4154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c710 tx=0x7f12bc00cbd0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 shutdown_connections 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f12a0073200 0x7f12a0075710 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12c4070620 0x7f12c4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb860 0x7f12c4154840 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 --2- 172.21.15.43:0/4109780921 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f12c406ce60 0x7f12c409ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 >> 172.21.15.43:0/4109780921 conn(0x7f12c40b23d0 msgr2=0x7f12c40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 shutdown_connections 2024-09-17T05:35:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.321+0000 7f12cb600640 1 -- 172.21.15.43:0/4109780921 wait complete. 2024-09-17T05:35:14.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:35:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- 172.21.15.43:0/1918051487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 msgr2=0x7fdbe00cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 --2- 172.21.15.43:0/1918051487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe00cf270 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0072bd0 tx=0x7fdbc801c800 comp rx=0 tx=0).stop 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- 172.21.15.43:0/1918051487 shutdown_connections 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 --2- 172.21.15.43:0/1918051487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe00cf270 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 --2- 172.21.15.43:0/1918051487 >> v2:172.21.15.62:3300/0 conn(0x7fdbe0074c60 0x7fdbe00b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- 172.21.15.43:0/1918051487 >> 172.21.15.43:0/1918051487 conn(0x7fdbe006ce60 msgr2=0x7fdbe006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- 172.21.15.43:0/1918051487 shutdown_connections 2024-09-17T05:35:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- 172.21.15.43:0/1918051487 wait complete. 2024-09-17T05:35:14.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 Processor -- start 2024-09-17T05:35:14.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.517+0000 7fdbe5600640 1 -- start start 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe0074c60 0x7fdbe0159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe00cf7b0 0x7fdbe015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0168d10 con 0x7fdbe015b7e0 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0168b90 con 0x7fdbe00cf7b0 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe0168e90 con 0x7fdbe0074c60 2024-09-17T05:35:14.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46680/0 (socket says 172.21.15.43:46680) 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 -- 172.21.15.43:0/3193775818 learned_addr learned my addr 172.21.15.43:0/3193775818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbd7e00640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe00cf7b0 0x7fdbe015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdea00640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe0074c60 0x7fdbe0159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 -- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe0074c60 msgr2=0x7fdbe0159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe0074c60 0x7fdbe0159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 -- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe00cf7b0 msgr2=0x7fdbe015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe00cf7b0 0x7fdbe015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 -- 172.21.15.43:0/3193775818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe01fc820 con 0x7fdbe015b7e0 2024-09-17T05:35:14.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbdf400640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c790 tx=0x7fdbd800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8003db0 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbd800ce40 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80052d0 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe01fcb00 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01fd0c0 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.521+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe015cd40 con 0x7fdbe015b7e0 2024-09-17T05:35:14.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.525+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdbd8005470 con 0x7fdbe015b7e0 2024-09-17T05:35:14.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.525+0000 7fdbd6a00640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 0x7fdbbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.525+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5939+0+0 (secure 0 0 0) 0x7fdbd8093fb0 con 0x7fdbe015b7e0 2024-09-17T05:35:14.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.525+0000 7fdbdea00640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 0x7fdbbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.525+0000 7fdbdea00640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 0x7fdbbc075740 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009b40 tx=0x7fdbd00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.529+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdbe00cfbc0 con 0x7fdbe015b7e0 2024-09-17T05:35:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:14.705+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdbe006a720 con 0x7fdbe015b7e0 2024-09-17T05:35:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.321+0000 7fdbd6a00640 1 -- 172.21.15.43:0/3193775818 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) ==== 181+0+0 (secure 0 0 0) 0x7fdbd805e880 con 0x7fdbe015b7e0 2024-09-17T05:35:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 msgr2=0x7fdbbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 0x7fdbbc075740 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009b40 tx=0x7fdbd00091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 msgr2=0x7fdbe0150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800c790 tx=0x7fdbd800cc50 comp rx=0 tx=0).stop 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 shutdown_connections 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdbbc073230 0x7fdbbc075740 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbe00cf7b0 0x7fdbe015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe015b7e0 0x7fdbe0150170 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 --2- 172.21.15.43:0/3193775818 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdbe0074c60 0x7fdbe0159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 >> 172.21.15.43:0/3193775818 conn(0x7fdbe006ce60 msgr2=0x7fdbe0075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 shutdown_connections 2024-09-17T05:35:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.325+0000 7fdbe5600640 1 -- 172.21.15.43:0/3193775818 wait complete. 2024-09-17T05:35:15.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 -- 172.21.15.43:0/1836721787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c8074c60 msgr2=0x7f66c80b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 --2- 172.21.15.43:0/1836721787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c8074c60 0x7f66c80b3b80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009840 tx=0x7f66bc01c820 comp rx=0 tx=0).stop 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 -- 172.21.15.43:0/1836721787 shutdown_connections 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 --2- 172.21.15.43:0/1836721787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c8074c60 0x7f66c80b3b80 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 -- 172.21.15.43:0/1836721787 >> 172.21.15.43:0/1836721787 conn(0x7f66c806ce60 msgr2=0x7f66c806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 -- 172.21.15.43:0/1836721787 shutdown_connections 2024-09-17T05:35:15.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.517+0000 7f66d0600640 1 -- 172.21.15.43:0/1836721787 wait complete. 2024-09-17T05:35:15.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 Processor -- start 2024-09-17T05:35:15.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 -- start start 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c8074c60 0x7f66c80cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66c815b7e0 0x7f66c815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8169f00 con 0x7f66c81fcbf0 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8169d80 con 0x7f66c8074c60 2024-09-17T05:35:15.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66d0600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f66c816a080 con 0x7f66c815b7e0 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66ce600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66cdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c8074c60 0x7f66c80cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66ce600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46700/0 (socket says 172.21.15.43:46700) 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66ce600640 1 -- 172.21.15.43:0/3921096855 learned_addr learned my addr 172.21.15.43:0/3921096855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.521+0000 7f66cd200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66c815b7e0 0x7f66c815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 -- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66c815b7e0 msgr2=0x7f66c815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66c815b7e0 0x7f66c815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 -- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c8074c60 msgr2=0x7f66c80cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c8074c60 0x7f66c80cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 -- 172.21.15.43:0/3921096855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c815c0b0 con 0x7f66c81fcbf0 2024-09-17T05:35:15.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66ce600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cb10 tx=0x7f66c000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0016020 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c000ce50 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c815d700 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c000ad60 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c815d9c0 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c81fd0a0 con 0x7f66c81fcbf0 2024-09-17T05:35:15.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66c0019020 con 0x7f66c81fcbf0 2024-09-17T05:35:15.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.525+0000 7f66c5600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 0x7f66a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.529+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5443+0+0 (secure 0 0 0) 0x7f66c0094300 con 0x7f66c81fcbf0 2024-09-17T05:35:15.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.529+0000 7f66cdc00640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 0x7f66a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.529+0000 7f66cdc00640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 0x7f66a8075710 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0af000 tx=0x7f66bc01b060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.533+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f66c0020020 con 0x7f66c81fcbf0 2024-09-17T05:35:15.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:15.705+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66c80b3b60 con 0x7f66c81fcbf0 2024-09-17T05:35:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.333+0000 7f66c5600640 1 -- 172.21.15.43:0/3921096855 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) ==== 169+0+0 (secure 0 0 0) 0x7f66c815c0b0 con 0x7f66c81fcbf0 2024-09-17T05:35:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-17T05:35:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 msgr2=0x7f66a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 0x7f66a8075710 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0af000 tx=0x7f66bc01b060 comp rx=0 tx=0).stop 2024-09-17T05:35:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 msgr2=0x7f66c80997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cb10 tx=0x7f66c000cc00 comp rx=0 tx=0).stop 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 shutdown_connections 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f66a8073200 0x7f66a8075710 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c81fcbf0 0x7f66c80997d0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f66c815b7e0 0x7f66c815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 --2- 172.21.15.43:0/3921096855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66c8074c60 0x7f66c80cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 >> 172.21.15.43:0/3921096855 conn(0x7f66c806ce60 msgr2=0x7f66c80afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.337+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 shutdown_connections 2024-09-17T05:35:16.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.341+0000 7f66d0600640 1 -- 172.21.15.43:0/3921096855 wait complete. 2024-09-17T05:35:16.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:35:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 -- 172.21.15.43:0/1665699734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae406d570 msgr2=0x7f8ae40cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 --2- 172.21.15.43:0/1665699734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae406d570 0x7f8ae40cb860 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009bc0 tx=0x7f8adc01cb10 comp rx=0 tx=0).stop 2024-09-17T05:35:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 -- 172.21.15.43:0/1665699734 shutdown_connections 2024-09-17T05:35:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 --2- 172.21.15.43:0/1665699734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae406d570 0x7f8ae40cb860 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 -- 172.21.15.43:0/1665699734 >> 172.21.15.43:0/1665699734 conn(0x7f8ae40b23d0 msgr2=0x7f8ae40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 -- 172.21.15.43:0/1665699734 shutdown_connections 2024-09-17T05:35:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.529+0000 7f8aeba00640 1 -- 172.21.15.43:0/1665699734 wait complete. 2024-09-17T05:35:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 Processor -- start 2024-09-17T05:35:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 -- start start 2024-09-17T05:35:16.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 0x7f8ae4155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ae415cb30 0x7f8ae4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae40ce980 con 0x7f8ae4070620 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae40ce800 con 0x7f8ae40d0910 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8aeba00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae40ceb00 con 0x7f8ae415cb30 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46708/0 (socket says 172.21.15.43:46708) 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 -- 172.21.15.43:0/3078730958 learned_addr learned my addr 172.21.15.43:0/3078730958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8adbe00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 0x7f8ae4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9a00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ae415cb30 0x7f8ae4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 -- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ae415cb30 msgr2=0x7f8ae4156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ae415cb30 0x7f8ae4156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 -- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 msgr2=0x7f8ae4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 0x7f8ae4155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 -- 172.21.15.43:0/3078730958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4155760 con 0x7f8ae4070620 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8adbe00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 0x7f8ae4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae9000640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c430 tx=0x7f8ad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.533+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011790 con 0x7f8ae4070620 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad4017070 con 0x7f8ae4070620 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae40af480 con 0x7f8ae4070620 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011ca0 con 0x7f8ae4070620 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae40ae080 con 0x7f8ae4070620 2024-09-17T05:35:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae40cb860 con 0x7f8ae4070620 2024-09-17T05:35:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ad401c460 con 0x7f8ae4070620 2024-09-17T05:35:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8ae1600640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 0x7f8ac0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f8ad4093e00 con 0x7f8ae4070620 2024-09-17T05:35:16.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.537+0000 7f8adbe00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 0x7f8ac0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.541+0000 7f8adbe00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 0x7f8ac0075710 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c10 tx=0x7f8acc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:16.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.541+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ad405f020 con 0x7f8ae4070620 2024-09-17T05:35:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.725+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f8ae4159e40 con 0x7f8ae4070620 2024-09-17T05:35:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.729+0000 7f8ae1600640 1 -- 172.21.15.43:0/3078730958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f8ad405e920 con 0x7f8ae4070620 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 msgr2=0x7f8ac0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 0x7f8ac0075710 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000c10 tx=0x7f8acc009040 comp rx=0 tx=0).stop 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 msgr2=0x7f8ae40ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c430 tx=0x7f8ad400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 shutdown_connections 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ac0073200 0x7f8ac0075710 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ae40d0910 0x7f8ae4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ae415cb30 0x7f8ae4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 --2- 172.21.15.43:0/3078730958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae4070620 0x7f8ae40ae880 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 >> 172.21.15.43:0/3078730958 conn(0x7f8ae40b23d0 msgr2=0x7f8ae415a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 shutdown_connections 2024-09-17T05:35:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.733+0000 7f8aeba00640 1 -- 172.21.15.43:0/3078730958 wait complete. 2024-09-17T05:35:16.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-17T05:35:16.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- 172.21.15.43:0/1822547418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a2c005250 msgr2=0x7f8a2c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1822547418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a2c005250 0x7f8a2c00b9d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f8a440c7280 tx=0x7f8a3c01f390 comp rx=0 tx=0).stop 2024-09-17T05:35:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4a000640 1 -- 172.21.15.43:0/1822547418 reap_dead start 2024-09-17T05:35:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- 172.21.15.43:0/1822547418 shutdown_connections 2024-09-17T05:35:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- 172.21.15.43:0/1822547418 >> 172.21.15.43:0/1822547418 conn(0x7f8a440b23d0 msgr2=0x7f8a440d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- 172.21.15.43:0/1822547418 shutdown_connections 2024-09-17T05:35:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- 172.21.15.43:0/1822547418 wait complete. 2024-09-17T05:35:16.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 Processor -- start 2024-09-17T05:35:16.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.913+0000 7f8a4c000640 1 -- start start 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a2c005250 0x7f8a440c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a441f9a30 0x7f8a441ec930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4415c960 con 0x7f8a441ece70 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4415c7e0 con 0x7f8a441f9a30 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4415cae0 con 0x7f8a2c005250 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46732/0 (socket says 172.21.15.43:46732) 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 -- 172.21.15.43:0/1035957412 learned_addr learned my addr 172.21.15.43:0/1035957412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a49600640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a2c005250 0x7f8a440c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a48c00640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a441f9a30 0x7f8a441ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 -- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a2c005250 msgr2=0x7f8a440c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a2c005250 0x7f8a440c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 -- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a441f9a30 msgr2=0x7f8a441ec930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a441f9a30 0x7f8a441ec930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 -- 172.21.15.43:0/1035957412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a441f5c50 con 0x7f8a441ece70 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4a000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01fc30 tx=0x7f8a3c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c001ba0 con 0x7f8a441ece70 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a3c006380 con 0x7f8a441ece70 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c006c50 con 0x7f8a441ece70 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a441f5f30 con 0x7f8a441ece70 2024-09-17T05:35:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441f64f0 con 0x7f8a441ece70 2024-09-17T05:35:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.917+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a440ada40 con 0x7f8a441ece70 2024-09-17T05:35:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.921+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a3c0be460 con 0x7f8a441ece70 2024-09-17T05:35:16.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.921+0000 7f8a41600640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 0x7f8a24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:16.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.921+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f8a3c1319f0 con 0x7f8a441ece70 2024-09-17T05:35:16.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.921+0000 7f8a49600640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 0x7f8a24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.921+0000 7f8a49600640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 0x7f8a24075740 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c10 tx=0x7f8a38000f80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:16.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:16.925+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8a3c0c5030 con 0x7f8a441ece70 2024-09-17T05:35:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.161+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f8a441558e0 con 0x7f8a441ece70 2024-09-17T05:35:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a41600640 1 -- 172.21.15.43:0/1035957412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v622) ==== 135+0+0 (secure 0 0 0) 0x7f8a3c019140 con 0x7f8a441ece70 2024-09-17T05:35:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 msgr2=0x7f8a24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 0x7f8a24075740 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8a38000c10 tx=0x7f8a38000f80 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 msgr2=0x7f8a4409d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01fc30 tx=0x7f8a3c003f40 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 shutdown_connections 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8a24073230 0x7f8a24075740 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a441f9a30 0x7f8a441ec930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a441ece70 0x7f8a4409d0f0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 --2- 172.21.15.43:0/1035957412 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8a2c005250 0x7f8a440c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.337+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 >> 172.21.15.43:0/1035957412 conn(0x7f8a440b23d0 msgr2=0x7f8a440b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.341+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 shutdown_connections 2024-09-17T05:35:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:17.341+0000 7f8a4c000640 1 -- 172.21.15.43:0/1035957412 wait complete. 2024-09-17T05:35:17.364 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2024-09-17T05:35:17.364 INFO:tasks.cephfs_test_runner: 2024-09-17T05:35:17.365 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T05:35:17.365 INFO:tasks.cephfs_test_runner:Ran 6 tests in 685.505s 2024-09-17T05:35:17.365 INFO:tasks.cephfs_test_runner: 2024-09-17T05:35:17.365 INFO:tasks.cephfs_test_runner:OK 2024-09-17T05:35:17.365 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T05:35:17.374 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-17T05:35:17.400 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-17T05:35:17.400 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:17.400 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:17.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:35:17.409 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-17T05:35:17.409 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:17.463 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6185 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:35:17.463 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:35:17.463 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:17.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:35:17.473 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-09-17T05:35:17.473 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:35:17.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4305 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:35:17.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:17.523 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:35:17.532 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:35:17.532 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:35:17.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:17.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:35:17.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:17.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:35:18.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-17T05:35:18.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.250 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-17T05:35:18.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-17T05:35:18.756 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-17T05:35:18.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:35:18.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.759 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:35:18.769 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:35:18.769 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:35:18.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:35:18.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:18.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:35:19.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:19.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-17T05:35:19.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:19.454 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-09-17T05:35:19.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:35:19.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-09-17T05:35:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-09-17T05:35:19.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:35:19.967 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T05:35:19.981 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:35:19.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:35:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- 172.21.15.43:0/621553926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff40b3af0 msgr2=0x7f7ff40ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/621553926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff40b3af0 0x7f7ff40ad0b0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009c80 tx=0x7f7fec01cce0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- 172.21.15.43:0/621553926 shutdown_connections 2024-09-17T05:35:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/621553926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff40b3af0 0x7f7ff40ad0b0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/621553926 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4155d70 0x7f7ff415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- 172.21.15.43:0/621553926 >> 172.21.15.43:0/621553926 conn(0x7f7ff40b23d0 msgr2=0x7f7ff40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- 172.21.15.43:0/621553926 shutdown_connections 2024-09-17T05:35:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- 172.21.15.43:0/621553926 wait complete. 2024-09-17T05:35:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 Processor -- start 2024-09-17T05:35:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.149+0000 7f7ff9c00640 1 -- start start 2024-09-17T05:35:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 0x7f7ff4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 0x7f7ff415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4168c10 con 0x7f7ff4155d70 2024-09-17T05:35:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4168a90 con 0x7f7ff406f110 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff4168d90 con 0x7f7ff4151a30 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46754/0 (socket says 172.21.15.43:46754) 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 -- 172.21.15.43:0/488725599 learned_addr learned my addr 172.21.15.43:0/488725599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff2a00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 0x7f7ff4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff3400640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 0x7f7ff415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 -- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 msgr2=0x7f7ff415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 0x7f7ff415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 -- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 msgr2=0x7f7ff4154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 0x7f7ff4154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 -- 172.21.15.43:0/488725599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff408f110 con 0x7f7ff4155d70 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff3400640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 0x7f7ff415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff2a00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 0x7f7ff4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7febe00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00b410 tx=0x7f7fdc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc015030 con 0x7f7ff4155d70 2024-09-17T05:35:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fdc004020 con 0x7f7ff4155d70 2024-09-17T05:35:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff4074d20 con 0x7f7ff4155d70 2024-09-17T05:35:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004470 con 0x7f7ff4155d70 2024-09-17T05:35:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff40731c0 con 0x7f7ff4155d70 2024-09-17T05:35:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.153+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff40ad0b0 con 0x7f7ff4155d70 2024-09-17T05:35:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.157+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7fdc018070 con 0x7f7ff4155d70 2024-09-17T05:35:20.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.157+0000 7f7feaa00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 0x7f7fd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.157+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f7fdc093b80 con 0x7f7ff4155d70 2024-09-17T05:35:20.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.157+0000 7f7ff2a00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 0x7f7fd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.157+0000 7f7ff2a00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 0x7f7fd0075710 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40715c0 tx=0x7f7fe400b5b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.161+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7fdc05ee30 con 0x7f7ff4155d70 2024-09-17T05:35:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.333+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ff415ea90 con 0x7f7fd0073200 2024-09-17T05:35:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.333+0000 7f7feaa00640 1 -- 172.21.15.43:0/488725599 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13501 (secure 0 0 0) 0x7f7ff415ea90 con 0x7f7fd0073200 2024-09-17T05:35:20.338 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 msgr2=0x7f7fd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 0x7f7fd0075710 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40715c0 tx=0x7f7fe400b5b0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 msgr2=0x7f7ff41514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00b410 tx=0x7f7fdc00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 shutdown_connections 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7fd0073200 0x7f7fd0075710 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f7ff4151a30 0x7f7ff415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff4155d70 0x7f7ff41514f0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 --2- 172.21.15.43:0/488725599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ff406f110 0x7f7ff4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 >> 172.21.15.43:0/488725599 conn(0x7f7ff40b23d0 msgr2=0x7f7ff40ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 shutdown_connections 2024-09-17T05:35:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.337+0000 7f7ff9c00640 1 -- 172.21.15.43:0/488725599 wait complete. 2024-09-17T05:35:20.361 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-17T05:35:20.040322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294236,"kb_used_data":15592,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680484,"statfs":{"total":773094113280,"available":772792815616,"internally_reserved":0,"allocated":15966208,"data_stored":8251688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":285134386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34876,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-86,"num_read_kb":-84,"num_write":-98,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-50,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.752827"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:35:16.347074+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:35:16.347074+0000","last_peered":"2024-09-17T05:35:16.347074+0000","last_clean":"2024-09-17T05:35:16.347074+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:35:16.347074+0000","last_undegraded":"2024-09-17T05:35:16.347074+0000","last_fullsized":"2024-09-17T05:35:16.347074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28836,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41312,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330528,"statfs":{"total":96636764160,"available":96594460672,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:35:20.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- 172.21.15.43:0/2839349451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b2330 msgr2=0x7f78540b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 --2- 172.21.15.43:0/2839349451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b2330 0x7f78540b2740 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f783c009870 tx=0x7f783c01c790 comp rx=0 tx=0).stop 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- 172.21.15.43:0/2839349451 shutdown_connections 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 --2- 172.21.15.43:0/2839349451 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540cd9a0 0x7f78540cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 --2- 172.21.15.43:0/2839349451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b2330 0x7f78540b2740 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- 172.21.15.43:0/2839349451 >> 172.21.15.43:0/2839349451 conn(0x7f78540b2e80 msgr2=0x7f78540b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- 172.21.15.43:0/2839349451 shutdown_connections 2024-09-17T05:35:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- 172.21.15.43:0/2839349451 wait complete. 2024-09-17T05:35:20.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 Processor -- start 2024-09-17T05:35:20.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 -- start start 2024-09-17T05:35:20.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.517+0000 7f785bc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540b2330 0x7f78541f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78540cd9a0 0x7f7854152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f785416a0e0 con 0x7f78540b6b10 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7854169f60 con 0x7f78540cd9a0 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f785416a260 con 0x7f78540b2330 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46778/0 (socket says 172.21.15.43:46778) 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 -- 172.21.15.43:0/2288188452 learned_addr learned my addr 172.21.15.43:0/2288188452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7859200640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540b2330 0x7f78541f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7859c00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78540cd9a0 0x7f7854152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 -- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540b2330 msgr2=0x7f78541f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540b2330 0x7f78541f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 -- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78540cd9a0 msgr2=0x7f7854152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78540cd9a0 0x7f7854152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 -- 172.21.15.43:0/2288188452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78540cfa00 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f784be00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f783c01cc70 tx=0x7f783c0a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c0b2040 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f783c0bb070 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78540cfce0 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c003600 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78540d02a0 con 0x7f78540b6b10 2024-09-17T05:35:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78540b2740 con 0x7f78540b6b10 2024-09-17T05:35:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.521+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f783c0037a0 con 0x7f78540b6b10 2024-09-17T05:35:20.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.525+0000 7f7851600640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.525+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f783c12e400 con 0x7f78540b6b10 2024-09-17T05:35:20.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.525+0000 7f7859200640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.525+0000 7f7859200640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 0x7f7830075710 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7844000f70 tx=0x7f7844009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.529+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f783c016410 con 0x7f78540b6b10 2024-09-17T05:35:20.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.701+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f785406dd50 con 0x7f7830073200 2024-09-17T05:35:20.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.701+0000 7f7851600640 1 -- 172.21.15.43:0/2288188452 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13501 (secure 0 0 0) 0x7f785406dd50 con 0x7f7830073200 2024-09-17T05:35:20.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:20.706 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 msgr2=0x7f7830075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 0x7f7830075710 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7844000f70 tx=0x7f7844009040 comp rx=0 tx=0).stop 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 msgr2=0x7f7854152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f783c01cc70 tx=0x7f783c0a56f0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 shutdown_connections 2024-09-17T05:35:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78540cd9a0 0x7f7854152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540b6b10 0x7f7854152700 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 --2- 172.21.15.43:0/2288188452 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f78540b2330 0x7f78541f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 >> 172.21.15.43:0/2288188452 conn(0x7f78540b2e80 msgr2=0x7f785406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 shutdown_connections 2024-09-17T05:35:20.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.705+0000 7f785bc00640 1 -- 172.21.15.43:0/2288188452 wait complete. 2024-09-17T05:35:20.728 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-17T05:35:20.040322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294236,"kb_used_data":15592,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680484,"statfs":{"total":773094113280,"available":772792815616,"internally_reserved":0,"allocated":15966208,"data_stored":8251688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":285134386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34876,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-86,"num_read_kb":-84,"num_write":-98,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-50,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.752827"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:35:16.347074+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:35:16.347074+0000","last_peered":"2024-09-17T05:35:16.347074+0000","last_clean":"2024-09-17T05:35:16.347074+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:35:16.347074+0000","last_undegraded":"2024-09-17T05:35:16.347074+0000","last_fullsized":"2024-09-17T05:35:16.347074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28836,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41312,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330528,"statfs":{"total":96636764160,"available":96594460672,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:35:20.729 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:35:20.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 -- 172.21.15.43:0/1160505782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110004810 msgr2=0x7fb110004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 --2- 172.21.15.43:0/1160505782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110004810 0x7fb110004c20 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fb114009b40 tx=0x7fb11401d240 comp rx=0 tx=0).stop 2024-09-17T05:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb12b400640 1 -- 172.21.15.43:0/1160505782 reap_dead start 2024-09-17T05:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 -- 172.21.15.43:0/1160505782 shutdown_connections 2024-09-17T05:35:20.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 -- 172.21.15.43:0/1160505782 >> 172.21.15.43:0/1160505782 conn(0x7fb12c0b23d0 msgr2=0x7fb12c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 -- 172.21.15.43:0/1160505782 shutdown_connections 2024-09-17T05:35:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 -- 172.21.15.43:0/1160505782 wait complete. 2024-09-17T05:35:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.885+0000 7fb131600640 1 Processor -- start 2024-09-17T05:35:20.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- start start 2024-09-17T05:35:20.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb110004810 0x7fb12c092890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c09d750 0x7fb12c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c15cb50 con 0x7fb12c0b0e30 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c15c9d0 con 0x7fb12c09d750 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c1663b0 con 0x7fb110004810 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46804/0 (socket says 172.21.15.43:46804) 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 -- 172.21.15.43:0/3734118628 learned_addr learned my addr 172.21.15.43:0/3734118628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12aa00640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb110004810 0x7fb12c092890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb122000640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c09d750 0x7fb12c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 -- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb110004810 msgr2=0x7fb12c092890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb110004810 0x7fb12c092890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 -- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c09d750 msgr2=0x7fb12c0b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:20.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c09d750 0x7fb12c0b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 -- 172.21.15.43:0/3734118628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c0b1240 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb12b400640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0b5290 tx=0x7fb124010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124004d30 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb12400eaf0 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c1f6bd0 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124005020 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1f7190 con 0x7fb12c0b0e30 2024-09-17T05:35:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.889+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12c0ad0b0 con 0x7fb12c0b0e30 2024-09-17T05:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.893+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb12401a030 con 0x7fb12c0b0e30 2024-09-17T05:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.893+0000 7fb123400640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 0x7fb108075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:20.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.893+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fb124094b60 con 0x7fb12c0b0e30 2024-09-17T05:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.893+0000 7fb12aa00640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 0x7fb108075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:20.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.893+0000 7fb12aa00640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 0x7fb108075710 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000fc0 tx=0x7fb11c001560 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:20.897+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb12c0b1240 con 0x7fb12c0b0e30 2024-09-17T05:35:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.069+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb12c165300 con 0x7fb108073200 2024-09-17T05:35:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.069+0000 7fb123400640 1 -- 172.21.15.43:0/3734118628 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13501 (secure 0 0 0) 0x7fb12c165300 con 0x7fb108073200 2024-09-17T05:35:21.075 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:21.075 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:35:21.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 msgr2=0x7fb108075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 0x7fb108075710 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000fc0 tx=0x7fb11c001560 comp rx=0 tx=0).stop 2024-09-17T05:35:21.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 msgr2=0x7fb12c0c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0b5290 tx=0x7fb124010e10 comp rx=0 tx=0).stop 2024-09-17T05:35:21.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 shutdown_connections 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fb108073200 0x7fb108075710 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb12c09d750 0x7fb12c0b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12c0b0e30 0x7fb12c0c85b0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 --2- 172.21.15.43:0/3734118628 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fb110004810 0x7fb12c092890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 >> 172.21.15.43:0/3734118628 conn(0x7fb12c0b23d0 msgr2=0x7fb12c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 shutdown_connections 2024-09-17T05:35:21.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.073+0000 7fb131600640 1 -- 172.21.15.43:0/3734118628 wait complete. 2024-09-17T05:35:21.097 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-17T05:35:20.040322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294236,"kb_used_data":15592,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680484,"statfs":{"total":773094113280,"available":772792815616,"internally_reserved":0,"allocated":15966208,"data_stored":8251688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":285134386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34876,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-86,"num_read_kb":-84,"num_write":-98,"num_write_kb":-67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-50,"num_keys_recovered":-10,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.752827"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:35:16.347074+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:35:16.347074+0000","last_peered":"2024-09-17T05:35:16.347074+0000","last_clean":"2024-09-17T05:35:16.347074+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:35:16.347074+0000","last_undegraded":"2024-09-17T05:35:16.347074+0000","last_fullsized":"2024-09-17T05:35:16.347074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28836,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41312,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330528,"statfs":{"total":96636764160,"available":96594460672,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:35:21.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:35:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- 172.21.15.43:0/3814563313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f040b3af0 msgr2=0x7f5f040ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3814563313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f040b3af0 0x7f5f040ad0b0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f5f04072b60 tx=0x7f5efc01c800 comp rx=0 tx=0).stop 2024-09-17T05:35:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- 172.21.15.43:0/3814563313 shutdown_connections 2024-09-17T05:35:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3814563313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f040b3af0 0x7f5f040ad0b0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3814563313 >> v2:172.21.15.62:3300/0 conn(0x7f5f04155d70 0x7f5f0415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- 172.21.15.43:0/3814563313 >> 172.21.15.43:0/3814563313 conn(0x7f5f040b23d0 msgr2=0x7f5f040d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- 172.21.15.43:0/3814563313 shutdown_connections 2024-09-17T05:35:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- 172.21.15.43:0/3814563313 wait complete. 2024-09-17T05:35:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 Processor -- start 2024-09-17T05:35:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.253+0000 7f5f09c00640 1 -- start start 2024-09-17T05:35:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 0x7f5f040cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 0x7f5f040cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f04168c60 con 0x7f5f0406f110 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f04168ae0 con 0x7f5f040d0000 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5f04168de0 con 0x7f5f04155d70 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46820/0 (socket says 172.21.15.43:46820) 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 -- 172.21.15.43:0/3905496541 learned_addr learned my addr 172.21.15.43:0/3905496541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f03400640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 0x7f5f040cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5efa000640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 0x7f5f040cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 -- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 msgr2=0x7f5f040cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 0x7f5f040cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 -- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 msgr2=0x7f5f040cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 0x7f5f040cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 -- 172.21.15.43:0/3905496541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f040c4010 con 0x7f5f0406f110 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f03400640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 0x7f5f040cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5efa000640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 0x7f5f040cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f02a00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9f0 tx=0x7f5ef400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400cd80 con 0x7f5f0406f110 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef4003a70 con 0x7f5f0406f110 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004020 con 0x7f5f0406f110 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f040c42f0 con 0x7f5f0406f110 2024-09-17T05:35:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f040c6750 con 0x7f5f0406f110 2024-09-17T05:35:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.257+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f040ad0b0 con 0x7f5f0406f110 2024-09-17T05:35:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.261+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ef40041c0 con 0x7f5f0406f110 2024-09-17T05:35:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.261+0000 7f5efb400640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 0x7f5ee0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.261+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f5ef4094be0 con 0x7f5f0406f110 2024-09-17T05:35:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.261+0000 7f5efa000640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 0x7f5ee0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.261+0000 7f5efa000640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 0x7f5ee0075710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5f04166220 tx=0x7f5eec008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.265+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ef4016030 con 0x7f5f0406f110 2024-09-17T05:35:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.441+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f040750a0 con 0x7f5f0406f110 2024-09-17T05:35:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.441+0000 7f5efb400640 1 -- 172.21.15.43:0/3905496541 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+13112 (secure 0 0 0) 0x7f5ef405ffa0 con 0x7f5f0406f110 2024-09-17T05:35:21.445 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:21.446 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":145,"fsid":"1cac0027-7a1b-472a-9fc8-e0027668aa6d","created":"2024-09-17T05:22:58.373181+0000","modified":"2024-09-17T05:35:16.333682+0000","last_up_change":"2024-09-17T05:23:13.586494+0000","last_in_change":"2024-09-17T05:23:02.815265+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:23:13.868734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58bb03d-6df4-4384-9392-7f75c166a6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6801","nonce":4272274264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6803","nonce":4272274264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6807","nonce":4272274264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":4272274264},{"type":"v1","addr":"172.21.15.43:6805","nonce":4272274264}]},"public_addr":"172.21.15.43:6801/4272274264","cluster_addr":"172.21.15.43:6803/4272274264","heartbeat_back_addr":"172.21.15.43:6807/4272274264","heartbeat_front_addr":"172.21.15.43:6805/4272274264","state":["exists","up"]},{"osd":1,"uuid":"14750228-325c-43bb-bd26-84cfbf867ab3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6818","nonce":1939991414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6822","nonce":1939991414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6831","nonce":1939991414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1939991414},{"type":"v1","addr":"172.21.15.43:6826","nonce":1939991414}]},"public_addr":"172.21.15.43:6818/1939991414","cluster_addr":"172.21.15.43:6822/1939991414","heartbeat_back_addr":"172.21.15.43:6831/1939991414","heartbeat_front_addr":"172.21.15.43:6826/1939991414","state":["exists","up"]},{"osd":2,"uuid":"a10f45b8-705a-48e6-8862-8f92b0bc1225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6809","nonce":3128122281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6811","nonce":3128122281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6815","nonce":3128122281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3128122281},{"type":"v1","addr":"172.21.15.43:6813","nonce":3128122281}]},"public_addr":"172.21.15.43:6809/3128122281","cluster_addr":"172.21.15.43:6811/3128122281","heartbeat_back_addr":"172.21.15.43:6815/3128122281","heartbeat_front_addr":"172.21.15.43:6813/3128122281","state":["exists","up"]},{"osd":3,"uuid":"359bdccb-91ae-428a-864b-50613ad48610","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6819","nonce":6899353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6823","nonce":6899353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6830","nonce":6899353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":6899353},{"type":"v1","addr":"172.21.15.43:6827","nonce":6899353}]},"public_addr":"172.21.15.43:6819/6899353","cluster_addr":"172.21.15.43:6823/6899353","heartbeat_back_addr":"172.21.15.43:6830/6899353","heartbeat_front_addr":"172.21.15.43:6827/6899353","state":["exists","up"]},{"osd":4,"uuid":"5fcdbf08-53ee-4331-b1a9-c868a69c40f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6819","nonce":1078882315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6821","nonce":1078882315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6825","nonce":1078882315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1078882315},{"type":"v1","addr":"172.21.15.62:6823","nonce":1078882315}]},"public_addr":"172.21.15.62:6819/1078882315","cluster_addr":"172.21.15.62:6821/1078882315","heartbeat_back_addr":"172.21.15.62:6825/1078882315","heartbeat_front_addr":"172.21.15.62:6823/1078882315","state":["exists","up"]},{"osd":5,"uuid":"64597b7a-54bb-46e0-8d71-aebe6fc6a6b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6827","nonce":3886236614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6829","nonce":3886236614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6833","nonce":3886236614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3886236614},{"type":"v1","addr":"172.21.15.62:6831","nonce":3886236614}]},"public_addr":"172.21.15.62:6827/3886236614","cluster_addr":"172.21.15.62:6829/3886236614","heartbeat_back_addr":"172.21.15.62:6833/3886236614","heartbeat_front_addr":"172.21.15.62:6831/3886236614","state":["exists","up"]},{"osd":6,"uuid":"42791e8e-8675-4f6d-b6ac-c5f4525fcb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6803","nonce":2388352512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6805","nonce":2388352512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6809","nonce":2388352512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2388352512},{"type":"v1","addr":"172.21.15.62:6807","nonce":2388352512}]},"public_addr":"172.21.15.62:6803/2388352512","cluster_addr":"172.21.15.62:6805/2388352512","heartbeat_back_addr":"172.21.15.62:6809/2388352512","heartbeat_front_addr":"172.21.15.62:6807/2388352512","state":["exists","up"]},{"osd":7,"uuid":"a3e0f337-0f35-4867-b65c-47703989edb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6811","nonce":1426001971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6813","nonce":1426001971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6817","nonce":1426001971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1426001971},{"type":"v1","addr":"172.21.15.62:6815","nonce":1426001971}]},"public_addr":"172.21.15.62:6811/1426001971","cluster_addr":"172.21.15.62:6813/1426001971","heartbeat_back_addr":"172.21.15.62:6817/1426001971","heartbeat_front_addr":"172.21.15.62:6815/1426001971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.846895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.071027+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.893134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.945687+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.038496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.915382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:09.748630+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:23:10.320963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/1735972562":"2024-09-18T05:35:13.279362+0000","172.21.15.43:6832/1735972562":"2024-09-18T05:35:13.279362+0000","172.21.15.43:6834/4038368577":"2024-09-18T05:34:59.058452+0000","172.21.15.43:6832/4038368577":"2024-09-18T05:34:59.058452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 msgr2=0x7f5ee0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 0x7f5ee0075710 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5f04166220 tx=0x7f5eec008040 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 msgr2=0x7f5f0409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400e9f0 tx=0x7f5ef400eeb0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 shutdown_connections 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5ee0073200 0x7f5ee0075710 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f040d0000 0x7f5f040cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5f04155d70 0x7f5f040cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 --2- 172.21.15.43:0/3905496541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0406f110 0x7f5f0409cbc0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 >> 172.21.15.43:0/3905496541 conn(0x7f5f040b23d0 msgr2=0x7f5f0415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 shutdown_connections 2024-09-17T05:35:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:21.445+0000 7f5f09c00640 1 -- 172.21.15.43:0/3905496541 wait complete. 2024-09-17T05:35:22.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:22.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:22.469 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T05:35:22.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 -- 172.21.15.43:0/2330808481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e9015b7e0 msgr2=0x7f8e900731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 --2- 172.21.15.43:0/2330808481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e9015b7e0 0x7f8e900731d0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009c80 tx=0x7f8e8801cda0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 -- 172.21.15.43:0/2330808481 shutdown_connections 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 --2- 172.21.15.43:0/2330808481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e9015b7e0 0x7f8e900731d0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 --2- 172.21.15.43:0/2330808481 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e900cf980 0x7f8e900b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 -- 172.21.15.43:0/2330808481 >> 172.21.15.43:0/2330808481 conn(0x7f8e9006ce60 msgr2=0x7f8e9006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 -- 172.21.15.43:0/2330808481 shutdown_connections 2024-09-17T05:35:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.641+0000 7f8e96200640 1 -- 172.21.15.43:0/2330808481 wait complete. 2024-09-17T05:35:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 Processor -- start 2024-09-17T05:35:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 -- start start 2024-09-17T05:35:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 0x7f8e90159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e9015b7e0 0x7f8e900aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e901f90d0 0x7f8e900cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90168fa0 con 0x7f8e90074c60 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e90168e20 con 0x7f8e9015b7e0 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8e90169120 con 0x7f8e901f90d0 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 0x7f8e90159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 0x7f8e90159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46834/0 (socket says 172.21.15.43:46834) 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 -- 172.21.15.43:0/1990376213 learned_addr learned my addr 172.21.15.43:0/1990376213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e86a00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e9015b7e0 0x7f8e900aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e901f90d0 0x7f8e900cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 -- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e901f90d0 msgr2=0x7f8e900cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8e901f90d0 0x7f8e900cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 -- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e9015b7e0 msgr2=0x7f8e900aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e9015b7e0 0x7f8e900aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e900aefb0 con 0x7f8e90074c60 2024-09-17T05:35:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e8f400640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 0x7f8e90159f90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000c3c0 tx=0x7f8e8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80017070 con 0x7f8e90074c60 2024-09-17T05:35:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e800053c0 con 0x7f8e90074c60 2024-09-17T05:35:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80003860 con 0x7f8e90074c60 2024-09-17T05:35:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.645+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e900ad150 con 0x7f8e90074c60 2024-09-17T05:35:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e900ad8a0 con 0x7f8e90074c60 2024-09-17T05:35:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e900b3b80 con 0x7f8e90074c60 2024-09-17T05:35:22.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8e8001a070 con 0x7f8e90074c60 2024-09-17T05:35:22.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8e6c073230 0x7f8e6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f8e800935c0 con 0x7f8e90074c60 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7f8e6c0791e0 0x7f8e6c07b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e80015e70 con 0x7f8e6c0791e0 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f8e80093970 con 0x7f8e90074c60 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e86a00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8e6c073230 0x7f8e6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.649+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7f8e6c0791e0 0x7f8e6c07b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.653+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7f8e6c0791e0 0x7f8e6c07b630 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.653+0000 7f8e86a00640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8e6c073230 0x7f8e6c075740 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8e9015eeb0 tx=0x7f8e7800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.653+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== osd.0 v2:172.21.15.43:6800/4272274264 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e80015e70 con 0x7f8e6c0791e0 2024-09-17T05:35:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 --> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8e9006dd20 con 0x7f8e6c0791e0 2024-09-17T05:35:22.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e87e00640 1 -- 172.21.15.43:0/1990376213 <== osd.0 v2:172.21.15.43:6800/4272274264 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8e9006dd20 con 0x7f8e6c0791e0 2024-09-17T05:35:22.670 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:22.670 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:22.670 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7f8e6c0791e0 msgr2=0x7f8e6c07b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:6800/4272274264,v1:172.21.15.43:6801/4272274264] conn(0x7f8e6c0791e0 0x7f8e6c07b630 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8e6c073230 msgr2=0x7f8e6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8e6c073230 0x7f8e6c075740 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8e9015eeb0 tx=0x7f8e7800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 msgr2=0x7f8e90159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 --2- 172.21.15.43:0/1990376213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e90074c60 0x7f8e90159f90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000c3c0 tx=0x7f8e8000c880 comp rx=0 tx=0).stop 2024-09-17T05:35:22.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1990376213 reap_dead start 2024-09-17T05:35:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 shutdown_connections 2024-09-17T05:35:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 >> 172.21.15.43:0/1990376213 conn(0x7f8e9006ce60 msgr2=0x7f8e9006fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 shutdown_connections 2024-09-17T05:35:22.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.665+0000 7f8e96200640 1 -- 172.21.15.43:0/1990376213 wait complete. 2024-09-17T05:35:22.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T05:35:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 -- 172.21.15.43:0/2532971860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809415ca80 msgr2=0x7f809415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 --2- 172.21.15.43:0/2532971860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809415ca80 0x7f809415ce90 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f80940d2830 tx=0x7f808001c870 comp rx=0 tx=0).stop 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 -- 172.21.15.43:0/2532971860 shutdown_connections 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 --2- 172.21.15.43:0/2532971860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809415ca80 0x7f809415ce90 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 --2- 172.21.15.43:0/2532971860 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f80941605e0 0x7f80941609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 --2- 172.21.15.43:0/2532971860 >> v2:172.21.15.62:3300/0 conn(0x7f80940d1cb0 0x7f80940ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 -- 172.21.15.43:0/2532971860 >> 172.21.15.43:0/2532971860 conn(0x7f8094068fe0 msgr2=0x7f8094069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 -- 172.21.15.43:0/2532971860 shutdown_connections 2024-09-17T05:35:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.845+0000 7f809ac00640 1 -- 172.21.15.43:0/2532971860 wait complete. 2024-09-17T05:35:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 Processor -- start 2024-09-17T05:35:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- start start 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80940d1cb0 0x7f809414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 0x7f80940b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80941532e0 con 0x7f80941605e0 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8094153160 con 0x7f80940d1cb0 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8094153460 con 0x7f809415ca80 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46854/0 (socket says 172.21.15.43:46854) 2024-09-17T05:35:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 -- 172.21.15.43:0/208338868 learned_addr learned my addr 172.21.15.43:0/208338868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8098e00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 0x7f80940b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8099800640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80940d1cb0 0x7f809414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 -- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 msgr2=0x7f80940b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 0x7f80940b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 -- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80940d1cb0 msgr2=0x7f809414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80940d1cb0 0x7f809414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 -- 172.21.15.43:0/208338868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80940b2c50 con 0x7f80941605e0 2024-09-17T05:35:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8098e00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 0x7f80940b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809a200640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f808c00ea80 tx=0x7f808c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00ce40 con 0x7f80941605e0 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f808c004b30 con 0x7f80941605e0 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c005020 con 0x7f80941605e0 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80940b2ed0 con 0x7f80941605e0 2024-09-17T05:35:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.849+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809414bd10 con 0x7f80941605e0 2024-09-17T05:35:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.853+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809414b7a0 con 0x7f80941605e0 2024-09-17T05:35:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.853+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f808c0051c0 con 0x7f80941605e0 2024-09-17T05:35:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.853+0000 7f8091600640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 0x7f8074075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.853+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f808c094c80 con 0x7f80941605e0 2024-09-17T05:35:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.853+0000 7f8099800640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 0x7f8074075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.857+0000 7f8099800640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 0x7f8074075740 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8088005fd0 tx=0x7f8088005d60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:22.857+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80940b2c50 con 0x7f80941605e0 2024-09-17T05:35:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.029+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f809406a240 con 0x7f8074073230 2024-09-17T05:35:23.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f8091600640 1 -- 172.21.15.43:0/208338868 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f808800faa0 con 0x7f8074073230 2024-09-17T05:35:23.036 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 msgr2=0x7f8074075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 0x7f8074075740 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8088005fd0 tx=0x7f8088005d60 comp rx=0 tx=0).stop 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 msgr2=0x7f80940c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f808c00ea80 tx=0x7f808c00ef40 comp rx=0 tx=0).stop 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 shutdown_connections 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8074073230 0x7f8074075740 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80941605e0 0x7f80940c9cb0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f809415ca80 0x7f80940b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 --2- 172.21.15.43:0/208338868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80940d1cb0 0x7f809414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 >> 172.21.15.43:0/208338868 conn(0x7f8094068fe0 msgr2=0x7f80940d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 shutdown_connections 2024-09-17T05:35:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.033+0000 7f809ac00640 1 -- 172.21.15.43:0/208338868 wait complete. 2024-09-17T05:35:23.058 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T05:35:23.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.217+0000 7f9dfd200640 1 -- 172.21.15.43:0/3340700162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc004860 msgr2=0x7f9ddc004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.217+0000 7f9dfd200640 1 --2- 172.21.15.43:0/3340700162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc004860 0x7f9ddc004d30 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c3f0 tx=0x7f9de401cee0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- 172.21.15.43:0/3340700162 shutdown_connections 2024-09-17T05:35:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 --2- 172.21.15.43:0/3340700162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ddc004860 0x7f9ddc004d30 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- 172.21.15.43:0/3340700162 >> 172.21.15.43:0/3340700162 conn(0x7f9df806ce60 msgr2=0x7f9df806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- 172.21.15.43:0/3340700162 shutdown_connections 2024-09-17T05:35:23.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- 172.21.15.43:0/3340700162 wait complete. 2024-09-17T05:35:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 Processor -- start 2024-09-17T05:35:23.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- start start 2024-09-17T05:35:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ddc004860 0x7f9df8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8154fb0 0x7f9df81f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 0x7f9df80b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9df80cce60 con 0x7f9df814f360 2024-09-17T05:35:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9df80ccce0 con 0x7f9ddc004860 2024-09-17T05:35:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.221+0000 7f9dfd200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9df80ccfe0 con 0x7f9df8154fb0 2024-09-17T05:35:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 0x7f9df80b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 0x7f9df80b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46868/0 (socket says 172.21.15.43:46868) 2024-09-17T05:35:23.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 -- 172.21.15.43:0/2572139597 learned_addr learned my addr 172.21.15.43:0/2572139597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df5e00640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ddc004860 0x7f9df8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df5400640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8154fb0 0x7f9df81f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 -- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8154fb0 msgr2=0x7f9df81f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9df8154fb0 0x7f9df81f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 -- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ddc004860 msgr2=0x7f9df8154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ddc004860 0x7f9df8154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df8155690 con 0x7f9df814f360 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df5e00640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ddc004860 0x7f9df8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9df6800640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 0x7f9df80b0fe0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f9df815faa0 tx=0x7f9df000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df000de20 con 0x7f9df814f360 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9df0003980 con 0x7f9df814f360 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0003db0 con 0x7f9df814f360 2024-09-17T05:35:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df8155970 con 0x7f9df814f360 2024-09-17T05:35:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df814f770 con 0x7f9df814f360 2024-09-17T05:35:23.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9df814fab0 con 0x7f9df814f360 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.225+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9df0004030 con 0x7f9df814f360 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9dd8073200 0x7f9dd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f9df0095040 con 0x7f9df814f360 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f9dd80791b0 0x7f9dd807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9df001be50 con 0x7f9dd80791b0 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f9df00954a0 con 0x7f9df814f360 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9df5e00640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9dd8073200 0x7f9dd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9df5400640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f9dd80791b0 0x7f9dd807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9df5400640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f9dd80791b0 0x7f9dd807b600 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9df5e00640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9dd8073200 0x7f9dd8075710 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9de800afd0 tx=0x7f9de800ab70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.229+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== osd.1 v2:172.21.15.43:6816/1939991414 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9df001be50 con 0x7f9dd80791b0 2024-09-17T05:35:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.241+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 --> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9df8155440 con 0x7f9dd80791b0 2024-09-17T05:35:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.241+0000 7f9dee000640 1 -- 172.21.15.43:0/2572139597 <== osd.1 v2:172.21.15.43:6816/1939991414 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9df8155440 con 0x7f9dd80791b0 2024-09-17T05:35:23.248 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:23.249 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f9dd80791b0 msgr2=0x7f9dd807b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:6816/1939991414,v1:172.21.15.43:6818/1939991414] conn(0x7f9dd80791b0 0x7f9dd807b600 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9dd8073200 msgr2=0x7f9dd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9dd8073200 0x7f9dd8075710 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9de800afd0 tx=0x7f9de800ab70 comp rx=0 tx=0).stop 2024-09-17T05:35:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 msgr2=0x7f9df80b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 --2- 172.21.15.43:0/2572139597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df814f360 0x7f9df80b0fe0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f9df815faa0 tx=0x7f9df000d7b0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9df6800640 1 -- 172.21.15.43:0/2572139597 reap_dead start 2024-09-17T05:35:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 shutdown_connections 2024-09-17T05:35:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 >> 172.21.15.43:0/2572139597 conn(0x7f9df806ce60 msgr2=0x7f9df80cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 shutdown_connections 2024-09-17T05:35:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.245+0000 7f9dfd200640 1 -- 172.21.15.43:0/2572139597 wait complete. 2024-09-17T05:35:23.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 -- 172.21.15.43:0/3864647086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5540b3af0 msgr2=0x7fd5540ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 --2- 172.21.15.43:0/3864647086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5540b3af0 0x7fd5540ad0b0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7fd554072b60 tx=0x7fd54c01cac0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 -- 172.21.15.43:0/3864647086 shutdown_connections 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 --2- 172.21.15.43:0/3864647086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5540b3af0 0x7fd5540ad0b0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 --2- 172.21.15.43:0/3864647086 >> v2:172.21.15.62:3300/0 conn(0x7fd554155d70 0x7fd55415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 -- 172.21.15.43:0/3864647086 >> 172.21.15.43:0/3864647086 conn(0x7fd5540b23d0 msgr2=0x7fd5540d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 -- 172.21.15.43:0/3864647086 shutdown_connections 2024-09-17T05:35:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.421+0000 7fd558e00640 1 -- 172.21.15.43:0/3864647086 wait complete. 2024-09-17T05:35:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 Processor -- start 2024-09-17T05:35:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 -- start start 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 0x7fd5540c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd554155d70 0x7fd554154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd554168b90 con 0x7fd55409d950 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd554168a10 con 0x7fd554155d70 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd558e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fd554168d10 con 0x7fd55406f110 2024-09-17T05:35:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46888/0 (socket says 172.21.15.43:46888) 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 -- 172.21.15.43:0/2739046126 learned_addr learned my addr 172.21.15.43:0/2739046126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd551e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 0x7fd5540c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd551400640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd554155d70 0x7fd554154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 -- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 msgr2=0x7fd5540c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 0x7fd5540c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 -- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd554155d70 msgr2=0x7fd554154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd554155d70 0x7fd554154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd552800640 1 -- 172.21.15.43:0/2739046126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5540ca030 con 0x7fd55409d950 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.425+0000 7fd551e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 0x7fd5540c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd552800640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fd54c01c530 tx=0x7fd54c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0b2040 con 0x7fd55409d950 2024-09-17T05:35:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd54c0bb070 con 0x7fd55409d950 2024-09-17T05:35:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004330 con 0x7fd55409d950 2024-09-17T05:35:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5540731c0 con 0x7fd55409d950 2024-09-17T05:35:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd554073780 con 0x7fd55409d950 2024-09-17T05:35:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5540ad0b0 con 0x7fd55409d950 2024-09-17T05:35:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd54c0044d0 con 0x7fd55409d950 2024-09-17T05:35:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 0x7fd534075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.429+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fd54c12e960 con 0x7fd55409d950 2024-09-17T05:35:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.433+0000 7fd551e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 0x7fd534075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.433+0000 7fd551e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 0x7fd534075710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd5540715c0 tx=0x7fd5440091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.437+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd54c016770 con 0x7fd55409d950 2024-09-17T05:35:23.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.609+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd5540ca310 con 0x7fd534073200 2024-09-17T05:35:23.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.609+0000 7fd54a000640 1 -- 172.21.15.43:0/2739046126 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5540ca310 con 0x7fd534073200 2024-09-17T05:35:23.615 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 msgr2=0x7fd534075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 0x7fd534075710 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd5540715c0 tx=0x7fd5440091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 msgr2=0x7fd55409c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fd54c01c530 tx=0x7fd54c003f40 comp rx=0 tx=0).stop 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 shutdown_connections 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fd534073200 0x7fd534075710 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd554155d70 0x7fd554154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd55409d950 0x7fd55409c360 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 --2- 172.21.15.43:0/2739046126 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fd55406f110 0x7fd5540c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 >> 172.21.15.43:0/2739046126 conn(0x7fd5540b23d0 msgr2=0x7fd5540afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 shutdown_connections 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.613+0000 7fd558e00640 1 -- 172.21.15.43:0/2739046126 wait complete. 2024-09-17T05:35:23.637 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T05:35:23.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- 172.21.15.43:0/3048284934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0080cf980 msgr2=0x7ff0080731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 --2- 172.21.15.43:0/3048284934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0080cf980 0x7ff0080731d0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7ff000009c80 tx=0x7ff00001cda0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- 172.21.15.43:0/3048284934 shutdown_connections 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 --2- 172.21.15.43:0/3048284934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0080cf980 0x7ff0080731d0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 --2- 172.21.15.43:0/3048284934 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff008074c60 0x7ff0080b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- 172.21.15.43:0/3048284934 >> 172.21.15.43:0/3048284934 conn(0x7ff00806ce60 msgr2=0x7ff00806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- 172.21.15.43:0/3048284934 shutdown_connections 2024-09-17T05:35:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- 172.21.15.43:0/3048284934 wait complete. 2024-09-17T05:35:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 Processor -- start 2024-09-17T05:35:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.797+0000 7ff010600640 1 -- start start 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 0x7ff00814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00815b7e0 0x7ff008155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff00814b9c0 0x7ff008156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff008168f00 con 0x7ff008074c60 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff008168d80 con 0x7ff00815b7e0 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7ff008169080 con 0x7ff00814b9c0 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 0x7ff00814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 0x7ff00814b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55422/0 (socket says 172.21.15.43:55422) 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 -- 172.21.15.43:0/780085822 learned_addr learned my addr 172.21.15.43:0/780085822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00e600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff00814b9c0 0x7ff008156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00d200640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00815b7e0 0x7ff008155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 -- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff00814b9c0 msgr2=0x7ff008156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7ff00814b9c0 0x7ff008156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 -- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00815b7e0 msgr2=0x7ff008155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00815b7e0 0x7ff008155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008160060 con 0x7ff008074c60 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00d200640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00815b7e0 0x7ff008155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff00dc00640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 0x7ff00814b390 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d720 tx=0x7feffc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc017070 con 0x7ff008074c60 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feffc009d20 con 0x7ff008074c60 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081f6cc0 con 0x7ff008074c60 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc005280 con 0x7ff008074c60 2024-09-17T05:35:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081f7280 con 0x7ff008074c60 2024-09-17T05:35:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.801+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0080b3b80 con 0x7ff008074c60 2024-09-17T05:35:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feffc005420 con 0x7ff008074c60 2024-09-17T05:35:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefe8073200 0x7fefe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7feffc0945b0 con 0x7ff008074c60 2024-09-17T05:35:23.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff00d200640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefe8073200 0x7fefe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fefe80791b0 0x7fefe807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefe807bcd0 con 0x7fefe80791b0 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7feffc094960 con 0x7ff008074c60 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff00e600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fefe80791b0 0x7fefe807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff00e600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fefe80791b0 0x7fefe807b600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff00d200640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefe8073200 0x7fefe8075710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff0081f7e70 tx=0x7feff4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.805+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== osd.2 v2:172.21.15.43:6808/3128122281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefe807bcd0 con 0x7fefe80791b0 2024-09-17T05:35:23.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 --> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff0080734f0 con 0x7fefe80791b0 2024-09-17T05:35:23.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff005600640 1 -- 172.21.15.43:0/780085822 <== osd.2 v2:172.21.15.43:6808/3128122281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff0080734f0 con 0x7fefe80791b0 2024-09-17T05:35:23.825 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:23.825 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:23.825 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fefe80791b0 msgr2=0x7fefe807b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:6808/3128122281,v1:172.21.15.43:6809/3128122281] conn(0x7fefe80791b0 0x7fefe807b600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefe8073200 msgr2=0x7fefe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fefe8073200 0x7fefe8075710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff0081f7e70 tx=0x7feff4009040 comp rx=0 tx=0).stop 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 msgr2=0x7ff00814b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 --2- 172.21.15.43:0/780085822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff008074c60 0x7ff00814b390 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d720 tx=0x7feffc00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff00e600640 1 -- 172.21.15.43:0/780085822 reap_dead start 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 shutdown_connections 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 >> 172.21.15.43:0/780085822 conn(0x7ff00806ce60 msgr2=0x7ff0080ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 shutdown_connections 2024-09-17T05:35:23.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:23.821+0000 7ff010600640 1 -- 172.21.15.43:0/780085822 wait complete. 2024-09-17T05:35:23.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T05:35:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 -- 172.21.15.43:0/40109868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf1806ce60 msgr2=0x7fdf1806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 --2- 172.21.15.43:0/40109868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf1806ce60 0x7fdf1806d270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009900 tx=0x7fdf0401c850 comp rx=0 tx=0).stop 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 -- 172.21.15.43:0/40109868 shutdown_connections 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 --2- 172.21.15.43:0/40109868 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf1806ce60 0x7fdf1806d270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 --2- 172.21.15.43:0/40109868 >> v2:172.21.15.62:3300/0 conn(0x7fdf18070620 0x7fdf180b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 -- 172.21.15.43:0/40109868 >> 172.21.15.43:0/40109868 conn(0x7fdf180b23d0 msgr2=0x7fdf180d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 -- 172.21.15.43:0/40109868 shutdown_connections 2024-09-17T05:35:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.001+0000 7fdf1d400640 1 -- 172.21.15.43:0/40109868 wait complete. 2024-09-17T05:35:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 Processor -- start 2024-09-17T05:35:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- start start 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf18070620 0x7fdf180c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf180cb860 0x7fdf180c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf180ce600 con 0x7fdf1806ce60 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf180ce480 con 0x7fdf18070620 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdf180ce780 con 0x7fdf180cb860 2024-09-17T05:35:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55426/0 (socket says 172.21.15.43:55426) 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 -- 172.21.15.43:0/428870356 learned_addr learned my addr 172.21.15.43:0/428870356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf15e00640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf18070620 0x7fdf180c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf17200640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf180cb860 0x7fdf180c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 -- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf180cb860 msgr2=0x7fdf180c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf180cb860 0x7fdf180c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 -- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf18070620 msgr2=0x7fdf180c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf18070620 0x7fdf180c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 -- 172.21.15.43:0/428870356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf180c44a0 con 0x7fdf1806ce60 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf16800640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ea90 tx=0x7fdf0800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800cc80 con 0x7fdf1806ce60 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf08005020 con 0x7fdf1806ce60 2024-09-17T05:35:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf180c41c0 con 0x7fdf1806ce60 2024-09-17T05:35:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08005470 con 0x7fdf1806ce60 2024-09-17T05:35:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf180c7630 con 0x7fdf1806ce60 2024-09-17T05:35:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.005+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf180ced30 con 0x7fdf1806ce60 2024-09-17T05:35:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.009+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf080047b0 con 0x7fdf1806ce60 2024-09-17T05:35:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.009+0000 7fdf0ea00640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 0x7fdef8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.009+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fdf08094420 con 0x7fdf1806ce60 2024-09-17T05:35:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.009+0000 7fdf15e00640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 0x7fdef8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.009+0000 7fdf15e00640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 0x7fdef8075740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ae000 tx=0x7fdf0401b090 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.013+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf0805f640 con 0x7fdf1806ce60 2024-09-17T05:35:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.189+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdf1809e190 con 0x7fdef8073230 2024-09-17T05:35:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.189+0000 7fdf0ea00640 1 -- 172.21.15.43:0/428870356 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdf1809e190 con 0x7fdef8073230 2024-09-17T05:35:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T05:35:24.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 msgr2=0x7fdef8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 0x7fdef8075740 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ae000 tx=0x7fdf0401b090 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 msgr2=0x7fdf180c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ea90 tx=0x7fdf0800ef50 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 shutdown_connections 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdef8073230 0x7fdef8075740 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf18070620 0x7fdf180c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdf180cb860 0x7fdf180c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 --2- 172.21.15.43:0/428870356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf1806ce60 0x7fdf180c7e30 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 >> 172.21.15.43:0/428870356 conn(0x7fdf180b23d0 msgr2=0x7fdf18166320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 shutdown_connections 2024-09-17T05:35:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.193+0000 7fdf1d400640 1 -- 172.21.15.43:0/428870356 wait complete. 2024-09-17T05:35:24.217 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T05:35:24.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- 172.21.15.43:0/4150045037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 msgr2=0x7f3ec8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 --2- 172.21.15.43:0/4150045037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ec8005160 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40b7690 tx=0x7f3edc01d770 comp rx=0 tx=0).stop 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee3200640 1 -- 172.21.15.43:0/4150045037 reap_dead start 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- 172.21.15.43:0/4150045037 shutdown_connections 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- 172.21.15.43:0/4150045037 >> 172.21.15.43:0/4150045037 conn(0x7f3ee40ceef0 msgr2=0x7f3ee40cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- 172.21.15.43:0/4150045037 shutdown_connections 2024-09-17T05:35:24.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- 172.21.15.43:0/4150045037 wait complete. 2024-09-17T05:35:24.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 Processor -- start 2024-09-17T05:35:24.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.377+0000 7f3ee9400640 1 -- start start 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ee41f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee415aa40 0x7f3ee415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee415d940 0x7f3ee41f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40d01c0 con 0x7f3ec8004d50 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40d0040 con 0x7f3ee415aa40 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee40d0340 con 0x7f3ee415d940 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ee41f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ee41f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55436/0 (socket says 172.21.15.43:55436) 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 -- 172.21.15.43:0/36606739 learned_addr learned my addr 172.21.15.43:0/36606739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3edbe00640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee415aa40 0x7f3ee415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee3200640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee415d940 0x7f3ee41f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 -- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee415d940 msgr2=0x7f3ee41f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f3ee415d940 0x7f3ee41f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 -- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee415aa40 msgr2=0x7f3ee415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee415aa40 0x7f3ee415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee41f4340 con 0x7f3ec8004d50 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee2800640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ee41f3e00 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4011ae0 tx=0x7f3ed4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed401e070 con 0x7f3ec8004d50 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed400fe70 con 0x7f3ec8004d50 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee415dd50 con 0x7f3ec8004d50 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40052b0 con 0x7f3ec8004d50 2024-09-17T05:35:24.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee415b820 con 0x7f3ec8004d50 2024-09-17T05:35:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.381+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ee40d0990 con 0x7f3ec8004d50 2024-09-17T05:35:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ed4021070 con 0x7f3ec8004d50 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ec0073230 0x7f3ec0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f3ed4094b10 con 0x7f3ec8004d50 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f3ec00791e0 0x7f3ec007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ec007bd00 con 0x7f3ec00791e0 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f3ed40992a0 con 0x7f3ec8004d50 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edbe00640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ec0073230 0x7f3ec0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3ee3200640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f3ec00791e0 0x7f3ec007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edbe00640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ec0073230 0x7f3ec0075740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000fc0 tx=0x7f3ecc0067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3ee3200640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f3ec00791e0 0x7f3ec007b630 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.385+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== osd.3 v2:172.21.15.43:6817/6899353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ec007bd00 con 0x7f3ec00791e0 2024-09-17T05:35:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 --> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3ee40d3300 con 0x7f3ec00791e0 2024-09-17T05:35:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3edaa00640 1 -- 172.21.15.43:0/36606739 <== osd.3 v2:172.21.15.43:6817/6899353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3ee40d3300 con 0x7f3ec00791e0 2024-09-17T05:35:24.404 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:24.405 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:24.405 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f3ec00791e0 msgr2=0x7f3ec007b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:6817/6899353,v1:172.21.15.43:6819/6899353] conn(0x7f3ec00791e0 0x7f3ec007b630 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ec0073230 msgr2=0x7f3ec0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f3ec0073230 0x7f3ec0075740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000fc0 tx=0x7f3ecc0067b0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 msgr2=0x7f3ee41f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 --2- 172.21.15.43:0/36606739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8004d50 0x7f3ee41f3e00 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4011ae0 tx=0x7f3ed4009e90 comp rx=0 tx=0).stop 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee3200640 1 -- 172.21.15.43:0/36606739 reap_dead start 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 shutdown_connections 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 >> 172.21.15.43:0/36606739 conn(0x7f3ee40ceef0 msgr2=0x7f3ee40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 shutdown_connections 2024-09-17T05:35:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.401+0000 7f3ee9400640 1 -- 172.21.15.43:0/36606739 wait complete. 2024-09-17T05:35:24.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T05:35:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.577+0000 7f986a800640 1 -- 172.21.15.43:0/867965397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f986406ce60 msgr2=0x7f986406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.577+0000 7f986a800640 1 --2- 172.21.15.43:0/867965397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f986406ce60 0x7f986406d270 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f9850009870 tx=0x7f985001c760 comp rx=0 tx=0).stop 2024-09-17T05:35:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 -- 172.21.15.43:0/867965397 shutdown_connections 2024-09-17T05:35:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 --2- 172.21.15.43:0/867965397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f986406ce60 0x7f986406d270 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 -- 172.21.15.43:0/867965397 >> 172.21.15.43:0/867965397 conn(0x7f98640b2e80 msgr2=0x7f98640b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 -- 172.21.15.43:0/867965397 shutdown_connections 2024-09-17T05:35:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 -- 172.21.15.43:0/867965397 wait complete. 2024-09-17T05:35:24.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 Processor -- start 2024-09-17T05:35:24.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 -- start start 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.581+0000 7f986a800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f986406ce60 0x7f986414a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 0x7f98641fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f986416ae70 con 0x7f98640b4180 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f986416acf0 con 0x7f98640b58e0 2024-09-17T05:35:24.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f986416aff0 con 0x7f986406ce60 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55464/0 (socket says 172.21.15.43:55464) 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 -- 172.21.15.43:0/1452339579 learned_addr learned my addr 172.21.15.43:0/1452339579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 -- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f986406ce60 msgr2=0x7f986414a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f986406ce60 0x7f986414a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 -- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 msgr2=0x7f98641fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9863e00640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 0x7f98641fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 0x7f98641fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 -- 172.21.15.43:0/1452339579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641536c0 con 0x7f98640b4180 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9863e00640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 0x7f98641fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f9862a00640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f9850009790 tx=0x7f98500a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500bb070 con 0x7f98640b4180 2024-09-17T05:35:24.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9850004470 con 0x7f98640b4180 2024-09-17T05:35:24.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98500b2040 con 0x7f98640b4180 2024-09-17T05:35:24.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986415b1e0 con 0x7f98640b4180 2024-09-17T05:35:24.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986414a540 con 0x7f98640b4180 2024-09-17T05:35:24.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986406d270 con 0x7f98640b4180 2024-09-17T05:35:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.585+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9850005290 con 0x7f98640b4180 2024-09-17T05:35:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.589+0000 7f985b400640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 0x7f9844075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.589+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f985012d930 con 0x7f98640b4180 2024-09-17T05:35:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.589+0000 7f9863400640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 0x7f9844075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.589+0000 7f9863400640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 0x7f9844075740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f98540046c0 tx=0x7f98540012e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.593+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9850016410 con 0x7f98640b4180 2024-09-17T05:35:24.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9864153200 con 0x7f9844073230 2024-09-17T05:35:24.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f985b400640 1 -- 172.21.15.43:0/1452339579 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f98540068a0 con 0x7f9844073230 2024-09-17T05:35:24.772 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T05:35:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 msgr2=0x7f9844075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 0x7f9844075740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f98540046c0 tx=0x7f98540012e0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 msgr2=0x7f98641f9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f9850009790 tx=0x7f98500a56a0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 shutdown_connections 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9844073230 0x7f9844075740 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98640b58e0 0x7f98641fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640b4180 0x7f98641f9ef0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 --2- 172.21.15.43:0/1452339579 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f986406ce60 0x7f986414a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 >> 172.21.15.43:0/1452339579 conn(0x7f98640b2e80 msgr2=0x7f98640c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 shutdown_connections 2024-09-17T05:35:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.769+0000 7f986a800640 1 -- 172.21.15.43:0/1452339579 wait complete. 2024-09-17T05:35:24.794 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T05:35:24.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 -- 172.21.15.43:0/1690769771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 msgr2=0x7f8abc0052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 --2- 172.21.15.43:0/1690769771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8abc0052e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009ec0 tx=0x7f8ad001d5f0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8add000640 1 -- 172.21.15.43:0/1690769771 reap_dead start 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 -- 172.21.15.43:0/1690769771 shutdown_connections 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 -- 172.21.15.43:0/1690769771 >> 172.21.15.43:0/1690769771 conn(0x7f8ad80b2c60 msgr2=0x7f8ad80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 -- 172.21.15.43:0/1690769771 shutdown_connections 2024-09-17T05:35:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.953+0000 7f8adf000640 1 -- 172.21.15.43:0/1690769771 wait complete. 2024-09-17T05:35:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 Processor -- start 2024-09-17T05:35:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 -- start start 2024-09-17T05:35:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8ad81f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ad80af1d0 0x7f8ad815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad80af5e0 0x7f8ad815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b5800 con 0x7f8abc004e70 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b5680 con 0x7f8ad80af5e0 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8adf000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80b5980 con 0x7f8ad80af1d0 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8ad81f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8ad81f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55490/0 (socket says 172.21.15.43:55490) 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 -- 172.21.15.43:0/563540532 learned_addr learned my addr 172.21.15.43:0/563540532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8acfe00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ad80af1d0 0x7f8ad815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8add000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad80af5e0 0x7f8ad815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 -- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ad80af1d0 msgr2=0x7f8ad815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f8ad80af1d0 0x7f8ad815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 -- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad80af5e0 msgr2=0x7f8ad815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad80af5e0 0x7f8ad815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.957+0000 7f8ad7e00640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad81f36b0 con 0x7f8abc004e70 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad7e00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8ad81f74f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8003ef0 tx=0x7f8ac8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8011640 con 0x7f8abc004e70 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ac8016020 con 0x7f8abc004e70 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad81f3b20 con 0x7f8abc004e70 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8011a80 con 0x7f8abc004e70 2024-09-17T05:35:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81f40e0 con 0x7f8abc004e70 2024-09-17T05:35:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ad80b4d60 con 0x7f8abc004e70 2024-09-17T05:35:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ac8011c20 con 0x7f8abc004e70 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ab4073200 0x7f8ab4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f8ac8094c90 con 0x7f8abc004e70 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f8ab40791b0 0x7f8ab407b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ac800ee50 con 0x7f8ab40791b0 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.961+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f8ac809a2a0 con 0x7f8abc004e70 2024-09-17T05:35:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.965+0000 7f8acfe00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ab4073200 0x7f8ab4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.965+0000 7f8add000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f8ab40791b0 0x7f8ab407b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.965+0000 7f8acfe00640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ab4073200 0x7f8ab4075710 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8159250 tx=0x7f8ac00067b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.965+0000 7f8add000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f8ab40791b0 0x7f8ab407b600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.965+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== osd.4 v2:172.21.15.62:6818/1078882315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ac800ee50 con 0x7f8ab40791b0 2024-09-17T05:35:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.977+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 --> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8ad8156fd0 con 0x7f8ab40791b0 2024-09-17T05:35:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8ad4c00640 1 -- 172.21.15.43:0/563540532 <== osd.4 v2:172.21.15.62:6818/1078882315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8ad8156fd0 con 0x7f8ab40791b0 2024-09-17T05:35:24.984 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:24.984 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:24.984 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f8ab40791b0 msgr2=0x7f8ab407b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6818/1078882315,v1:172.21.15.62:6819/1078882315] conn(0x7f8ab40791b0 0x7f8ab407b600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ab4073200 msgr2=0x7f8ab4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8ab4073200 0x7f8ab4075710 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8159250 tx=0x7f8ac00067b0 comp rx=0 tx=0).stop 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 msgr2=0x7f8ad81f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 --2- 172.21.15.43:0/563540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc004e70 0x7f8ad81f74f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8003ef0 tx=0x7f8ac8001640 comp rx=0 tx=0).stop 2024-09-17T05:35:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8add000640 1 -- 172.21.15.43:0/563540532 reap_dead start 2024-09-17T05:35:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 shutdown_connections 2024-09-17T05:35:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 >> 172.21.15.43:0/563540532 conn(0x7f8ad80b2c60 msgr2=0x7f8ad815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 shutdown_connections 2024-09-17T05:35:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:24.981+0000 7f8adf000640 1 -- 172.21.15.43:0/563540532 wait complete. 2024-09-17T05:35:25.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T05:35:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 -- 172.21.15.43:0/629982000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 msgr2=0x7fdfe4160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 --2- 172.21.15.43:0/629982000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4160620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009bc0 tx=0x7fdfdc01cb10 comp rx=0 tx=0).stop 2024-09-17T05:35:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 -- 172.21.15.43:0/629982000 shutdown_connections 2024-09-17T05:35:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 --2- 172.21.15.43:0/629982000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4160620 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 --2- 172.21.15.43:0/629982000 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 0x7fdfe40b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 -- 172.21.15.43:0/629982000 >> 172.21.15.43:0/629982000 conn(0x7fdfe40b2810 msgr2=0x7fdfe40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 -- 172.21.15.43:0/629982000 shutdown_connections 2024-09-17T05:35:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.157+0000 7fdfec200640 1 -- 172.21.15.43:0/629982000 wait complete. 2024-09-17T05:35:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 Processor -- start 2024-09-17T05:35:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 -- start start 2024-09-17T05:35:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 0x7fdfe40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe416bea0 con 0x7fdfe406f940 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe416c020 con 0x7fdfe4160210 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfec200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe416c1a0 con 0x7fdfe40b4570 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfe9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.161+0000 7fdfe9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55506/0 (socket says 172.21.15.43:55506) 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 -- 172.21.15.43:0/3384135255 learned_addr learned my addr 172.21.15.43:0/3384135255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfea200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe8e00640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 0x7fdfe40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 -- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 msgr2=0x7fdfe40b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 0x7fdfe40b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 -- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 msgr2=0x7fdfe4145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 -- 172.21.15.43:0/3384135255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe40b0b40 con 0x7fdfe406f940 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfea200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe9800640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c760 tx=0x7fdfd800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8016020 con 0x7fdfe406f940 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd800a950 con 0x7fdfe406f940 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800ada0 con 0x7fdfe406f940 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe40b0e20 con 0x7fdfe406f940 2024-09-17T05:35:25.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe40b13e0 con 0x7fdfe406f940 2024-09-17T05:35:25.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe40b2120 con 0x7fdfe406f940 2024-09-17T05:35:25.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.165+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdfd8019020 con 0x7fdfe406f940 2024-09-17T05:35:25.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.169+0000 7fdfe1600640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 0x7fdfc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.169+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fdfd8093ed0 con 0x7fdfe406f940 2024-09-17T05:35:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.169+0000 7fdfe8e00640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 0x7fdfc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.169+0000 7fdfe8e00640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 0x7fdfc4075710 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdfe414f160 tx=0x7fdfd00091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.173+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdfd8020020 con 0x7fdfe406f940 2024-09-17T05:35:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fdfe40b4980 con 0x7fdfc4073200 2024-09-17T05:35:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfe1600640 1 -- 172.21.15.43:0/3384135255 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdfe40b4980 con 0x7fdfc4073200 2024-09-17T05:35:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T05:35:25.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 msgr2=0x7fdfc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 0x7fdfc4075710 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdfe414f160 tx=0x7fdfd00091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 msgr2=0x7fdfe414a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.349+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c760 tx=0x7fdfd800cc20 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 shutdown_connections 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fdfc4073200 0x7fdfc4075710 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfe4160210 0x7fdfe4145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fdfe40b4570 0x7fdfe40b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 --2- 172.21.15.43:0/3384135255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406f940 0x7fdfe414a5a0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 >> 172.21.15.43:0/3384135255 conn(0x7fdfe40b2810 msgr2=0x7fdfe4071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 shutdown_connections 2024-09-17T05:35:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.353+0000 7fdfec200640 1 -- 172.21.15.43:0/3384135255 wait complete. 2024-09-17T05:35:25.376 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T05:35:25.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 -- 172.21.15.43:0/3162872939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44160140 msgr2=0x7f2a440ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 --2- 172.21.15.43:0/3162872939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44160140 0x7f2a440ccdc0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f2a4415d090 tx=0x7f2a3801c790 comp rx=0 tx=0).stop 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 -- 172.21.15.43:0/3162872939 shutdown_connections 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 --2- 172.21.15.43:0/3162872939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44160140 0x7f2a440ccdc0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 -- 172.21.15.43:0/3162872939 >> 172.21.15.43:0/3162872939 conn(0x7f2a440691b0 msgr2=0x7f2a440695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 -- 172.21.15.43:0/3162872939 shutdown_connections 2024-09-17T05:35:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.533+0000 7f2a4b000640 1 -- 172.21.15.43:0/3162872939 wait complete. 2024-09-17T05:35:25.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 Processor -- start 2024-09-17T05:35:25.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 -- start start 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 0x7f2a4414e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44160140 0x7f2a4414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a440cb2d0 0x7f2a441fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4416b470 con 0x7f2a4415d3a0 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4416b2f0 con 0x7f2a44160140 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4b000640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4416b5f0 con 0x7f2a440cb2d0 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a49c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 0x7f2a4414e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a49c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 0x7f2a4414e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55526/0 (socket says 172.21.15.43:55526) 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a49c00640 1 -- 172.21.15.43:0/2293046121 learned_addr learned my addr 172.21.15.43:0/2293046121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a49200640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44160140 0x7f2a4414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a49c00640 1 -- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a440cb2d0 msgr2=0x7f2a441fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.537+0000 7f2a4a600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a440cb2d0 0x7f2a441fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49c00640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a440cb2d0 0x7f2a441fc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49c00640 1 -- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44160140 msgr2=0x7f2a4414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49c00640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44160140 0x7f2a4414ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49c00640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440d1830 con 0x7f2a4415d3a0 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49200640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a44160140 0x7f2a4414ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a4a600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f2a440cb2d0 0x7f2a441fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a49c00640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 0x7f2a4414e7d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c10 tx=0x7f2a38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380bc070 con 0x7f2a4415d3a0 2024-09-17T05:35:25.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a38003600 con 0x7f2a4415d3a0 2024-09-17T05:35:25.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380b2040 con 0x7f2a4415d3a0 2024-09-17T05:35:25.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a440b34d0 con 0x7f2a4415d3a0 2024-09-17T05:35:25.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a440b55b0 con 0x7f2a4415d3a0 2024-09-17T05:35:25.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a440ccdc0 con 0x7f2a4415d3a0 2024-09-17T05:35:25.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a380bf070 con 0x7f2a4415d3a0 2024-09-17T05:35:25.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2a24073230 0x7f2a24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.541+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f2a3812db80 con 0x7f2a4415d3a0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a41600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f2a240791e0 0x7f2a2407b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a380c6ea0 con 0x7f2a240791e0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f2a3812df30 con 0x7f2a4415d3a0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a49200640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2a24073230 0x7f2a24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a4a600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f2a240791e0 0x7f2a2407b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a49200640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2a24073230 0x7f2a24075740 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b80a0 tx=0x7f2a30009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a4a600640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f2a240791e0 0x7f2a2407b630 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.545+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== osd.5 v2:172.21.15.62:6826/3886236614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a380c6ea0 con 0x7f2a240791e0 2024-09-17T05:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.557+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 --> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2a4406a1e0 con 0x7f2a240791e0 2024-09-17T05:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a41600640 1 -- 172.21.15.43:0/2293046121 <== osd.5 v2:172.21.15.62:6826/3886236614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2a4406a1e0 con 0x7f2a240791e0 2024-09-17T05:35:25.564 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:25.564 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:25.564 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f2a240791e0 msgr2=0x7f2a2407b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6826/3886236614,v1:172.21.15.62:6827/3886236614] conn(0x7f2a240791e0 0x7f2a2407b630 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2a24073230 msgr2=0x7f2a24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f2a24073230 0x7f2a24075740 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b80a0 tx=0x7f2a30009040 comp rx=0 tx=0).stop 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 msgr2=0x7f2a4414e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 --2- 172.21.15.43:0/2293046121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a4415d3a0 0x7f2a4414e7d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c10 tx=0x7f2a38004040 comp rx=0 tx=0).stop 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4a600640 1 -- 172.21.15.43:0/2293046121 reap_dead start 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 shutdown_connections 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 >> 172.21.15.43:0/2293046121 conn(0x7f2a440691b0 msgr2=0x7f2a440d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 shutdown_connections 2024-09-17T05:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.561+0000 7f2a4b000640 1 -- 172.21.15.43:0/2293046121 wait complete. 2024-09-17T05:35:25.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T05:35:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 -- 172.21.15.43:0/1880247599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc06fc90 msgr2=0x7fc5dc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 --2- 172.21.15.43:0/1880247599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0700a0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0b3bb0 tx=0x7fc5c401c6e0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 -- 172.21.15.43:0/1880247599 shutdown_connections 2024-09-17T05:35:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 --2- 172.21.15.43:0/1880247599 >> v2:172.21.15.62:3300/0 conn(0x7fc5dc0730b0 0x7fc5dc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 --2- 172.21.15.43:0/1880247599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0700a0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 -- 172.21.15.43:0/1880247599 >> 172.21.15.43:0/1880247599 conn(0x7fc5dc0b23d0 msgr2=0x7fc5dc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 -- 172.21.15.43:0/1880247599 shutdown_connections 2024-09-17T05:35:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.737+0000 7fc5e1200640 1 -- 172.21.15.43:0/1880247599 wait complete. 2024-09-17T05:35:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 Processor -- start 2024-09-17T05:35:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 -- start start 2024-09-17T05:35:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5dc15fe10 0x7fc5dc0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc16a000 con 0x7fc5dc0730b0 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc169e80 con 0x7fc5dc06fc90 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc16a180 con 0x7fc5dc15fe10 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55546/0 (socket says 172.21.15.43:55546) 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 -- 172.21.15.43:0/676888358 learned_addr learned my addr 172.21.15.43:0/676888358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5da800640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5dc15fe10 0x7fc5dc0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d9e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 -- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5dc15fe10 msgr2=0x7fc5dc0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5dc15fe10 0x7fc5dc0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 -- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 msgr2=0x7fc5dc0997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 -- 172.21.15.43:0/676888358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0ad700 con 0x7fc5dc0730b0 2024-09-17T05:35:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d9e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d3e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b0040 tx=0x7fc5c40a6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40be020 con 0x7fc5dc0730b0 2024-09-17T05:35:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c4004040 con 0x7fc5dc0730b0 2024-09-17T05:35:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc15d520 con 0x7fc5dc0730b0 2024-09-17T05:35:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.741+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4004450 con 0x7fc5dc0730b0 2024-09-17T05:35:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc0b08c0 con 0x7fc5dc0730b0 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc1fda50 con 0x7fc5dc0730b0 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5c4003600 con 0x7fc5dc0730b0 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5d2000640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 0x7fc5b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fc5c412e5a0 con 0x7fc5dc0730b0 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.745+0000 7fc5d9e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 0x7fc5b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.749+0000 7fc5d9e00640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 0x7fc5b8075710 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000f70 tx=0x7fc5cc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.749+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5c40163f0 con 0x7fc5dc0730b0 2024-09-17T05:35:25.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.925+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fc5dc0b3570 con 0x7fc5b8073200 2024-09-17T05:35:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.925+0000 7fc5d2000640 1 -- 172.21.15.43:0/676888358 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5dc0b3570 con 0x7fc5b8073200 2024-09-17T05:35:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T05:35:25.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 msgr2=0x7fc5b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 0x7fc5b8075710 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000f70 tx=0x7fc5cc009040 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 msgr2=0x7fc5dc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b0040 tx=0x7fc5c40a6540 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 shutdown_connections 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc5b8073200 0x7fc5b8075710 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc5dc15fe10 0x7fc5dc0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc0730b0 0x7fc5dc0cb5d0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 --2- 172.21.15.43:0/676888358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5dc06fc90 0x7fc5dc0997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 >> 172.21.15.43:0/676888358 conn(0x7fc5dc0b23d0 msgr2=0x7fc5dc0b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 shutdown_connections 2024-09-17T05:35:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:25.929+0000 7fc5e1200640 1 -- 172.21.15.43:0/676888358 wait complete. 2024-09-17T05:35:25.954 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T05:35:25.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 -- 172.21.15.43:0/3220606929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e180b67b0 msgr2=0x7f9e1815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 --2- 172.21.15.43:0/3220606929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e180b67b0 0x7f9e1815fa50 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009840 tx=0x7f9e0c01c700 comp rx=0 tx=0).stop 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 -- 172.21.15.43:0/3220606929 shutdown_connections 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 --2- 172.21.15.43:0/3220606929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e180b67b0 0x7f9e1815fa50 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 -- 172.21.15.43:0/3220606929 >> 172.21.15.43:0/3220606929 conn(0x7f9e180b2f60 msgr2=0x7f9e180b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 -- 172.21.15.43:0/3220606929 shutdown_connections 2024-09-17T05:35:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.113+0000 7f9e20400640 1 -- 172.21.15.43:0/3220606929 wait complete. 2024-09-17T05:35:26.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 Processor -- start 2024-09-17T05:35:26.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- start start 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e180b67b0 0x7f9e181fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e1815ff90 0x7f9e1813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 0x7f9e1814dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18169e30 con 0x7f9e181603a0 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18169cb0 con 0x7f9e1815ff90 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f9e18169fb0 con 0x7f9e180b67b0 2024-09-17T05:35:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 0x7f9e1814dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 0x7f9e1814dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55562/0 (socket says 172.21.15.43:55562) 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 -- 172.21.15.43:0/3488478875 learned_addr learned my addr 172.21.15.43:0/3488478875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1da00640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e180b67b0 0x7f9e181fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1d000640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e1815ff90 0x7f9e1813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 -- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e180b67b0 msgr2=0x7f9e181fbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f9e180b67b0 0x7f9e181fbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 -- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e1815ff90 msgr2=0x7f9e1813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e1815ff90 0x7f9e1813d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1813de30 con 0x7f9e181603a0 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1d000640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e1815ff90 0x7f9e1813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e1e400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 0x7f9e1814dd60 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c730 tx=0x7f9e1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10003b80 con 0x7f9e181603a0 2024-09-17T05:35:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e10003d40 con 0x7f9e181603a0 2024-09-17T05:35:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1814e2a0 con 0x7f9e181603a0 2024-09-17T05:35:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e100052d0 con 0x7f9e181603a0 2024-09-17T05:35:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.117+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e181fcad0 con 0x7f9e181603a0 2024-09-17T05:35:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e1815fa50 con 0x7f9e181603a0 2024-09-17T05:35:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9e10005680 con 0x7f9e181603a0 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9df8073200 0x7f9df8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f9e100949f0 con 0x7f9e181603a0 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f9df80791b0 0x7f9df807b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e10013e70 con 0x7f9df80791b0 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f9e10094da0 con 0x7f9e181603a0 2024-09-17T05:35:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e1da00640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9df8073200 0x7f9df8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.121+0000 7f9e1d000640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f9df80791b0 0x7f9df807b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.125+0000 7f9e1da00640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9df8073200 0x7f9df8075710 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0af000 tx=0x7f9e0c0b2040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.125+0000 7f9e1d000640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f9df80791b0 0x7f9df807b600 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.125+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== osd.6 v2:172.21.15.62:6802/2388352512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e10013e70 con 0x7f9df80791b0 2024-09-17T05:35:26.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 --> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9e180b2d80 con 0x7f9df80791b0 2024-09-17T05:35:26.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e15600640 1 -- 172.21.15.43:0/3488478875 <== osd.6 v2:172.21.15.62:6802/2388352512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9e040050e0 con 0x7f9df80791b0 2024-09-17T05:35:26.142 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:26.143 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:26.143 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f9df80791b0 msgr2=0x7f9df807b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6802/2388352512,v1:172.21.15.62:6803/2388352512] conn(0x7f9df80791b0 0x7f9df807b600 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9df8073200 msgr2=0x7f9df8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f9df8073200 0x7f9df8075710 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0af000 tx=0x7f9e0c0b2040 comp rx=0 tx=0).stop 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 msgr2=0x7f9e1814dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.137+0000 7f9e20400640 1 --2- 172.21.15.43:0/3488478875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e181603a0 0x7f9e1814dd60 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c730 tx=0x7f9e1000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.141+0000 7f9e1e400640 1 -- 172.21.15.43:0/3488478875 reap_dead start 2024-09-17T05:35:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.141+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 shutdown_connections 2024-09-17T05:35:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.141+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 >> 172.21.15.43:0/3488478875 conn(0x7f9e180b2f60 msgr2=0x7f9e1806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.141+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 shutdown_connections 2024-09-17T05:35:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.141+0000 7f9e20400640 1 -- 172.21.15.43:0/3488478875 wait complete. 2024-09-17T05:35:26.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 -- 172.21.15.43:0/2656486560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd880b3af0 msgr2=0x7efd880ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 --2- 172.21.15.43:0/2656486560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd880b3af0 0x7efd880ad0b0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7efd88072b60 tx=0x7efd8001cac0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 -- 172.21.15.43:0/2656486560 shutdown_connections 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 --2- 172.21.15.43:0/2656486560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd880b3af0 0x7efd880ad0b0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 --2- 172.21.15.43:0/2656486560 >> v2:172.21.15.62:3300/0 conn(0x7efd88155d70 0x7efd8815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 -- 172.21.15.43:0/2656486560 >> 172.21.15.43:0/2656486560 conn(0x7efd880b23d0 msgr2=0x7efd880d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 -- 172.21.15.43:0/2656486560 shutdown_connections 2024-09-17T05:35:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.317+0000 7efd8f400640 1 -- 172.21.15.43:0/2656486560 wait complete. 2024-09-17T05:35:26.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 Processor -- start 2024-09-17T05:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- start start 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd8806f110 0x7efd880c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd88155d70 0x7efd880c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd88168bc0 con 0x7efd8809cf40 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd88168a40 con 0x7efd88155d70 2024-09-17T05:35:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7efd88168d40 con 0x7efd8806f110 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55580/0 (socket says 172.21.15.43:55580) 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 -- 172.21.15.43:0/65998378 learned_addr learned my addr 172.21.15.43:0/65998378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8ca00640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd8806f110 0x7efd880c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd87e00640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd88155d70 0x7efd880c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 -- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd8806f110 msgr2=0x7efd880c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd8806f110 0x7efd880c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 -- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd88155d70 msgr2=0x7efd880c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd88155d70 0x7efd880c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 -- 172.21.15.43:0/65998378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8809e150 con 0x7efd8809cf40 2024-09-17T05:35:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8d400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7efd800986b0 tx=0x7efd80005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800b2040 con 0x7efd8809cf40 2024-09-17T05:35:26.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd800bb070 con 0x7efd8809cf40 2024-09-17T05:35:26.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80004530 con 0x7efd8809cf40 2024-09-17T05:35:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd880c9cf0 con 0x7efd8809cf40 2024-09-17T05:35:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.321+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8809dc30 con 0x7efd8809cf40 2024-09-17T05:35:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd880c97c0 con 0x7efd8809cf40 2024-09-17T05:35:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd800046d0 con 0x7efd8809cf40 2024-09-17T05:35:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd84c00640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 0x7efd68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7efd8012e920 con 0x7efd8809cf40 2024-09-17T05:35:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd8ca00640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 0x7efd68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.325+0000 7efd8ca00640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 0x7efd68075710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efd880715c0 tx=0x7efd7c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.329+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd880c97c0 con 0x7efd8809cf40 2024-09-17T05:35:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efd880ca210 con 0x7efd68073200 2024-09-17T05:35:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd84c00640 1 -- 172.21.15.43:0/65998378 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd880ca210 con 0x7efd68073200 2024-09-17T05:35:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 msgr2=0x7efd68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 0x7efd68075710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efd880715c0 tx=0x7efd7c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 msgr2=0x7efd880cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7efd800986b0 tx=0x7efd80005a60 comp rx=0 tx=0).stop 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 shutdown_connections 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7efd68073200 0x7efd68075710 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd88155d70 0x7efd880c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8809cf40 0x7efd880cc9a0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 --2- 172.21.15.43:0/65998378 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7efd8806f110 0x7efd880c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 >> 172.21.15.43:0/65998378 conn(0x7efd880b23d0 msgr2=0x7efd880d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 shutdown_connections 2024-09-17T05:35:26.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.505+0000 7efd8f400640 1 -- 172.21.15.43:0/65998378 wait complete. 2024-09-17T05:35:26.531 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T05:35:26.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:35:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 -- 172.21.15.43:0/2626792794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d840cf980 msgr2=0x7f5d840b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 --2- 172.21.15.43:0/2626792794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d840cf980 0x7f5d840b3b80 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009840 tx=0x7f5d6c01c820 comp rx=0 tx=0).stop 2024-09-17T05:35:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 -- 172.21.15.43:0/2626792794 shutdown_connections 2024-09-17T05:35:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 --2- 172.21.15.43:0/2626792794 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d840cf980 0x7f5d840b3b80 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 -- 172.21.15.43:0/2626792794 >> 172.21.15.43:0/2626792794 conn(0x7f5d8406ce60 msgr2=0x7f5d8406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 -- 172.21.15.43:0/2626792794 shutdown_connections 2024-09-17T05:35:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.689+0000 7f5d8bc00640 1 -- 172.21.15.43:0/2626792794 wait complete. 2024-09-17T05:35:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 Processor -- start 2024-09-17T05:35:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 -- start start 2024-09-17T05:35:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d84074c60 0x7f5d8415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 0x7f5d84149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d8414e2d0 0x7f5d8415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8416a000 con 0x7f5d8415b7e0 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84169e80 con 0x7f5d84074c60 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d8bc00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8416a180 con 0x7f5d8414e2d0 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 0x7f5d84149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 0x7f5d84149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55596/0 (socket says 172.21.15.43:55596) 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d7be00640 1 -- 172.21.15.43:0/4067137157 learned_addr learned my addr 172.21.15.43:0/4067137157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d89200640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d84074c60 0x7f5d8415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.693+0000 7f5d89c00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d8414e2d0 0x7f5d8415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 -- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d8414e2d0 msgr2=0x7f5d8415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f5d8414e2d0 0x7f5d8415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 -- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d84074c60 msgr2=0x7f5d8415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d84074c60 0x7f5d8415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d84149e90 con 0x7f5d8415b7e0 2024-09-17T05:35:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d7be00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 0x7f5d84149590 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009790 tx=0x7f5d6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0bb070 con 0x7f5d8415b7e0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d6c005030 con 0x7f5d8415b7e0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8415cd30 con 0x7f5d8415b7e0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0b2040 con 0x7f5d8415b7e0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8414a450 con 0x7f5d8415b7e0 2024-09-17T05:35:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d8414ab10 con 0x7f5d8415b7e0 2024-09-17T05:35:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.697+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d6c0be070 con 0x7f5d8415b7e0 2024-09-17T05:35:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d60073230 0x7f5d60075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f5d6c12dca0 con 0x7f5d8415b7e0 2024-09-17T05:35:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f5d600791e0 0x7f5d6007b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d6007bd00 con 0x7f5d600791e0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=145) ==== 24+0+0 (secure 0 0 0) 0x7f5d6c12e050 con 0x7f5d8415b7e0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d89200640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d60073230 0x7f5d60075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d89c00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f5d600791e0 0x7f5d6007b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d89200640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d60073230 0x7f5d60075740 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5d74007fd0 tx=0x7f5d74009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d89c00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f5d600791e0 0x7f5d6007b630 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.701+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== osd.7 v2:172.21.15.62:6810/1426001971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d6007bd00 con 0x7f5d600791e0 2024-09-17T05:35:26.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.713+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 --> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5d840b3b80 con 0x7f5d600791e0 2024-09-17T05:35:26.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d81600640 1 -- 172.21.15.43:0/4067137157 <== osd.7 v2:172.21.15.62:6810/1426001971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5d840b3b80 con 0x7f5d600791e0 2024-09-17T05:35:26.719 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-17T05:35:26.720 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:35:26.720 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f5d600791e0 msgr2=0x7f5d6007b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6810/1426001971,v1:172.21.15.62:6811/1426001971] conn(0x7f5d600791e0 0x7f5d6007b630 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d60073230 msgr2=0x7f5d60075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f5d60073230 0x7f5d60075740 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5d74007fd0 tx=0x7f5d74009040 comp rx=0 tx=0).stop 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 msgr2=0x7f5d84149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 --2- 172.21.15.43:0/4067137157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8415b7e0 0x7f5d84149590 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009790 tx=0x7f5d6c004040 comp rx=0 tx=0).stop 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d89c00640 1 -- 172.21.15.43:0/4067137157 reap_dead start 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 shutdown_connections 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 >> 172.21.15.43:0/4067137157 conn(0x7f5d8406ce60 msgr2=0x7f5d840ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 shutdown_connections 2024-09-17T05:35:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.717+0000 7f5d8bc00640 1 -- 172.21.15.43:0/4067137157 wait complete. 2024-09-17T05:35:26.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T05:35:26.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- 172.21.15.43:0/4003321997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869415fe10 msgr2=0x7f86940b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 --2- 172.21.15.43:0/4003321997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869415fe10 0x7f86940b4b80 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f86940d0190 tx=0x7f868401c6e0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- 172.21.15.43:0/4003321997 shutdown_connections 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 --2- 172.21.15.43:0/4003321997 >> v2:172.21.15.62:3300/0 conn(0x7f869406fc90 0x7f86940700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 --2- 172.21.15.43:0/4003321997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869415fe10 0x7f86940b4b80 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- 172.21.15.43:0/4003321997 >> 172.21.15.43:0/4003321997 conn(0x7f86940b23d0 msgr2=0x7f86940d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- 172.21.15.43:0/4003321997 shutdown_connections 2024-09-17T05:35:26.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- 172.21.15.43:0/4003321997 wait complete. 2024-09-17T05:35:26.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 Processor -- start 2024-09-17T05:35:26.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.897+0000 7f8698e00640 1 -- start start 2024-09-17T05:35:26.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 0x7f86940b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f869415fe10 0x7f8694150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f869416a020 con 0x7f869406fc90 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8694169ea0 con 0x7f86940730b0 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7f869416a1a0 con 0x7f869415fe10 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55606/0 (socket says 172.21.15.43:55606) 2024-09-17T05:35:26.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 -- 172.21.15.43:0/652757051 learned_addr learned my addr 172.21.15.43:0/652757051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:26.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8692800640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f869415fe10 0x7f8694150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f868be00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 0x7f86940b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 -- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f869415fe10 msgr2=0x7f8694150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f869415fe10 0x7f8694150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 -- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 msgr2=0x7f86940b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:26.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 0x7f86940b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:26.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 -- 172.21.15.43:0/652757051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86940ca630 con 0x7f869406fc90 2024-09-17T05:35:26.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f868be00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 0x7f86940b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:26.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8691e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f868401ad90 tx=0x7f86840a6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86840bd030 con 0x7f869406fc90 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8684004040 con 0x7f869406fc90 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684004250 con 0x7f869406fc90 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869415b850 con 0x7f869406fc90 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869415be10 con 0x7f869406fc90 2024-09-17T05:35:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.901+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86940700a0 con 0x7f869406fc90 2024-09-17T05:35:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.905+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f86840c0070 con 0x7f869406fc90 2024-09-17T05:35:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.905+0000 7f868a000640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 0x7f8670075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.905+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7f868412f1e0 con 0x7f869406fc90 2024-09-17T05:35:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.905+0000 7f868be00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 0x7f8670075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.905+0000 7f868be00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 0x7f8670075710 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f86940b15c0 tx=0x7f867c0091e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:26.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:26.909+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86840c7020 con 0x7f869406fc90 2024-09-17T05:35:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.081+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f86940b1b70 con 0x7f8670073200 2024-09-17T05:35:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.081+0000 7f868a000640 1 -- 172.21.15.43:0/652757051 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f86940b1b70 con 0x7f8670073200 2024-09-17T05:35:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 msgr2=0x7f8670075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 0x7f8670075710 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f86940b15c0 tx=0x7f867c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 msgr2=0x7f86941f6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f868401ad90 tx=0x7f86840a6540 comp rx=0 tx=0).stop 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 shutdown_connections 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7f8670073200 0x7f8670075710 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86940730b0 0x7f86940b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7f869415fe10 0x7f8694150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 --2- 172.21.15.43:0/652757051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f869406fc90 0x7f86941f6cb0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 >> 172.21.15.43:0/652757051 conn(0x7f86940b23d0 msgr2=0x7f86940ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 shutdown_connections 2024-09-17T05:35:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.085+0000 7f8698e00640 1 -- 172.21.15.43:0/652757051 wait complete. 2024-09-17T05:35:27.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:35:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- 172.21.15.43:0/3595231698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe464004810 msgr2=0x7fe464004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 --2- 172.21.15.43:0/3595231698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe464004810 0x7fe464004c20 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c2e0 tx=0x7fe46801f6a0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- 172.21.15.43:0/3595231698 shutdown_connections 2024-09-17T05:35:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 --2- 172.21.15.43:0/3595231698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe464004810 0x7fe464004c20 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- 172.21.15.43:0/3595231698 >> 172.21.15.43:0/3595231698 conn(0x7fe4800b2e80 msgr2=0x7fe4800b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- 172.21.15.43:0/3595231698 shutdown_connections 2024-09-17T05:35:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- 172.21.15.43:0/3595231698 wait complete. 2024-09-17T05:35:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 Processor -- start 2024-09-17T05:35:27.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.265+0000 7fe485800640 1 -- start start 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe464004810 0x7fe480154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe48014f660 0x7fe4800c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4800b6570 con 0x7fe4800c78f0 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4800b63f0 con 0x7fe48014f660 2024-09-17T05:35:27.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe48015f530 con 0x7fe464004810 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55630/0 (socket says 172.21.15.43:55630) 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 -- 172.21.15.43:0/162879077 learned_addr learned my addr 172.21.15.43:0/162879077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe476000640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe48014f660 0x7fe4800c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47ea00640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe464004810 0x7fe480154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 -- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe464004810 msgr2=0x7fe480154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe464004810 0x7fe480154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 -- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe48014f660 msgr2=0x7fe4800c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe48014f660 0x7fe4800c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 -- 172.21.15.43:0/162879077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4800c7d00 con 0x7fe4800c78f0 2024-09-17T05:35:27.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe47f400640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fe480075360 tx=0x7fe47800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478003d90 con 0x7fe4800c78f0 2024-09-17T05:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe478004b30 con 0x7fe4800c78f0 2024-09-17T05:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478004030 con 0x7fe4800c78f0 2024-09-17T05:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe480156100 con 0x7fe4800c78f0 2024-09-17T05:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801566c0 con 0x7fe4800c78f0 2024-09-17T05:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.269+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48015fa90 con 0x7fe4800c78f0 2024-09-17T05:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.273+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4780041d0 con 0x7fe4800c78f0 2024-09-17T05:35:27.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.273+0000 7fe477400640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 0x7fe45c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:27.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.273+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fe478094d90 con 0x7fe4800c78f0 2024-09-17T05:35:27.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.273+0000 7fe47ea00640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 0x7fe45c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:27.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.273+0000 7fe47ea00640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 0x7fe45c075710 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe48006a400 tx=0x7fe470006b80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:27.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.277+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe478020020 con 0x7fe4800c78f0 2024-09-17T05:35:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.449+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4800b7120 con 0x7fe45c073200 2024-09-17T05:35:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.449+0000 7fe477400640 1 -- 172.21.15.43:0/162879077 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13363 (secure 0 0 0) 0x7fe4800b7120 con 0x7fe45c073200 2024-09-17T05:35:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 msgr2=0x7fe45c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 0x7fe45c075710 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe48006a400 tx=0x7fe470006b80 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 msgr2=0x7fe480156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fe480075360 tx=0x7fe47800f790 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 shutdown_connections 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe45c073200 0x7fe45c075710 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe48014f660 0x7fe4800c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4800c78f0 0x7fe480156ec0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 --2- 172.21.15.43:0/162879077 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe464004810 0x7fe480154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 >> 172.21.15.43:0/162879077 conn(0x7fe4800b2e80 msgr2=0x7fe48006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 shutdown_connections 2024-09-17T05:35:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:27.453+0000 7fe485800640 1 -- 172.21.15.43:0/162879077 wait complete. 2024-09-17T05:35:27.477 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-17T05:35:26.041709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294236,"kb_used_data":15592,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680484,"statfs":{"total":773094113280,"available":772792815616,"internally_reserved":0,"allocated":15966208,"data_stored":8251688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":285134386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002380"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:35:16.347074+0000","last_change":"2024-09-17T05:23:15.971367+0000","last_active":"2024-09-17T05:35:16.347074+0000","last_peered":"2024-09-17T05:35:16.347074+0000","last_clean":"2024-09-17T05:35:16.347074+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:35:16.347074+0000","last_undegraded":"2024-09-17T05:35:16.347074+0000","last_fullsized":"2024-09-17T05:35:16.347074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:23:14.851058+0000","last_clean_scrub_stamp":"2024-09-17T05:23:14.851058+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:15.085038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28836,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37732,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334108,"statfs":{"total":96636764160,"available":96598126592,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41312,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330528,"statfs":{"total":96636764160,"available":96594460672,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:35:27.480 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T05:23:14.851058+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=23, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=35, tm_sec=21, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T05:35:27.481 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T05:35:47.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:35:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 -- 172.21.15.43:0/3019644521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b5fe0 msgr2=0x7fa7e015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 --2- 172.21.15.43:0/3019644521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b5fe0 0x7fa7e015f530 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009900 tx=0x7fa7d401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 -- 172.21.15.43:0/3019644521 shutdown_connections 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 --2- 172.21.15.43:0/3019644521 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b43c0 0x7fa7e00b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 --2- 172.21.15.43:0/3019644521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b5fe0 0x7fa7e015f530 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 -- 172.21.15.43:0/3019644521 >> 172.21.15.43:0/3019644521 conn(0x7fa7e00b2e80 msgr2=0x7fa7e00b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 -- 172.21.15.43:0/3019644521 shutdown_connections 2024-09-17T05:35:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.649+0000 7fa7e8800640 1 -- 172.21.15.43:0/3019644521 wait complete. 2024-09-17T05:35:47.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 Processor -- start 2024-09-17T05:35:47.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 -- start start 2024-09-17T05:35:47.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:47.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b5fe0 0x7fa7e00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e015fa70 0x7fa7e01f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0169130 con 0x7fa7e00b43c0 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0168fb0 con 0x7fa7e015fa70 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e8800640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e01692b0 con 0x7fa7e00b5fe0 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51276/0 (socket says 172.21.15.43:51276) 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.653+0000 7fa7e5e00640 1 -- 172.21.15.43:0/343230608 learned_addr learned my addr 172.21.15.43:0/343230608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5400640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b5fe0 0x7fa7e00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e6800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e015fa70 0x7fa7e01f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:47.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 -- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b5fe0 msgr2=0x7fa7e00c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b5fe0 0x7fa7e00c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 -- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e015fa70 msgr2=0x7fa7e01f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e015fa70 0x7fa7e01f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 -- 172.21.15.43:0/343230608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e015bc90 con 0x7fa7e00b43c0 2024-09-17T05:35:47.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e5e00640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009850 tx=0x7fa7d4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b2040 con 0x7fa7e00b43c0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7d4005320 con 0x7fa7e00b43c0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e01f7830 con 0x7fa7e00b43c0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40046d0 con 0x7fa7e00b43c0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e00ca830 con 0x7fa7e00b43c0 2024-09-17T05:35:47.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e00b47d0 con 0x7fa7e00b43c0 2024-09-17T05:35:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.657+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7d4004870 con 0x7fa7e00b43c0 2024-09-17T05:35:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.661+0000 7fa7da000640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:47.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.661+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fa7d412df20 con 0x7fa7e00b43c0 2024-09-17T05:35:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.661+0000 7fa7e5400640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.661+0000 7fa7e5400640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 0x7fa7c4075740 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00c7b30 tx=0x7fa7d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:47.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.665+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7d40bc060 con 0x7fa7e00b43c0 2024-09-17T05:35:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.837+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 --> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7e01f7b10 con 0x7fa7c4073230 2024-09-17T05:35:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.837+0000 7fa7da000640 1 -- 172.21.15.43:0/343230608 <== mgr.4117 v2:172.21.15.62:6800/1563535289 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7fa7e01f7b10 con 0x7fa7c4073230 2024-09-17T05:35:47.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 msgr2=0x7fa7c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 0x7fa7c4075740 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00c7b30 tx=0x7fa7d0009040 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 msgr2=0x7fa7e00cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009850 tx=0x7fa7d4005760 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 shutdown_connections 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e015fa70 0x7fa7e01f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fa7e00b5fe0 0x7fa7e00c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 --2- 172.21.15.43:0/343230608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00b43c0 0x7fa7e00cb030 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 >> 172.21.15.43:0/343230608 conn(0x7fa7e00b2e80 msgr2=0x7fa7e015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 shutdown_connections 2024-09-17T05:35:47.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:47.841+0000 7fa7e8800640 1 -- 172.21.15.43:0/343230608 wait complete. 2024-09-17T05:35:47.864 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-17T05:35:46.047106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":294228,"kb_used_data":15592,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754680492,"statfs":{"total":773094113280,"available":772792823808,"internally_reserved":0,"allocated":15966208,"data_stored":8251688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":285134386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003103"},"pg_stats":[{"pgid":"1.0","version":"145'74","reported_seq":364,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:35:23.907487+0000","last_change":"2024-09-17T05:35:23.907487+0000","last_active":"2024-09-17T05:35:23.907487+0000","last_peered":"2024-09-17T05:35:23.907487+0000","last_clean":"2024-09-17T05:35:23.907487+0000","last_became_active":"2024-09-17T05:23:15.969830+0000","last_became_peered":"2024-09-17T05:23:15.969830+0000","last_unstale":"2024-09-17T05:35:23.907487+0000","last_undegraded":"2024-09-17T05:35:23.907487+0000","last_fullsized":"2024-09-17T05:35:23.907487+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'74","last_scrub_stamp":"2024-09-17T05:35:23.907405+0000","last_deep_scrub":"145'74","last_deep_scrub_stamp":"2024-09-17T05:35:23.907405+0000","last_clean_scrub_stamp":"2024-09-17T05:35:23.907405+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:37.379452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509594,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28836,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509593,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1474237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37156,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334684,"statfs":{"total":96636764160,"available":96598716416,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542297,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41312,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330528,"statfs":{"total":96636764160,"available":96594460672,"internally_reserved":0,"allocated":1847296,"data_stored":883869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:35:47.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- 172.21.15.43:0/257039390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540b7ce0 msgr2=0x7fc6540d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- 172.21.15.43:0/257039390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540b7ce0 0x7fc6540d0840 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fc65409bbe0 tx=0x7fc64801c600 comp rx=0 tx=0).stop 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- 172.21.15.43:0/257039390 shutdown_connections 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- 172.21.15.43:0/257039390 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540cd580 0x7fc6541609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- 172.21.15.43:0/257039390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540b7ce0 0x7fc6540d0840 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- 172.21.15.43:0/257039390 >> 172.21.15.43:0/257039390 conn(0x7fc6540cee80 msgr2=0x7fc6540cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- 172.21.15.43:0/257039390 shutdown_connections 2024-09-17T05:35:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- 172.21.15.43:0/257039390 wait complete. 2024-09-17T05:35:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 Processor -- start 2024-09-17T05:35:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- start start 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6540b7ce0 0x7fc654152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540d0d80 0x7fc6541582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc65416a020 con 0x7fc6540cd580 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc654169ea0 con 0x7fc6540b7ce0 2024-09-17T05:35:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.021+0000 7fc65c200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fc65416a1a0 con 0x7fc6540d0d80 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51302/0 (socket says 172.21.15.43:51302) 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 -- 172.21.15.43:0/112899599 learned_addr learned my addr 172.21.15.43:0/112899599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc659800640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6540b7ce0 0x7fc654152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc65a200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540d0d80 0x7fc6541582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 -- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540d0d80 msgr2=0x7fc6541582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540d0d80 0x7fc6541582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 -- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6540b7ce0 msgr2=0x7fc654152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6540b7ce0 0x7fc654152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 -- 172.21.15.43:0/112899599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654159cf0 con 0x7fc6540cd580 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc658e00640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b3c0 tx=0x7fc64000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640015020 con 0x7fc6540cd580 2024-09-17T05:35:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc640011920 con 0x7fc6540cd580 2024-09-17T05:35:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc654159fd0 con 0x7fc6540cd580 2024-09-17T05:35:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640011d90 con 0x7fc6540cd580 2024-09-17T05:35:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65415a590 con 0x7fc6540cd580 2024-09-17T05:35:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6540d0840 con 0x7fc6540cd580 2024-09-17T05:35:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.025+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc640018020 con 0x7fc6540cd580 2024-09-17T05:35:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.029+0000 7fc651600640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 0x7fc634075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.029+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fc640093cf0 con 0x7fc6540cd580 2024-09-17T05:35:48.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.029+0000 7fc659800640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 0x7fc634075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.029+0000 7fc659800640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 0x7fc634075710 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ae000 tx=0x7fc64801ae40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.033+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc64001f030 con 0x7fc6540cd580 2024-09-17T05:35:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.213+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc654158ca0 con 0x7fc6540cd580 2024-09-17T05:35:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.217+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v51)=0 v51) ==== 123+0+0 (secure 0 0 0) 0x7fc64005ef10 con 0x7fc6540cd580 2024-09-17T05:35:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.217+0000 7fc651600640 1 -- 172.21.15.43:0/112899599 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fc64005e320 con 0x7fc6540cd580 2024-09-17T05:35:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 msgr2=0x7fc634075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 0x7fc634075710 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ae000 tx=0x7fc64801ae40 comp rx=0 tx=0).stop 2024-09-17T05:35:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 msgr2=0x7fc654157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc64000b3c0 tx=0x7fc64000b880 comp rx=0 tx=0).stop 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 shutdown_connections 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fc634073200 0x7fc634075710 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fc6540d0d80 0x7fc6541582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6540cd580 0x7fc654157d70 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 --2- 172.21.15.43:0/112899599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6540b7ce0 0x7fc654152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 >> 172.21.15.43:0/112899599 conn(0x7fc6540cee80 msgr2=0x7fc6541591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 shutdown_connections 2024-09-17T05:35:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.221+0000 7fc65c200640 1 -- 172.21.15.43:0/112899599 wait complete. 2024-09-17T05:35:48.245 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:35:48.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- 172.21.15.43:0/298714596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 msgr2=0x7fe8bc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 --2- 172.21.15.43:0/298714596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc06d270 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0d12a0 tx=0x7fe8a401c830 comp rx=0 tx=0).stop 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- 172.21.15.43:0/298714596 shutdown_connections 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 --2- 172.21.15.43:0/298714596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc06d270 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 --2- 172.21.15.43:0/298714596 >> v2:172.21.15.62:3300/0 conn(0x7fe8bc070620 0x7fe8bc0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- 172.21.15.43:0/298714596 >> 172.21.15.43:0/298714596 conn(0x7fe8bc0b23d0 msgr2=0x7fe8bc0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- 172.21.15.43:0/298714596 shutdown_connections 2024-09-17T05:35:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- 172.21.15.43:0/298714596 wait complete. 2024-09-17T05:35:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 Processor -- start 2024-09-17T05:35:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.401+0000 7fe8c2200640 1 -- start start 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 --2- >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8bc070620 0x7fe8bc0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 0x7fe8bc0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0cede0 con 0x7fe8bc06ce60 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0cec60 con 0x7fe8bc0cb860 2024-09-17T05:35:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- --> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc168b40 con 0x7fe8bc070620 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51320/0 (socket says 172.21.15.43:51320) 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 -- 172.21.15.43:0/3767741344 learned_addr learned my addr 172.21.15.43:0/3767741344 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8b2a00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8bc070620 0x7fe8bc0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bbe00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 0x7fe8bc0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 -- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8bc070620 msgr2=0x7fe8bc0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8bc070620 0x7fe8bc0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 -- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 msgr2=0x7fe8bc0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 0x7fe8bc0c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 -- 172.21.15.43:0/3767741344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0c4f20 con 0x7fe8bc06ce60 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bbe00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 0x7fe8bc0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8bb400640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e7f0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00ccc0 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fe8ac005020 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc0c7cd0 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac005490 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc0c8290 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc06d270 con 0x7fe8bc06ce60 2024-09-17T05:35:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.405+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8ac0047b0 con 0x7fe8bc06ce60 2024-09-17T05:35:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.409+0000 7fe8b3e00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 0x7fe898075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.409+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5029+0+0 (secure 0 0 0) 0x7fe8ac094990 con 0x7fe8bc06ce60 2024-09-17T05:35:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.409+0000 7fe8b2a00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 0x7fe898075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.409+0000 7fe8b2a00640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 0x7fe898075740 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40af000 tx=0x7fe8a40195d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.413+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8ac05fc40 con 0x7fe8bc06ce60 2024-09-17T05:35:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.621+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8bc09e190 con 0x7fe8bc06ce60 2024-09-17T05:35:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.621+0000 7fe8b3e00640 1 -- 172.21.15.43:0/3767741344 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 354 v354) ==== 95+0+3911 (secure 0 0 0) 0x7fe8ac066070 con 0x7fe8bc06ce60 2024-09-17T05:35:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-17T05:35:48.626 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":354,"btime":"2024-09-17T05:35:14:316759+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8753,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3845717422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6835","nonce":3845717422},{"type":"v1","addr":"172.21.15.62:6837","nonce":3845717422}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9862,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2790787946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2790787946},{"type":"v1","addr":"172.21.15.43:6835","nonce":2790787946}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":9871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6836/446428682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":446428682},{"type":"v1","addr":"172.21.15.62:6836","nonce":446428682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":340},{"gid":10953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/605179984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":605179984},{"type":"v1","addr":"172.21.15.43:6834","nonce":605179984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":354}],"filesystems":[]} 2024-09-17T05:35:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 354 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 msgr2=0x7fe898075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 0x7fe898075740 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40af000 tx=0x7fe8a40195d0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 msgr2=0x7fe8bc1576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e7f0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 shutdown_connections 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:6800/1563535289,v1:172.21.15.62:6801/1563535289] conn(0x7fe898073230 0x7fe898075740 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8bc0cb860 0x7fe8bc0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] conn(0x7fe8bc070620 0x7fe8bc0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 --2- 172.21.15.43:0/3767741344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8bc06ce60 0x7fe8bc1576b0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 >> 172.21.15.43:0/3767741344 conn(0x7fe8bc0b23d0 msgr2=0x7fe8bc15a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 shutdown_connections 2024-09-17T05:35:48.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-17T05:35:48.625+0000 7fe8c2200640 1 -- 172.21.15.43:0/3767741344 wait complete. 2024-09-17T05:35:48.648 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T05:35:48.648 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T05:35:48.648 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:35:48.649 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.661 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:35:48.661 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:35:48.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.726 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:35:48.726 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:35:48.727 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.741 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:35:48.741 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:35:48.742 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.798 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:35:48.798 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T05:35:48.798 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T05:35:48.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.894 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T05:35:48.894 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T05:35:48.894 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:48.975 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T05:35:48.975 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T05:35:48.975 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:49.048 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T05:35:49.049 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T05:35:49.049 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:49.148 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T05:35:49.148 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T05:35:49.148 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:49.246 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T05:35:49.246 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T05:35:49.246 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:52.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:52.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:52.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-17T05:35:55.348 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T05:35:55.348 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T05:35:55.348 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.423 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T05:35:55.423 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T05:35:55.423 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.521 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T05:35:55.522 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T05:35:55.522 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T05:35:55.522 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.586 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T05:35:55.586 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T05:35:55.586 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.651 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T05:35:55.651 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T05:35:55.651 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T05:35:55.651 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.674 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T05:35:55.674 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T05:35:55.674 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.692 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T05:35:55.692 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T05:35:55.692 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:35:55.751 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T05:35:55.751 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T05:35:55.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-09-17T05:35:55.803 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:35:55.803 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T05:35:56.172 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:35:56.172 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T05:35:56.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:35:56.384 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T05:35:56.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:35:56.609 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T05:35:56.809 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:35:56.809 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T05:35:57.048 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:35:57.048 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T05:35:57.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:35:57.250 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T05:35:57.442 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:35:57.442 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T05:35:57.608 INFO:tasks.ceph:Archiving mon data... 2024-09-17T05:35:57.609 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/data/mon.a.tgz 2024-09-17T05:35:57.610 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-17T05:35:57.619 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.IDfF6gV2zi 2024-09-17T05:35:57.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IDfF6gV2zi 2024-09-17T05:35:57.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.IDfF6gV2zi 2024-09-17T05:35:57.883 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.IDfF6gV2zi is 629KB 2024-09-17T05:35:57.949 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.IDfF6gV2zi 2024-09-17T05:35:57.958 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/data/mon.b.tgz 2024-09-17T05:35:57.958 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-09-17T05:35:57.967 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.djFpcaK2YR 2024-09-17T05:35:57.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.djFpcaK2YR 2024-09-17T05:35:57.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-17T05:35:58.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.djFpcaK2YR 2024-09-17T05:35:58.238 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.djFpcaK2YR is 653KB 2024-09-17T05:35:58.305 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.djFpcaK2YR 2024-09-17T05:35:58.314 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/data/mon.c.tgz 2024-09-17T05:35:58.314 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-09-17T05:35:58.362 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.oWifrLrvw9 2024-09-17T05:35:58.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.oWifrLrvw9 2024-09-17T05:35:58.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.oWifrLrvw9 2024-09-17T05:35:58.635 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.oWifrLrvw9 is 653KB 2024-09-17T05:35:58.701 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.oWifrLrvw9 2024-09-17T05:35:58.710 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T05:35:58.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:35:58.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:35:58.784 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:35:58.822 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:35:58.822 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T05:35:58.824 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi043/crash 2024-09-17T05:35:58.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:35:58.889 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi062/crash 2024-09-17T05:35:58.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:35:58.909 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T05:35:58.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:58.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:59.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:59.036 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:59.055 INFO:tasks.ceph:Compressing logs... 2024-09-17T05:35:59.055 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:35:59.086 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:35:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39153.log 2024-09-17T05:35:59.104 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2024-09-17T05:35:59.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-09-17T05:35:59.105 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39153.log.gz 2024-09-17T05:35:59.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49515.log 2024-09-17T05:35:59.106 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2024-09-17T05:35:59.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40103.log 2024-09-17T05:35:59.107 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-09-17T05:35:59.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-09-17T05:35:59.108 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49515.log.gz 2024-09-17T05:35:59.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56238.log 2024-09-17T05:35:59.109 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-09-17T05:35:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53472.log 2024-09-17T05:35:59.110 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-09-17T05:35:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40471.log 2024-09-17T05:35:59.111 INFO:teuthology.orchestra.run.smithi043.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.56238.log.gz 2024-09-17T05:35:59.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.53472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-09-17T05:35:59.112 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53472.log.gz 2024-09-17T05:35:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40870.log 2024-09-17T05:35:59.113 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40471.log.gz 2024-09-17T05:35:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2024-09-17T05:35:59.114 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-09-17T05:35:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44953.log 2024-09-17T05:35:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55465.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40870.log.gz 2024-09-17T05:35:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40575.log 2024-09-17T05:35:59.115 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2024-09-17T05:35:59.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2024-09-17T05:35:59.116 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44953.log.gz 2024-09-17T05:35:59.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40575.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23916.log 2024-09-17T05:35:59.117 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40575.log.gz 2024-09-17T05:35:59.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37206.log 87.5% 2024-09-17T05:35:59.118 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz 2024-09-17T05:35:59.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.tmp-client.admin.23916.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.43500.log 2024-09-17T05:35:59.119 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.23916.log.gz 2024-09-17T05:35:59.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36424.log 2024-09-17T05:35:59.120 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37206.log.gz 2024-09-17T05:35:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-09-17T05:35:59.121 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-09-17T05:35:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.36424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36424.log.gzgzip 2024-09-17T05:35:59.121 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52263.log 2024-09-17T05:35:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42975.log 2024-09-17T05:35:59.122 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-09-17T05:35:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47435.log 2024-09-17T05:35:59.123 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52263.log.gz 2024-09-17T05:35:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38884.log 2024-09-17T05:35:59.124 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42975.log.gz 2024-09-17T05:35:59.125 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-17T05:35:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45192.log 2024-09-17T05:35:59.126 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29113.log 2024-09-17T05:35:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38884.log: gzip 92.9% -- replaced with /var/log/ceph/ceph-client.admin.47435.log.gz -5 2024-09-17T05:35:59.126 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43643.log 2024-09-17T05:35:59.126 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38884.log.gz 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35841.log 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43643.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45192.log.gz 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43643.log.gz 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-09-17T05:35:59.129 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35841.log.gz 2024-09-17T05:35:59.130 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.29010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28962.log 2024-09-17T05:35:59.130 INFO:teuthology.orchestra.run.smithi062.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-17T05:35:59.130 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.29113.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T05:35:59.130 INFO:teuthology.orchestra.run.smithi062.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29113.log.gz 2024-09-17T05:35:59.130 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.28962.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T05:35:59.131 INFO:teuthology.orchestra.run.smithi062.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28962.log.gz 2024-09-17T05:35:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29089.log 2024-09-17T05:35:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29138.log 2024-09-17T05:35:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.29089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34334.log 2024-09-17T05:35:59.131 INFO:teuthology.orchestra.run.smithi062.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29089.log.gz 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.29138.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29138.log.gz 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.3.34334.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.3.34334.log.gz 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.1.34248.log 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T05:35:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.1.34248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34248.log.gz 2024-09-17T05:35:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T05:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47260.log 2024-09-17T05:35:59.136 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28741.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49371.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47260.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28741.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33311.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36937.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.49371.log.gzgzip 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33311.log.gz 2024-09-17T05:35:59.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53754.log 2024-09-17T05:35:59.138 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-09-17T05:35:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37141.log 2024-09-17T05:35:59.138 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2024-09-17T05:35:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28986.log 2024-09-17T05:35:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T05:35:59.142 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.28986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29041.log 2024-09-17T05:35:59.142 INFO:teuthology.orchestra.run.smithi062.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28986.log.gz 2024-09-17T05:35:59.142 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.29041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29065.log 2024-09-17T05:35:59.142 INFO:teuthology.orchestra.run.smithi062.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29041.log.gz 2024-09-17T05:35:59.142 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53754.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53754.log.gz 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.37141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37141.log.gzgzip 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28798.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50666.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28798.log.gz 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54076.log 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-17T05:35:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33565.log 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.29065.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29065.log.gz 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34230.log 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.1.34230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34230.log.gz 2024-09-17T05:35:59.146 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34339.log 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54076.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.50666.log.gz 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35919.log 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54076.log.gz 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38910.log 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33565.log.gz 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35919.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35919.log.gz 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-09-17T05:35:59.148 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T05:35:59.149 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38910.log.gz 2024-09-17T05:35:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35047.log 2024-09-17T05:35:59.149 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-09-17T05:35:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38000.log 2024-09-17T05:35:59.149 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-09-17T05:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.3.34339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34339.log.gz 2024-09-17T05:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T05:35:59.150 INFO:teuthology.orchestra.run.smithi062.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.35047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56186.log 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35047.log.gz 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38000.log.gz 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44315.log 2024-09-17T05:35:59.151 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56186.log.gz 2024-09-17T05:35:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-09-17T05:35:59.152 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-09-17T05:35:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-09-17T05:35:59.153 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44315.log.gz 2024-09-17T05:35:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34150.log 2024-09-17T05:35:59.153 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-09-17T05:35:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51684.log 2024-09-17T05:35:59.154 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-09-17T05:35:59.155 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T05:35:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-09-17T05:35:59.155 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34150.log.gz 2024-09-17T05:35:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58281.log 2024-09-17T05:35:59.156 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T05:35:59.157 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51684.log.gz 2024-09-17T05:35:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39663.log 2024-09-17T05:35:59.157 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-09-17T05:35:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38416.log 2024-09-17T05:35:59.158 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58281.log.gz 2024-09-17T05:35:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35788.log 2024-09-17T05:35:59.158 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39663.log.gz 2024-09-17T05:35:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-09-17T05:35:59.159 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38416.log.gz 2024-09-17T05:35:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 87.5% 2024-09-17T05:35:59.160 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35788.log.gz 2024-09-17T05:35:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56959.log 2024-09-17T05:35:59.161 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-09-17T05:35:59.161 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56565.log 2024-09-17T05:35:59.161 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-09-17T05:35:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45723.log 2024-09-17T05:35:59.162 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56959.log.gz 2024-09-17T05:35:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56565.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56565.log.gz 2024-09-17T05:35:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47549.log 2024-09-17T05:35:59.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48155.log 2024-09-17T05:35:59.164 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45723.log.gz 2024-09-17T05:35:59.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2024-09-17T05:35:59.165 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47549.log.gz 2024-09-17T05:35:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.48155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48155.log.gz 2024-09-17T05:35:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35999.log 2024-09-17T05:35:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28771.log 2024-09-17T05:35:59.167 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-09-17T05:35:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2024-09-17T05:35:59.168 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35999.log.gz 2024-09-17T05:35:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28771.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T05:35:59.169 INFO:teuthology.orchestra.run.smithi043.stderr: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.28771.log.gz 2024-09-17T05:35:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49930.log 2024-09-17T05:35:59.170 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2024-09-17T05:35:59.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55107.log 2024-09-17T05:35:59.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-09-17T05:35:59.171 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49930.log.gz 2024-09-17T05:35:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55107.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55107.log.gz 2024-09-17T05:35:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36223.log 2024-09-17T05:35:59.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37946.log 2024-09-17T05:35:59.173 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-09-17T05:35:59.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44579.log 2024-09-17T05:35:59.174 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36223.log.gz 2024-09-17T05:35:59.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55000.log 2024-09-17T05:35:59.175 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37946.log.gz 2024-09-17T05:35:59.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54849.log 2024-09-17T05:35:59.176 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44579.log.gz 2024-09-17T05:35:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55000.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.55000.log.gz -5 2024-09-17T05:35:59.176 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40655.log 2024-09-17T05:35:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2024-09-17T05:35:59.177 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54849.log.gz 2024-09-17T05:35:59.178 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33278.log 2024-09-17T05:35:59.178 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40655.log.gz 2024-09-17T05:35:59.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-09-17T05:35:59.180 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 87.3% 2024-09-17T05:35:59.180 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33278.log.gz 2024-09-17T05:35:59.181 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz 2024-09-17T05:35:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46090.log 2024-09-17T05:35:59.181 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-09-17T05:35:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49772.log 2024-09-17T05:35:59.182 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2024-09-17T05:35:59.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43079.log 2024-09-17T05:35:59.183 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46090.log.gz 2024-09-17T05:35:59.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55387.log 2024-09-17T05:35:59.184 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49772.log.gz 2024-09-17T05:35:59.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43079.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.43079.log.gz 2024-09-17T05:35:59.184 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-17T05:35:59.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2024-09-17T05:35:59.185 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55387.log.gz 2024-09-17T05:35:59.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47912.log 2024-09-17T05:35:59.186 INFO:teuthology.orchestra.run.smithi043.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-17T05:35:59.187 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2024-09-17T05:35:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49904.log 2024-09-17T05:35:59.188 INFO:teuthology.orchestra.run.smithi043.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2024-09-17T05:35:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51639.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 92.9% 2024-09-17T05:35:59.189 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47912.log.gz 2024-09-17T05:35:59.189 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2024-09-17T05:35:59.190 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2024-09-17T05:35:59.190 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49904.log.gz 2024-09-17T05:35:59.190 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48448.log 2024-09-17T05:35:59.190 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz 2024-09-17T05:35:59.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-09-17T05:35:59.191 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2024-09-17T05:35:59.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2024-09-17T05:35:59.193 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48448.log.gz 2024-09-17T05:35:59.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39205.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-09-17T05:35:59.193 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33513.log 2024-09-17T05:35:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2024-09-17T05:35:59.194 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz 2024-09-17T05:35:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-09-17T05:35:59.195 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33513.log.gz 2024-09-17T05:35:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52099.log 2024-09-17T05:35:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55569.log 2024-09-17T05:35:59.197 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-09-17T05:35:59.197 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2024-09-17T05:35:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-09-17T05:35:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55933.log 2024-09-17T05:35:59.199 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.52099.log.gz 2024-09-17T05:35:59.199 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55569.log.gz 2024-09-17T05:35:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2024-09-17T05:35:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55933.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-09-17T05:35:59.200 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55933.log.gz 2024-09-17T05:35:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47807.log 2024-09-17T05:35:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39283.log 2024-09-17T05:35:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53235.log 2024-09-17T05:35:59.202 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47807.log.gz 2024-09-17T05:35:59.202 INFO:teuthology.orchestra.run.smithi043.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-09-17T05:35:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2024-09-17T05:35:59.203 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39283.log.gz 2024-09-17T05:35:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39875.log 2024-09-17T05:35:59.204 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34579.log 2024-09-17T05:35:59.205 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.53235.log.gz 2024-09-17T05:35:59.205 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2024-09-17T05:35:59.205 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38943.log 2024-09-17T05:35:59.206 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39875.log.gz 2024-09-17T05:35:59.206 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34579.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34579.log.gz 2024-09-17T05:35:59.206 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-09-17T05:35:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-09-17T05:35:59.207 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38943.log.gz 2024-09-17T05:35:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38465.log 2024-09-17T05:35:59.208 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-09-17T05:35:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56897.log 2024-09-17T05:35:59.209 INFO:teuthology.orchestra.run.smithi043.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-09-17T05:35:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.38465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38465.log.gz 2024-09-17T05:35:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37552.log 2024-09-17T05:35:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.56897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56897.log.gz 2024-09-17T05:35:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-09-17T05:35:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41942.log 2024-09-17T05:35:59.212 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37552.log.gz 2024-09-17T05:35:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51103.log 2024-09-17T05:35:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48671.log 2024-09-17T05:35:59.213 INFO:teuthology.orchestra.run.smithi043.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-09-17T05:35:59.213 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41942.log.gz 2024-09-17T05:35:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-09-17T05:35:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52359.log 2024-09-17T05:35:59.215 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.51103.log.gz 2024-09-17T05:35:59.215 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48671.log.gz 2024-09-17T05:35:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-09-17T05:35:59.216 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-09-17T05:35:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52359.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52359.log.gz 2024-09-17T05:35:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-09-17T05:35:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28910.log 2024-09-17T05:35:59.217 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-09-17T05:35:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-09-17T05:35:59.218 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-09-17T05:35:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57402.log 2024-09-17T05:35:59.219 INFO:teuthology.orchestra.run.smithi043.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28910.log.gz 2024-09-17T05:35:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33877.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-09-17T05:35:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45495.log 2024-09-17T05:35:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-09-17T05:35:59.221 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57402.log.gz 2024-09-17T05:35:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-09-17T05:35:59.222 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2024-09-17T05:35:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34356.log 2024-09-17T05:35:59.223 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-09-17T05:35:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39520.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-09-17T05:35:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40276.log 2024-09-17T05:35:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46744.log 2024-09-17T05:35:59.224 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34356.log.gz 2024-09-17T05:35:59.225 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51974.log 2024-09-17T05:35:59.225 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40276.log.gz 2024-09-17T05:35:59.226 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-09-17T05:35:59.226 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46744.log.gz 2024-09-17T05:35:59.227 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51974.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51974.log.gz 2024-09-17T05:35:59.227 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2024-09-17T05:35:59.227 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40603.log 2024-09-17T05:35:59.228 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2024-09-17T05:35:59.228 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-09-17T05:35:59.229 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2024-09-17T05:35:59.229 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-09-17T05:35:59.230 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40603.log.gz 2024-09-17T05:35:59.230 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43925.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-09-17T05:35:59.230 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36562.log 2024-09-17T05:35:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39337.log 2024-09-17T05:35:59.231 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-09-17T05:35:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43901.log 2024-09-17T05:35:59.232 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36562.log.gz 2024-09-17T05:35:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45759.log 2024-09-17T05:35:59.233 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39337.log.gz 2024-09-17T05:35:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.43901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43901.log.gz 2024-09-17T05:35:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55081.log 2024-09-17T05:35:59.234 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50832.log 2024-09-17T05:35:59.234 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45759.log.gz 2024-09-17T05:35:59.235 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45608.log 2024-09-17T05:35:59.235 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55081.log.gz 2024-09-17T05:35:59.236 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51745.log 2024-09-17T05:35:59.236 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50832.log.gz 2024-09-17T05:35:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45608.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45608.log.gz 2024-09-17T05:35:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54051.log 2024-09-17T05:35:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51745.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T05:35:59.238 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51745.log.gz 2024-09-17T05:35:59.239 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49320.log 2024-09-17T05:35:59.239 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54051.log.gz 2024-09-17T05:35:59.239 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56108.log 2024-09-17T05:35:59.240 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49320.log: gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.45433.log 2024-09-17T05:35:59.240 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2024-09-17T05:35:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-09-17T05:35:59.241 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56108.log.gz 2024-09-17T05:35:59.242 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39745.log 2024-09-17T05:35:59.242 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45433.log.gz 2024-09-17T05:35:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-17T05:35:59.243 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-09-17T05:35:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42949.log 2024-09-17T05:35:59.244 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39745.log.gz 2024-09-17T05:35:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57710.log 2024-09-17T05:35:59.245 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-17T05:35:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56672.log 2024-09-17T05:35:59.246 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42949.log.gz 2024-09-17T05:35:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57506.log 2024-09-17T05:35:59.246 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57710.log.gz 2024-09-17T05:35:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47000.log 2024-09-17T05:35:59.247 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56672.log.gz 2024-09-17T05:35:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38184.log 2024-09-17T05:35:59.248 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57506.log.gz 2024-09-17T05:35:59.249 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-09-17T05:35:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38184.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.47000.log.gz 2024-09-17T05:35:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-09-17T05:35:59.250 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38184.log.gz 2024-09-17T05:35:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-09-17T05:35:59.251 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-09-17T05:35:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48720.log 2024-09-17T05:35:59.252 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-09-17T05:35:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33949.log 2024-09-17T05:35:59.253 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-09-17T05:35:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44491.log 2024-09-17T05:35:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33949.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33949.log.gz 2024-09-17T05:35:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 92.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.48720.log.gz --verbose 2024-09-17T05:35:59.255 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.0.42249.log 2024-09-17T05:35:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-09-17T05:35:59.256 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44491.log.gz 2024-09-17T05:35:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.42249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42249.log.gz 2024-09-17T05:35:59.257 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51390.log 2024-09-17T05:35:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56078.log 2024-09-17T05:35:59.258 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-09-17T05:35:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28937.log 2024-09-17T05:35:59.259 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51390.log.gz 2024-09-17T05:35:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57969.log 2024-09-17T05:35:59.260 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56078.log.gz 2024-09-17T05:35:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28937.log: gzip 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28937.log.gz -5 2024-09-17T05:35:59.260 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51462.log 2024-09-17T05:35:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37455.log 2024-09-17T05:35:59.261 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57969.log.gz 2024-09-17T05:35:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36514.log 2024-09-17T05:35:59.262 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51462.log.gz 2024-09-17T05:35:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37455.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37455.log.gz 2024-09-17T05:35:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-09-17T05:35:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56290.log 2024-09-17T05:35:59.264 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36514.log.gz 2024-09-17T05:35:59.265 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33953.log 2024-09-17T05:35:59.265 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-09-17T05:35:59.266 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46113.log 2024-09-17T05:35:59.266 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56290.log.gz 2024-09-17T05:35:59.267 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33953.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33953.log.gz 2024-09-17T05:35:59.267 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38564.log 2024-09-17T05:35:59.268 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52048.log 2024-09-17T05:35:59.269 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38564.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.46113.log.gz 2024-09-17T05:35:59.269 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38564.log.gz 2024-09-17T05:35:59.269 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44293.log 2024-09-17T05:35:59.270 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52048.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.52048.log.gz 2024-09-17T05:35:59.270 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-09-17T05:35:59.271 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-09-17T05:35:59.271 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44293.log.gz 2024-09-17T05:35:59.272 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58047.log 2024-09-17T05:35:59.272 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-09-17T05:35:59.273 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51364.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-09-17T05:35:59.273 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37180.log 2024-09-17T05:35:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49032.log 2024-09-17T05:35:59.274 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.58047.log.gz 2024-09-17T05:35:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43669.log 2024-09-17T05:35:59.275 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37180.log.gz 2024-09-17T05:35:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48108.log 2024-09-17T05:35:59.275 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49032.log.gz 2024-09-17T05:35:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55152.log 2024-09-17T05:35:59.276 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43669.log.gz 2024-09-17T05:35:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54555.log 2024-09-17T05:35:59.277 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48108.log.gz 2024-09-17T05:35:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45074.log 2024-09-17T05:35:59.278 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55152.log.gz 2024-09-17T05:35:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54555.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T05:35:59.279 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54555.log.gz 2024-09-17T05:35:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47781.log 2024-09-17T05:35:59.280 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45074.log.gz 2024-09-17T05:35:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41782.log 2024-09-17T05:35:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47781.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-09-17T05:35:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36849.log 2024-09-17T05:35:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34161.log 2024-09-17T05:35:59.283 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36849.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41782.log.gz 2024-09-17T05:35:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2024-09-17T05:35:59.284 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36849.log.gz 2024-09-17T05:35:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54316.log 2024-09-17T05:35:59.285 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34161.log.gz 2024-09-17T05:35:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.56396.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44072.log 2024-09-17T05:35:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54316.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gz 2024-09-17T05:35:59.287 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46135.log 2024-09-17T05:35:59.287 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54316.log.gz 2024-09-17T05:35:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45289.log 2024-09-17T05:35:59.289 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44072.log.gz 2024-09-17T05:35:59.289 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46249.log 2024-09-17T05:35:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46135.log: /var/log/ceph/ceph-client.0.45289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46135.log.gz 2024-09-17T05:35:59.290 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45289.log.gz 2024-09-17T05:35:59.290 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-09-17T05:35:59.292 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2024-09-17T05:35:59.292 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46249.log.gz 2024-09-17T05:35:59.293 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39309.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-09-17T05:35:59.293 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.44182.log 2024-09-17T05:35:59.293 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.49177.log: --verbose -- /var/log/ceph/ceph-client.admin.41350.log 2024-09-17T05:35:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44182.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2024-09-17T05:35:59.295 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35405.log 2024-09-17T05:35:59.295 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44182.log.gz 2024-09-17T05:35:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44601.log 2024-09-17T05:35:59.296 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41350.log.gz 2024-09-17T05:35:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.35405.log: -5 --verbose -- /var/log/ceph/ceph-client.0.49695.log 2024-09-17T05:35:59.297 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35405.log.gz 2024-09-17T05:35:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43443.log 2024-09-17T05:35:59.298 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44601.log.gz 2024-09-17T05:35:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.49695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49695.log.gz 2024-09-17T05:35:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43695.log 2024-09-17T05:35:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29006.log 2024-09-17T05:35:59.301 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43443.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43443.log.gz 2024-09-17T05:35:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52776.log 2024-09-17T05:35:59.302 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43695.log.gz 2024-09-17T05:35:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29006.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29006.log.gz 2024-09-17T05:35:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54197.log 2024-09-17T05:35:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47135.log 2024-09-17T05:35:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.52776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52776.log.gz 2024-09-17T05:35:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-09-17T05:35:59.305 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54197.log.gz 2024-09-17T05:35:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47135.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47135.log.gz 2024-09-17T05:35:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56909.log 2024-09-17T05:35:59.307 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39771.log 2024-09-17T05:35:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56909.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-09-17T05:35:59.308 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56909.log.gz 2024-09-17T05:35:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42563.log 2024-09-17T05:35:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29356.log 2024-09-17T05:35:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39771.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39771.log.gz 2024-09-17T05:35:59.310 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57061.log 2024-09-17T05:35:59.310 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42563.log.gz 2024-09-17T05:35:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29356.log: gzip 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29356.log.gz 2024-09-17T05:35:59.311 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.40155.log 2024-09-17T05:35:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57061.log: -- /var/log/ceph/ceph-client.admin.34152.log 2024-09-17T05:35:59.313 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.40155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40155.log.gz 2024-09-17T05:35:59.313 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57061.log.gz 2024-09-17T05:35:59.314 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38730.log 2024-09-17T05:35:59.315 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-09-17T05:35:59.315 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34152.log.gz 2024-09-17T05:35:59.316 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37403.log 2024-09-17T05:35:59.316 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38730.log.gz 2024-09-17T05:35:59.317 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53860.log: /var/log/ceph/ceph-client.admin.36275.log 2024-09-17T05:35:59.317 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-09-17T05:35:59.318 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37578.log 2024-09-17T05:35:59.318 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37403.log.gz 2024-09-17T05:35:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36275.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36275.log.gz 2024-09-17T05:35:59.319 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37429.log 2024-09-17T05:35:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55205.log 2024-09-17T05:35:59.320 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37578.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37578.log.gz 2024-09-17T05:35:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34553.log 2024-09-17T05:35:59.321 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37429.log.gz 2024-09-17T05:35:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55205.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55205.log.gz 2024-09-17T05:35:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49076.log 2024-09-17T05:35:59.323 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33695.log 2024-09-17T05:35:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34553.log.gz 2024-09-17T05:35:59.324 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51339.log 2024-09-17T05:35:59.324 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49076.log.gz 2024-09-17T05:35:59.325 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33695.log: gzip -5 87.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.33695.log.gz 2024-09-17T05:35:59.325 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.52511.log 2024-09-17T05:35:59.326 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33405.log 2024-09-17T05:35:59.327 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51339.log: /var/log/ceph/ceph-client.admin.52511.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51339.log.gz 2024-09-17T05:35:59.327 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2024-09-17T05:35:59.327 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54226.log 2024-09-17T05:35:59.328 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58308.log 2024-09-17T05:35:59.329 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33405.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33405.log.gz 2024-09-17T05:35:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54226.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54226.log.gz 2024-09-17T05:35:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51875.log 2024-09-17T05:35:59.330 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38026.log 2024-09-17T05:35:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58308.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58308.log.gz 2024-09-17T05:35:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-09-17T05:35:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T05:35:59.332 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38026.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.51875.log.gz 2024-09-17T05:35:59.333 INFO:teuthology.orchestra.run.smithi062.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T05:35:59.333 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-09-17T05:35:59.333 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38026.log.gz 2024-09-17T05:35:59.334 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.40844.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40500.log 2024-09-17T05:35:59.334 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-09-17T05:35:59.335 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-09-17T05:35:59.335 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-17T05:35:59.336 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38052.log 2024-09-17T05:35:59.337 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38652.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.40500.log.gz 2024-09-17T05:35:59.337 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-09-17T05:35:59.337 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-09-17T05:35:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53477.log 2024-09-17T05:35:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38052.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38052.log.gz 2024-09-17T05:35:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29007.log 2024-09-17T05:35:59.340 INFO:teuthology.orchestra.run.smithi043.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-09-17T05:35:59.340 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.53477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52121.log 2024-09-17T05:35:59.340 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53477.log.gz 2024-09-17T05:35:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46064.log 2024-09-17T05:35:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29007.log: /var/log/ceph/ceph-client.admin.52121.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29007.log.gz 2024-09-17T05:35:59.342 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41539.log 2024-09-17T05:35:59.342 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52121.log.gz 2024-09-17T05:35:59.343 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36088.log 2024-09-17T05:35:59.344 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46064.log.gz 2024-09-17T05:35:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41539.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41539.log.gz 2024-09-17T05:35:59.344 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35535.log 2024-09-17T05:35:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-09-17T05:35:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36088.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2024-09-17T05:35:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35535.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35535.log.gz 2024-09-17T05:35:59.346 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36454.log 2024-09-17T05:35:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-09-17T05:35:59.348 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-09-17T05:35:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36454.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36454.log.gz 2024-09-17T05:35:59.348 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38756.log 2024-09-17T05:35:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39998.log 2024-09-17T05:35:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44050.log: /var/log/ceph/ceph-client.admin.38756.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38756.log.gz 2024-09-17T05:35:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-09-17T05:35:59.351 INFO:teuthology.orchestra.run.smithi043.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-09-17T05:35:59.351 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.39998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51080.log 2024-09-17T05:35:59.352 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39998.log.gz 2024-09-17T05:35:59.352 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37630.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-09-17T05:35:59.353 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-09-17T05:35:59.353 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28997.log 2024-09-17T05:35:59.354 INFO:teuthology.orchestra.run.smithi062.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T05:35:59.355 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51080.log.gz 2024-09-17T05:35:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2024-09-17T05:35:59.355 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-09-17T05:35:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44403.log 2024-09-17T05:35:59.356 INFO:teuthology.orchestra.run.smithi043.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.28997.log.gz 2024-09-17T05:35:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58177.log 2024-09-17T05:35:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44403.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2024-09-17T05:35:59.358 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33459.log 2024-09-17T05:35:59.359 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44403.log.gz 2024-09-17T05:35:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56134.log 2024-09-17T05:35:59.360 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58177.log.gz 2024-09-17T05:35:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57865.log 2024-09-17T05:35:59.361 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33459.log.gz 2024-09-17T05:35:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-09-17T05:35:59.362 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56134.log.gz 2024-09-17T05:35:59.363 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57684.log 2024-09-17T05:35:59.363 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57865.log.gz 2024-09-17T05:35:59.364 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48160.log 2024-09-17T05:35:59.364 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-09-17T05:35:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41890.log 2024-09-17T05:35:59.365 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57684.log.gz 2024-09-17T05:35:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29035.log 2024-09-17T05:35:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.48160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48160.log.gz 2024-09-17T05:35:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-09-17T05:35:59.368 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41890.log.gz 2024-09-17T05:35:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55026.log 2024-09-17T05:35:59.369 INFO:teuthology.orchestra.run.smithi043.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29035.log.gz 2024-09-17T05:35:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54964.log 2024-09-17T05:35:59.370 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-09-17T05:35:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52618.log 2024-09-17T05:35:59.371 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55026.log.gz 2024-09-17T05:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55413.log 2024-09-17T05:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54964.log.gz 2024-09-17T05:35:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-09-17T05:35:59.373 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52618.log.gz 2024-09-17T05:35:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-09-17T05:35:59.374 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55413.log.gz 2024-09-17T05:35:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48589.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-09-17T05:35:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56493.log 2024-09-17T05:35:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28967.log 2024-09-17T05:35:59.376 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-09-17T05:35:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.56493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56493.log.gz 2024-09-17T05:35:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42254.log 2024-09-17T05:35:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53717.log 2024-09-17T05:35:59.378 INFO:teuthology.orchestra.run.smithi043.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.28967.log.gz 2024-09-17T05:35:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.42254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42254.log.gz 2024-09-17T05:35:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50858.log 2024-09-17T05:35:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34361.log 2024-09-17T05:35:59.381 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53717.log.gz 2024-09-17T05:35:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50858.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50858.log.gzgzip 2024-09-17T05:35:59.382 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2024-09-17T05:35:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48803.log 2024-09-17T05:35:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44711.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34361.log.gz 2024-09-17T05:35:59.384 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz 2024-09-17T05:35:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46568.log 2024-09-17T05:35:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-09-17T05:35:59.386 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48803.log.gz 2024-09-17T05:35:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.46568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46568.log.gz 2024-09-17T05:35:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-09-17T05:35:59.387 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45242.log 2024-09-17T05:35:59.389 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2024-09-17T05:35:59.389 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-09-17T05:35:59.389 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-09-17T05:35:59.390 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2024-09-17T05:35:59.390 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45242.log.gz 2024-09-17T05:35:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47356.log 2024-09-17T05:35:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52289.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2024-09-17T05:35:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2024-09-17T05:35:59.392 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2024-09-17T05:35:59.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47356.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.47356.log.gz 2024-09-17T05:35:59.393 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-09-17T05:35:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.48767.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49006.log 2024-09-17T05:35:59.395 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2024-09-17T05:35:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-09-17T05:35:59.396 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz 2024-09-17T05:35:59.396 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37892.log 2024-09-17T05:35:59.396 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49006.log.gz 2024-09-17T05:35:59.397 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51948.log 2024-09-17T05:35:59.398 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-09-17T05:35:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-09-17T05:35:59.399 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37892.log.gz 2024-09-17T05:35:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2024-09-17T05:35:59.400 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51948.log.gz 2024-09-17T05:35:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.43535.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35125.log 2024-09-17T05:35:59.401 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-09-17T05:35:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45964.log 2024-09-17T05:35:59.402 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2024-09-17T05:35:59.403 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48209.log 2024-09-17T05:35:59.403 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35125.log.gz 2024-09-17T05:35:59.404 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-09-17T05:35:59.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45964.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45964.log.gz 2024-09-17T05:35:59.405 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51709.log 2024-09-17T05:35:59.405 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48209.log.gz 2024-09-17T05:35:59.406 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34795.log 2024-09-17T05:35:59.406 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-09-17T05:35:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53282.log 2024-09-17T05:35:59.407 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51709.log.gz 2024-09-17T05:35:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58099.log 2024-09-17T05:35:59.408 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34795.log.gz 2024-09-17T05:35:59.409 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53054.log 2024-09-17T05:35:59.409 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53282.log.gz 2024-09-17T05:35:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.58099.log: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T05:35:59.411 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58099.log.gz 2024-09-17T05:35:59.411 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45294.log 2024-09-17T05:35:59.412 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53054.log.gz 2024-09-17T05:35:59.412 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29018.log 2024-09-17T05:35:59.414 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.45294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52819.log 2024-09-17T05:35:59.414 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45294.log.gz 2024-09-17T05:35:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49444.log 2024-09-17T05:35:59.415 INFO:teuthology.orchestra.run.smithi043.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29018.log.gz 2024-09-17T05:35:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40395.log 2024-09-17T05:35:59.417 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52819.log.gz 2024-09-17T05:35:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41454.log 2024-09-17T05:35:59.418 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49444.log.gz 2024-09-17T05:35:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-09-17T05:35:59.419 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40395.log.gz 2024-09-17T05:35:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49985.log 2024-09-17T05:35:59.420 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41454.log.gz 2024-09-17T05:35:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55673.log 2024-09-17T05:35:59.421 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-09-17T05:35:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48471.log 2024-09-17T05:35:59.422 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49985.log.gz 2024-09-17T05:35:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35613.log 2024-09-17T05:35:59.423 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55673.log.gz 2024-09-17T05:35:59.425 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-09-17T05:35:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38858.log 2024-09-17T05:35:59.426 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35613.log.gz 2024-09-17T05:35:59.426 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.48471.log.gz 2024-09-17T05:35:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57995.log 2024-09-17T05:35:59.427 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-09-17T05:35:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41059.log 2024-09-17T05:35:59.428 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38858.log.gz 2024-09-17T05:35:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35071.log 2024-09-17T05:35:59.429 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57995.log.gz 2024-09-17T05:35:59.430 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-09-17T05:35:59.431 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41059.log.gz 2024-09-17T05:35:59.431 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45579.log 2024-09-17T05:35:59.432 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35071.log.gz 2024-09-17T05:35:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29499.log 2024-09-17T05:35:59.433 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-09-17T05:35:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44799.log 2024-09-17T05:35:59.434 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45579.log.gz 2024-09-17T05:35:59.435 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53425.log 2024-09-17T05:35:59.435 INFO:teuthology.orchestra.run.smithi043.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29499.log.gz 2024-09-17T05:35:59.436 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37974.log 2024-09-17T05:35:59.436 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44799.log.gz 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53425.log.gz 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34914.log 2024-09-17T05:35:59.439 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37974.log.gz 2024-09-17T05:35:59.440 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35665.log 2024-09-17T05:35:59.440 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.34914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34914.log.gz 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35665.log.gz 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34709.log 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-09-17T05:35:59.444 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-09-17T05:35:59.444 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-09-17T05:35:59.446 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45458.log 2024-09-17T05:35:59.446 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34709.log.gz 2024-09-17T05:35:59.447 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44733.log 2024-09-17T05:35:59.448 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-09-17T05:35:59.448 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45458.log.gz 2024-09-17T05:35:59.448 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-09-17T05:35:59.449 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45913.log 2024-09-17T05:35:59.450 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44733.log.gz 2024-09-17T05:35:59.450 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-09-17T05:35:59.450 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-09-17T05:35:59.452 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55276.log 2024-09-17T05:35:59.453 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45913.log.gz 2024-09-17T05:35:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35814.log 2024-09-17T05:35:59.453 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-09-17T05:35:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-09-17T05:35:59.454 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55276.log.gz 2024-09-17T05:35:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54917.log 2024-09-17T05:35:59.455 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35814.log.gz 2024-09-17T05:35:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33955.log 2024-09-17T05:35:59.457 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-09-17T05:35:59.458 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41506.log 2024-09-17T05:35:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38132.log 2024-09-17T05:35:59.459 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33955.log.gz 2024-09-17T05:35:59.459 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.54917.log.gz 2024-09-17T05:35:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50777.log 2024-09-17T05:35:59.460 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41506.log.gz 2024-09-17T05:35:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50643.log 2024-09-17T05:35:59.461 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38132.log.gz 2024-09-17T05:35:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55491.log 2024-09-17T05:35:59.463 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50777.log.gz 2024-09-17T05:35:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56845.log 2024-09-17T05:35:59.464 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50643.log.gz 2024-09-17T05:35:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2024-09-17T05:35:59.465 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55491.log.gz 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56845.log.gz 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50713.log 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-09-17T05:35:59.469 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2024-09-17T05:35:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54531.log 2024-09-17T05:35:59.470 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50713.log.gz 2024-09-17T05:35:59.471 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52182.log 2024-09-17T05:35:59.471 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-09-17T05:35:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.54531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54531.log.gz 2024-09-17T05:35:59.472 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50107.log 2024-09-17T05:35:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-09-17T05:35:59.473 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52182.log.gz 2024-09-17T05:35:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49878.log 2024-09-17T05:35:59.474 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50107.log.gz 2024-09-17T05:35:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41402.log 2024-09-17T05:35:59.476 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-09-17T05:35:59.476 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-09-17T05:35:59.477 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2024-09-17T05:35:59.478 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58255.log 2024-09-17T05:35:59.478 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41402.log.gz 2024-09-17T05:35:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-09-17T05:35:59.479 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-09-17T05:35:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54171.log 2024-09-17T05:35:59.481 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58255.log.gz 2024-09-17T05:35:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-09-17T05:35:59.482 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-09-17T05:35:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57532.log 2024-09-17T05:35:59.483 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54171.log.gz 2024-09-17T05:35:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50881.log 2024-09-17T05:35:59.484 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-09-17T05:35:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49418.log 2024-09-17T05:35:59.485 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57532.log.gz 2024-09-17T05:35:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2024-09-17T05:35:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50055.log 2024-09-17T05:35:59.487 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49418.log.gz 2024-09-17T05:35:59.488 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.50881.log.gz 2024-09-17T05:35:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-09-17T05:35:59.489 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gz 2024-09-17T05:35:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54113.log 2024-09-17T05:35:59.490 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50055.log.gz 2024-09-17T05:35:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2024-09-17T05:35:59.491 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-09-17T05:35:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-09-17T05:35:59.492 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54113.log.gz 2024-09-17T05:35:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38366.log 2024-09-17T05:35:59.494 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz 2024-09-17T05:35:59.494 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33379.log 2024-09-17T05:35:59.495 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-09-17T05:35:59.495 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38366.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T05:35:59.496 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38366.log.gz 2024-09-17T05:35:59.497 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2024-09-17T05:35:59.497 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33379.log.gz 2024-09-17T05:35:59.498 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-09-17T05:35:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52648.log 2024-09-17T05:35:59.500 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2024-09-17T05:35:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52556.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-09-17T05:35:59.502 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52648.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52648.log.gz 2024-09-17T05:35:59.502 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-09-17T05:35:59.504 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40204.log 2024-09-17T05:35:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55250.log 2024-09-17T05:35:59.505 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-09-17T05:35:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40204.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40204.log.gz 2024-09-17T05:35:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55250.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55250.log.gz 2024-09-17T05:35:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51125.log 2024-09-17T05:35:59.509 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42422.log 2024-09-17T05:35:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38470.log 2024-09-17T05:35:59.511 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51125.log.gz 2024-09-17T05:35:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42422.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-09-17T05:35:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.38470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38470.log.gz 2024-09-17T05:35:59.512 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2024-09-17T05:35:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44138.log 2024-09-17T05:35:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35097.log 2024-09-17T05:35:59.516 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2024-09-17T05:35:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44138.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44138.log.gz 2024-09-17T05:35:59.517 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35097.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35097.log.gz 2024-09-17T05:35:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40445.log 2024-09-17T05:35:59.519 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35281.log 2024-09-17T05:35:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38314.log 2024-09-17T05:35:59.521 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40445.log.gz 2024-09-17T05:35:59.522 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35281.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35281.log.gz 2024-09-17T05:35:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38314.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38314.log.gz 2024-09-17T05:35:59.523 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-09-17T05:35:59.525 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40003.log 2024-09-17T05:35:59.527 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56446.log 2024-09-17T05:35:59.527 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-09-17T05:35:59.527 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.40003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40003.log.gz 2024-09-17T05:35:59.528 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56446.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56446.log.gz 2024-09-17T05:35:59.529 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33617.log 2024-09-17T05:35:59.530 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-09-17T05:35:59.531 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56344.log 2024-09-17T05:35:59.531 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33617.log.gz 2024-09-17T05:35:59.533 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44094.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-09-17T05:35:59.534 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56344.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56344.log.gz 2024-09-17T05:35:59.534 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-09-17T05:35:59.536 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43326.log 2024-09-17T05:35:59.537 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35255.log 2024-09-17T05:35:59.537 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz/var/log/ceph/ceph-client.admin.43326.log: 2024-09-17T05:35:59.538 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-09-17T05:35:59.539 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47109.log 2024-09-17T05:35:59.539 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35255.log.gz 2024-09-17T05:35:59.540 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53784.log 2024-09-17T05:35:59.542 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54484.log 2024-09-17T05:35:59.542 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47109.log.gz 2024-09-17T05:35:59.543 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53784.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53784.log.gz 2024-09-17T05:35:59.543 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35639.log 2024-09-17T05:35:59.544 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42604.log 2024-09-17T05:35:59.545 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54484.log.gz 2024-09-17T05:35:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50253.log 2024-09-17T05:35:59.546 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35639.log.gz 2024-09-17T05:35:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42604.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42604.log.gz 2024-09-17T05:35:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50253.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50253.log.gz 2024-09-17T05:35:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-09-17T05:35:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54536.log 2024-09-17T05:35:59.551 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45313.log 2024-09-17T05:35:59.552 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-09-17T05:35:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.54536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54536.log.gz 2024-09-17T05:35:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45313.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2024-09-17T05:35:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2024-09-17T05:35:59.554 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2024-09-17T05:35:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56318.log 2024-09-17T05:35:59.557 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2024-09-17T05:35:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49959.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2024-09-17T05:35:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56318.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56318.log.gz 2024-09-17T05:35:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33539.log 2024-09-17T05:35:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45938.log 2024-09-17T05:35:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55963.log 2024-09-17T05:35:59.562 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33539.log.gz 2024-09-17T05:35:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45938.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45938.log.gz 2024-09-17T05:35:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55963.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55963.log.gz 2024-09-17T05:35:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38210.log 2024-09-17T05:35:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-09-17T05:35:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57428.log 2024-09-17T05:35:59.567 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38210.log.gz 2024-09-17T05:35:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44931.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-09-17T05:35:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57428.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57428.log.gz 2024-09-17T05:35:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43896.log 2024-09-17T05:35:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54434.log 2024-09-17T05:35:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.43896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52534.log 2024-09-17T05:35:59.572 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43896.log.gz 2024-09-17T05:35:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54434.log: /var/log/ceph/ceph-client.admin.52534.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54434.log.gz 2024-09-17T05:35:59.575 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.52534.log.gz 2024-09-17T05:35:59.576 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41194.log 2024-09-17T05:35:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50207.log 2024-09-17T05:35:59.579 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57287.log 2024-09-17T05:35:59.579 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41194.log.gz 2024-09-17T05:35:59.579 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50207.log.gz 2024-09-17T05:35:59.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57287.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57287.log.gz 2024-09-17T05:35:59.581 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49291.log 2024-09-17T05:35:59.582 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57035.log 2024-09-17T05:35:59.584 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-17T05:35:59.584 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49291.log.gz 2024-09-17T05:35:59.585 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57035.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57035.log.gz 2024-09-17T05:35:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43721.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-17T05:35:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37918.log 2024-09-17T05:35:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48341.log 2024-09-17T05:35:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37708.log 2024-09-17T05:35:59.589 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37918.log.gz 2024-09-17T05:35:59.590 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48341.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48341.log.gz 2024-09-17T05:35:59.591 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37708.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37708.log.gz 2024-09-17T05:35:59.591 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52411.log 2024-09-17T05:35:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37265.log 2024-09-17T05:35:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35229.log 2024-09-17T05:35:59.594 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-09-17T05:35:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37265.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37265.log.gz 2024-09-17T05:35:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35229.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35229.log.gz 2024-09-17T05:35:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-09-17T05:35:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45785.log 2024-09-17T05:35:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50928.log 2024-09-17T05:35:59.601 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45785.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-09-17T05:35:59.601 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45785.log.gz 2024-09-17T05:35:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50928.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50928.log.gz 2024-09-17T05:35:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49643.log 2024-09-17T05:35:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29411.log 2024-09-17T05:35:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57480.log 2024-09-17T05:35:59.605 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49643.log.gz 2024-09-17T05:35:59.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29411.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29411.log.gz 2024-09-17T05:35:59.607 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57480.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57480.log.gz 2024-09-17T05:35:59.607 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-09-17T05:35:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54700.log 2024-09-17T05:35:59.610 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2024-09-17T05:35:59.610 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-09-17T05:35:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54700.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54700.log.gz 2024-09-17T05:35:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53931.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2024-09-17T05:35:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29332.log 2024-09-17T05:35:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51317.log 2024-09-17T05:35:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-09-17T05:35:59.616 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29332.log.gz 2024-09-17T05:35:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51317.log: /var/log/ceph/ceph-client.admin.41428.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-09-17T05:35:59.619 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.51317.log.gz 2024-09-17T05:35:59.619 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50008.log 2024-09-17T05:35:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33721.log 2024-09-17T05:35:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33903.log 2024-09-17T05:35:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33721.log: 92.9% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33721.log.gz 2024-09-17T05:35:59.622 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50008.log.gz 2024-09-17T05:35:59.623 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54271.log 2024-09-17T05:35:59.624 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33903.log.gz 2024-09-17T05:35:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-09-17T05:35:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57211.log 2024-09-17T05:35:59.626 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54271.log.gz 2024-09-17T05:35:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55816.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz 2024-09-17T05:35:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57211.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57211.log.gz 2024-09-17T05:35:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2024-09-17T05:35:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57891.log 2024-09-17T05:35:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28840.log 2024-09-17T05:35:59.631 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2024-09-17T05:35:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57891.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57891.log.gz 2024-09-17T05:35:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28840.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28840.log.gz 2024-09-17T05:35:59.634 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2024-09-17T05:35:59.635 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55699.log 2024-09-17T05:35:59.636 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37039.log 2024-09-17T05:35:59.637 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2024-09-17T05:35:59.637 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55699.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55699.log.gz 2024-09-17T05:35:59.639 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37039.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37039.log.gz 2024-09-17T05:35:59.639 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-09-17T05:35:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39396.log 2024-09-17T05:35:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-09-17T05:35:59.642 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-09-17T05:35:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39396.log.gz 2024-09-17T05:35:59.644 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39179.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-09-17T05:35:59.645 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49346.log 2024-09-17T05:35:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-09-17T05:35:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2024-09-17T05:35:59.647 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49346.log.gz 2024-09-17T05:35:59.648 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49239.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-09-17T05:35:59.649 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55621.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-09-17T05:35:59.650 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42676.log 2024-09-17T05:35:59.651 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35346.log 2024-09-17T05:35:59.652 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-09-17T05:35:59.653 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42676.log.gz 2024-09-17T05:35:59.653 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35346.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35346.log.gz 2024-09-17T05:35:59.655 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-09-17T05:35:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43987.log 2024-09-17T05:35:59.658 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50466.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-09-17T05:35:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54585.log 2024-09-17T05:35:59.659 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-09-17T05:35:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-09-17T05:35:59.659 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43987.log.gz 2024-09-17T05:35:59.660 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54585.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54585.log.gz 2024-09-17T05:35:59.661 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-09-17T05:35:59.662 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-09-17T05:35:59.663 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-09-17T05:35:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48280.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-09-17T05:35:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46327.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-09-17T05:35:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56739.log 2024-09-17T05:35:59.667 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50688.log 2024-09-17T05:35:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43391.log 2024-09-17T05:35:59.669 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56739.log.gz 2024-09-17T05:35:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50688.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50688.log.gz 2024-09-17T05:35:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43391.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43391.log.gz 2024-09-17T05:35:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52871.log 2024-09-17T05:35:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-09-17T05:35:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44755.log 2024-09-17T05:35:59.674 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52871.log.gz 2024-09-17T05:35:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47457.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-09-17T05:35:59.676 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44755.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44755.log.gz 2024-09-17T05:35:59.676 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35509.log 2024-09-17T05:35:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33957.log 2024-09-17T05:35:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53907.log 2024-09-17T05:35:59.680 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35509.log.gz 2024-09-17T05:35:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33957.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33957.log.gz 2024-09-17T05:35:59.681 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.53907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53907.log.gz 2024-09-17T05:35:59.681 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35151.log 2024-09-17T05:35:59.683 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48179.log 2024-09-17T05:35:59.684 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-09-17T05:35:59.685 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35151.log.gz 2024-09-17T05:35:59.685 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48179.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48179.log.gz 2024-09-17T05:35:59.687 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33851.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-09-17T05:35:59.687 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53017.log 2024-09-17T05:35:59.688 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-09-17T05:35:59.689 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41480.log 2024-09-17T05:35:59.689 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53017.log.gz 2024-09-17T05:35:59.691 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40023.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz 2024-09-17T05:35:59.692 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41480.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41480.log.gz 2024-09-17T05:35:59.692 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54894.log 2024-09-17T05:35:59.693 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56595.log 2024-09-17T05:35:59.695 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57837.log 2024-09-17T05:35:59.695 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54894.log.gz 2024-09-17T05:35:59.696 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56595.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56595.log.gz 2024-09-17T05:35:59.697 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57837.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57837.log.gz 2024-09-17T05:35:59.698 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50587.log 2024-09-17T05:35:59.699 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-09-17T05:35:59.700 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T05:35:59.700 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-09-17T05:35:59.700 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50587.log.gz 2024-09-17T05:35:59.703 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46671.log: /var/log/ceph/ceph-client.admin.44359.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-09-17T05:35:59.703 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-09-17T05:35:59.703 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47834.log 2024-09-17T05:35:59.704 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2024-09-17T05:35:59.705 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40629.log 2024-09-17T05:35:59.706 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47834.log.gz 2024-09-17T05:35:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44425.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2024-09-17T05:35:59.707 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45811.log 2024-09-17T05:35:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40629.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40629.log.gz 2024-09-17T05:35:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58229.log 2024-09-17T05:35:59.710 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45811.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45811.log.gz 2024-09-17T05:35:59.710 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56160.log 2024-09-17T05:35:59.712 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-09-17T05:35:59.712 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58229.log.gz 2024-09-17T05:35:59.712 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56160.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56160.log.gz 2024-09-17T05:35:59.715 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45676.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-09-17T05:35:59.715 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T05:35:59.715 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T05:35:59.716 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-09-17T05:35:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-09-17T05:35:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43561.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-09-17T05:35:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-09-17T05:35:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51826.log 2024-09-17T05:35:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2024-09-17T05:35:59.722 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:35:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51826.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51826.log.gz 2024-09-17T05:35:59.724 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49488.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2024-09-17T05:35:59.724 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2024-09-17T05:35:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34160.log 2024-09-17T05:35:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54408.log 2024-09-17T05:35:59.727 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2024-09-17T05:35:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34160.log.gz 2024-09-17T05:35:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54408.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54408.log.gz 2024-09-17T05:35:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49842.log 2024-09-17T05:35:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36429.log 2024-09-17T05:35:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43131.log 2024-09-17T05:35:59.732 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49842.log.gz 2024-09-17T05:35:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.36429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36429.log.gz 2024-09-17T05:35:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43131.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43131.log.gz 2024-09-17T05:35:59.735 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38288.log 2024-09-17T05:35:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55907.log 2024-09-17T05:35:59.737 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53961.log 2024-09-17T05:35:59.737 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38288.log.gz 2024-09-17T05:35:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55907.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55907.log.gz 2024-09-17T05:35:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44337.log 2024-09-17T05:35:59.740 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53961.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53961.log.gz 2024-09-17T05:35:59.741 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-09-17T05:35:59.742 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37325.log 2024-09-17T05:35:59.742 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44337.log.gz 2024-09-17T05:35:59.744 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40772.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-09-17T05:35:59.744 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37325.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37325.log.gz 2024-09-17T05:35:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58332.log 2024-09-17T05:35:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38236.log 2024-09-17T05:35:59.747 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49393.log 2024-09-17T05:35:59.748 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58332.log.gz 2024-09-17T05:35:59.748 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38236.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38236.log.gz 2024-09-17T05:35:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49393.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49393.log.gz 2024-09-17T05:35:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46951.log 2024-09-17T05:35:59.752 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-09-17T05:35:59.753 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-09-17T05:35:59.753 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46951.log.gz 2024-09-17T05:35:59.754 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37089.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-09-17T05:35:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36377.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-09-17T05:35:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52455.log 2024-09-17T05:35:59.757 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53912.log 2024-09-17T05:35:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46718.log 2024-09-17T05:35:59.759 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52455.log.gz 2024-09-17T05:35:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.53912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53912.log.gz 2024-09-17T05:35:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46718.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46718.log.gz 2024-09-17T05:35:59.761 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-09-17T05:35:59.762 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-09-17T05:35:59.763 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42897.log 2024-09-17T05:35:59.764 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-09-17T05:35:59.764 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41220.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 2024-09-17T05:35:59.766 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42897.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42897.log.gz 2024-09-17T05:35:59.766 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40053.log 2024-09-17T05:35:59.767 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2024-09-17T05:35:59.768 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-09-17T05:35:59.769 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53987.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40053.log.gz 2024-09-17T05:35:59.770 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2024-09-17T05:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36327.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-09-17T05:35:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-17T05:35:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48032.log 2024-09-17T05:35:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34353.log 2024-09-17T05:35:59.775 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-17T05:35:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48032.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48032.log.gz 2024-09-17T05:35:59.776 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34353.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34353.log.gz 2024-09-17T05:35:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-17T05:35:59.778 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-09-17T05:35:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51559.log 2024-09-17T05:35:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47312.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-17T05:35:59.781 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-09-17T05:35:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.51559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51559.log.gz 2024-09-17T05:35:59.782 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37136.log 2024-09-17T05:35:59.783 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2024-09-17T05:35:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.37136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-09-17T05:35:59.784 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37136.log.gz 2024-09-17T05:35:59.785 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48422.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2024-09-17T05:35:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34657.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-09-17T05:35:59.787 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47412.log 2024-09-17T05:35:59.788 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56618.log 2024-09-17T05:35:59.790 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33773.log 2024-09-17T05:35:59.790 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47412.log.gz 2024-09-17T05:35:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56618.log: /var/log/ceph/ceph-client.admin.33773.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56618.log.gz 2024-09-17T05:35:59.792 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33773.log.gz 2024-09-17T05:35:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-09-17T05:35:59.794 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-09-17T05:35:59.795 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-09-17T05:35:59.795 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-09-17T05:35:59.796 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46374.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-09-17T05:35:59.797 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52074.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-09-17T05:35:59.798 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44623.log 2024-09-17T05:35:59.799 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49541.log 2024-09-17T05:35:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35042.log 2024-09-17T05:35:59.801 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44623.log.gz 2024-09-17T05:35:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49541.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49541.log.gz 2024-09-17T05:35:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.35042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35042.log.gz 2024-09-17T05:35:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-17T05:35:59.804 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34932.log 2024-09-17T05:35:59.805 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39021.log 2024-09-17T05:35:59.805 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-17T05:35:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.34932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34932.log.gz 2024-09-17T05:35:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39021.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39021.log.gz 2024-09-17T05:35:59.808 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40174.log 2024-09-17T05:35:59.809 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-09-17T05:35:59.811 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36537.log 2024-09-17T05:35:59.811 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2024-09-17T05:35:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53670.log: /var/log/ceph/ceph-client.admin.36537.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-09-17T05:35:59.814 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36537.log.gz 2024-09-17T05:35:59.814 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-09-17T05:35:59.815 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39101.log 2024-09-17T05:35:59.816 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54378.log 2024-09-17T05:35:59.816 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-09-17T05:35:59.818 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39101.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-09-17T05:35:59.819 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54378.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54378.log.gz 2024-09-17T05:35:59.819 INFO:teuthology.orchestra.run.smithi062.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T05:35:59.819 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46471.log 2024-09-17T05:35:59.820 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-09-17T05:35:59.822 INFO:teuthology.orchestra.run.smithi062.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T05:35:59.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43294.log 2024-09-17T05:35:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41834.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46471.log.gz 2024-09-17T05:35:59.823 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-09-17T05:35:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34153.log 2024-09-17T05:35:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44160.log 2024-09-17T05:35:59.825 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43294.log.gz 2024-09-17T05:35:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-09-17T05:35:59.826 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34153.log.gz 2024-09-17T05:35:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44160.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44160.log.gz 2024-09-17T05:35:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49795.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-09-17T05:35:59.830 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51436.log 2024-09-17T05:35:59.831 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52771.log 2024-09-17T05:35:59.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46445.log 2024-09-17T05:35:59.833 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51436.log.gz 2024-09-17T05:35:59.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.52771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52771.log.gz 2024-09-17T05:35:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46445.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46445.log.gz 2024-09-17T05:35:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log 2024-09-17T05:35:59.836 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38808.log 2024-09-17T05:35:59.838 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54768.log 2024-09-17T05:35:59.838 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33825.log.gz 2024-09-17T05:35:59.839 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38808.log: /var/log/ceph/ceph-client.admin.54768.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38808.log.gz 2024-09-17T05:35:59.840 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54768.log.gz 2024-09-17T05:35:59.840 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34360.log 2024-09-17T05:35:59.841 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-09-17T05:35:59.843 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53520.log 2024-09-17T05:35:59.843 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34360.log.gz 2024-09-17T05:35:59.844 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47164.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-09-17T05:35:59.846 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53520.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53520.log.gz 2024-09-17T05:35:59.846 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-09-17T05:35:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55881.log 2024-09-17T05:35:59.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33643.log 2024-09-17T05:35:59.849 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-09-17T05:35:59.850 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55881.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55881.log.gz 2024-09-17T05:35:59.851 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33643.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33643.log.gz 2024-09-17T05:35:59.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41376.log 2024-09-17T05:35:59.852 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37155.log 2024-09-17T05:35:59.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-09-17T05:35:59.854 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41376.log.gz 2024-09-17T05:35:59.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37155.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37155.log.gz 2024-09-17T05:35:59.856 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56052.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-09-17T05:35:59.856 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51294.log 2024-09-17T05:35:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57237.log 2024-09-17T05:35:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-09-17T05:35:59.859 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51294.log.gz 2024-09-17T05:35:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57237.log: /var/log/ceph/ceph-client.admin.44535.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57237.log.gz 2024-09-17T05:35:59.861 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-09-17T05:35:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35177.log 2024-09-17T05:35:59.863 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57561.log 2024-09-17T05:35:59.864 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58151.log 2024-09-17T05:35:59.865 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35177.log.gz 2024-09-17T05:35:59.866 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57561.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57561.log.gz 2024-09-17T05:35:59.867 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58151.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58151.log.gz 2024-09-17T05:35:59.867 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-09-17T05:35:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52234.log 2024-09-17T05:35:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41090.log 2024-09-17T05:35:59.870 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-09-17T05:35:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52234.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52234.log.gz 2024-09-17T05:35:59.872 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41090.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41090.log.gz 2024-09-17T05:35:59.873 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37351.log 2024-09-17T05:35:59.874 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52674.log 2024-09-17T05:35:59.875 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44447.log 2024-09-17T05:35:59.875 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37351.log.gz 2024-09-17T05:35:59.877 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52674.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52674.log.gz 2024-09-17T05:35:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44447.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44447.log.gz 2024-09-17T05:35:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39428.log 2024-09-17T05:35:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-09-17T05:35:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35483.log 2024-09-17T05:35:59.881 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39428.log.gz 2024-09-17T05:35:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41593.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-09-17T05:35:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35483.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35483.log.gz 2024-09-17T05:35:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-09-17T05:35:59.884 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33431.log 2024-09-17T05:35:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28820.log 2024-09-17T05:35:59.886 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-09-17T05:35:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33431.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33431.log.gz 2024-09-17T05:35:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28820.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.28820.log.gz 2024-09-17T05:35:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55647.log 2024-09-17T05:35:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33943.log 2024-09-17T05:35:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58203.log 2024-09-17T05:35:59.891 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55647.log.gz 2024-09-17T05:35:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33943.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33943.log.gz 2024-09-17T05:35:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58203.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58203.log.gz 2024-09-17T05:35:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50803.log 2024-09-17T05:35:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39574.log 2024-09-17T05:35:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38078.log 2024-09-17T05:35:59.897 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50803.log.gz 2024-09-17T05:35:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39574.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39574.log.gz 2024-09-17T05:35:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38078.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-09-17T05:35:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-17T05:35:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2024-09-17T05:35:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50030.log 2024-09-17T05:35:59.902 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-17T05:35:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52793.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz 2024-09-17T05:35:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50030.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50030.log.gz 2024-09-17T05:35:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46900.log 2024-09-17T05:35:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2024-09-17T05:35:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.46900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-09-17T05:35:59.907 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46900.log.gz 2024-09-17T05:35:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50314.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2024-09-17T05:35:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34373.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-09-17T05:35:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-09-17T05:35:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-09-17T05:35:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49690.log 2024-09-17T05:35:59.912 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-09-17T05:35:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39719.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz 2024-09-17T05:35:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.49690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49690.log.gz 2024-09-17T05:35:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-09-17T05:35:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36907.log 2024-09-17T05:35:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-09-17T05:35:59.917 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-09-17T05:35:59.919 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36907.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36907.log.gz 2024-09-17T05:35:59.919 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39231.log 2024-09-17T05:35:59.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-09-17T05:35:59.921 INFO:teuthology.orchestra.run.smithi043.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T05:35:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44204.log 2024-09-17T05:35:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55335.log 2024-09-17T05:35:59.922 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39231.log.gz 2024-09-17T05:35:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-09-17T05:35:59.923 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44204.log.gz 2024-09-17T05:35:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48393.log 2024-09-17T05:35:59.925 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55335.log.gz 2024-09-17T05:35:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33747.log 2024-09-17T05:35:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45552.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-09-17T05:35:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39693.log 2024-09-17T05:35:59.927 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2024-09-17T05:35:59.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33747.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33747.log.gz 2024-09-17T05:35:59.927 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49106.log 2024-09-17T05:35:59.928 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55725.log 2024-09-17T05:35:59.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39693.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39693.log.gz 2024-09-17T05:35:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-09-17T05:35:59.930 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49106.log.gz 2024-09-17T05:35:59.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55725.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55725.log.gz 2024-09-17T05:35:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46400.log 2024-09-17T05:35:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40818.log 2024-09-17T05:35:59.932 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-09-17T05:35:59.933 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-09-17T05:35:59.933 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46400.log.gz 2024-09-17T05:35:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40818.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40818.log.gz 2024-09-17T05:35:59.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-09-17T05:35:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-09-17T05:35:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36114.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-09-17T05:35:59.937 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50395.log 2024-09-17T05:35:59.937 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-09-17T05:35:59.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47630.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-09-17T05:35:59.938 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.39493.log 2024-09-17T05:35:59.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.50395.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-09-17T05:35:59.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39493.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50395.log.gz 2024-09-17T05:35:59.939 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39493.log.gz 2024-09-17T05:35:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52927.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-09-17T05:35:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56932.log 2024-09-17T05:35:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-09-17T05:35:59.944 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55052.log 2024-09-17T05:35:59.944 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56932.log.gz 2024-09-17T05:35:59.945 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-17T05:35:59.945 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-09-17T05:35:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52901.log 2024-09-17T05:35:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55052.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2024-09-17T05:35:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-09-17T05:35:59.948 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz 2024-09-17T05:35:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52901.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52901.log.gz 2024-09-17T05:35:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:35:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.47482.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50751.log 2024-09-17T05:35:59.950 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-09-17T05:35:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35762.log 2024-09-17T05:35:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T05:35:59.952 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50751.log.gz 2024-09-17T05:35:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40981.log 2024-09-17T05:35:59.953 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35762.log.gz 2024-09-17T05:35:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.c.log: gzip 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:35:59.955 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53210.log 2024-09-17T05:35:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.40981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40981.log.gz 2024-09-17T05:35:59.956 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34149.log 2024-09-17T05:35:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47934.log 2024-09-17T05:35:59.957 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53210.log.gz 2024-09-17T05:35:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34149.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34149.log.gz 2024-09-17T05:35:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48235.log 2024-09-17T05:35:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42202.log 2024-09-17T05:35:59.960 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47934.log.gz 2024-09-17T05:35:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39602.log 2024-09-17T05:35:59.961 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48235.log.gz 2024-09-17T05:35:59.961 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42202.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42202.log.gz 2024-09-17T05:35:59.963 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39602.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39602.log.gz 2024-09-17T05:35:59.963 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51771.log 2024-09-17T05:35:59.964 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T05:35:59.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37656.log 2024-09-17T05:35:59.966 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51771.log.gz 2024-09-17T05:35:59.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.37656.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37656.log.gz 2024-09-17T05:35:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-09-17T05:35:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-09-17T05:35:59.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-09-17T05:35:59.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44909.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-09-17T05:35:59.973 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52208.log 2024-09-17T05:35:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55130.log 2024-09-17T05:35:59.976 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52208.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2024-09-17T05:35:59.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55130.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.55130.log.gz 2024-09-17T05:35:59.979 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45411.log 2024-09-17T05:35:59.980 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-09-17T05:35:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45411.log: /var/log/ceph/ceph-client.admin.37682.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.45411.log.gz 2024-09-17T05:35:59.983 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-09-17T05:35:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-09-17T05:35:59.984 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-09-17T05:35:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36987.log: 91.1%/var/log/ceph/ceph-client.admin.40707.log: -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-09-17T05:35:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56892.log 2024-09-17T05:35:59.987 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-09-17T05:35:59.988 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-09-17T05:35:59.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.56892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56892.log.gz 2024-09-17T05:35:59.990 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-09-17T05:35:59.991 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43587.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-09-17T05:35:59.992 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50517.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-09-17T05:35:59.992 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47575.log 2024-09-17T05:35:59.994 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33131.log 2024-09-17T05:35:59.995 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47575.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47575.log.gz 2024-09-17T05:35:59.996 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33131.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33131.log.gz 2024-09-17T05:35:59.997 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38538.log 2024-09-17T05:35:59.998 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-09-17T05:36:00.000 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38538.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38538.log.gz 2024-09-17T05:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42537.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-09-17T05:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51268.log 2024-09-17T05:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51268.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51268.log.gz 2024-09-17T05:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41916.log 2024-09-17T05:36:00.007 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41916.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41916.log.gz 2024-09-17T05:36:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-09-17T05:36:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47601.log 2024-09-17T05:36:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43472.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-09-17T05:36:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47601.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47601.log.gz 2024-09-17T05:36:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36823.log 2024-09-17T05:36:00.012 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T05:36:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-09-17T05:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55991.log 2024-09-17T05:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36823.log.gz 2024-09-17T05:36:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37481.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-09-17T05:36:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55991.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55991.log.gz 2024-09-17T05:36:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37734.log 2024-09-17T05:36:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28843.log 2024-09-17T05:36:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49567.log 2024-09-17T05:36:00.019 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37734.log.gz 2024-09-17T05:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-09-17T05:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28843.log.gz 2024-09-17T05:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49567.log: gzip -5 94.0% --verbose -- -- replaced with /var/log/ceph/ceph-osd.0.log.gz /var/log/ceph/ceph-client.admin.46197.log 2024-09-17T05:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-17T05:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49567.log.gz 2024-09-17T05:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-09-17T05:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-09-17T05:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33799.log 2024-09-17T05:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2024-09-17T05:36:00.025 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42295.log 2024-09-17T05:36:00.025 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-09-17T05:36:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57161.log 2024-09-17T05:36:00.026 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33799.log.gz 2024-09-17T05:36:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2024-09-17T05:36:00.027 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42295.log.gz 2024-09-17T05:36:00.028 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54341.log 2024-09-17T05:36:00.028 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57161.log.gz 2024-09-17T05:36:00.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50151.log 2024-09-17T05:36:00.030 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2024-09-17T05:36:00.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54341.log: gzip -5 87.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54341.log.gz 2024-09-17T05:36:00.030 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.28926.log 2024-09-17T05:36:00.031 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48641.log 2024-09-17T05:36:00.032 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50151.log.gz 2024-09-17T05:36:00.032 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57917.log 2024-09-17T05:36:00.033 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28926.log.gz 2024-09-17T05:36:00.033 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48641.log: /var/log/ceph/ceph-client.admin.33487.log 2024-09-17T05:36:00.034 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48641.log.gz 2024-09-17T05:36:00.035 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45029.log 2024-09-17T05:36:00.035 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57917.log.gz 2024-09-17T05:36:00.035 INFO:teuthology.orchestra.run.smithi062.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T05:36:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33487.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-09-17T05:36:00.036 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53257.log 2024-09-17T05:36:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28784.log 2024-09-17T05:36:00.038 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-09-17T05:36:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35971.log 2024-09-17T05:36:00.038 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53257.log.gz 2024-09-17T05:36:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52018.log 2024-09-17T05:36:00.039 INFO:teuthology.orchestra.run.smithi043.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.28784.log.gz 2024-09-17T05:36:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57370.log 2024-09-17T05:36:00.041 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35971.log.gz 2024-09-17T05:36:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36171.log 2024-09-17T05:36:00.042 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52018.log.gz 2024-09-17T05:36:00.043 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55177.log 2024-09-17T05:36:00.043 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57370.log.gz 2024-09-17T05:36:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-09-17T05:36:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36171.log: /var/log/ceph/ceph-client.admin.55177.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36171.log.gz 2024-09-17T05:36:00.045 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55177.log.gz 2024-09-17T05:36:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47959.log 2024-09-17T05:36:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52724.log 2024-09-17T05:36:00.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42845.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-09-17T05:36:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47959.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.47959.log.gz -5 2024-09-17T05:36:00.047 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-09-17T05:36:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-09-17T05:36:00.048 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52724.log.gz 2024-09-17T05:36:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50366.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-09-17T05:36:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33669.log 2024-09-17T05:36:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57454.log 2024-09-17T05:36:00.051 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-09-17T05:36:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33669.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33669.log.gz 2024-09-17T05:36:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49132.log 2024-09-17T05:36:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-17T05:36:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57454.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57454.log.gz 2024-09-17T05:36:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49132.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49132.log.gz 2024-09-17T05:36:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49716.log 2024-09-17T05:36:00.054 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47889.log 2024-09-17T05:36:00.055 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-09-17T05:36:00.055 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49716.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.49716.log.gz -5 2024-09-17T05:36:00.055 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-09-17T05:36:00.056 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2024-09-17T05:36:00.057 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47889.log.gz 2024-09-17T05:36:00.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48884.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz -5 2024-09-17T05:36:00.058 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-09-17T05:36:00.059 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-09-17T05:36:00.059 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2024-09-17T05:36:00.060 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50491.log 2024-09-17T05:36:00.060 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-09-17T05:36:00.061 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57632.log 2024-09-17T05:36:00.061 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50491.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-09-17T05:36:00.062 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37800.log 87.1% 2024-09-17T05:36:00.062 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50491.log.gz 2024-09-17T05:36:00.063 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-09-17T05:36:00.063 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57632.log.gz 2024-09-17T05:36:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37800.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37800.log.gz -5 2024-09-17T05:36:00.064 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34735.log 2024-09-17T05:36:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53136.log 2024-09-17T05:36:00.065 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-09-17T05:36:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2024-09-17T05:36:00.067 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34735.log.gz 2024-09-17T05:36:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51662.log 2024-09-17T05:36:00.068 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53136.log.gz 2024-09-17T05:36:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48910.log 2024-09-17T05:36:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57811.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2024-09-17T05:36:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2024-09-17T05:36:00.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48910.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.51662.log.gz 2024-09-17T05:36:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48518.log 2024-09-17T05:36:00.071 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-09-17T05:36:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41719.log 2024-09-17T05:36:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52485.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2024-09-17T05:36:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48518.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48518.log.gz 2024-09-17T05:36:00.073 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45343.log 2024-09-17T05:36:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33942.log 2024-09-17T05:36:00.075 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41719.log.gz 2024-09-17T05:36:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45343.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.45343.log.gz -5 2024-09-17T05:36:00.075 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-17T05:36:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38106.log 2024-09-17T05:36:00.077 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33942.log.gz 2024-09-17T05:36:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52992.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-17T05:36:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55361.log 2024-09-17T05:36:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-09-17T05:36:00.079 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38106.log.gz 2024-09-17T05:36:00.079 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44381.log 87.8% 2024-09-17T05:36:00.080 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55361.log.gz 2024-09-17T05:36:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44689.log 2024-09-17T05:36:00.081 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-09-17T05:36:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44381.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44381.log.gz 2024-09-17T05:36:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41747.log 2024-09-17T05:36:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33940.log 2024-09-17T05:36:00.083 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44689.log.gz 2024-09-17T05:36:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41747.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41747.log.gz 2024-09-17T05:36:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-09-17T05:36:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57943.log 2024-09-17T05:36:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33940.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33940.log.gz 2024-09-17T05:36:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-09-17T05:36:00.086 INFO:teuthology.orchestra.run.smithi043.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-09-17T05:36:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38340.log 2024-09-17T05:36:00.087 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57943.log.gz 2024-09-17T05:36:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-17T05:36:00.088 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-09-17T05:36:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56026.log 2024-09-17T05:36:00.089 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38340.log.gz 2024-09-17T05:36:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43235.log 2024-09-17T05:36:00.091 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-17T05:36:00.092 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56026.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.56026.log.gz 2024-09-17T05:36:00.092 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.40369.log 2024-09-17T05:36:00.092 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38626.log 2024-09-17T05:36:00.093 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43235.log.gz 2024-09-17T05:36:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48367.log 2024-09-17T05:36:00.094 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40369.log.gz 2024-09-17T05:36:00.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38626.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38626.log.gz 2024-09-17T05:36:00.095 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47213.log 2024-09-17T05:36:00.096 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35945.log 2024-09-17T05:36:00.096 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48367.log.gz 2024-09-17T05:36:00.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51897.log 2024-09-17T05:36:00.098 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50340.log 2024-09-17T05:36:00.099 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.47213.log.gz 2024-09-17T05:36:00.099 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35945.log.gz 2024-09-17T05:36:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-09-17T05:36:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50340.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51897.log.gz 2024-09-17T05:36:00.101 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2024-09-17T05:36:00.101 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47047.log 2024-09-17T05:36:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55517.log 2024-09-17T05:36:00.102 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-09-17T05:36:00.103 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58125.log 2024-09-17T05:36:00.103 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47047.log.gz 2024-09-17T05:36:00.104 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34368.log 2024-09-17T05:36:00.104 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55517.log.gz 2024-09-17T05:36:00.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43261.log 2024-09-17T05:36:00.106 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58125.log.gz 2024-09-17T05:36:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34368.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34368.log.gz 2024-09-17T05:36:00.106 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43417.log 2024-09-17T05:36:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.43261.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56212.log 2024-09-17T05:36:00.108 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43261.log.gz 2024-09-17T05:36:00.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53110.log 2024-09-17T05:36:00.109 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43417.log.gz 2024-09-17T05:36:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56212.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56212.log.gz 2024-09-17T05:36:00.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29008.log 2024-09-17T05:36:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56498.log 2024-09-17T05:36:00.112 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53110.log.gz 2024-09-17T05:36:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29008.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29008.log.gz 2024-09-17T05:36:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56264.log 2024-09-17T05:36:00.113 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-09-17T05:36:00.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.56498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56498.log.gz 2024-09-17T05:36:00.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56264.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56264.log.gz 2024-09-17T05:36:00.114 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36197.log 2024-09-17T05:36:00.115 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.43799.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42458.log 2024-09-17T05:36:00.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36197.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-09-17T05:36:00.116 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36197.log.gz 2024-09-17T05:36:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-09-17T05:36:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57108.log 2024-09-17T05:36:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42458.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42458.log.gz 2024-09-17T05:36:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-09-17T05:36:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46573.log 2024-09-17T05:36:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54029.log 2024-09-17T05:36:00.120 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57108.log.gz 2024-09-17T05:36:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.46573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46573.log.gz 2024-09-17T05:36:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-09-17T05:36:00.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33956.log 2024-09-17T05:36:00.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54029.log: /var/log/ceph/ceph-client.admin.56535.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-09-17T05:36:00.123 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-17T05:36:00.123 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.54029.log.gz 2024-09-17T05:36:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37526.log 2024-09-17T05:36:00.124 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33956.log.gz 2024-09-17T05:36:00.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37377.log 2024-09-17T05:36:00.125 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-17T05:36:00.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2024-09-17T05:36:00.126 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37526.log.gz 2024-09-17T05:36:00.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37377.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37377.log.gz 2024-09-17T05:36:00.127 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55751.log 2024-09-17T05:36:00.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-09-17T05:36:00.128 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2024-09-17T05:36:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55751.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55751.log.gz 2024-09-17T05:36:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-17T05:36:00.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38969.log 2024-09-17T05:36:00.131 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-09-17T05:36:00.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-17T05:36:00.131 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51564.log 2024-09-17T05:36:00.132 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36799.log 2024-09-17T05:36:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38969.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38969.log.gz 2024-09-17T05:36:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.51564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51564.log.gz 2024-09-17T05:36:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36752.log 2024-09-17T05:36:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39075.log 2024-09-17T05:36:00.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.36799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36799.log.gz 2024-09-17T05:36:00.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41324.log 2024-09-17T05:36:00.135 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36752.log.gz 2024-09-17T05:36:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-09-17T05:36:00.137 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39075.log.gz 2024-09-17T05:36:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41324.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41324.log.gz 2024-09-17T05:36:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36027.log 2024-09-17T05:36:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44777.log 2024-09-17T05:36:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42871.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-09-17T05:36:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36027.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36027.log.gz 2024-09-17T05:36:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-09-17T05:36:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2024-09-17T05:36:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44777.log: /var/log/ceph/ceph-client.admin.42046.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44777.log.gz 2024-09-17T05:36:00.142 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-09-17T05:36:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56795.log 2024-09-17T05:36:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41994.log 2024-09-17T05:36:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2024-09-17T05:36:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56769.log 2024-09-17T05:36:00.144 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56795.log.gz 2024-09-17T05:36:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39823.log 2024-09-17T05:36:00.145 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41994.log.gz 2024-09-17T05:36:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57658.log 2024-09-17T05:36:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56769.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56769.log.gz 2024-09-17T05:36:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39823.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39823.log.gz 2024-09-17T05:36:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-09-17T05:36:00.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57658.log: /var/log/ceph/ceph-client.admin.39363.log 2024-09-17T05:36:00.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57658.log.gz 2024-09-17T05:36:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38704.log 2024-09-17T05:36:00.149 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-09-17T05:36:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29013.log 2024-09-17T05:36:00.150 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39363.log.gz 2024-09-17T05:36:00.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48305.log 2024-09-17T05:36:00.151 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38704.log.gz 2024-09-17T05:36:00.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33591.log 2024-09-17T05:36:00.153 INFO:teuthology.orchestra.run.smithi043.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29013.log.gz 2024-09-17T05:36:00.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48305.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48305.log.gz 2024-09-17T05:36:00.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53349.log 2024-09-17T05:36:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42650.log 2024-09-17T05:36:00.155 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33591.log.gz 2024-09-17T05:36:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.53349.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53349.log.gz 2024-09-17T05:36:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-09-17T05:36:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54145.log 2024-09-17T05:36:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42650.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42650.log.gz 2024-09-17T05:36:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42126.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-09-17T05:36:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53494.log 2024-09-17T05:36:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-09-17T05:36:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54145.log: /var/log/ceph/ceph-client.admin.53494.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54145.log.gz 2024-09-17T05:36:00.160 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53494.log.gz 2024-09-17T05:36:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47720.log 2024-09-17T05:36:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.50543.log: -5 --verbose -- /var/log/ceph/ceph-client.0.36804.log 2024-09-17T05:36:00.161 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-09-17T05:36:00.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51797.log 2024-09-17T05:36:00.162 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47720.log.gz 2024-09-17T05:36:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.36804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-17T05:36:00.163 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36804.log.gz 2024-09-17T05:36:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51797.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51797.log.gz 2024-09-17T05:36:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48742.log 2024-09-17T05:36:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-09-17T05:36:00.165 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-17T05:36:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29002.log 2024-09-17T05:36:00.166 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48742.log.gz 2024-09-17T05:36:00.167 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50421.log: -- /var/log/ceph/ceph-client.admin.49202.log 2024-09-17T05:36:00.168 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-09-17T05:36:00.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29002.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29002.log.gz 2024-09-17T05:36:00.168 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38678.log 2024-09-17T05:36:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-09-17T05:36:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49202.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49202.log.gz 2024-09-17T05:36:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38678.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38678.log.gzgzip 2024-09-17T05:36:00.170 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55543.log 2024-09-17T05:36:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-09-17T05:36:00.172 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-09-17T05:36:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-09-17T05:36:00.173 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55543.log.gz 2024-09-17T05:36:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-09-17T05:36:00.174 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-09-17T05:36:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48980.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-09-17T05:36:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-09-17T05:36:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58358.log 2024-09-17T05:36:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43027.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-09-17T05:36:00.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43053.log 2024-09-17T05:36:00.177 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-09-17T05:36:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55439.log 2024-09-17T05:36:00.178 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58358.log.gz 2024-09-17T05:36:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43053.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43053.log.gz 2024-09-17T05:36:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-09-17T05:36:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58073.log 2024-09-17T05:36:00.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55439.log: /var/log/ceph/ceph-client.admin.44469.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55439.log.gz 2024-09-17T05:36:00.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-09-17T05:36:00.181 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-09-17T05:36:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47656.log 2024-09-17T05:36:00.182 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58073.log.gz 2024-09-17T05:36:00.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.49817.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-09-17T05:36:00.184 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54820.log 2024-09-17T05:36:00.185 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40929.log 2024-09-17T05:36:00.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47656.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47656.log.gz 2024-09-17T05:36:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54820.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54820.log.gz 2024-09-17T05:36:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40502.log 2024-09-17T05:36:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57587.log 2024-09-17T05:36:00.187 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40929.log.gz 2024-09-17T05:36:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40502.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.40502.log.gz 2024-09-17T05:36:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51054.log 2024-09-17T05:36:00.189 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57587.log: -- /var/log/ceph/ceph-client.admin.50278.log 2024-09-17T05:36:00.189 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57587.log.gz 2024-09-17T05:36:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51054.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51054.log.gz 2024-09-17T05:36:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-09-17T05:36:00.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28808.log 2024-09-17T05:36:00.191 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-09-17T05:36:00.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40150.log 87.1% 2024-09-17T05:36:00.192 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-09-17T05:36:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.28808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37604.log 2024-09-17T05:36:00.193 INFO:teuthology.orchestra.run.smithi043.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.28808.log.gz 2024-09-17T05:36:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.40150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40150.log.gz 2024-09-17T05:36:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-09-17T05:36:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50903.log 2024-09-17T05:36:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37604.log: /var/log/ceph/ceph-client.admin.48493.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37604.log.gz 2024-09-17T05:36:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-09-17T05:36:00.197 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-09-17T05:36:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.50903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-09-17T05:36:00.198 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50903.log.gz 2024-09-17T05:36:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38488.log 2024-09-17T05:36:00.199 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-09-17T05:36:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48955.log: gzip -5 --verbose 89.2% -- /var/log/ceph/ceph-client.admin.36301.log -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-09-17T05:36:00.200 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-17T05:36:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45099.log 2024-09-17T05:36:00.202 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38488.log.gz 2024-09-17T05:36:00.202 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35561.log 2024-09-17T05:36:00.203 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36301.log.gz 2024-09-17T05:36:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 2024-09-17T05:36:00.204 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45099.log.gz 2024-09-17T05:36:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35561.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35561.log.gz 2024-09-17T05:36:00.207 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52312.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2024-09-17T05:36:00.208 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48058.log 2024-09-17T05:36:00.209 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39901.log 2024-09-17T05:36:00.210 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39546.log 2024-09-17T05:36:00.211 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48058.log.gz 2024-09-17T05:36:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54611.log 2024-09-17T05:36:00.212 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39901.log.gz 2024-09-17T05:36:00.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-09-17T05:36:00.213 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39546.log.gz 2024-09-17T05:36:00.214 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-09-17T05:36:00.214 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54611.log.gz 2024-09-17T05:36:00.215 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57135.log 2024-09-17T05:36:00.215 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz 2024-09-17T05:36:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51583.log 2024-09-17T05:36:00.216 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-09-17T05:36:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35867.log 2024-09-17T05:36:00.217 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57135.log.gz 2024-09-17T05:36:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51583.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51583.log.gz 2024-09-17T05:36:00.218 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-17T05:36:00.219 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35867.log.gz 2024-09-17T05:36:00.221 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.29012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-09-17T05:36:00.221 INFO:teuthology.orchestra.run.smithi043.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gz 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43183.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36145.log: 87.2%/var/log/ceph/ceph-client.admin.45369.log: -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35375.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-09-17T05:36:00.226 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38595.log 2024-09-17T05:36:00.227 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-09-17T05:36:00.227 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-09-17T05:36:00.227 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52146.log 2024-09-17T05:36:00.227 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38595.log.gz 2024-09-17T05:36:00.228 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46521.log 2024-09-17T05:36:00.228 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gz 2024-09-17T05:36:00.229 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52146.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52146.log.gz 2024-09-17T05:36:00.231 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46521.log.gz 2024-09-17T05:36:00.231 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-09-17T05:36:00.232 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51922.log 2024-09-17T05:36:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42374.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-09-17T05:36:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-09-17T05:36:00.234 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-09-17T05:36:00.234 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51922.log.gz 2024-09-17T05:36:00.235 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52581.log 2024-09-17T05:36:00.235 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-09-17T05:36:00.236 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46977.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-09-17T05:36:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2024-09-17T05:36:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52581.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52581.log.gz 2024-09-17T05:36:00.239 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47022.log 2024-09-17T05:36:00.240 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56370.log 2024-09-17T05:36:00.241 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48829.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-09-17T05:36:00.241 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-09-17T05:36:00.242 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47022.log.gz 2024-09-17T05:36:00.243 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52334.log 2024-09-17T05:36:00.243 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56370.log.gz 2024-09-17T05:36:00.243 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48855.log 2024-09-17T05:36:00.244 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44557.log: /var/log/ceph/ceph-client.admin.52334.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-09-17T05:36:00.245 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-09-17T05:36:00.245 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52334.log.gz 2024-09-17T05:36:00.246 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48855.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48855.log.gz 2024-09-17T05:36:00.247 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45868.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-09-17T05:36:00.248 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45990.log 2024-09-17T05:36:00.248 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34345.log 2024-09-17T05:36:00.250 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34956.log 2024-09-17T05:36:00.250 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45990.log.gz 2024-09-17T05:36:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34345.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34345.log.gz 2024-09-17T05:36:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46895.log 2024-09-17T05:36:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34141.log 2024-09-17T05:36:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34956.log: /var/log/ceph/ceph-client.0.46895.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34956.log.gz 2024-09-17T05:36:00.253 INFO:teuthology.orchestra.run.smithi062.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T05:36:00.255 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46895.log.gz 2024-09-17T05:36:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38158.log 2024-09-17T05:36:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54794.log 2024-09-17T05:36:00.255 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34141.log.gz 2024-09-17T05:36:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38158.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38158.log.gz 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47386.log 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.54794.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51187.log 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m1.150s 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m4.569s 2024-09-17T05:36:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m0.433s 2024-09-17T05:36:00.258 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54794.log.gz 2024-09-17T05:36:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58021.log 2024-09-17T05:36:00.258 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47386.log.gz 2024-09-17T05:36:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.51187.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.51187.log.gz 2024-09-17T05:36:00.258 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2024-09-17T05:36:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-09-17T05:36:00.260 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58021.log.gz 2024-09-17T05:36:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54939.log 2024-09-17T05:36:00.261 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz 2024-09-17T05:36:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57759.log 2024-09-17T05:36:00.262 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-09-17T05:36:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.54939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36249.log 2024-09-17T05:36:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57759.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54939.log.gz 2024-09-17T05:36:00.264 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-09-17T05:36:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-09-17T05:36:00.265 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40976.log 2024-09-17T05:36:00.265 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36249.log.gz 2024-09-17T05:36:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36646.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-09-17T05:36:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52385.log 2024-09-17T05:36:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.40976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41808.log 2024-09-17T05:36:00.268 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40976.log.gz 2024-09-17T05:36:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.52385.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52385.log.gz 2024-09-17T05:36:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41808.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41808.log.gz 2024-09-17T05:36:00.300 INFO:teuthology.orchestra.run.smithi043.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T05:36:00.330 INFO:teuthology.orchestra.run.smithi043.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T05:36:00.408 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T05:36:00.550 INFO:teuthology.orchestra.run.smithi043.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T05:36:00.552 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-17T05:36:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m1.463s 2024-09-17T05:36:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m5.694s 2024-09-17T05:36:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m1.911s 2024-09-17T05:36:00.554 INFO:tasks.ceph:Archiving logs... 2024-09-17T05:36:00.554 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi043/log 2024-09-17T05:36:00.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:36:01.811 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi062/log 2024-09-17T05:36:01.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:36:02.302 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T05:36:02.311 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T05:36:02.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:36:02.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:36:02.398 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T05:36:02.407 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T05:36:02.407 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:36:02.410 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout:*hv01.front.sepi 67.205.162.81 3 u 3 64 377 0.070 -1.574 0.231 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout:+hv02.front.sepi 63.231.80.2 3 u 9 64 377 0.078 -1.173 0.523 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout:+hv03.front.sepi 74.6.168.72 3 u 67 64 377 0.102 +1.443 0.218 2024-09-17T05:36:02.423 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================== 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout:+hv01.front.sepi 67.205.162.81 3 u 59 64 377 0.109 -2.354 0.981 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout:*hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.079 -2.708 0.665 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout:+hv03.front.sepi 74.6.168.72 3 u 61 64 377 0.066 -0.043 0.768 2024-09-17T05:36:02.426 INFO:teuthology.orchestra.run.smithi062.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-17T05:36:02.427 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T05:36:02.436 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T05:36:02.437 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T05:36:02.486 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T05:36:02.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T05:36:02.587 INFO:teuthology.task.internal:Duration was 1499.336500 seconds 2024-09-17T05:36:02.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T05:36:02.637 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T05:36:02.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:36:02.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:36:02.738 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T05:36:02.739 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T05:36:02.739 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:36:02.760 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-09-17T05:36:02.760 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:36:02.781 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T05:36:02.781 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:36:02.809 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:36:02.862 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T05:36:02.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:36:02.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:36:02.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T05:36:03.001 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T05:36:03.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:36:03.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:36:03.042 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T05:36:03.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:36:03.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:36:03.084 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-17T05:36:03.099 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-09-17T05:36:03.118 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:36:03.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:36:03.153 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:36:03.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:36:03.169 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T05:36:03.179 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T05:36:03.180 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi043 2024-09-17T05:36:03.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:36:03.226 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908606/remote/smithi062 2024-09-17T05:36:03.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:36:03.252 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T05:36:03.252 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:36:03.270 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:36:03.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T05:36:03.309 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T05:36:03.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T05:36:03.337 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T05:36:03.338 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:36:03.341 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:36 /home/ubuntu/cephtest 2024-09-17T05:36:03.350 INFO:teuthology.orchestra.run.smithi062.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:36 /home/ubuntu/cephtest 2024-09-17T05:36:03.352 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T05:36:03.388 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T05:36:03.491 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} duration: 1499.3365001678467 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T05:36:03.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T05:36:03.599 INFO:teuthology.run:pass